多点触摸技术3

Demo Applets

Conclusion and Remarks

Something new can be expected to emerge from this interactive experience: a new perspective on the recorded events and their temporal relationship – something perhaps not even accessible to the film maker in the first place. The spectator will be an explorer, waking her/his own path on the bulk spatio-temporal data of the movie, not helped nor constrained by the temporal arrow that fills space, dictates causality and creates understanding in a usual movie projection. At the same time, he will certainly feel lost: for to understand what’s going on, will now demand an active participation, an exhaustive and conscious exploration of each portion of the visual space. We can say that by shattering the spatio-temporal volume of the movie, the Khronos projector forces our conscious attention over silent yet powerful perceptual-unification processes that encompass intensive recalling, spatial memory skills and logic reasoning. From my own experience with the Khronos Projector, I believe this can be a puzzling and recreational experience for the public. Presently, even the film maker less prone to linear-narratives is forced to integrate the screen-ubiquitous time arrow as a fact in her/his work. Therefore, from the creator’s side, something new may also emerge from the idea of having a Khronos Projector accessible to her/his spectators: a new way of approaching and conceiving a visual narrative altogether. (By the way, I wonder how many film makers have really integrated the fact that a large audience can nowadays see their work using a digital player – and thus being able to stop, rewind, replay and even skip some parts, just as if their were reading an illustrated book. The result is an interactive “audience’s cut”: people edit and post-produce the movie at home.) For instance, imagine a version of Hitchcock’s “The Rear Window” prepared for the Khronos-projector: as in the original movie, the whole story is to be reconstructed from pieces of sequences taking place simultaneously, but because of the hidden correlation between facts, understanding must heavily rely on memory. Now, thanks to the Khronos interface, we will be able to go backwards or forward in time at – literally – the desired window. It would probably destroy the interest of the whole movie, since suspense is maintained thanks to our actual ignorance and perpetual conjecturing. However, the movie in question was shot only to be seen in a movie theater, using conventional reproducing hardware, and having in mind that the projecting hardware was precisely the kind that does not allow the public to freely explore the space-time volume. The “Rear Window” for the Khronos projector will perhaps contain several possible interleaved stories, depending on the way the space-time volume is explored.

Future works

Last, I would like to point to some ideas for improvement of the present interface as well as further development of the whole idea:

• Full-body interface, capable of capturing complex and large deformations of the fabric. Using it, it would be possible to physically plunge into the video cube. A sensitive and large screen is currently under development using thin latex sheets instead of spandex (a similar use of this material has been magnificently demonstrated by multimedia artist H. Tanahashi [16]).

• Add other modalities to enhance the experience, such as sound or voice. A sample sound like stretching-rubber may be heard as the surface is stretched and deformed, contributing to an organic, non-digital experience. In the case of an interaction controlled by throwing pebbles onto the deformable display, the formation of “time-ripples” on the surface could be accompanied by the sound of a stone plunging in the water. In sequences involving talking persons, it would be interesting to intelligently integrate the audio-track, for instance by clipping and attaching phrases or words to precise spatio-temporal coordinates (i.e. events). Then, sounds, words or phrases will be triggered and modulated by the user pressure onto the screen.

• 3D Khronos projector, using GLUT Stereo frames (with LCD shuttered goggles). Fed with stereoscopic video, the Khronos Projector would display time-warped three-dimensional objects. (I cannot resist citing here T. Iwai’s tour de force: he built the real thing, i.e. a real three-dimensional “buffer” – a rotating house – that is time-sliced thanks to synchronized light beams, resulting in the illusion of a is physical “morphed” object thanks to image retention in the retina [15]). In an interesting variation of the idea, parallax could even be used to adjust the time coordinate (so that when trying to see “behind” some thing, would in fact be transposed as to seeing “before” (by rotating the space-time volume).

• Use image databases not necessarily coming from “slicing” video or time-lapse photography. As said above, an interesting example database may be a set of body scanner images; but it would also be interesting to apply the interactive and smooth “image-fusion” algorithm of the khronos projector to geological/aerial/historical maps, architectural or mechanical drawings, etc: in a word, to any set of images corresponding each to a different information layer. (Additional “data” would appear under the pressed area, smoothly blended/fused with the rest of the image.)

• Develop a completely different interface, that would enable sculpting and carving into the spatio-temporal volume of the movie as if it were made of clay; this simple deformable tissue-based screen could be replaced by a more sophisticated “tangible” display (using the amazing “GelForce” interface developed at the University of Tokyo [11], or a “tangible” display developed at MIT [12], or even an interface capable of complex haptic feedback (such as Sony “TouchEngine”) to achieve direct “tactile” sensing of the spatio-temporal object being sculpted and manipulated (e.g. spatio-temporal contours). This work is closely related to [3].

• Use the pressure=time paradigm not just to explore spatio-temporal content, but also to create new content. For instance, a similar interface could be used to search and place images, alphanumeric data or sound in 3D space, thus converting the space into a Volumetric Note-Pad. Of course, the depth coordinate could readily represent “time” (hours, days or months), making for a Volumetric Agenda.

• Last but not least, the Khronos-projector can develop into an interface for decrypting what’s going on in complex physical phenomena: for instance, recorded activity of neurons in the surface of the brain or femto-second quantum interaction phenomena in a lattice of atoms. In both cases, to be able to “evolve” in time some parts of the image but keep others in the present could be helpful to uncover long-range, causal relationships.

Contributions & Acknowledgements

• Mike Posehn from Granite Bay Software (I’ve been using his Time-Lapse software for Canon cameras. He also contributed with some superb time-lapse sequences. Check his website).
• Olivier & Saikal
• Nelson Balbela (independent artist).
• Takahito Ito, Master Student, University of Tokyo.
• Roger P. Hangarter, Associate Professor, Indiana University (contributed with superb time-lapse photographic sequences of growing plants. Check his awesome website “Plants in Motion” – plants are alive and restlessly moving!).
• Also, very special thanks to Naoko Itoh and all the dancers from the “Mademoiselle Cinema” dance company.

Exhibition/Awards History

• Split Film Festival (Croatia), 3/6-10/7/2005 (presented by Luc Foubert and Alex)
• SIGGRAPH 2005, Emerging Technologies, Los Angeles, 31/7-5/8/2005.
• Interactive Tokyo 2005 (25-26/8/2005).
• NHK Digital Stadium, Best Selection Award (aired at NHK-BS2, 8/10/2005).
• NHK Digital Stadium, Finalist Award (aired at NHK-BS, 14/1/2006).
• Panasonic Prize at the Digital Art Festival 2005 (9-13/12/2005).
• Grand Prize [Art Division] at the 9th Japan Media Art Festival (24/2-5/3/2006).
• Laval Virtual (France), (25-30/4/ 2006)
• Aired in “On a tout essaye” TV program, France2 television (4/5/2006).
• Collaboration with architect Hiroshi Naito: TIMESCAPE installation at the Okamura Design Space, Tokyo (13-28/07/2006) and featured on TV-Tokyo, 知恵の和 program 13/08/2006. Picture Gallery here.
• Honorary Mention at Prix Ars Electronica 2006 (Interactive Art Category). Piece exhibited at the O.K. Center for Contemporary Art in Linz, Austria. (Prix Ars and Cyberarts 2006 exhibition, 31/08-8/10/2006).
• Khronos (for a year or two) at the Ars Electronica Center (AEC) in Linz, Austria.
• Future of Design Pavillon at WIRED NEXTFEST 2006 in New York (20/09-01/10/2006).
• Animatronica – Microwave International New Media Arts Festival, Hong Kong (4-15/11/2006). (Invited talk here.)
• NTT InterCommunication Center “Sight & Sound: Watch the sound, listen to the images” exhibition, 14/7-2/9 2007. Featured a version of Khronos Projector with spatialized sound on the screen.
• Future of Design Pavillon at WIRED NEXTFEST 2007, Los Angeles, (13-16/09/2007).
• Festival ArtFutura, Barcelona 25-28/10 (2007).
• “Openplay”, Digital Art Festival Taipei, 23/11 – 2/12 Taipei, Taiwan (2007).
• VIA Festival International (20-22/3 in Maubeuge), and EXIT08 festival international (28/3-5/4, Maison des Arts de Créteil), France (2008).

Contact

Alvaro Cassinelli. Assistant Professor (Research Associate).
Ishikawa- Komuro Laboratory, Department of Information Physics and Computing. The University of Tokyo. 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033 Japan.
Tel: +81-3-5841-6937 / Fax: +81-3-5841-6952
Email: alvaro (at) k2.t.u-tokyo.ac.jp / cassinelli.alvaro (at) gmail.com

Some interesting references and links

[1] S. Jaschko. Space-Time Correlations Focused in Film Objects and Interactive Video. Published in: ISEA Papers, Nagoya/Japan (2002). Also published in Future Cinema.The Cinematic Imaginary after Film. Edited by Peter Weibel, MIT Press, (2003). [2] A. W. Klein et al. Stylized video cubes. Proc. the ACM SIGGRAPH Symposium on Comp. Animation, pp. 15-22, (2002). [3] S. Fels, E. Lee, and K. Mase. Techniques for interactive cubism. In Proceedings of the ACM Conference on Multimedia, pages 368–370, October (2000). [4] Camille Utterback. Liquid Time Series (2000). [5] Toshio Iwai, Another Time, Another Space (1993). [6] Z. B.. Simpson. Novel Infrared Touch-Screen Technology and Associated Artwork. Emerging Technologies, SIGGRAPH 2004, Los Angeles, (2004). [7] J. Angesleva and R. Cooper. Last Clock. Emerging Technologies, SIGGRAPH 2004, Los Angeles, August (2004). [8] T. Komuro, I. Ishii, M. Ishikawa and A. Yoshida. A Digital Vision Chip Specialized for High-speed Target Tracking. IEEE transaction on Electron Devices, Vol.50, No.1, pp.191-199 (2003). [9] A. Cassinelli, S. Perrin and M. Ishikawa. Markerless Laser-based Tracking for Real-Time 3D Gesture Acquisition. SIGGRAPH 2004, Los Angeles, (2004). [10] Y. Ugai, A. Namiki and M. Ishikawa. 3D Shape Recognition Using High Speed Vision and Active Lighting Device. The 4th SICE System Integration Division Annual Conference (SI2003) (Tokyo, Japan, 2003.12.19) / Proceedings, pp. 113-114, Dec. (2003). [11] K. Kamiyama, K. Vlack, T. Mizota, H. Kajimoto, N. Kawakami and S. Tachi. GelForce. Emerging Technologies, SIGGRAPH 2004, Los Angeles, (2004). [12] H. Ishii, C Ratti, B. Piper, Y. Wang, A. Biderman and E. Ben-Joseph. Bringing clay and sand into digital design – continuous tangible user interfaces. BT Technology Journal, Vol. 22, No. 4, pp. 287-299, October (2004). [13] Levin, Golan. An Informal Catalogue of Slit-Scan Video Artworks, 2005. Web: <http://www.flong.com/writings/lists/list_slit_scan.html>. [14] A. Cassinelli and M. Ishikawa. Khronos Projector. Emerging Technologies, SIGGRAPH 2005, Los Angeles (2005). One page abstract [PDF-0.5MB]. Video Demo [WMB-40MB]. Power Point presentation (with video) [PPT-10MB]. [15] Toshio Iwai with NHK Science & Technical Research and Laboratories, Morphovision: Distorted House (2005) [16] H. Tanahashi (from Post Theater - a theater company without theater and without a company), SkinSites: a site-specific multi-media dance performance series (2005) [17] See for instance Reflectance Map: Shape from Shading, Robot Vision, Berhold Klaus Paul Horn, The MIT Press, McGraw-Hill, pp: 243-277 (Chapter 11). [18] A. Cassinelli, Time Delayed Cinema [PPT-28MB], invited talk at Microwave/Animatronica New Media Art Festival, Hong Kong 4-15 Nov. (2006)