A Paper-Based Interface for Video Browsing and Retrieval
The output of video category recognition is a ranked list of categories and specific instances for each video frame. For example, it might determine that there is an 85% chance that an image contains a face, a 3% chance that it contains a building and a 1% chance that it contains a logo. A relevant image filter calculates a numerical ranking of the results of video category recognition. The layout module chooses the key frames that are printed based on the ranking calculated by the relevant image filter, the desired size of each key frame, the presence of nearby keywords from a user's personal interest profile [4] and the page area that should be filled with key frames.
5. Experimental Results and Applications
A complete working prototype of the video paper system has been developed that includes a PC server that records television programs from a satellite receiver and automatically generates video paper. Running on a 2 GHz Windows 2000 system, less than one minute of runtime is required to produce a video paper PDF file for a one-hour recording. The MPEG2 video file can also be trans-coded to a low bit-rate version for the portable system. These files typically require about 100 MB per hour of recorded data. Thus, up to 50 hours of video can be saved on a 5GB PCMCIA disk.
This system has been running on a regular basis in our lab and has generated Video Paper for approximately 150 news broadcasts and financial news programs. The remote control shown in Figure 2 is connected to our WiFi LAN and allows users to replay any of the video paper documents on demand. We also developed a version of video paper for our Meeting Recorder [7]. This system captures 360 degree panoramic video and presentation slides while they're being displayed in our conference room. The Video Paper representation includes key frames from both sources and a transcript for the audio track. Both versions are used regularly and help people quickly find useful information in long recordings.
Video Paper is also being used for oral histories [6]. These are interviews of the observers or participants in historical events. Typically, the interviews are recorded and transcribed. Because of the difficulty in accessing the audio, subsequent analysis is mostly limited to reading the transcript. Oral historians have started video taping their interviews, but these are even more difficult to access afterward because of the difficulty in synchronizing the video replay with the transcript.
A recent experiment supplied Video Paper to 13 subjects who reviewed two 75-minute interviews. They were given a list of questions and used Video Paper to find the correct answers. Post-hoc analysis of their experiences showed they were completely at ease with the concept of Video Paper and reported that it provided substantial benefits with minimal overhead. They used the audio to hear particularly compelling passages and used to video to help get an idea about the personality of the interviewee.
Interestingly enough, on their own several people established a multi-tasking search strategy in which they read one part of the transcript while listening to the audio from another. This unintended result demonstrates the usefulness of the two-dimensional display provided by Video Paper.
