Introduction
"Tokyo Story" is a monochrome film produced in 1953, directed by Yasujiro Ozu (Figure 1). Yuharu Atsuta is the cameraman of the film. It is famous in the world as one of the best works of Yasujiro Ozu. Unfortunately, its original negative film was lost by fire of the development office. The playing film in movie theaters and video products of today were replicated from an intermediate negative film produced from the original positive film [Hasumi, 83]. When we watch at a screen of "Tokyo Story", we will soon notice not only natural aging of films in a long period, but also deterioration of image contrast. This would not be the image representation that Yasujiro Ozu and Yuharu Atsuta intended. It is said that, formerly, the image quality of the film was much better. Two living materials escaped from the fire can make us imagine the original quality of the "Tokyo Story" film. One is its preview film for promotion preserved in Shochiku Co., Ltd. Another is a collection of strips of original films removed in the editing process (Figure 2). This is preserved in the University of Tokyo. In spite of the passing of long time, both keep good quality of images.
Fig.1 A scene from "Tokyo Story" Fig.2 Strips We have tried film restoration of "Tokyo Story", one of the most famous Japanese films, in order to bring back its beauty of images by using the forefront of digital technologies such as computer technologies, multimedia techniques, and signal processing.
Meaning of Film Restoration and Related Works in the World
Origin of Movies
Movie is a motion picture system that projects multiple photographs on a screen very fast. Photograph, which is a base technology of the movie, was invented in 1820th. In photographs in those days, images are projected on metal boards like "Daguerreotypes", or on glass boards like Wet-Plate and Dry-Plate Negatives. By using these mediums, we could not realize movies. The most important technology revolution for the movie was an invention of celluloid in 1869 and succeeding invention of roll films by Eastman Codak in 1889. Then, Thomas Edison created the first movie in 1889, and the Lumiere brothers in France firstly showed a movie on a screen in front of many people by using projection technology in 1895.
The Number of Movies
In the beginning of the movie, for example 1920th to 1930th, we might think that only a few movies were produced because the movie technologies were much more primitive than today. This is wrong. For example in Japan, in 1920th, 600 to 800 feature-films were produced in one year. In 1930th, while the pace of film movie production decreases, 400 to 600 feature-films were produced in a year. This means that about 10,000 feature-films had already been produced until World War II. By the way, Japan, in those days, was one of the most active countries for movie production in the world.
Fig.3 Fig.3(2) Fig.3(3) Fig.3(4) Fig.3 Flaws in a film, (1) a whole image, (2) flaws at upper portion of a sliding door, (3) a rather thick flaw at the foot of a person, and (4) horizontal black line by the film dirt
Degradation of Motion Picture Films and Loosing Movies
Movie and photograph are based on the technology to record images by chemical transformation of materials on a film occurred by exposure to the light. A monochrome film, as used in "Tokyo Story", mainly consists of two kinds of materials: a film-base produced from polyester etc., and monochrome photographic materials coating the film-base. Exposing the photographic materials to the light leads to chemical changes of the photographic materials. Then, by the development of the film, silver particles, which record the image, are generated at the exposed portions.It is passing about a hundred years since the birth of movies. The film degradation has already lost huge amount of movie art properties. According to the survey of the Library of Congress of the United States, about eighty percents of silent movies produced in the United States have already been lost or in inrestorable states [Okamura, 98]. There is a report that about ninety percents of silent movies produced before 1930 and about fifty percents before 1950 have been lost [Frame]. Yasujiro Ozu was active just during the 1920th and the 1960th, from his first work, "An Edge of Penitence (Zange no Ha)" (1927), to his posthumous work, "An Autumn Afternoon (Sanma no Aji)" (1962). The number of his known works is fifty four, and only thirty three works remains today. This means that the rest eighteen works have been lost [Shochiku, 93]. Even in the remained works, parts of some movies are lost, or the quality of some films is as bad as we cannot enjoy the works.
Fig.4-1 Fig.4-2 Fig.4-3 Fig.4 Example of the difference of image brightness, (1) darker frame, (2) brighter frame, and (3) histograms of pixel values of both images
Preservation and Restoration of Movies
The Japanese movie community, who worried about the situation in which a large amount of movie properties have been lost and were now loosing, established National Filmcenter in The National Museum of Modern Art, Tokyo. It preserves not only movie films but also secondary materials such as still pictures, posters, and scenarios. In the world, The International Federation of Film Archives (FIAF) and The Museum of Modern Art, New York (MoMA) are famous as film preservation facilities. In establishing the film center, these were good models [Arai, 96].As well as the preservation movement, there are many attempts to restore old damaged films. History of simple film restoration of pictures is very old. Even in 1858, there were many research reports on film resttation. Main methods until today are chemical and physical methods [Arai, 96].
Recently, the most advancing film restoration method is "digital film restoration" using image processing technologies of computer science. The most famous example is a digital restoration of a popular Disney movie "Snow White" (1937) in 1993 by "Cinesite", which is the digital image center of Eastman Codak Corporation opened in 1992. This restoration project spent seven million dollars, and runs forty graphic workstations for eighteen weeks in twenty-four hours a day. Total amount of processed data is more than fifteen tera bytes [FRAME][Yagyu, 95]. Moreover, there are many digital film restoration projects in the western such as AURORA Project (AUtomatic Restoration of ORiginal film and video Archives) [Suter, 97] and The FRAME Project of EU (European Union) [FRAME].
Goal of Restoring "Tokyo Story"
This section describes basic issues in digital film restoration and several characteristics of films closely related to the restoration.
Restoring Points
First, we present points of digital restoration in the "Tokyo Story" film.(1) Deleting flaws, cracks, blots by mold, etc.
More than forty years have already been past since "Tokyo Story" was produced. During the period, many flaws and cracks have been created on the films on account of physical factors in handling the films (Figure 3). Additionally, large flaws lose parts of images on the film. Fortunately, films of "Tokyo Story" have been restored under rather good condition, and there are little molds on them.(2) Correction of contrast
On account of the lost of original films, contrast of images is not good in the remaining "Tokyo Story" films. Answering the interview by Shigehiko Hasumi, Mr. Yuharu Atsuta told in retrospect that he represented "summer" by the contrast between a bright windows and dark faces of persons in the scene in which all family members in Tokyo meets and talks in the oldest brother's house. However, currently remaining film does not maintain the contrast in the scene as Mr. Yuharau Atsuta intended.(3) Difference of brightness in each film frame (seen as glimmers)
In the "Tokyo Story" films, difference of image brightness in film frames is very large (Figure 4). Audience in the theaters would see this difference of brightness as glimmers. We can guess several causes of this difference: difference of exposition time of each frame in taking the film, error in printing, and uneven quality of films at that time.(4) Incorrect image position
Watching at "Tokyo Story", in spite of static scenes, a screen sometimes quakes horizontally and vertically a little. We cannot know whether this difference was caused by the camera or printing. As a result, we can identify a few percents of difference of image positions among film frames.(5) Restoration of sound: deleting noise
A movie film records sound optically in its sound tracks. The sound tracks also have damages such as flaws and cracks. This causes sound noise and sound missing.
Analyzing "Tokyo Story"
Here, we shortly overview technical characteristics of "Tokyo Story" closely related to our digital restoration.
"Tokyo Story" is a monochrome film. This removes color correction problem from our film restoration. A normal color film of today records all colors on the same roll film. In this color system, color balance changes in a long period. Another representative color system, "technicolor system", decomposes colors into three primary colors when taking the film in real-time, and records each primary color on a different monochrome film. "Gone with the Wind" is a representative work using "technicolor system". Most important advantage of "technicolor system" is that its color will not fade because color is composed at playing time from the three monochrome films of the three primary colors. On the other hand, it has another problem. Three films will warp in different. This causes misfit of three films and makes uneven images on a screen. Restoration of "Tokyo Story" is free from the typical color film restoration problems such as correction of color balance or unevenness of images. On the other hand, structure of Ozu films is very stoic: static camera angle, simple image sequence only with cut connection, etc. This simpleness makes our restoration process such as flaw detection and motion analysis so easy. We have not only the playing film of "Tokyo Story" but also its preview films for promotion and cut-pieces of original films removed in the editing process which remain better image quality than the playing film. For restoration, existence of references indicating the restoration is very important. The existence of many reference films of "Tokyo Story" is a very good condition for our digital restoration.
Yasujiro Ozu had been using 35mm standard size films in his all works. "Tokyo Story" was not an exception and produced by using the 35mm standard size film. The size of this film is the same as the usual 35mm roll films for home-use. The size of a frame of 35mm movie films is just as half as the one of 35mm photo camera films. This means that common scanner products for 35mm films are available for digitizing Ozu films.
Restoration Process
This section presents the basic process of digital film restoration.
Data scanning
The first process is to digitize image information recorded on films. By using film scanners, we translate film images into digital image data. In our case, films for digitization are 35mm playing films of "Tokyo Story" provided by Shochiku Co., Ltd. We have digitized these films into the 3072x2048 dots data by using film scanners for the Photo CD production. This generates about 6.1 MB data per one frame in PGM format. For immediate image processing, all digital image data are stored in the disk array of our special computers for image processing equipped in the University Museum, The University of Tokyo.
Pre-processing
The second process is pre-processing in advance of the main restoration process.
- Trimming
The trimming process removes extra image portions in the scanned, and clips just the image portion.
- Regulation of size and position
As we cannot scan films completely correctly, an image in a scanned data rotates a little or shifts horizontally and/or vertically. So, we must adjust the image position and rotation among adjacent frames. As a result, the same static object should be projected correctly on the same position of a screen. In addition, we must correctly adjust the size of all frames.
- Correction of brightness
Scanning films, we have noticed that image brightness differs for frames. Especially in the "Tokyo Story" film, the difference of brightness is large. To correct this difference, we have built a histogram of pixel intensity for each frame and tuned the peak positions of histograms of adjacent frames (Figure 4-3).
Restoring flaws, cracks, and blots
Next is the main restoration process: to restore flaws, cracks, and blots. To restore flaws, we must firstly detect positions of flaws in each frame, then restore missing pixels by the flaws. The summary of the flaw detection mechanism is as follows. Analyzing flaws in films by using image analysis techniques, they generate sudden changes of pixel values both in space dimensions and a time dimension. We can detect flaws by using this characteristic. However, this sudden change would be made not only by flaws but also by hard actions of image objects and a special effect or special camera work. First, to detect object actions, we adopt motion analysis technologies. Second, we do not have to take into account the special effect and special camera work because, in all Ozu film, no special effect and special camera work was used.Next is the main restoration process. Image restoration techniques have been used in many applications such as correcting pictures taken by artificial satellites and restoring old photo pictures. These techniques handle static pictures and restore an image independently. In case of movies, there are a large number of continuous images. Current standard motion picture films include twenty-four image frames in one second. In our film restoration, we can utilize mostly same images recorded in adjacent frames. As the flaw positions of different frames must be completely different, the flaw pixels can be complemented not only by neighboring pixel values of the frame but also by the pixel values of the previous and next frames. This is the most remarkable point in the movie film restoration technique.
Conclusion
At the dawn of the cinema age, in the Taisho era and in the early years in the Showa era, many movies were produced not only in Japan but also all over the world. However, many of these works have already been lost and are now loosing because of film degradation. Digital technologies are powerful methods to protect our movie property from deterioration. For example, digital recording is the best way to inherit the "current" state of a film as it is. Digital restoration can bring back the original beauty of images in the films from the "past" by using the slight traces remaining on a film. We have met the representative movies produced by Yasujiro Ozu and Yuharu Atsuta, named "Tokyo Story", and challenged to recover its beauty of images. We would be very happy if this challenge could bring back the original image beauty intended by Yasujiro Ozu and Yuharu Atsuta even a little. Then, we would appreciate much if our challenge could promote the importance of the film preservation, and make the film restoration movement in Japan more active.
AcknowledgementWe would like to appreciate Shochiku Co., Ltd. for their offering of the "Tokyo Story" films. We thank Mr. Shigeki Kohno for his kind cooperation in digitizing the "Tokyo Story" film.
 |
 |
 |
