Japan Broadcasting Corporation (NHK) Science & Technology Research Laboratories (STRL) has long been consistently opening up new horizons for broadcasting. Its recent R&D on 8K ultra-high-definition television―the ultimate two-dimensional television―came to fruition as a regular satellite broadcasting service in Japan in December 2018, with 8K gradually penetrating media and other industries globally. The 8K specifications were determined based on psychophysical experiments to effectively produce an immersive and realistic experience with a two-dimensional screen. STRL is continuing to enhance user experiences. Aside from two-dimensional displays, today there are media devices such as head-mounted displays, augmented reality glasses, three-dimensional displays, and haptic devices. STRL is researching these devices to enhance their performance from the perspective of visual psychology and cognitive science. It is also developing three-dimensional information processing technologies and artificial intelligence aiming for new content presentations with new immersive devices. Conveying sensations other than sight and sound will create innovative sensory experiences that provide unprecedented immersion. A new media scheme that utilizes various delivery platforms such as broadcasting, the internet, and 5G is also being studied to produce new viewing experiences.
To develop an ultrahigh-definition television (UHDTV) camera-with a resolution 16 times higher than that of HDTV
resolution and a frame rate of 60 Hz (progressive)-a compact and high-mobility UHDTV camera using a 33M-pixel
CMOS image sensor to provide single-chip color imaging was developed. The sensor has a Bayer color-filter array
(CFA), and its output signal format is compatible with the conventional UHDTV camera that uses four 8M-pixel image
sensors. The theoretical MTF characteristics of the single-chip camera and a conventional four-8M-pixel CMOS camera
were first calculated. A new technique for Bayer CFA demosaicing used for the single-chip UHDTV camera was then
evaluated. Finally, a pick-up system for single-chip imaging with a 33M-pixel color CMOS image sensor was
measured. The measurement results show that the resolution of this is equivalent to or surpasses that of the conventional
four-8M-pixel CMOS camera. The possibility of a practical compact UHDTV camera that makes use of single-chip
color imaging was thereby confirmed.
We have been developing an ultra high definition television (UHDTV) system with a 7,680 horizontal by 4,320 vertical pixel resolution and a 60 Hz frame rate. This system, which is called Super Hi-vision (SHV), is expected to serve the next generation of broadcasting services. We have just completed the world's first imaging equipment that is capable of capturing video at a full SHV resolution. In designing this equipment, we decided to develop three new devices, taking into account the camera performance and the ease of implementation. First, we developed a 33-megapixel CMOS image sensor. Its pixel size of 3.8 &mgr;m sq. retained the dynamic range of the sensor above 60 dB even with a 3-transistor pixel structure. Second, a fixed focal length lens was developed to create an adequate MTF right up to the limiting resolution of the sensor. Third, we developed a signal-processing device capable of handling 72 Gbps signals and cascading boards to expand the process. SHV images with a modulation of 20% at the Nyquist frequency were obtained by using these three key technologies.
We developed an experimental ultrahigh-definition color
video camera 7680H4320V pixels using four 8-million-pixel
charge-coupled devices (CCDs) to increase the camera’s resolution.
This involves attaching four CCDs to a special color separation
prism. Two CCDs are used for the green image; the other two
are used for the red and blue images. Our prototype camera attains
a limiting resolution of more than 2700 television lines, both horizontally
and vertically. Camera sensitivity is F/2.8 at 2000 lux, with a
luminance signal dark-noise level of approximately 50 dB in high
definition television format. To analyze camera performance, we estimated
the spatial position error between the two green CCDs and
the chromatic aberration. Based on these estimations, the cause of
resolution deterioration and ways to improve resolution are
discussed.
We developed an experimental single chip color HDTV video image acquisition system with 8M-pixel CMOS
image sensor. The imager has 3840 (H) × 2160 (V) effective pixels and built-in analog-to-digital converters, and its
frame rate is 60-fps with progressive scanning. The MTF characteristic we measured with this system on luminance
signal in horizontal direction was about 45% on 800 TV lines. This MTF was better than conventional three-pickup
broadcasting cameras, therefore the enhancement gain (the "enhancement area" in MTF) of the 8M single-chip HDTV
system was about a half of the three-pickup cameras. We also measured the color characteristics and corrected the color
gamut using matrix gain on primary colors. We set the color correction target similar to that of three-pickup color
cameras in order to use multiple cameras to shoot for broadcasting, where all cameras are controlled in the same manner.
The color error between the single-chip system and three-pickup cameras after the correction became 2.7, which could
be useful in practice.
We have developed color camera for an 8k x 4k-pixel ultrahigh-definition video system, which is called Super Hi- Vision, with a 5x zoom lens and a signal-processing system incorporating a function for real-time lateral chromatic aberration correction. The chromatic aberration of the lens degrades color image resolution. So in order to develop a compact zoom lens consistent with ultrahigh-resolution characteristics, we incorporated a real-time correction function in the signal-processing system. The signal-processing system has eight memory tables to store the correction data at eight focal length points on the blue and red channels. When the focal length data is inputted from the lens control units, the relevant correction data are interpolated from two of eights correction data tables. This system performs geometrical conversion on both channels using this correction data. This paper describes that the correction function can successfully reduce the lateral chromatic aberration, to an amount small enough to ensure the desired image resolution was achieved over the entire range of the lens in real time.
We have developed an experimental single-chip color HDTV image acquisition system using 8M-pixel CMOS image sensor. The sensor has 3840 × 2160 effective pixels and is progressively scanned at 60 frames per second. We describe the color filter array and interpolation method to improve image quality with a high-pixel-count single-chip sensor. We also describe an experimental image acquisition system we used to measured spatial frequency characteristics in the horizontal direction. The results indicate good prospects for achieving a high quality single chip HDTV camera that reduces pseudo signals and maintains high spatial frequency characteristics within the frequency band for HDTV.
The authors propose a technique for performing real-time correction of the gain nonuniformity at output taps in an area-parallel-type frame transfer CCD (FT-CCD) by incorporating a reference optical signal. This technique activates a reference light source every vertical blanking (V-blanking) period, during which time the image sensor is shielded from incident light, and uses that output signal level to automatically adjust the gain of each block. For this method, a color-separation prism with reference light sources has been developed. In addition, to obtain a reference signal from the reference light source emitting for about 11% of the V-blanking period, a new method of accumulating the electron signal generated by the reference light source is described for the CCD. The results of a camera experiment using an 8-million-pixel CCD are described, and the compensation accuracy of this method is discussed.
We describe a precise alignment method of attaching imagers to a prism to produce an ultra-high definition color camera system. We have already developed a prototype camera with 4-k scanning lines using this alignment method.
To increase its spatial resolution, this camera has four 8-megapixel imagers (GGBR), which are attached to a prism with a half-pixel pitch offset so that their pixel arrangement is equivalent to that of a single-chip color-imaging sensor with a Bayer-pattern color filter. The precision of their positioning influences the resolution of the reproduced images. The small pixels in the latest imager make it more difficult to maintain precise imager positions. A precise alignment method for attaching imagers to prism is therefore essential for developing a camera system with high resolution. We propose a method with high detectivity using a sinusoidal pattern chart that easily reproduced by one imager, and a signal process. Images from this camera can attain a limiting resolution of more than 3200 TV lines.
In an integral three-dimensional television (integral 3-D TV) system, 3-D images are reconstructed by integrating the light beams from elemental images captured by a pickup system. 160(H) x 118(V) elemental images are used for reconstruction in this system. We use a camera with 2000 scanning lines for the pickup system and a high-resolution liquid crystal display for the display system and have achieved an integral 3-D TV system with approximately 3000(H) x 2000(V) effective pixels. Comparisons with theoretical resolution and viewing angle are performed, and it is shown that the resolution and viewing angle of 3-D images are improved about 2 times and 1.5 times respectively compared to previous system. The accuracy of alignment of microlenses is another factor that should be considered for integral 3-D TV system. If the lens array of the pickup system or display system is not aligned accurately, positional errors of elemental images may occur, which cause the 3-D image to be reconstructed at an incorrect position. The relation between positional errors of elemental image and reconstructed image is also shown. As a result, the 3-D images reconstructed far from the lens array are greatly influenced by such positional error.
An experimental ultrahigh-definition color video camera system with 7680(H) × 4320(V) pixels has been developed using four 8-million-pixel CCDs. The 8-million-pixel CCD with a progressive scanning rate of 60 frames per second has 4046(H) × 2048(V) effective imaging pixels, each of which is 8.4 micron2. We applied the four-imager pickup method to increase the camera’s resolution. This involves attaching four CCDs to a special color-separation prism. Two CCDs are used for the green image, and the other two are used for red and blue. The spatial image sampling pattern of these CCDs to the optical image is equivalent to one with 32 million pixels in the Bayer pattern color filter. The prototype camera attains a limiting resolution of more than 2700 TV lines both horizontally and vertically, which is higher than that of an 8-million-CCD. The sensitivity of the camera is 2000 lux, F 2.8 at approx. 50 dB of dark-noise level on the HDTV format. Its other specifications are a dynamic range of 200%, a power consumption of about 600 W and a weight, with lens, of 76 kg.
A wide dynamic range camera for high picture quality use is proposed. The camera is equipped with a novel optical beam splitting system. It first divides incident light into two different intensity lights. Small intensity light is taken by a single-chip color imager. The other large intensity light is further led to a tri-color prism and taken by three imagers. These functions are integrated into one-piece optical block, which is suited for 2/3-inch optical format standard. An experimental HDTV camera has been developed. The exposure ratio was set as 9:1. A high exposure image is taken by three 2M-pixel CCDs and a low exposure image is taken by a single-chip color 2M-pixel CCD with an on-chip stripe color filter. The results have shown that the validity of the proposed method for obtaining wide dynamic range images with high picture quality.
KEYWORDS: Cameras, Imaging systems, Charge-coupled devices, Video, Digital signal processing, Signal processing, CCD cameras, Image resolution, Analog electronics, Image quality
An ultra-high definition experimental camera system has been designed with double the horizontal and vertical resolution of HDTV. An 8M-pixel CCD with a progressive 60 frame-per- second scan-rate has been developed for the system. The 34 mm X 17.2 mm image area has 4046 (H) X 2048 (V) active imaging pixels with 8.4-micrometers squares. This CCD has a split- frame transfer structure and sixteen 37.125 MHz outputs so that the vertical and horizontal transfer frequencies are almost the same as those of HDTV. The split-frame transfer structure halves the required VCCD clock speeds and thus improves charge transfer efficiency. The multiple-output structure with its 16 outputs enables high data-rate imaging for ultra-high resolution moving pictures. In the signal processing section, analog gain adjustment circuits correct for the mismatches in the characteristics of outputs, and a correlated double-sampling technology is employed on each of the 16 CCD output signals. The output signals are digitized by 12-bit ADCs. The converted signals are then sent to the digital signal processing (DSP) circuits. In the DSP circuits, the upper half of the captured image is vertically inverted. All of the output data is then merged into a 4K X 2K pixel image and reformatted to create twenty-four 640 (H) X 480 (V) pixel sub-images for image processing. After contour compensation processing, the video signals are converted into an analog signal and presented on two ultra high resolution video monitors.
This paper describes the development of an experimental super- high-definition color video camera system. During the past several years there has been much interest in super-high- definition images as the next generation image media. One of the difficulties in implementing a super-high-definition motion imaging system is constructing the image-capturing section (camera). Even the state-of-the-art semiconductor technology can not realize the image sensor which has enough pixels and output data rate for super-high-definition images. The present study is an attempt to fill the gap in this respect. The authors intend to solve the problem by using new imaging method in which four HDTV sensors are attached on a new color separation optics so that their pixel sample pattern forms checkerboard pattern. A series of imaging experiments demonstrate that this technique is an effective approach to capturing super-high-definition moving images in the present situation where no image sensors exist for such images.
A dynamic range expansion method for charge modulation device (CMD) imagers has been proposed and an experimental circuit was evaluated. As active pixel sensors, CMD imagers offer significant advantages in terms of non-destructive readout and high-speed operation. Furthermore, the CMD imager used in this study has built-in dual vertical shift registers which can start their scanning independently. We applied our newly developed source reset method to the imager, which enables each pixel charge to be reset by external circuits. By using these functions, we have obtained our dynamic range expansion method. THe source reset method enables each pixel of a CMD imager to be reset selectively. We use it for a regional electrical shutter. For instance, if there is an overexposed pixel, its accumulated charge is reset from outside the imager, so that the pixel charge is prevented from becoming saturated. The non-destructive readout and high-speed operation enable the CMD imager to be read multiple times without destruction of accumulated charges. For instance, if the imager is driven twice in one field period, each pixel's accumulations can be checked and reset selectively from external of the device at the first scanning, then, all the signals are obtained at the second scanning and output through a small process circuit. This dynamic range expansion method for CMD imagers needs only a simple feedback circuit but does not need complex equipment externally nor adding reset circuits on a chip. An experimental circuit with this method was successfully evaluated here.
KEYWORDS: Imaging systems, Cameras, Image resolution, Video, Prisms, Signal processing, Spatial resolution, Video processing, Modulation, RGB color model
We have developed a 60 frame/sec 2 K multiplied by 2 K progressive-scan color camera system. Two new key technologies have been applied in the design process. One is high-data- rate imager operation technology in which each of four charge modulation device (CMD) chips is driven at 167 M pixel/sec in progressive scan mode. One chip consists of 1920 (H) multiplied by 1035 (V) pixels. The other technology is the four-imager pickup method in which two CMD imagers are used for green and the other two for red and blue. Spatial offset imaging is applied in the vertical direction to the two green imagers so that the equivalent number of vertical lines reaches 2070, twice that of one CMD. The above technologies enable the construction of a very-high-resolution camera with a data rate of 334 M pixel/sec and a vertical limiting resolution on a color monitor of more than 1500 lines.
In an effort to realize a compact HDTV camera with high performance, we have developed a prototype equipped with four 2/3-inch CCDs. A smaller image format is preferable for downsizing TV cameras. However, this causes shrinkage of the unit pixel size and inevitably makes it more difficult to produce an image-pickup device with required HDTV qualities, especially sensitivity, and dynamic range. We have overcome this problem by using CCD imagers with high performance but with a relatively small number of pixels and by increasing the number of CCD chips used in a camera to secure the necessary spatial sampling points for HDTV. In the newly developed color-separating system of the camera, two of the four CCDs are assigned for the green (G) light component and one each for red (R) and blue (B). We succeeded in improving the resolution by introducing spatial pixel offset imaging. This new method has two major advantages: it prevents resolution degradation caused by chromatic aberration and improves the resolution of colored signals over a wide range.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.