Pixel mixture

Japanese






What is a pixel mixture?

Recently, digital still cameras have several millions pixels. A video capture function is one of necessary functions. However, it is very hard task to read-out such several millions pixels in video rate. We need to reduce data amount to be read for the video capture function when the cameras have the sensor which contains such several millions pixels.
A pixel mixture is a technique to reduce the amount of read-out data. Figure in the right side is example of nine pixel mixture. Pixels which are filled in same color are mixed or averaged. The output is the average value. In this case, the pixel mixture reduces the ninth compared to the case of all pixel read-out. In the read-out rate, it accelerates nine times faster than normal read-out method. The pixel mixture is very popular technique and already implemented to consumer digital cameras.
The pixel mixture can captures with larger area of photo diode. The area of photo diode for the pixel mixture can be considered nine times larger than that of the normal capture. It allows us high-sensitive capturing.
The pixel mixture degrades the spatial resolution. It is the drawback of the pixel mixture.



Neighbor pixel mxiture

The drawback of the pixel mixture is the degradation of the spatial resolution. Therefore, we have proposed a novel pixel mixture which we call the neighbor pixel mixture. The key of the proposed neighbor pixel mixture is that it mixes the nine pixels with are located in the smallest region of 3x3. In the normal pixel mixture, the nine pixels which are mixed are located within 5x5 regions. This smallest region of the proposed neighbor pixel mixture reserves the original spatial resolution a lot. The acceleration of the proposed neighbor pixel mixture is identical to that of the normal pixel mixture.

Original image Normal pixel mixutre Proposed neighbor pixel mxiture



Multi mixture

A multi-mixture is a novel pixel mixture method and a very unique imaging system. The multi-mixture can simultaneously generate two different types of images as shown in the figure. It is very unique to capture multiple different types of images at one shot.
In the aspect of PSF (Point Spread Function), the capture with the broad spectrum of the PSF (down sampling) and the capture with the narrow spectrum of the PSF (pixel mixture) are simultaneously performed. The down sampling images usually contain sever aliasing effect. In the contrast, the pixel mixture reduces the aliasing effect. In addition, we can consider that the down sampling is low-sensitive capturing and that the pixel mixture is high-sensitive capturing.

Block diagram of multi mixture

We apply the multi-mixture to the super-resolution. The super-resolution consists of two processes: the registration process and the reconstruction process. For the registration process, the aliasing effects lower the precision of the registration process. On the contrast, the reconstruction process requires the aliasing effect to improve the spatial resolution. The requirements for the successful super-resolution are completely conflicted.
However, we can satisfy these conflicting requirements by using the multi-mixture. The multi-mixture can simultaneously generate the pixel mixture image and the down sampling image. We can apply the pixel mixture image to the registration process. Then, the down sampling images are used for the reconstruction process. This scheme greatly improves the spatial resolution because the pixel mixture images are not aliased and the down sampling images are severely aliased.
In actual applications, users view the pixel mixture image sequence as a video. If users want to see in the detail, users can apply the new scheme of the super-resolution.

New scheme of the super-resolution using the multi-mixture

Comparisons of the super-resolution
Normal pixel mixture Super-resolution result of
normal pixel mixture
Super-resolution result of multi-mixture



Publications




Go to the top of this page