Lei Feng Network (Search "Lei Feng Net" public concern) : This article originally contained in those familiar with the column semiconductor .
In the past two years, the highlights of mobile phones have become less and less. High-end smart phones are now basically no longer competing with Thunder Rabbit, but began to PK camera performance. Ever since, each of these two years has begun to start a dual Camera.
| Dual Camera can do anything?
The question is, what can the dual Camera do?
1. Dual Camera can measure distance, so it can be used for distance related applications
As shown in the figure above, since Dual Camera can determine the distance of the subject by the algorithm, it is easy to make some special effects through this feature, such as:
A: Background blur
One of the most outstanding features of SLR cameras is the large aperture. Because the Dual Camera can measure the distances of different objects, it is easy to achieve a large aperture by aligning objects that require large apertures and blurring other objects at different distances.
Original image
(focusing on beauty, blurring background and wounded mushrooms)
(focus on mushroom, blur background and beauty)
B: Background replacement
Since the distance can be measured, the subject in the object to be photographed can be extracted and the background can be changed, and it can be simpler than the PS to carry out the map.
C: Background effects
Since it is possible to map, it is still very easy to do something about the background. For example, this drawing uses sketch effects.
D: Measurement distance
This figure clearly identifies the distances of different objects. This distance information is identified in different colors. When the AP has obtained the distance information of different objects, it can do the above-mentioned various functions.
2. Double Camera can do optical zoom
If the two cameras have different FOVs, a large FOV, a small FOV, and then an algorithm to achieve the effect between the two optical lenses, optical zoom can be easily achieved.
If you do not use Dual Camera, the text is not clear after you enlarge the picture.
If using Dual Camera, the text remains clear after zooming in.
This figure is a combination of a wide-angle map and telephoto map, calculated by the algorithm of the intermediate attitude photos, so that the details are not distorted.
3. Enhanced dark light
This generally uses a color + black and white camera. The light intensity of the picture is obtained by the black and white camera to compensate the dark light of the picture.
4. 3D shooting and 3D modeling
It is different from shooting of general 3D movie. The two cameras on the mobile phone cannot produce sufficient visual difference during the shooting of the image because the distance between the two cameras is different from that of the human eye. And in order to make people more obvious 3D visual effects. So often the algorithm needs to be enhanced.
Because the distance can be measured, the subsequent dual Camera can not only achieve 3D photography, but also can perform 3D modeling. By this time, I think the importance of dual Camera will be more important.
Other effects are enhanced, such as HDR, to increase the resolution. In fact, single camera can also be achieved. However, dual cameras can make the effect better, and they will not be listed one by one.
summary:
Currently, these features are the most common features of dual Camera phones. Bokeh/replacement, the effect of dark light brings more photo effects to the user. Optical zoom let us feel the camera function of the zoom function. But personally feel that the most exciting of the future is the 3D function. This year VR is so hot. Where does the VR material come from? Still rely on the optimization of the dual Camera algorithm. If the algorithm for 3D photographing and modeling matures, dual cameras will become more popular.
| The principle of dual camera
As mentioned earlier, the dual camera application is mainly divided into: distance-related applications, optical zoom, dark light compensation, and 3D shooting and modeling. The principle of each application is somewhat different, and we will introduce the relevant principles separately:
1. Distance related applications
The human eye can easily locate the distance of an object, but when one person closes one of the eyes, the positioning ability will drop a lot. Dual cameras are applications that simulate human eyes. Simply put, if the distance is measured, it is calculated by an algorithm. The angle θ1 and θ2 of the object to be photographed with the left/right camera, plus a fixed y value (ie, the center distance between the two cameras), it is very easy to calculate the z value. (ie the distance from the object to the Camera)
However, it is also easy to predict that if the center distance between the two cameras is too small, the calculable object distance will be very close. If you want to work out a long distance, you must make the distance between the left and right cameras far away.
2. Optical zoom
The optical zoom is mainly about using different FOVs (viewing angles) for the left and right cameras, so that the two cameras have different views. When the user needs a wide-angle photo, the left camera with a viewing angle of 85 degrees is used to obtain a wide-angle effect. When the user needs a telephoto photograph, a right camera with a viewing angle of 45 degrees is used to obtain a telephoto effect.
In order to make the overlap of objects photographed by the left and right cameras high, the dual camera modules of the optical zoom cannot be as large as the distance from the camera module of the application, but the closer the left and right cameras are, the better.
3. Dark light enhancement
In general, doing dimming enhancement is to use two cameras with a standard camera of RGBG and a black and white camera with RGBG filters removed. RGBG is used to obtain the color of the object, while the black and white camera is used to obtain a better amount of light to determine the intensity of the light intensity of the object being photographed. Then merge the two pictures for better dark light enhancement.
Just generally speaking, there are two fusion methods:
1. Using black and white pictures as the main body, the color of each pixel obtained on the color picture is pasted on the black and white picture, and the two pictures are merged.
2. With the color picture as the main body, the brightness intensity of each image obtained on the black and white picture is compensated to the color picture, and the two pictures are merged.
As to which method is more suitable for integration, it is possible that the benevolent sees the wise and sees the wisdom and does not discuss it.
In the same way, for the dim light enhancement, such dual camera modules are also required to be as close as possible in order to allow the objects photographed by the left and right cameras to overlap.
It should be noted that Huawei P9 actually chooses the module of this method.
Of course, some people in the industry also said that the current results of this algorithm are not obvious. Dim light compensation is really helpful for users, especially when shooting night scenes. However, some customers think that the Dual PD technology of Sony and Samsung is very good, and they prefer to use the Dual PD camera for dark light compensation.
In the end is dual camera or Dual PD dim light compensation effect is good, we can compare Huawei P9 and Samsung's Galaxy S7 edge, there will be an answer.
4.3D shooting and 3D modeling
The algorithm of 3D shooting and 3D modeling is actually similar to the distance application, but its accuracy is even higher, and sometimes even infrared distance measurement is needed to make a more accurate distance judgment. It will not be introduced in detail here.
5. Requirements for ISP
Referring to the dual-camera algorithm, we must mention ISP (Image Signal Processing). The main function of the ISP is to post-process the signals output by the front-end image sensor. The main functions include linear correction, noise removal, and dead pixel removal. Interpolation, white balance, automatic exposure control, etc., depend on the ISP can restore the site details in different optical conditions, ISP technology to a large extent determines the imaging quality of the phone.
In the era of functional machines, ISPs are all made on cameras. Different pixel cameras have different performance ISPs. As the mobile phone camera pixels are getting higher and higher, the requirements for ISP performance are getting higher and higher. If the ISP is integrated into the camera Sensor, it will inevitably cause the camera's module to be too large and even affect the camera effect. So in the era of smart machines, ISPs are generally on the main chip SoC. In order to achieve better results, some brand customers even resort to cost plus an ISP to achieve better and more professional photographic results.
A good camera algorithm requires a good ISP, and ISPs and algorithms complement each other and are indispensable. Dual cameras require more ISP performance. First of all, in order to enable the left and right camera signals to be processed at the same time, a single ISP can no longer meet the needs of dual cameras. This requires a dual ISP to implement this feature.
Take the dark light enhancement as an example, the color/black and white images enter the respective ISP channels and calibration channels respectively, and then the two pictures are matched (for example, the same part of the two pictures is extracted to remove only the part captured by one camera). Then, the relevant pictures are processed through algorithms such as occlusion, detection, and compensation. Finally, the two pictures are merged to achieve color enhancement. Of course, in fact, the ISP cooperates with the algorithm to do far more than this picture. I really do not know, not to mislead everyone.
Of course, there is also a small episode in it. After all, there are two ISPs. The two ISPs have some problems in processing speed and processing capacity. In order to ensure that the two ISPs can sample at the same time, the pictures taken by the dual cameras need to be taken at the same time. One of the solutions is to have the Sensor have a sync signal pin. The synchronization signals of the two cameras are connected, and each time the image is read, the pictures are time stamped, and the ISP ensures that the pictures taken by the left and right cameras are shot at the same time through time stamps, and finally the fusion is performed.
6. Camera interface
In general, the camera interfaces of current smart phones are MIPI interfaces. Before the mobile phone platform has only 2 way MIPI interface, respectively to the front camera and rear camera. To do dual cameras requires the platform to support at least three MIPI interfaces. In fact, in the previous high-end platform, in order to achieve a higher pixel, has used a dual ISP (for example, in order to support 16M camera, will use 2 8M ISP), this type of platform is likely to only two MIPI. But this can't stop the engineer from doing the front single camera + the rear dual camera.
Yes, adding a small Swticch makes it easy to implement dual cameras.
Related Reading
Dual camera depth analysis (below)
Lei Feng Net Note: This article was authorized by the author Zhang Jingyang released Lei Feng network. Zhang Jingyang, founder and CEO of Moore Elite. Reproduced please contact the authorizing and retain the source and the author, not to delete the content.