
1. Re: D435 Resolution VS Depth Image Quality
Aug 16, 2018 3:35 AM (in response to jakeding)This message was posted on behalf of Intel CorporationHello jakeding,
Thank you for your interest in the Intel RealSense D435 camera.
We have acknowledged your request and will provide a resolution a soon as possible.
Thank you,
Eliza 
2. Re: D435 Resolution VS Depth Image Quality
Aug 16, 2018 8:59 AM (in response to Intel Corporation)This message was posted on behalf of Intel Corporation1 of 1 people found this helpfulHello jakeding,
I will treat every question one by one. I need to know the impact of the higher resolution on the depth quality
The only compelling reason to set the D435 cameras into a lower resolution mode would be to reduce the minimum operating range, which will scale with the xy resolution and to reduce the bandwidth on the USB3.0 bus, to allow, for example, more cameras to operate at the same time, if the bandwidth limit has been reached.
This information is also shown at page 1 from BestKnownMethods for Tuning Intel® RealSense™ D400 Depth Cameras for Best Performance paper. Could you take a look at two depth quality test pics in attachments to see if the camera is well calibrated?
 What's use of limits we get here? Are there any relationship between the Depth RMS error and Plane Fit RMS error?
This information can also be found on Depth Testing Methodology paper at pages 9 and 10.
Please let me know if I can provide you any other information!
Kindest regards,
Eliza 
3. Re: D435 Resolution VS Depth Image Quality
jakeding Aug 16, 2018 11:21 AM (in response to Intel Corporation)Dear Eliza,
I do appreciate your professional answers here!! Actually, I posted my questions after I read all white papers provided by Intel. I am confused because I found there are some contradictions between my experiment results and the corresponding theories in Intel papers. Let me clarify all my questions in the following way.
 I need to know the impact of the higher resolution on the depth quality
The reason why I ask this: As you can see in the depth quality snapshots in the original attachments, both the Subpixel RMS error and Planefit RMS error are actually bigger under 1280x720 resolution than those under 848x480 resolution with the same camera configurations and same physical location.
Like the plane fit RMS is 3.32mm for 1280x720 and 2.71mm for 848x480 and the Subpixel RMS is 0.09 for 1280x720 and 0.05 for 848x480. It's obviously against with the saying that the higher the input resolution, the better the input image, the better the depth precision. I got bigger noise in higher resolution, why is it?
 Are there any relationship between the Depth RMS error and Plane Fit RMS error?
The reason why I ask this: You successfully clarified part of my question here but not the whole picture. Now I know the unit and usage of Plane Fit RMS error and Subpixel RMS error.
I calculated the Depth RMS error from my experiment as mentioned in the original post, and it seems has nothing to do with the Plane Fit RMS error and Subpixel RMS error limit. So how do I get the theoretical limit of (or relate to) Plane Fit RMS error and Subpixel RMS error from the theoretical limit of Depth RMS error equation?
Thanks!!

4. Re: D435 Resolution VS Depth Image Quality
Aug 20, 2018 10:58 AM (in response to jakeding)This message was posted on behalf of Intel CorporationHello jakeding,
We have forwarded your questions to the RealSense engineering team. We will update this thread as soon as we have a response from them.
Regards,
Jesus G.
Intel Customer Support

5. Re: D435 Resolution VS Depth Image Quality
Aug 22, 2018 2:40 AM (in response to Intel Corporation)This message was posted on behalf of Intel Corporation1 of 1 people found this helpfulHello jakeding,
Thank you for your patience.
The Depth RMS error formula in the BKMs for Tuning RealSense whitepaper shows how to translate between the Plane Fit RMS Error and the Subpixel RMS Error. The formula on Page 3 of the BKMs for Tuning whitepaper can be interpreted as:
I hope this information helps you!
Depth RMS error (mm) = Plane Fit RMS error = Distance(mm)^2 x Subpixel RMS Error/(focal length * Baseline)
The reason the Plane Fit RMS error increases with resolution on the D435 camera is because the active pixels (see page 9 of the BKMs for tuning whitepaper and page 33 of the D400 datasheet) on the D435 are 1280 x 800. When the camera returns depth, it uses rectified images. During the rectification process, the image gets downsized. In order to get the 1280x720 resolution after rectification, the extra pixels are extrapolated. This is why the whitepapers say that the optimal depth resolution for the D435 is 848x480. Using a resolution higher than 848x480 should give you similar or slightly worse depth quality.
The D415 camera's depth sensors have active pixels of 1920x1080, which is higher than the D435, so it's optimal depth is 1280x720 (as can be seen on page 1 of the BKMs for Tuning whitepaper).
The subpixel RMS error goes up with resolution because the pixels get smaller and there are more pixels, therefore the disparity between them gets larger.
Best regards,
Eliza 
6. Re: D435 Resolution VS Depth Image Quality
jakeding Aug 23, 2018 2:29 PM (in response to Intel Corporation)Hi Eliza,
Thank you very much!!!
My last question about his topic is: What the meaning or usage of the theoretical error limit calculated from Depth RMS error (mm) = Plane Fit RMS error = Distance(mm)^2 x Subpixel RMS Error/(focal length * Baseline)?
Like I got 2.88mm theoretical limitation of Depth RMS error(Plane fit error) calculated using the above equationusing D435 with HFOV=90deg, Xres=848, baseline=50mm, subpixel=0.08 and Distance = 1074.44mm (1074.44mm is provided by Realsense Viewer and you can see it in original attachment. I don't know what's the definition of the Distance in the equation. Should I use the ground truth value measured by myself instead of the distance shown in Viewer?) while the Realsense Viewer gives me 3.32mm(Plane fit error).
And I got 4.35mm for HFOV=90deg, Xres=848, baseline=50mm, subpixel=0.08 and Distance = 1073.62mm while Realsense Viewer gives me 2.71mm(Plane fit error).
How do I understand the relationship between the theoretical Depth RMS error limit and the Depth RMS error provided by the Realsense Viewer? And please tell me should I use the ground truth value to put into the above equation?
Thanks for your time and effort!
Jake

7. Re: D435 Resolution VS Depth Image Quality
Aug 24, 2018 7:46 AM (in response to jakeding)This message was posted on behalf of Intel CorporationHello jakeding,
When calculating the theoretical limits of the Depth RMS error will be better to use the ground truth value.
I have also calculated the Depth RMS Error using the values from the screenshots of the Depth Quality Tool and this is what I got: (1280x720) HFOV = 90deg, Xres = 1280, subpixel = 0.09, baseline = 50mm, distance = 1111mm. The Plane Fit RMS error is: 3.47mm (The Depth Quality Tool indicated 3.32mm)
 (848x480) HFOV = 90deg, Xres = 848, subpixels = 0.05, baseline = 50mm, distance = 1111mm. The Plane Fit RMS error is: 2.91mm (The Depth Quality Tool indicated 2.71mm)
As you can see, the Xres value will change with the resolution. Also, you have different values for subpixels for 1280x720 and for 848x480.
The difference between the values received from the theoretical calculus and the values provided by the Depth Quality Tool are caused by the fact that every camera has a slightly difference focal length, field of view, etc.
Kindest regards,
Eliza 
8. Re: D435 Resolution VS Depth Image Quality
jakeding Aug 24, 2018 7:59 AM (in response to Intel Corporation)You are awesome! Thanks again!

9. Re: D435 Resolution VS Depth Image Quality
jakeding Aug 24, 2018 10:14 AM (in response to Intel Corporation)Hi Eliza,
Last two questions:
Why do you use Subpixel error instead of Subpixel in the equation? In P3 of BKMs white paper, I found they recommend us to use Subpixel = 0.08.
Also, in my understanding, the Depth error limit should be the best case scenario. This means the theoretical limit should always be smaller than the realistic error computed from Realsense Viewer. How do you think the definition of the depth error limit?
Best,
Jake

10. Re: D435 Resolution VS Depth Image Quality
Aug 24, 2018 11:02 AM (in response to jakeding)This message was posted on behalf of Intel CorporationHello jakeding,
Subpixel RMS error and Subpixel represents the same value on the BKMs paper. This value was set as 0.08 in order to provide with a better understanding of the relationship between Depth RMS Error and the distance for D415 and D435 cameras. (the graph on page 4)
Regarding your last question, I do agree with you. The theoretical limit should be the best scenario, but as I have mentioned before, all the cameras are slightly different due to manufacturing differences. In the datasheet, you can also see that there are tolerances for FOV, at page 56. When we calculate the theoretical limit of the Depth RMS error, we are using some specific values, which may or may not be the same as yours. For example, if your camera has a smaller HFOV value than 90deg, then your Depth RMS error from the Depth Quality Tool will be smaller as well.
I hope this answers your questions!
Thank you in advance,
Eliza 
11. Re: D435 Resolution VS Depth Image Quality
jakeding Aug 30, 2018 8:59 AM (in response to Intel Corporation)Hi Eliza,
Sorry for the late reply. Now I know the Subpixel RMS error and Subpixel represents are the same value.
But I am still confused about theoretical depth RMS error limit equation on BKMs paper. My question is: Why is this equation a limit? I think we just input the real physical parameters like focal length or distance and get the error. So, what's the definition of the limit here? If the limit here indicates the best scenario, why did I have the smaller RMS error (3.32 & 2.71) generated from Depth Quality tool than the theoretical RMS error (3.47 & 2.91)?
I found a paper which derive the theoretical depth RMS error equation(please see attachment) derived from depth equation. I don't know if you derive in the same way.
Best,
Jake

RMS derivation.jpg 217.4 K


12. Re: D435 Resolution VS Depth Image Quality
Aug 30, 2018 11:21 AM (in response to jakeding)This message was posted on behalf of Intel CorporationHello
Please note that the refer of "limit" appears because it is a mathematical equation that does not take into account all physical imperfections in the actual hardware. The equation will come very close to the reported values by the tool but the sensor hardware may introduce tiny variations.
Please let me know if you have further questions!
Best regards,
Eliza 
13. Re: D435 Resolution VS Depth Image Quality
jakeding Aug 30, 2018 11:26 AM (in response to Intel Corporation)Thanks! So can I understand the imperfections in the camera may cause the realistic error less than the theoretical limit value?

14. Re: D435 Resolution VS Depth Image Quality
Aug 30, 2018 11:49 AM (in response to jakeding)This message was posted on behalf of Intel CorporationHello jakeding,
Yes, this is correct.
Thank you,
Eliza