5 Replies Latest reply on Jul 15, 2018 12:25 PM by MartyG

    D435 Geometry: Focal point seems to be 285mm BEHIND the camera

    smirkingman

      I have been struggling for a full week, 10 hours a day, trying to interpret the depth data from my D435, without success.

      To try and identify the issue I performed an experiment, as follows.

       

      The D435(which I have successfully calibrated using Intel.Realsense.DynamicCalibrator.exe) is mounted 420mm above the floor:

      robot.jpg

      It is facing a door with markers 420mm above the floor, the left marker at 220mm left, right marker at 560mm right, there's a tape measure starting against the door:

      door.jpg

      The D435 front glass is 3840mm from the door (this picture is taken at an angle, looking vertically, it IS 3840mm):

      distance.jpg

      My application reads depth at 848x480, colour at 1920x1080 and uses Align to make the colour frame line up with the depth:

      code.jpg

      I view the colour and depth streams and read off the depths of the 3 markers:

      screenshot.jpg

      The results are (I count rows like a human, from the bottom: row 0 is at the BOTTOM of the images, row 479 it the topmost row):

      Left marker: Row 237, column 388, depth 4147

      Centre marker: Row 237, column 412, depth 4128

      Right marker: Row 236, column 472, depth 4135

      It's worth noting:

      • The rows are almost at the vertical centre of the D435
      • The centre column is almost at the horizontal centre of the D435

      so it's fair to say that vertical errors are negligable.

       

      I then drew this setup, here's the plan view.

      Black are known measurements made with a ruler, Red is the right depth ray, blue is the centre depth ray and green is the left depth ray:

      D435Geometry.png

      (The D435 is deliberately off-centre, 52.50mm is the distance from the edge of the 435 to the focal point of the depth map.)

      The 3 depth measurements are consistent (there is only 17.42mm difference between the red and green arrow tips) with a point 285mm behind the D435.

      Interestingly, at the face of the D435, the red and green rays are 53.51mm apart, which is awfully close the distance between the infrared cameras.

       

      Now, this is all rather empirical and my measurements are probably no better than +/- 1mm, but 285mm is a relatively huge discrepancy.

       

      Is this possible (if so, what is the EXACT value of my 285mm, to the µM?), or have I made a glaring blunder?

       

      P.S. The Intel-RealSense-D400-Series-Datasheet says, table 4-4 on page 54, that the D430's HFOV is 86° and VFOV is 57° and adds "Due to mechanical tolerances of +/-5%, Max and Min FOV values can vary from lens to lens and module to module by ~ +/- 3 degrees.".

             I'm assuming that by 430 they meant 435... How can I get the EXACT numbers?

        • 1. Re: D435 Geometry: Focal point seems to be 285mm BEHIND the camera
          MartyG

          The 'Librealsense2' section of code in the link below can give you script instructions for getting the field of view from 400 Series cameras.

           

          API How To · IntelRealSense/librealsense Wiki · GitHub

           

          Ideally, I would recommend setting color and depth to the same resolution if possible to aid alignment.  If this is not possible, then you could use a post-processing filter called a Decimation Filter to adjust the scale that a stream is displayed at.  You can test this without writing code by enabling the Advanced Mode in the RealSense Viewer, expanding open the 'Advanced Controls' section and expanding open the 'Decimation filter' option to reveal a slider.

           

          If the Decimation Filter works for you in RealSense Viewer tests then you can control it in your application via code.

           

          Also bear in mind that on the 400 Series models, the closer the camera gets to an object, the more error that may be introduced into the accuracy.  This is because of how the camera's pixels are scaled to enable the camera to 'see' up to 65 meters in its "expressive range" (and up to 10 m maximum for depth), compared to around 2 m max on the older SR300 camera model.  It is possible to change the 400 Series camera's scaling to match the greater close-range accuracy of the SR300.  Details of this can be found here:

           

          Re: Problem aligning color to depth

          • 2. Re: D435 Geometry: Focal point seems to be 285mm BEHIND the camera
            smirkingman

            Hi Marty,

            thanks for the super-quick reply, I appreciate it.

            I upped the colour frame resolution so that when aligned, I get more coverage; the results are identical with the same resolutions.

             

            My depth intrinsics are:

            Width 848

            Height 480

            PPX 422.4793

            PPY 241.7834

            FX 423.1294

            FY 423.1294

             

            Using the viewer, I place my cursor over a known point on the floor, at the base of the sheet of paper, exactly in the centre of the camera; it's at column 424 row 388.

            It correctly says the point is 1200mm away (along the floor):

            viewer.jpg

            My D435 is 420mm from the floor and I have verified that it is looking horizontally.

            Knowing the intrinsics above, how do I calculate the HFOV and VFOV?

            and, knowing the row, column, intrinsics and the raw depth value is 1230, how do I calculate 1200mm?

             

            When I draw it, it's wrong; a 1230mm depth ray simply isn't long enough to get to a point 1200mm away (I'm fairly sure that the angle of row 388 is just under 18.5°):

            wrongangle.png

             

            I've searched the C++ source code of the viewer but I can't find out how it gets the right number, I'd be extremely grateful for any assistance.

            • 3. Re: D435 Geometry: Focal point seems to be 285mm BEHIND the camera
              MartyG

              On the 400 Series cameras, the depth pixel value is a measurement from the parallel plane of the imagers and not the absolute range.

               

               

              In the 400 Series' data sheet document, the values given for the D435's left and right imagers are HFOV = 91.2 and VFOV = 65.5.  Diagonal FOV is 100.6

               

              The FOV values of the color camera on the D435 are HFOV =69.4 and VFOV = 42.5.  Diagonal FOV = 77

               

              The IR projector FOV values are HFOV = 100.4, VFOV = 69 and diagonal FOV = 110.4

               

              Another document I found added that the above FOV values are  +/- 3 degrees (image FOV can vary).

               

              This link may be useful to you:

               

              Computed FOV (angle of view) doesn't match the specificatoins

              • 4. Re: D435 Geometry: Focal point seems to be 285mm BEHIND the camera
                smirkingman

                [FacePalm!] What a colossal blunder, that'll teach me to RTFM

                In my defence, I was using a Kinect before, where 'depth' is 'range' in Intel-speak and I blindly assumed the RealSense would be the same.

                All works delightfully now:

                fixed.jpg

                Many thanks Marty, if you're once in Geneva (Switzerland), I'll buy you dinner

                 

                P.S. The depth/range reported has it's origin about 32mm behind the front face of the D435.

                • 5. Re: D435 Geometry: Focal point seems to be 285mm BEHIND the camera
                  MartyG

                  Awesome news that you got it working.  We learn as we go.  Or to put it another way, Pobody's Nerfect.