7 Replies Latest reply on Jul 24, 2018 5:47 AM by MartyG

    Analyze Depth images in Unity


      I just installed the relasense SDK 2.0 on unity. I'm using 2 depth cameras (Intel d415 and Intel d435).

      Now, I've the colored depth image as in the examples. How can I analyze this images? I need to find the nearest object and draw a square around it.

      Is there something similar to Opencv in Unity? I can't find anything but a plugin with very bad rates...

        • 1. Re: Analyze Depth images in Unity

          Was the plugin that you saw the OpenCV For Unity one?


          OpenCV for Unity - Asset Store


          If you were able to use OpenCV within Unity then you may be able to draw a box simply by accessing the Rectangle function of OpenCV.


          Re: A simple C++ code to obtain bounding box.

          • 2. Re: Analyze Depth images in Unity

            Yes, I see now that the ratings are not so low...but price is a little to high.


            I wanted to see if there is something else and then I will try that add-on.


            Thank you

            • 3. Re: Analyze Depth images in Unity

              Unity has an inbuilt function called GUI.Box that can draw a box on the screen.  The problem would be programming an event to trigger this function, such as when the camera reads the depth of an object.


              Unity - Scripting API: GUI.Box

              1 of 1 people found this helpful
              • 4. Re: Analyze Depth images in Unity

                Good! this is interesting, but the main problem remains how to analyze the depth image. I think that the add-on with opencv is the only method...

                • 5. Re: Analyze Depth images in Unity

                  As the Unity wrapper is based on C# (the language that Unity uses), if I were writing your program then I would attempt to integrate the basic 'Hello World' C# example for analyzing depth into a Unity script.




                  var pipe = new Pipeline();



                  while (true)


                      using (var frames = pipe.WaitForFrames())

                      using (var depth = frames.First(x => x.Profile.Stream == Stream.Depth) as DepthFrame)


                          Console.WriteLine("The camera is pointing at an object " +

                              depth.GetDistance(depth.Width / 2, depth.Height / 2) + " meters away\t");


                          Console.SetCursorPosition(0, 0);



                  1 of 1 people found this helpful
                  • 6. Re: Analyze Depth images in Unity

                    I tried to modify the "RealsenseDevice" script adding the depth.GetDistance method and it works. I just analyze one pixel every 50 and I misure the distance. the problem is that it isn't very precise. Should I calibrate the camera or something similar?

                    • 7. Re: Analyze Depth images in Unity

                      RealSense cameras are usually well calibrated and only need re-calibrating if something unusual happens to them or they take a hard knock / drop.


                      Accuracy may be influenced by how near the camera is to what you are measuring.  At very close range, the accuracy of the 400 Series can reduce because of how its pixel scaling is set up to enable it to see a distance of around 65 meters (though depth scanning is limited to 10 meters maximum by default).  If you are doing close-range scanning, you can change the camera's depth scale to improve accuracy.


                      Re: Problem aligning color to depth


                      A less complex method of reducing the camera's minimum distance is to reduce the resolution to a low one, less than 848x400.