4 Replies Latest reply on Nov 6, 2018 3:19 AM by carvster

    Get depth data from D415 within a Unity script

    anonldlm

      I'm currently working with the ARBackground unity project from realsense's official repo, and wanted to get the distance from the camera to the object in the middle. So I used the following C++ code from the samples and compiled it into plugin so I can call the function from Unity:

       

      float getDepth()

      {

           // Create a Pipeline, which serves as a top-level API for streaming and processing frames

           pipeline p;

       

           // Configure and start the pipeline

           p.start();

       

           float center_distance = 0.0f;

       

           while (true)

           {

                // Block program until frames arrive

                frameset frames = p.wait_for_frames();

       

                // Try to get a frame of a depth image

                depth_frame depth = frames.get_depth_frame();

                // The frameset might not contain a depth frame, if so continue until it does

                if (!depth) continue;

       

                // Get he depth frame's dimensions

                float width = depth.get_width();

                float height = depth.get_height();

       

                // Query the distance from the camera to the object in the center of the image

                float dist_to_center = depth.get_distance(width / 2, height / 2);

                center_distance = dist_to_center;

       

                break;

           }

       

           return center_distance;

      }

       

      However, when I call this function from inside a Unity script, the original streaming to ARBackground stops and freezes:

       

          [DllImport("DepthPlug")]

          private static extern float getDepth();

       

       

          public void MoveZ ()

          {

              Vector3 newPos = transform.position;

       

       

              newPos.z = getDepth() * 0.98f;

              newPos.y = 0.0f;

       

       

              transform.position = newPos;

          }

       

      What is the proper way to call the realsense SDK from a Unity script?

        • 1. Re: Get depth data from D415 within a Unity script
          Intel Corporation
          This message was posted on behalf of Intel Corporation

          Hi amonldlm,

          Thank you for your interest in the Intel RealSense D415 camera. 
          Did you follow the steps described here?
          https://github.com/IntelRealSense/librealsense/tree/master/wrappers/unity

          Regards,
          Alexandra

          • 2. Re: Get depth data from D415 within a Unity script
            anonldlm

            Hi Alexandra,

             

            Yeah I did, I can receive the depth data correctly. The problem is that the other C# scripts in ARBackground in charge of streaming stops working immediately after calling the plugin.

            • 3. Re: Get depth data from D415 within a Unity script
              Intel Corporation
              This message was posted on behalf of Intel Corporation

              Hi amonldlm,

              Please refer to the provided script at .\ibrealsense-master\wrappers\unity\Assets\RealSenseSDK2.0\Scenes\StartHere.unity to see how to call the ARBackground plugin.

              Regards,
              Alexandra

              • 4. Re: Get depth data from D415 within a Unity script
                carvster

                Hi anonldlm,

                 

                I think the problem with this set up is that in the ARBackground sample the camera is already streaming and the C++ DLL then also tries to setup an additional pipeline stream on top which then confuses the camera stream with two pipelines

                 

                I have been looking to do the same thing and eventually found a much simpler way of doing this by changing the RsDevice.cs script within the Unity wrapper by creating a new float variable (CentrePixelDepthDistance) and then modifying the WaitForFrames() function:

                 

                private void WaitForFrames()

                    {

                        while (!stopEvent.WaitOne(0))

                        {

                            using (var frames = m_pipeline.WaitForFrames())

                            {

                                using (var depthDist = frames.First(x => x.Profile.Stream == Stream.Depth) as DepthFrame)

                                {              

                                    CentrePixelDepthDistance = depthDist.GetDistance(depthDist.Width / 2, depthDist.Height / 2);

                                    //Debug.Log("The camera is pointing at an object " + CentrePixelDepthDistance + " meters away\t");

                                }

                                OnNewFrameSet(frames);

                            }

                        }

                    }

                 

                 

                You can then access your depth distance float variable from any other script using:

                 

                RealSenseDevice.GetComponent<RsDevice>().CentrePixelDepthDistance

                 

                Hope this helps!