4 Replies Latest reply on Sep 17, 2018 8:28 PM by westa

    Unity Could not connect pins - RenderStream() to Intel Realsense D435

    snovan

      Hello

       

      First off, I realize that this question is maybe closer related to Nuitrack and Unity3d, so I apologize if this is the wrong place for this.

       

      I am trying to play around with this camera in unity, but when I try to use it's RGB features, I get the following error:

           Could not connect pins - RenderStream()

           UnityEngine.WebCamTexture:Play()

           RenderCamera:Start() (at Assets/Scripts/Camera/RenderCamera.cs:75)

       

       

      where the script is as follows:

          72    webcamTexture.deviceName = devices[RealsenseD435].name;

          73    Renderer renderer = GetComponent<Renderer>();

          74    renderer.material.mainTexture = webcamTexture;

          75    webcamTexture.Play();

       

      additionally, when I try to use it's depth features, I get the following error:

           ModuleNotInitializedException: NuitrackException: ModuleNotInitializedException

           nuitrack.NativeImporter.throwException (ExceptionType type)

           nuitrack.NativeDepthSensor..ctor ()

           nuitrack.DepthSensor.Create ()

           NuitrackManager.NuitrackInit () (at Assets/NuitrackSDK/Nuitrack/NuitrackManager.cs:187)

           NuitrackManager.Awake () (at Assets/NuitrackSDK/Nuitrack/NuitrackManager.cs:110)

       

      where NuitrackManager.cs:187 =

           depthSensor = nuitrack.DepthSensor.Create();

       

      So far, I know that the Intel Realsense D415 works with this exact same project.

       

      any help or experiences with the D435 would be greatly appreciated!

        • 1. Re: Unity Could not connect pins - RenderStream() to Intel Realsense D435
          MartyG

          You could test whether the problem is with the camera device name you are using by changing 'RealsenseD435' to '0'.

           

          webcamTexture.deviceName = devices[0].name;

           

          This will cause Unity to connect to the first webcam available.  If you are using a laptop with a built in webcam and Unity picks that up as the first camera, you could try '1' instead, assuming that Unity looks for all webcam-capable devices available and puts them in a numbered list.

           

           

           

          • 2. Re: Unity Could not connect pins - RenderStream() to Intel Realsense D435
            snovan

            I have done that, in my code 'RealsenseD435' == 1

            0 is my default cam, and 1 is the Intel realsense D435. I have quadruply checked this and have logging to confirm that this is not the issue.
            I have also tried disconnecting my default cam so that the Intel realsense D435 is 0, this changes nothing and the error persists

            • 3. Re: Unity Could not connect pins - RenderStream() to Intel Realsense D435
              MartyG

              May I confirm that you are using Unity and the Nuitrack SDK, and that you are not using the Unity wrapper in RealSense SDK 2.0?  So in other words, you are using the D435 hardware as your depth camera?

               

              The Nuitrack is of course a separate program from RealSense SDK 2.0 and so the camera can only use functions that are supported by Nuitrack SDK.  This makes this case a bit more difficult to diagnose, as if it is a software related incompatibility then that could only be corrected on Nuitrack's side, and the RealSense SDK development team cannot do anything.  However, looking at D435-related posts on the Nuitrack forum, people do seem to be able to use their D435s with it without too much trouble.

               

              A user called Westa is on both this Intel RealSense support forum and the Nuitrack forum, so they may be the best person to offer suggestions about your question.  I have linked westa  into this discussion so they can see it.

              • 4. Re: Unity Could not connect pins - RenderStream() to Intel Realsense D435
                westa

                FWIW - you cannot use the Unity Webcam control and the Nuitrack control at the same time.

                When Nuitrack initialises it attempts to take full control of the sensor - so that it can set the resolutions and preset for filtering.

                If you have the Webcam control running at the same time - this is likely to cause all sorts of issues with nuitrack.

                If you need to view the RGB stream while using Nuitrack then you will need to access the nuitrack color stream and process it yourself.

                Westa