9 Replies Latest reply on Feb 8, 2018 3:07 PM by rosme

    Windows UWP support of D415 and D435

    rosme

      I tested the new D415 and D435 to see if they supported the Windows UWP API and was pleasantly surprised that they do.

      However there seem to be a few differences with what I would expect.

       

      1. Each device reports three MediaFrameSourceGroups where I would expect only one per device.

       

      D415:

      Intel(R) RealSense(TM) 415 RGB

      Intel(R) RealSense(TM) 415 Depth

      Intel RS400 Cameras

       

      D435:

      Intel(R) RealSense(TM) 430 with RGB Module RGB

      Intel(R) RealSense(TM) 430 with RGB Module Depth

      Intel RS400 Cameras

       

      The generic RS400 Cameras are the ones that combine the RGB and Depth into the single MediaFrameSourceGroup that I would expect, but I would be better if they carried the specific type in their displayname.

       

      2. Infrared is reported as color

      The "Intel RS400 Cameras" reports the support of three MediaFrameSources, two color and one depth.

      It turns out that one of the two color MediaFrameSources actually contains the infrared image of the right IR camera (when seen from the front).

      I would expect this MediaFrameSource to have the Infrared type.

       

      Is this the best place to report issues like this?

       

      I am eagerly awaiting support for synchronized streams, but I believe we will have to wait for new firmware before this can be tested.

      kind regards,

       

      Roland Smeenk

        • 1. Re: Windows UWP support of D415 and D435
          Intel Corporation
          This message was posted on behalf of Intel Corporation

          Hello Rosme, 

          Thanks for contacting Intel customer support. 

          I am escalating your case to engineering for them to check the information you have provided. 

          Thanks for your patience and understanding. 

          Best Regards, 
          Juan N. 

          • 2. Re: Windows UWP support of D415 and D435
            Intel Corporation
            This message was posted on behalf of Intel Corporation

            Hello Rosme, 

            I have just received a notification from the engineering team with the following information: 

            "Regarding Issue 1. This is by design. By definition each camera device creates a Sensor Group (SG) on its own. For two cameras (asrc, awgc etc.) on same module (dedicated PID), we add the SG that includes them both. So if we take only depth you’ll have a single SG. If depth and color you’ll end up with Color SG, Depth SG and Depth+Color SG (that should be used to sync depth and RGB).
             
            We are digging into the second issue but it may be due to a Microsoft limitation."

            Thanks for your patience. 

            Best Regards, 
            Juan N. 

            • 3. Re: Windows UWP support of D415 and D435
              Intel Corporation
              This message was posted on behalf of Intel Corporation

              Hello Rosme,

              We received further clarification from the RealSense team on question 2.

              " The image streams from Left and Right Imagers in D400 series cameras (which include D415 and D435) are not just IR and include visible + IR. Y8 is more of grayscale color than IR."

              Regards,
              Jesus
              Intel Customer Support

              • 4. Re: Windows UWP support of D415 and D435
                erith835

                Thank you for the information provided in this thread. I am very interested in this development.

                 

                I use the "CameraStreamCorrelation" sample from the Windows-universal-samples (GitHub - Microsoft/Windows-universal-samples: API samples for the Universal Windows Platform.) to test the UWP API with a RealSense D435 camera. But for me the call "MediaFrameSourceGroup::FindAllAsync()" only reports the two items:

                 

                Intel(R) RealSense(TM) 430 with RGB Module RGB

                Intel(R) RealSense(TM) 430 with RGB Module Depth

                 

                but not the "Intel RS400 Cameras" which is said above to contain both RGB and Depth (which is what I need). What do I need to do to get the "Intel RS400 Cameras" group?

                 

                Where can I find the latest information and documentation about the UWP API for RealSense D400 cameras?

                 

                Thanks,

                 

                Erik

                 

                Here is my setup:

                 

                Windows 10 Pro, Version 1709, OS Build 16299.192

                Visual Studio Community 2017, Version 15.5.6

                Windows 10 SDK 10.0.16299.0 (as reported by Visual Studio installer)

                Intel RealSense 435, Firmware version: 05.08.15.00

                 

                 

                • 5. Re: Windows UWP support of D415 and D435
                  rosme

                  Hi Jesus,

                   

                  I understand the technical reasoning behind it, but I it makes it harder to select the real color source. Now the RGB and the "visible + IR" camera both report that they are a color source. The Microsoft example CameraFrames (Windows-universal-samples/Samples/CameraFrames at master · Microsoft/Windows-universal-samples · GitHub ) automatically takes the first color source which happens to be the "visible + IR" grayscale color.

                   

                  kind regards,

                   

                  Roland

                  • 6. Re: Windows UWP support of D415 and D435
                    Intel Corporation
                    This message was posted on behalf of Intel Corporation

                    Hello Erik,

                     

                    We just learned that Intel took down the UWP driver from the Windows Update servers on January 28 to make functional changes. This may be while the Windows UWP app isn't working for you. Check your Device Manager if you have entries for the D400 camera under Imaging Devices. If the driver is installed correctly you will see three entries.

                     

                    Also, there is no UWP API that is specific to the RealSense cameras. The purpose of the driver is to allow the cameras to work with any UWP app using the Microsoft APIs.

                     

                    Regards,
                    Jesus
                    Intel Customer Support

                    • 7. Re: Windows UWP support of D415 and D435
                      rosme

                      Hi Jesus,

                       

                      I also tested the CameraStreamCorrelation example and I ran into an issue where there is no valid CoordinateSystem in the provided colorframe. This will cause the TryCreateCoordinateMapper to fail. That coordinatemapper is needed to map depth pixels from depth space to color space. I suspect that the implementation of this API is still a work in progress.

                       

                      kind regards,

                       

                      Roland

                      • 8. Re: Windows UWP support of D415 and D435
                        Intel Corporation
                        This message was posted on behalf of Intel Corporation

                        Hello Rosme,

                        Thanks for the bug report. I will report it to the team. Support on the UWP driver is limited now since it has been taken down temporarily.

                        Regards,
                        Jesus
                        Intel Customer Support

                        • 9. Re: Windows UWP support of D415 and D435
                          rosme

                          " The image streams from Left and Right Imagers in D400 series cameras (which include D415 and D435) are not just IR and include visible + IR. Y8 is more of grayscale color than IR."

                           

                          Funny that you use this statement to defend that the stream should not be of the infrared type and then in the Unity wrapper simply call it "infrared".

                          Not really consistent.

                          librealsense/wrappers/unity at development · IntelRealSense/librealsense · GitHub

                           

                          kind regards,

                           

                          Roland