14 Replies Latest reply on Mar 21, 2018 3:34 PM by Ted_T

    No RGB stream available (D435)

    n_carey

      I added this comment on a previous thread (D415 is showing only depth stream and not RGB color stream ) but thought it might get lost, and it's an issue I'd like to solve - my apologies for the double post.

       

      My D435 has no colour sensor available. I'm running it on a NUC 5i7RYH with Ubuntu 16.10, and have tried all four USB3.0 ports. Realsense viewer shows no colour stream; if I run the rs-sensor-control example it only shows the stereo depth sensor on the device, no RGB camera.

       

      This same machine has run all previous iterations of the RealSense technology pretty well. Using the newest SDK with an older SR300 works fine, so I don't think it's the software or uvcvideo. It could be a power problem, but I thought USB3.1 was supposed to have the same power draw requirements as USB3.0?

      Possibly related issue: when viewing sensor streams using the included /example/rs-sensor-control, choosing an IR stream causes a segmentation fault (though I can view IR output using my own apps and the realsense viewer).

       

      CPU usage: When streaming depth only, if I use highest depth resolution the CPU (for a single core) spikes to 100%, but using a lower resolution it sits between 40-70%. I've tried compiling with OpenMP both on and off, and neither setting made a  difference to RGB stream availability or IR crashes.

       

      edit: Just noticed that when I pull up the realsense viewer it says the source is an Intel Realsense 430 (not 435) ???

        • 1. Re: No RGB stream available (D435)
          MartyG

          Although I do not have a D435 to test with, a search for Viewer pictures on Google Images shows that the camera is usually displayed as ''Intel RealSense 435'.  The D430 (the circuit board camera module inside the D435 casing) does not have an integrated RGB sensor, though it is a part of the D435's casing innards.  So if your camera is being detected as a D430 configuration, that might be why RGB is not made available (this is just a theory though).

          • 2. Re: No RGB stream available (D435)
            n_carey

            Yes, that seems very likely! Not sure what to do about it, unfortunately.

            • 3. Re: No RGB stream available (D435)
              MartyG

              In the Viewer's device information option just below the displayed camera name, there is an option to reset the camera hardware.  You could give that a try.

              • 4. Re: No RGB stream available (D435)
                n_carey

                Resetting the hardware didn't fix the issue, but it did throw up some interesting errors about holding onto frames. So I re-patched uvcvideo (since that's usually the culprit for camera issues) and remade the librealsense build.

                This did not fix the problem (still D430, still no RGB),  but now if I reset the hardware through realsense-viewer I'm at least getting different, more informative errors:

                 

                16/03 17:38:41,113 WARNING [139736904591104] (sensor.cpp:313) Unregistered Media formats : [ UYVY ]; Supported: [ ]

                16/03 17:38:47,457 ERROR [139736819689216] (types.h:178) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such device

                16/03 17:38:47,457 ERROR [139736819689216] (error-handling.cpp:68) Error during polling error handler: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such device

                16/03 17:38:48,457 ERROR [139736819689216] (types.h:178) Cannot open '/dev/video0 Last Error: No such file or directory

                16/03 17:38:48,457 ERROR [139736819689216] (error-handling.cpp:68) Error during polling error handler: Cannot open '/dev/video0 Last Error: No such file or directory

                 

                Following through on these:
                - Trying to open the RGB directly with v4l2-ctl also gives "/dev/video0: No such file",

                - the camera is not being registered/used  by uvcvideo, according to lsmod.

                - v4l2-ctl-dev is the latest version (1.10.0-1)
                - lsusb gives the following:

                 

                Bus 001 Device 002: ID 8087:8001 Intel Corp.

                Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

                Bus 003 Device 018: ID 8086:0ad4 Intel Corp.

                Bus 003 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub

                Bus 002 Device 005: ID 8087:0a2a Intel Corp.

                Bus 002 Device 004: ID 04f2:0939 Chicony Electronics Co., Ltd

                Bus 002 Device 003: ID 413c:2107 Dell Computer Corp.

                Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

                 

                When I plug in the realsense and check dmesg I get the following:

                USB Video Class driver (1.1.2.realsense-1.2.0)

                [24785.814215] usb 2-1: new high-speed USB device number 12 using xhci_hcd

                [24785.942911] usb 2-1: New USB device found, idVendor=8086, idProduct=0ad6

                [24785.942913] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0

                [24785.942914] usb 2-1: Product: Intel(R) RealSense(TM) 430

                [24785.942915] usb 2-1: Manufacturer: Intel(R) RealSense(TM) 430

                [24785.943511] uvcvideo: Found UVC 1.50 device Intel(R) RealSense(TM) 430 (8086:0ad6)

                [24785.944351] uvcvideo: Unable to create debugfs 2-12 directory.

                [24785.944410] input: Intel(R) RealSense(TM) 430 as /devices/pci0000:00/0000:00:14.0/usb2/2-1/2-1:1.0/input/input38

                 

                 

                So not yet solved, but getting closer to the source. Also there's a possibly related problem here: Frames didn't arrive error - after improper shutdown · Issue #1086 · IntelRealSense/librealsense · GitHub

                I'll keep plugging at it.

                 

                edit: Other people seem to have had the same errors with various streams, there is some indication it is a kernel patch problem.

                • 5. Re: No RGB stream available (D435)
                  MartyG

                  Thanks for the great detailed feedback _ it is sure to be helpful for Intel support team members when looking at your problem.

                   

                  When I get a wrong camera name displayed in the Viewer, I find it can help to unplug the camera from the USB port, wait a couple of seconds and then push it back in firmly all the way in.

                  • 6. Re: No RGB stream available (D435)
                    n_carey

                    Unfortunately unplugging and replugging does not help - but it's starting to look more and more like this might be related to a known firmware bug, cf
                    Frames not appearing and Camera unresponsive on realsense viewer and ros wrapper · Issue #1131 · IntelRealSense/libreals…


                    so hopefully it's being worked on! I will keep an eye out for any updates.

                    • 7. Re: No RGB stream available (D435)
                      thePaintedSnipe

                      Hi Carey,

                       

                      I have seen stability issues with D435 on various platforms. However, nothing exactly which matches the behavior you mention. I am not sure if this will really help you, but I would have tried following

                       

                      (1) Cross check all Linux Kernel patches are in place.

                      (2) run "rs-enumerate-devices"

                          - This will give you a device name. What is the name associated with your device?

                          - What is the firmware version?

                      (3) run "ls /dev/vid*", before and after you plug-in your device.

                          - For D435, if everything is ok, you will see 3 new device nodes being created

                          - 2 device nodes are for stereo IR cameras, and 3rd is for RGB device

                      (4) I have seen an issue with D435 where at some times it is not enumerated properly. But if you remove & replug it, it may work. For me, at times it require process be repeated 2-3 times.

                      (5) My D435 is more stable on PC, and less stable on embedded board I use. This could be because of either Power issue, or may be some kind of noise on USB lines. I am not fully sure.

                          - But connecting D435 via external USB hub seem to have helped

                          - Though I have not yet tried it, but I think using a shorter USB cable will help too. The cable which came with the D435 is rather too long.

                       

                      Regards,

                      PS

                      • 8. Re: No RGB stream available (D435)
                        n_carey

                        Hi PS,
                        Unfortunately none of these steps added much information. I've checked the kernel patches several times, and other RealSense devices are working ok.

                         

                        Output of rs-enumerate-devices gives same info as before (notably, only the board name 430 rather than the full device name 435). Firmware version as below:


                        Device info:

                            Name                          :     Intel RealSense 430

                            Serial Number                 :     746612070042

                            Firmware Version              :     05.08.15.00

                            Physical Port                 :     /sys/devices/pci0000:00/0000:00:14.0/usb3/3-1/3-1:1.0/video4linux/video0

                            Debug Op Code                 :     15

                            Advanced Mode                 :     YES

                            Product Id                    :     0AD4

                         

                        After connecting the D435 there are only ever two device nodes in use (dev/video0 and dev/video1) which are presumably the IR nodes
                        Obviously I've disconnected and reconnected the hardware several times now, and rebooted, nothing changes. I did try an external hub just on the off-chance it had a more stable connection, but no luck.

                         

                        I will see if i can find a shorter connecting cable, and maybe try it on another CPU, to see if it is a power issue with the USB3 input ports of the NUC.

                        • 9. Re: No RGB stream available (D435)
                          thePaintedSnipe

                          Hi Carey,

                           

                          I am not sure what should be the next debug step for you. Since you mention that only 2 device nodes are getting created when you plug in the device, so RGB sensor may be really missing in the signal chain. I think there are several possibilities

                          (1) You indeed have a device without RGB Sensor. On the D435, you can see two stereo IR sensors [these are identical and easy to spot], one IR projector [this is in middle of two IR sensors], and a RGB sensor [which is the right most]. Do you see all these sensors physically available?

                          (2) Next possibility is that RGB sensor is physically present but it is non functional. I am not sure what can be the reason, but may be some loose connection inside the casing, or not enough power? To debug any such problem, I think only Intel can help.

                          (3) Something specifically incompatible with the Hardware where you are connecting the camera, thereby not identifying the RGB sensor. However, if you have cross checked all patches etc, then this is less likely to be case.

                           

                          I am not sure what you could try next, but it may be worth trying to see how camera works on any other system.

                          (1 If you have access to Win 10 PC, then you should probably see how realsense-viewer is working.

                          (2) Likewise if you can get access to an Ubuntu PC, just check if realsense-viewer still detects only Stereo camera.

                           

                          I can understand the tricky situation you are in. I hope that some one from Intel will be able to help you out

                          • 10. Re: No RGB stream available (D435)
                            n_carey

                            update: Intel support have told me that the D435 doesn't have RGB.

                            To me this sounds like nonsense? I can clearly see an RGB camera/sensor on the front of the camera, and from the forums it seems that other people seem to have received colour streams from their D435. The specifications page clearly lists D4 vision processor with RGB resolution and FoV.

                             

                             

                            Main Intel® RealSense™ component

                            Intel® RealSense™ Vision Processor D4

                            Intel® RealSense™ module D430

                            Depth Field of View (FOV)—(Horizontal × Vertical × Diagonal)91.2 x 65.5 x 100.6 (+/- 3°)
                            Depth Stream Output ResolutionUp to 1280 x 720
                            Depth Stream Output Frame RateUp to 90 fps
                            Minimum Depth Distance (Min-Z)0.2m
                            Sensor Shutter TypeGlobal shutter
                            Maximum RangeApprox.10 meters; Varies depending on calibration, scene, and lighting condition
                            RGB Sensor Resolution and Frame Rate1920 x 1080 at 30 fps
                            RGB Sensor FOV (Horizontal x Vertical x Diagonal)69.4° x 42.5° x 77° (+/- 3°)

                             

                            Can anyone shed some light on this? Hopefully it's a communication issue.

                             

                            @PS: I'm using Ubuntu. I've tried plugging the realsense into both windows and OSX, none of them recognise an RGB camera. The RGB is visible externally, as noted. Hardware/power/loose connection is looking likely, unless Intel is right about D435 not having RGB at all.

                            • 11. Re: No RGB stream available (D435)
                              MartyG

                              The D435 does have an RGB sensor.  It is the circuit board inside (called the D430 Module) that does not have an RGB sensor integrated into it.  So the RGB sensor is connected to the cased USB D435 camera as a separate component rather than being an integrated part of the circuit board.

                              • 12. Re: No RGB stream available (D435)
                                n_carey

                                That was my understanding!

                                Intel customer support have told me the following:

                                 

                                     The D435 camera does not have an active RGB module. This information is stated on the following link:

                                     https://realsense.intel.com/stereo/

                                     Click on "Compare Depth Modules" to see a comparison table of features. It shows that the D415 is the only depth module with an RGB sensor.

                                 

                                I think they must not have understood my problem. Hopefully they will get back to me.

                                • 13. Re: No RGB stream available (D435)
                                  thePaintedSnipe

                                  Yes, they might have not understood you correctly.

                                  I have a D435 camera and RGB stream works just fine.

                                  As Marty mentioned, RGB sensor is connected to Asic Board via a connector. In case connector is loose inside, you may see this problem.

                                  • 14. Re: No RGB stream available (D435)
                                    Ted_T

                                    Odd choice for nomenclature and numbering; modules are only the imager boards and cameras are modules plus processor. The depth module, D430 module does not have an RGB camera; the D415 module has an integrated RGB camera. The D435 camera contains a D430 module AND a standalone RGB camera. The D4 processor board determines its mode by what is connected and it appears that there is something wrong with your RGB module. As someone else suggested, it might be that the flex cable is not firmly connected; perhaps the RGB camera itself is bad.

                                    In any case the D4 thinks it is a D430 camera because your earlier information indicated a PID of 0x0ad4; a D435 camera (D430+RGB) has a PID 0x0B07.