Although I do not have a D435 to test with, a search for Viewer pictures on Google Images shows that the camera is usually displayed as ''Intel RealSense 435'. The D430 (the circuit board camera module inside the D435 casing) does not have an integrated RGB sensor, though it is a part of the D435's casing innards. So if your camera is being detected as a D430 configuration, that might be why RGB is not made available (this is just a theory though).
Resetting the hardware didn't fix the issue, but it did throw up some interesting errors about holding onto frames. So I re-patched uvcvideo (since that's usually the culprit for camera issues) and remade the librealsense build.
This did not fix the problem (still D430, still no RGB), but now if I reset the hardware through realsense-viewer I'm at least getting different, more informative errors:
16/03 17:38:41,113 WARNING  (sensor.cpp:313) Unregistered Media formats : [ UYVY ]; Supported: [ ]
16/03 17:38:47,457 ERROR  (types.h:178) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such device
16/03 17:38:47,457 ERROR  (error-handling.cpp:68) Error during polling error handler: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such device
16/03 17:38:48,457 ERROR  (types.h:178) Cannot open '/dev/video0 Last Error: No such file or directory
16/03 17:38:48,457 ERROR  (error-handling.cpp:68) Error during polling error handler: Cannot open '/dev/video0 Last Error: No such file or directory
Following through on these:
- Trying to open the RGB directly with v4l2-ctl also gives "/dev/video0: No such file",
- the camera is not being registered/used by uvcvideo, according to lsmod.
- v4l2-ctl-dev is the latest version (1.10.0-1)
- lsusb gives the following:
Bus 001 Device 002: ID 8087:8001 Intel Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 018: ID 8086:0ad4 Intel Corp.
Bus 003 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 002 Device 005: ID 8087:0a2a Intel Corp.
Bus 002 Device 004: ID 04f2:0939 Chicony Electronics Co., Ltd
Bus 002 Device 003: ID 413c:2107 Dell Computer Corp.
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
When I plug in the realsense and check dmesg I get the following:
USB Video Class driver (1.1.2.realsense-1.2.0)
[24785.814215] usb 2-1: new high-speed USB device number 12 using xhci_hcd
[24785.942911] usb 2-1: New USB device found, idVendor=8086, idProduct=0ad6
[24785.942913] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[24785.942914] usb 2-1: Product: Intel(R) RealSense(TM) 430
[24785.942915] usb 2-1: Manufacturer: Intel(R) RealSense(TM) 430
[24785.943511] uvcvideo: Found UVC 1.50 device Intel(R) RealSense(TM) 430 (8086:0ad6)
[24785.944351] uvcvideo: Unable to create debugfs 2-12 directory.
[24785.944410] input: Intel(R) RealSense(TM) 430 as /devices/pci0000:00/0000:00:14.0/usb2/2-1/2-1:1.0/input/input38
So not yet solved, but getting closer to the source. Also there's a possibly related problem here: Frames didn't arrive error - after improper shutdown · Issue #1086 · IntelRealSense/librealsense · GitHub
I'll keep plugging at it.
edit: Other people seem to have had the same errors with various streams, there is some indication it is a kernel patch problem.
Thanks for the great detailed feedback _ it is sure to be helpful for Intel support team members when looking at your problem.
When I get a wrong camera name displayed in the Viewer, I find it can help to unplug the camera from the USB port, wait a couple of seconds and then push it back in firmly all the way in.
Unfortunately unplugging and replugging does not help - but it's starting to look more and more like this might be related to a known firmware bug, cf
Frames not appearing and Camera unresponsive on realsense viewer and ros wrapper · Issue #1131 · IntelRealSense/libreals…
so hopefully it's being worked on! I will keep an eye out for any updates.
I have seen stability issues with D435 on various platforms. However, nothing exactly which matches the behavior you mention. I am not sure if this will really help you, but I would have tried following
(1) Cross check all Linux Kernel patches are in place.
(2) run "rs-enumerate-devices"
- This will give you a device name. What is the name associated with your device?
- What is the firmware version?
(3) run "ls /dev/vid*", before and after you plug-in your device.
- For D435, if everything is ok, you will see 3 new device nodes being created
- 2 device nodes are for stereo IR cameras, and 3rd is for RGB device
(4) I have seen an issue with D435 where at some times it is not enumerated properly. But if you remove & replug it, it may work. For me, at times it require process be repeated 2-3 times.
(5) My D435 is more stable on PC, and less stable on embedded board I use. This could be because of either Power issue, or may be some kind of noise on USB lines. I am not fully sure.
- But connecting D435 via external USB hub seem to have helped
- Though I have not yet tried it, but I think using a shorter USB cable will help too. The cable which came with the D435 is rather too long.
Unfortunately none of these steps added much information. I've checked the kernel patches several times, and other RealSense devices are working ok.
Output of rs-enumerate-devices gives same info as before (notably, only the board name 430 rather than the full device name 435). Firmware version as below:
Name : Intel RealSense 430
Serial Number : 746612070042
Firmware Version : 05.08.15.00
Physical Port : /sys/devices/pci0000:00/0000:00:14.0/usb3/3-1/3-1:1.0/video4linux/video0
Debug Op Code : 15
Advanced Mode : YES
Product Id : 0AD4
After connecting the D435 there are only ever two device nodes in use (dev/video0 and dev/video1) which are presumably the IR nodes
Obviously I've disconnected and reconnected the hardware several times now, and rebooted, nothing changes. I did try an external hub just on the off-chance it had a more stable connection, but no luck.
I will see if i can find a shorter connecting cable, and maybe try it on another CPU, to see if it is a power issue with the USB3 input ports of the NUC.
I am not sure what should be the next debug step for you. Since you mention that only 2 device nodes are getting created when you plug in the device, so RGB sensor may be really missing in the signal chain. I think there are several possibilities
(1) You indeed have a device without RGB Sensor. On the D435, you can see two stereo IR sensors [these are identical and easy to spot], one IR projector [this is in middle of two IR sensors], and a RGB sensor [which is the right most]. Do you see all these sensors physically available?
(2) Next possibility is that RGB sensor is physically present but it is non functional. I am not sure what can be the reason, but may be some loose connection inside the casing, or not enough power? To debug any such problem, I think only Intel can help.
(3) Something specifically incompatible with the Hardware where you are connecting the camera, thereby not identifying the RGB sensor. However, if you have cross checked all patches etc, then this is less likely to be case.
I am not sure what you could try next, but it may be worth trying to see how camera works on any other system.
(1 If you have access to Win 10 PC, then you should probably see how realsense-viewer is working.
(2) Likewise if you can get access to an Ubuntu PC, just check if realsense-viewer still detects only Stereo camera.
I can understand the tricky situation you are in. I hope that some one from Intel will be able to help you out
update: Intel support have told me that the D435 doesn't have RGB.
To me this sounds like nonsense? I can clearly see an RGB camera/sensor on the front of the camera, and from the forums it seems that other people seem to have received colour streams from their D435. The specifications page clearly lists D4 vision processor with RGB resolution and FoV.
Main Intel® RealSense™ component
Intel® RealSense™ Vision Processor D4
Intel® RealSense™ module D430
Depth Field of View (FOV)—(Horizontal × Vertical × Diagonal) 91.2 x 65.5 x 100.6 (+/- 3°) Depth Stream Output Resolution Up to 1280 x 720 Depth Stream Output Frame Rate Up to 90 fps Minimum Depth Distance (Min-Z) 0.2m Sensor Shutter Type Global shutter Maximum Range Approx.10 meters; Varies depending on calibration, scene, and lighting condition RGB Sensor Resolution and Frame Rate 1920 x 1080 at 30 fps RGB Sensor FOV (Horizontal x Vertical x Diagonal) 69.4° x 42.5° x 77° (+/- 3°)
Can anyone shed some light on this? Hopefully it's a communication issue.
@PS: I'm using Ubuntu. I've tried plugging the realsense into both windows and OSX, none of them recognise an RGB camera. The RGB is visible externally, as noted. Hardware/power/loose connection is looking likely, unless Intel is right about D435 not having RGB at all.
That was my understanding!
Intel customer support have told me the following:
The D435 camera does not have an active RGB module. This information is stated on the following link:
Click on "Compare Depth Modules" to see a comparison table of features. It shows that the D415 is the only depth module with an RGB sensor.
I think they must not have understood my problem. Hopefully they will get back to me.
Odd choice for nomenclature and numbering; modules are only the imager boards and cameras are modules plus processor. The depth module, D430 module does not have an RGB camera; the D415 module has an integrated RGB camera. The D435 camera contains a D430 module AND a standalone RGB camera. The D4 processor board determines its mode by what is connected and it appears that there is something wrong with your RGB module. As someone else suggested, it might be that the flex cable is not firmly connected; perhaps the RGB camera itself is bad.
In any case the D4 thinks it is a D430 camera because your earlier information indicated a PID of 0x0ad4; a D435 camera (D430+RGB) has a PID 0x0B07.