Realsense R200 RGBD camera not working well with Nvidia Jetson TX2

[RLee36]

Hi there!

I am using R200 on one of my projects, running on Nvidia Jetson TX2, Ubuntu 16.04 via ROS Kinetic.

I have successfully download, installed the necessary drivers and have been updated to the latest stable releases. My issue is that after one successful run of the R200, Ctrl C gives a SIGTERM error. And unless I physically unplug and replug the USB, I am unable to rerun the R200.

Conversations with the Nvidia Jetson guys have led to nowhere and I'm here hoping someone could point me in the right direction to getting it fixed.

[MartyG]

SIGTERM is a signal sent by Ubuntu to shut down active processes. The circumstances of your error make me think that the camera may be 'going to sleep' after the first run and making the camera undetectable until the USB is unplugged and re-plugged. This issue has occurred in the past with using RealSense cameras for logging into Windows with the face using a program called Windows Hello.

The first thing I would advise trying is running an update, even if you have just freshly installed Ubuntu. Instructions on doing so can be found here:

https://github.com/IntelRealSense/librealsense/blob/master/doc/installation.md librealsense/installation.md at master · IntelRealSense/librealsense · GitHub

[RLee36]

Hey MartyG!

Thanks for the prompt reply! Wished I had checked back earlier. Would you recommended doing the steps you linked before or after http://www.jetsonhacks.com/2017/08/14/intel-realsense-camera-librealsense-nvidia-jetson-tx-dev-kits/ JetsonHack's 3 step process for the Intel RealSense R200, or dont follow JetsonHack at all.

[MartyG]

Since the Librealsense installation script provided with the JetsonHack does automated setup of permissions for using the camera with the Jetson, that would seem to be the easiest route to take. And since the article was only published in August 2017, the information in it is probably pretty current.

I would run the JetsonHack script to install Librealsense and the Jetson permissions, and then do an update patch instruction after that to ensure that Ubuntu and the newly installed Librealsense have the latest files.

[RLee36]

Am i right to understand that the JetsonHack's scripts are all I'd need to run? And to disregard the link you've sent?

I have done as suggested by the JetsonHacks and have bumped up to the original problem

[MartyG]

I haven't got direct experience of using a Jetson, so I can only make educated guesses. Looking at the Jetson information and the link to the installation guide that I provided earlier, I would revise my advice to the following steps:

1. Run the Ubuntu patch update first, as described in step one of the guide I linked to.

sudo apt-get update && sudo apt-get upgrade && sudo apt-get dist-upgrade

sudo update-grub && sudo reboot

2. Skip the rest of the installation guide (which covers the usual installation method for Librealsense) and instead use the JetsonHack's script to install Librealsense and the permissions for Jetson.

[RLee36]

Awesome! I'll do just that! Is the http://www.jetsonhacks.com/2017/03/29/intel-realsense-package-for-ros-on-nvidia-jetson-tx2/ RealSenseROSTX2 needed?

Also, could you ask internally on troubleshooting this issue should it not work. Thanks alot!

By the way, I do not have a "sudo update-grub" it's a custom version of ubuntu and some otherwise-available commands are not available

[MartyG]

I am not an Intel employee, so do not have access to internal Intel reference sources, unfortunately. The position stated in the past about Jetson inquiries has often been to ask the Jetson forum, since Jetson is not a product that Intel provides official tech support for. Unofficially, advice may be provided by a support team member if they have personal knowledge of the subject..

Regarding ROS, things seem to get a bit complicated at this point. My interpretation of the instructions is:

1. Install the ROS package after Librealsense has been installed.

2. The ROS package's guide says that "the ros-kinetic-librealsense package installs linux-headers in the /usr/src directory. These headers DO NOT match the Jetson TX2, so you should consider deleting them. Same with the uvcvideo realsense directory." The instructions for doing so are:

$ cd /usr/src

$ sudo rm -r linux-headers-4.4.0-70

$ sudo rm -r linux-headers-4.4.0-70-generic

$ sudo rm -r uvcvideo-1.1.1-3-realsense

3. Apparently, when the JetsonHacks script installs ros-kinetic,librealsense, you end up with two separate Librealsense installations on your Jetson. You can remove the original Librealsense installation if you wish (it seems to be optional) with:

$ cd librealsense

$ sudo make uninstall

[RLee36]

Oh! Thanks alot for the help!

I tried those commands and it still didnt work and that's why I am here. Dont understand why I am the only one who seem to have it work only ONCE. If we went back to your theory on having it sleeping, do you have other fixes? Updating everything didnt work.

[MartyG]

Are you running the stream at a high frames per second (FPS) value such as 60 FPS? With the R200 camera model, apps had a tendency to sometimes freeze up or crash if running at 60 FPS because the amount of data being transmitted through the USB cable overwhelmed the USB port. The fix was to run the stream at 30 FPS instead.

[RLee36]

hmmm... How could I reduce that?

[MartyG]

You can manually configure the resolution and FPS in a launch file. Lines 14 to 20 on the script in the ROS documentation page linked to below show how to do this,.

http://wiki.ros.org/realsense_camera/Tutorials/change_camera_parameters# Set_Camera_Mode_to_Manual realsense_camera/Tutorials/change_camera_parameters - ROS Wiki

[RLee36]

yea it's 30 by default

[MartyG]

Let's return to the possibility that the camera is going to sleep after its first successful run. In that case, we would want to try to prevent sleep mode from activating at all. The next thing I would suggest trying is turning off all power-saving functions in Ubuntu, setting their activation to 'never'.

https://askubuntu.com/questions/47311/how-do-i-disable-my-system-from-going-to-sleep/858617 power management - How do I disable my system from going to sleep? - Ask Ubuntu

[RLee36]

Hey sorry for the absence. Had some trouble with my TX2 cause i wrongly edited the kernel. Have it back to the original state as when I opened the ticket. I have uploaded a recording of the terminal where I observed the error. Hopefully it gives some clue

[RLee36]

https://asciinema.org/a/48tAYBorJcXb6XDjglEJV93n7

[MartyG]

Looking at the video frame by frame, on each attempt to launch, the shutdown of the processes is preceded by a warning message saying " /camera/driver - detected unvalidated firmware: 2521007705's current firmware is 1.0.71.06, validated camera firmware is 1.0.72.06.

2521007705 is the serial number of the R200 camera, according to a message directly above the error.

Someone else who had the same error was told "The warning you are seeing is from the realsense_camera ROS package, not librealsense. It is only a Warning message denoting that the version of FW on hour camera is different from the version used to validated that release of realsense_camera ROS package. This is used as an indicator to our QA team to update the version of FW used for validation. In general, the warning can be ignored; it is a warning, not an error."

It does indicate though that you don't have the newest version of the R200's firmware driver installed inside your camera (the firmware is stored in the camera, not the PC). As the error message suggested, the most recent available is 1.0.72.06. This newest firmware fixes an issue where "the stream is aborted after large number of Sleep / Hibernate cycles. "

It was considered earlier in this discussion that the connection might be failing because the camera was going to sleep.

If you want to update the firmware in your R200 to 1.0.72.06, you would have to plug the camera into a Windows PC and run the Depth Camera Manager (DCM) program. The DCM that installs the newer firmware in the camera can be downloaded with the 'Driver' link in the R200 section of the page linked to below.

https://software.intel.com/en-us/realsense-sdk-windows-eol Intel® RealSense™ SDK for Windows (Discontinued) | Intel® Software

[RLee36]

I have updated the firmware, but the error persists. Strangely I am on 1.0.72.10 now. I didnt really have anything to click using the DCM, Everything was automatically downloaded, installed and updated.

I have noticed that launches after the first launch failed to initialize Depth Control Preset.

New video: https://asciinema.org/a/eT7UHkWnnhdduTn3RPHoX4125

[MartyG]

The closing down procedure occurs after the program makes a call to the Dynamic Camera Options. Others have had this problem too, I found when I researched it. It may occur when the depth camera is shut down and then turned on again, which could account for why the camera works for you on the first run but not after.

The next step I would suggest is to try setting the depth control to be turned off by default. The instruction for that is:

r200_dc_preset (int, default: 1)

The current default you have is 5, which equates to a High setting: "Higher number of outliers removed. Minimal false positives". On an exotic setup like an Nvidia TX2 where results may be more unpredictable, this may be a less stable option than using a more balanced setting.

[RLee36]

Sorry, I dont follow. I couldnt find anywhere to change the preset.

[MartyG]

If it is setting the preset to 5 automatically, I would imagine that the instruction is in a batch file, perhaps the ROS launch file, and could be changed with a text editor. EDIT: confirmed, it is done in the launch file.