Realsense SR300 - Hand Tracking starts and immediately stops

[CMedi6]

Hey,

I'm trying to use the Hand Tracking demo on the following machine:

Intel Core I7 6800k

32GB DDR4 Ram

Quadro 5000

Windows 10.1

All other demos work fine except for ones that involve hand tracking. Ive tried it on another machine 3960x and it instantly crashed when detecting a hand, and on this new machine it instantly starts then stops. Ive installed all the drivers etc required, its simply refusing to work. Its in a USB 3.0 port, ive tried about 4 of them now.

Any help would be hugely appreciated

Regards,

[MartyG]

The minimum processor specifications for RealSense are 4th generation Haswell architecture for the F200 camera and sixth generation Skylake for the SR-300). If a processor does not meet the spec then it only usable for very basic RealSense functions such as raw data streaming and RGB video, but not more complex operations such as tracking.

Your 6800k processor is an Intel Core launched in February 2016. You would expect this to mean that it is a 6th generation Skylake, but it is actually 5th generation Broadwell-E architecture. So whilst it would work perfectly with an older F200 camera, an SR-300 would have problems with it as it needs Skylake as a minimum. Sadly, it seems that you have been caught in the changeover between chip generations.

The 3960X is a late 2011 2nd Generation Sandy Bridge architecture processor, which is too old to work with any RealSense camera except for the previously mentioned simple functions.

.

[idata]

Hi ConstructiveMedia,

Thank you so much for contacting us. I'm not sure if the processor's architecture is causing the issue. So, we will like to try a little experiment. Could you use the sdk_info.exe to record the logs using an SDK sample? In order to do it after you install the SDK, go to "C:\Program Files (x86)\Intel\RSSDK\bin\x64\", run SDK_Info.exe as an Administrator, navigate to the Logging Tab and Click "Enable All Logs" button. Reproduce the issue by running any example that includes hand tracking (make sure not to close sdk_info, it can be minimized, closing sdk_info will stop the logging). After the failure has occurred return to SDK_Info and click "Copy all Logs" button (in sdk_info logging tab), zip the created directory and attach it to your next reply. We will be waiting for your reply. Regards, -Leonardo

[TMorr2]

Hey,

I've attached the files following your instructions. The weird thing is, the eye tracking demo etc works fine, its literally just the hands! (The one function we need combined with background removal). We have several systems in office, sadly all are either X99 LGA2011-3 or LGA2011. It was an R&D investment for us to get realsense, I don't think we can swing an entire new system ontop of it too!

Hopefully the logs give some answers.

[MartyG]

I did some research into the difference between Broadwell-E and Skylake architecture. Because Broadwell-E was released around the same time as Skylake, there is apparently architecturally little difference between them, apart from how they prioritize processing in the chip's cores. In some applications, particularly ones involving multi-threading, Broadwell-E does better than Skylake because it has 10 cores, whereas Skylake has 4. So this creates some confusion, because Broadwell-E is technically 5th generation and so "older" than Skylake, but they are actually more or less born at the same time, like twins where one is born 10 minutes after the first!

The existence of the two architectures at the same time seems to come about as a result of Skylake being aimed at the consumer market, whilst Broadwell-E is aimed at enterprise. Broadwell-E has some serious power in it on the 'monster' level, so much so that Intel have described it as "megatasking" rather than multitasking.

So whilst generation numbers are usually a good guide to whether a processor will work well with RealSense or not, that means of measurement is not so useful in this case. So Leonardo has a good point that it may not be the architecture that is the problem, especially since everything else runs fine except hand tracking.

Edit: looking at your attached logs, I see the computer name is Renderfarm 28. That explains a lot about why you need the grunt of a chip like Broadwell-E.

[MartyG]

Ok, here's my own thoughts on the logs.

* As hugely lengthy as they are, only the handful of top entries need to be focused on, as the ones below that just list successful events such as the installing of the RealSense SDK.

* It looks like immediately after the SDK completes installation, the first program that generates a red error is 3DS Max. If your setup is a graphics render farm, I understand why you'd have 3DS Max on your system.

* Shortly after that, it disconnects from the VNC Server due to a time-out. I guess this means you are using a Virtual Machine setup?

* It scans for updates and fails.

* It tries to talk to the VNC Server again, and again times-out.

* 'Source MvWNI' errors. This error is related to not being able to find something related to the Nvidia graphics, such as a driver or GPU.

http://nvidia.custhelp.com/app/answers/detail/a_id/4103/~/what-causes-error:-source---nvwmi-%7C-level---warning-%7C-instance-and-method What causes error: Source - NVWMI | Level - Warning | Instance and Method provider registration failed - (null) hr=0x…

* The computer runs the Intel update utility

* The error 'Event ID 2 from source ESRV_SVC_QUEENCREEK' appears. The Queencreek error is apparently related to the Intel update utility and the Intel Energy Checker. It is most likely to be the update utility in this case, since that application immediately preceded the error. there are 6 yellow warning errors about Queencreek, indicating that the system was repeatedly pinging the update utility and having no luck.

* It pings the Nvidia graphics again and still cannot find what it is looking for, and then states again that it cannot connect to the VNC.

* It tries again a number of times to talk to the Intel update utility and fails.

* It has another conflict with 3DS Max.

* Scan for updates fails again.

* The Software Protection Service starts up and does a licence check. It then states that licence activation failed (possibly related to not being able to connect to the VNC). The Software Protection service tries to start again and stops. It starts again and stops, seemingly in a loop of start-check-fail, start-check-fail.

* Update check fails again. It then checks for the VNC and Nvidia and fails again.

* Finally it gets to looking at RealSense. It seems to be looking at the SR300's DCM but cannot find what it is looking for. Then the log ends.

***********

Overall, it is somewhat of a mystery, considering the other RealSense samples run fine. Looking at the log, one might think that hand tracking does not like the VNC setup.

A question comes to mind that I would have asked from the start if not mislead by the processor architecture issue. Are you using RealSense SDK R3? That new SDK has a significantly different structure from the previous SDK versions, in that it is modular instead of all the components being in a single install package. Hand tracking is one of those optional components that is not in the main 'Essentials' core module and has to be dowloaded and installed separately once the Essentials module has been installed.

[CMedi6]

Thankyou for the extremely detailed answer, you're bang on with the software running, it's a node in our farm (about 40 when they're all up and running) 28 was our test machine when swapping out to the 6800K. It was next to my desk running VNC so I didn't need to hook up any peripherals bar the camera.

3DS Max and VNC were both running (part of a farm, no one was rendering at the time so max shouldn't have even had an instance running, so im not sure why it was). When i'm in the office tomorrow i'll run without VNC running and all related services stopped just in case. All our machines are running VNC so it may explain an awful lot.

The graphics driver wan'st installed on the machine, as it's LGA2011 theirs no on board graphics so i just slapped a spare Quadro we had laying around in there for monitor use whilst initially setting up the machine, I just forgot to remove it!

With regards to the SDK R3 i'm definitely running it, I installed the 2016R3 essentials and then went on to install the additional packages to enable the functionality. It's a relief to hear it may not be a architecture issue. The deadline for the Proof of concept was monday so I may be working some OT at this rate!

[MartyG]

If time becomes critical then I'd recommend rolling back to SDK R2, which installs as a single package instead of being modular. It's much more of a known quantity and rock-solid, whereas R3 is on the bleeding edge of things and hasn't had all the kinks smoothed out yet (it isn't working with the Unity game engine yet, for example). At present, there is no clear advantage to using R3 over R2 other than the smaller install footprint of the modular setup. You would still have the option of upgrading your project to R3 once the proof of concept is approved and you have more time to work on it.

Using quick and dirty shortcuts held together with digital duct-tape is common in proof of concept pitches - it doesn't have to represent the quality of the final product, just a visualization of what you are aiming to achieve.

There isn't much information on running RealSense over a VNC. A veteran developer called Samontab who is deeply skilled in the more arcane elements of vision technology was of the opinion that some VNC software may have trouble with RealSense's requirement of sending camera data over a USB 3.0 connection. Apparently the solution may be as simple as enabling USB 3.0 support in the VNC software's settings. Here's how someone configured USB 3 on VMWare.

https://datanpixels.wordpress.com/2015/03/05/intel-realsense-sdk-on-macbook-pro/ Intel realsense SDK on Macbook pro – Shayne Wang

And another person's guide to doing so on Parallels.

http://kb.parallels.com/en/115008 KB Parallels: USB 3.0 support for Virtual Machines starting from Parallels Desktop 8.0.18305

Edit: the 'USB 3.0' explanation does not quite make sense as a cause of malfunction, given that the other sample programs worked fine. But the info's there if you want it anyway!

If you need further help, please feel free to keep commenting here and we'll try to work through the issues together. Best of luck!

[CMedi6]

Hey,

So, reverted back to R2 and all hell broke loose, no demos now work at all, even when updating to R3 again. However something interesting happened, it seems the Quadro I had in there doesn't support open CL1.2 (Only up to CL1.1) so i've swapped it out for a spare 970 and it's still being reconfigured however the device is failing due to a malfunction, exact same system (except the GPU). I'm going to try R3 again and see if it fixes atleast that issue. I've uploaded some more logs!

[MartyG]

I'm sorry about your trouble after reverting. Did you uninstall the DCM as well as the SDK? Recommended procedure would be to uninstall both the DCM and SDK, then re-install the DCM first and the SDK second.

The log looks like similar circumstances to the first. The VNC Server reports a drop and then the RealSense DCM can't be found.

[CMedi6]

Hey,

I'm 99% sure I removed it all then did a clean install and VNC was closed (I guess some services must have been runing in the back). But i'll try again on monday.

I found another machine (a dell AOI supporting a http://ark.intel.com/products/80808 Intel® Core™ i7-4790S Processor (8M Cache, up to 4.00 GHz) Specifications and the bloody thing worked on SDK 2016R2 every single demo worked. VNC was running on that machine too. It's not ideal to develop on but its proof the camera atleast works. I've taken the camera home to try on my own rigs (3930K with 1080) I'm wondering if its a powered USB Issue now we're gettig a complete failure?

Regards,

Tom

[MartyG]

It's quite a common occurrence that the camera may work fine on one machine but not another, even if they have similar specs. It is thought to be related to either the amount of power the USB port is providing to devices (it can vary between machines, hence the favorable results achieved with powered hubs) or the particular USB chipset used on that machine.

You can get cheap USB 3.0 PCI Express expansion boards for desktop computers that only cost about $5-10, for fellows such as yourself who don't mind getting in the case and pushing and pulling boards.

They typically come with a set of pins on the board that you can connect a 5 volt wire from the PC's internal PSU to in order to provide extra juice to the USB port on the board when the camera is performing high-drain activities such as depth sensing that the USB ports that came with the machine may not be able to cope with. I installed one of these boards in one of my own PCs when I had camera trouble when RealSense first came out and the camera worked immediately afterwards.

Edit: there's a "sticky" about USB drivers at the top of this forum on the front page.

[idata]

Hi ConstructiveMedia,

Do you have any updates about this?Regards,-Leonardo

[AStre]

We do have a very similar problem running the Intel RealSense SR300 on a Mac Pro (Late 2013, using a Intel Xeon e5-1620v2) using Bootcamp with Windows 10.1.

When we start any of the sample applications using the hand cursor module, the application crashes as soon as a hand is moving into the field of view of the camera. I have attached a zip file with the logs captured as described above. We could also provide a crash dump if that would help, it is just too large (150MB) to be attached to the post.

Any help would be appreciated.

Thanks!

Achim

[MartyG]

Your Intel Xeon e5-1620v2 2013 processor is Ivy Bridge architecture (3rd generation). The official minimum spec for full operation of RealSense is 4th generation Haswell architecture (F200 and R200 cameras) and 6th generation Skylake (SR-300 camera). If a camera is used on a machine with a non-compatible processor then it may be able to be used for simple functions such as raw camera data streaming and RGB video processing but not access the camera's more advanced functions such as hand and face tracking.

[LBill]

Does this run on Ubuntu 14.04? How about Ubuntu 16.04?

[idata]

Hi Pototo,

We noticed that you have opened a thread asking about this:https://communities.intel.com/thread/112013 https://communities.intel.com/thread/112013.And the information provided there is true, for Linux developing you'll have to use Librealsense https://github.com/IntelRealSense/librealsense, it has a lot of features and there's a lot of information around the web that you'll find useful.Have a nice day.Regards,Leonardo R.