Items with no label
3335 Discussions

SR300 C# Visual Studio depth stream

NMant1
Beginner
1,985 Views

Hi,

I'm having trouble implementing a simple program, which should record a raw depth stream and render it to a window, preferably saving images in an editable format. The RawStream sample included in the Sample Browser is what I'm looking for, but I'm having a hard time to find the right pieces of code in this project. I tried copying the parts I need into a new project but was unable to get it to work without errors. Tutorials on the internet covering this subject seem to be for R200, and don't give an image at all. Can someone point me to a simple tutorial, or give me an idea on how to do this?

Thanks,

Nahuel

P.S.

The RawStream Sample offered by Intel uses:

RS.SenseManager sm = RS.SenseManager.CreateInstance();

to create a sensemanager, while many tutorials use

private PXCMSenseManager senseManager;

to make one. Does this have to do with which RealSense device I'm using?

0 Kudos
11 Replies
MartyG
Honored Contributor III
500 Views

Have you seen this page of the documentation, which provides scripts for each aspect of raw stream processing?

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?doc_essential_raw_stream_capture_and_process.html Intel® RealSense™ SDK 2016 R2 Documentation

When looking at tutorial scripts on the internet, you should also be careful that you are selecting a script for the correct SDK. There are scripts for the official RealSense SDK (e.g 2016 R2 and 2016 R3), and there is also the open-source Librealsense SDK. An easy way to tell if a script is for Librealsense is to do a Find search on the browser page for the word 'librealsense'. If a script has this word in it then it is probably a Librealsense script and so should not be used with the official RealSense SDKs.

NMant1
Beginner
500 Views

Thanks a lot! I don't know how I missed that page. With this page, which uses the PXCMSenseManager, I managed to get a depth image with very low depth. As in, the depth was either black, grey or white. The Intel.RealSense.SenseManager, which I think uses the RealSense library, used in the RawStreams example does give a much better gradient. I attached the two images to illustrate the difference.

Is this problem indeed caused by using the PXCMSenseManager? How do I solve this?

Thanks, Nahuel

0 Kudos
MartyG
Honored Contributor III
500 Views

I would usually associate a very dark / black image with too much or too little lighting in the environment that the image was taken in. But as you were able to also take a bright image with Rawstreams, presumably in the same room at around the same time of day, that would cast doubt on the lighting level being the cause. I think I will need to refer this question to our resident stream programming expert jb455 - hopefully JB can offer useful advice when he sees this message. Good luck!

0 Kudos
NMant1
Beginner
500 Views

Thanks for your help! I've found out the problem lies somewhere else: I save the data to a WriteableBitmap, which I then use as the source of the PictureBox. The RawStreams sample renders the image. While I don't know the details of either method, I'm pretty certain this is the problem after some testing. Do you have advice on how to get the data in an individual pixel editable format?

Thanks again, you've been of great help.

0 Kudos
MartyG
Honored Contributor III
500 Views

I'm glad you were able to make progress!

Advanced stream programming is sadly outside of my field of experience, hence why I linked rb455 (our stream coding expert) into this discussion. In the meantime, you may find some useful information in a recent discussion on this forum about converting recorded streams in the .RSSDK format to image formats.

0 Kudos
jb455
Valued Contributor II
500 Views

The https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?tobitmap_imagedata_pxcmimage.html ToBitmap method, as the name suggests, converts the image data into a bitmap which you can then render using your picturebox's paint event. It's not the most efficient though, especially with higher-resolution images, so if you're worried about performance you may be better off going the WriteableBitmap route in a WPF app instead of WinForms. Alternatively, you could use LibSampleDX, which the SDK samples use for rendering - going through the RawStreams sample will show you how to use it.

0 Kudos
NMant1
Beginner
500 Views

Thanks, I'm using WriteableBitmap now. While the images still come out flat, I think I know the cause now. The renderer in RawStreams seems to normalize the brightness between white and black, while the bitmap is a fixed value for a fixed distance.

Is the mapping between the brightness of a pixel and distance to the camera specified?

0 Kudos
jb455
Valued Contributor II
500 Views

Ah right, your screenshot in the first post was so dark I thought that it wasn't rendering at all but I can see a faint silhouette now.

I do remember seeing a snippet somewhere which coloured the depth image manually, but I'm struggling to find it again now - I think it was in one of the samples. IIRC, the gist of it was to work out the min & max depth values then scale that to the range 0-255 and colour each pixel using its depth and that scale (so a pixel at max distance will be (0,0,0) and the closest will be (255,255,255)). You'd probably have to use pointers to make the bitmap pixel operations efficient enough to run at an acceptable framerate..

0 Kudos
NMant1
Beginner
500 Views

The framerate won't be the problem. I probably need to do operations on no more than 10 images. I would like to use the real (non-normalized) depth in the images. I read in the specifications that the SR300 has a range of 20 – 120cm. Does this mean 20cm is (0,0,0) and 12cm is (255,255,255)?

0 Kudos
jb455
Valued Contributor II
500 Views

Yes, something like that would work - or once you have all the depth values you can find the Max and Min and use those?

To be honest, I have a feeling I'm sending you down the wrong path. This would certainly work, but I feel like it should be much easier than this! I've only ever dealt with rendering the colour images so I don't know exactly how it all works, but this seems like something the SDK should do for you. Have you searched through the forum & the docs?

0 Kudos
NMant1
Beginner
500 Views

This worked out well. I did some measurements, and now have a fine mapping between distance and bit value:

Thank you for your help so far. I've found that the average of the depth result is quite good, but there's, understandably, quite some deviation between samples. Do you know of a built-in solution to average over 10 samples? Right now I'm trying to convert them to WriteableBitmaps first, but this seems inefficient.

Thanks,

Nahuel

0 Kudos
Reply