11 Replies Latest reply on Sep 6, 2018 4:11 PM by MartyG

How to find 3 or 5 center pixels from depth image

Hi everyone,

I am trying to find few center pixels from the depth image. I am trying to implement segmentation, I need to find distance of the segmented object.

This is how I was thinking, this will work. while the colored image is used for segmentation, the full frame depth image will be used to find the distance.

In my code below. I found the diagonal of the depth image, I am trying to find 3 or 5 pixels. I do not how to find the value from the array. I only have the index printed on my console.as print 40,35 but no value.

I hope i did not confuse you. Everything is new to me.

I have tried this;

void SampleArrived(object sender, SampleArrivedEventArgs args)

{

if (args.sample.Depth == null)

{

return;

}

else

{

var depthImageArray = args.sample.Depth;

Debug.Log("my depth value is" + depthImageArray);

var diagonal = Math.Sqrt((depthWidth*depthWidth)+ (depthHeight*depthHeight))/2;

texPlugin.UpdateTextureNative(args.sample.Depth, depthTex2DPtr);

for (int y =0; y < diagonal; ++y)

{

for (int x = 0; x < diagonal; ++x)

{

Debug.Log(string.Format("print {0}, {1}", x, y ));

var content = (string.Format("print {0}, {1}", x, y));

StreamWriter File = new StreamWriter(path)

File.WriteLine(content);

File.Close();

}

}

Debug.Log("diagonal pixels"  + StreamType.ToIndex[40,35];

}

• 1. Re: How to find 3 or 5 center pixels from depth image

The link below provides code for getting the depth of a pixel at a specific XY coordinate on the image (e.g 100, 100).

• 2. Re: How to find 3 or 5 center pixels from depth image

Hi MartyG,

The syntax does not look familiar to Intel RealSense R3000 SDK. I really do not understand how she is getting her raw depth value

• 3. Re: How to find 3 or 5 center pixels from depth image

I apologize, I did not see that you had tagged your entry as SR300 and 2016 R3 SDK.  I do not read the tags at the bottom of posts.

SR300 is compatible with the current RealSense SDK 2.0.  Would it be possible for your project to use it instead of R3?

If not: it has been a while since I used R2 and R3, but I believe the appropriate instruction for mapping 3D depth coordinates to 2D image coordinates is ProjectCameraToDepth.

• 4. Re: How to find 3 or 5 center pixels from depth image

That is no problem I did not tag the question properly myself.

The documentation is not clear on how the syntax should be. Like with a sample code showing how it is done.

I will play around with it and see what it will be returning.

• 5. Re: How to find 3 or 5 center pixels from depth image

I located a previous example where ProjectCameraToDepth was discussed on this forum.

Re: I want to get the real x, y, and z coordinates of the object.

On the link below, which has example scripts, example 6 shows what you were asking for - mapping depth to a few pixels.

• 6. Re: How to find 3 or 5 center pixels from depth image

Hi MartyG,

Please how can I use R2 since R3 is compatible with R2?

• 7. Re: How to find 3 or 5 center pixels from depth image

Yes, they are closely compatible.  The main difference is that in '2016 R2' you download the entire SDK as a 1.8 gb download, whilst R3 was split into a series of optional modules.  You can launch a download of R2 directly in your browser by using the link below.

• 9. Re: How to find 3 or 5 center pixels from depth image

To use Visual Studio with R2 or R3, you should build a 'development environment'.  To do this, please choose a link below depending on whether you plan to use C++ or C# in Visual Studio.

C++

With C++ you can set the dev environment up with a Proert Sheet  OR do the settings directly.

Using a Proerty Sheet

Intel® RealSense™ SDK 2016 R2 Documentation

Setting the project settings directly

C#

Intel® RealSense™ SDK 2016 R2 Documentation

• 10. Re: How to find 3 or 5 center pixels from depth image

Hi MartG,

I tried to setup R2 but I realized i do not have UnityCSharp.unitypackage like this in the directory " \$(RSSDK_DIR)/framework/Unity/UnityCSharp.unitypackage" so I went ahead with the working sample codes others got working.

Please have a look at my codes. I am trying to get the distance of the depth image. But anytime I run unity it crashes

void OnSampleArrived(object sender, SampleArrivedEventArgs args)

{

Image image = args.sample.Depth;

if (image == null)

{

Debug.Log("Null depthImage");

}

else

{

if (image != null)

{

ImageInfo cinfo = new ImageInfo() { width = colorWidth, height = colorHeight, format = PixelFormat.PIXEL_FORMAT_BGR }

ImageData outBuffer;

var imageHeight = image.Info.height;

var imageWidth = image.Info.width;

float[] pixel = outBuffer.ToFloatArray(0, imageWidth * imageHeight);

image.ReleaseAccess(outBuffer);

Projection projection = device.CreateProjection();

PointF32[] uvmap = new PointF32[cinfo.height * cinfo.width];

projection.QueryUVMap(image, uvmap);

var mappedPixels = new float[cinfo.height * cinfo.width];

for (int i = 0; i < uvmap.Length; i++)

{

int u = (int)(uvmap[i].x * imageWidth);

int v = (int)(uvmap[i].y * imageHeight);

if (u >= 0 && v >= 0 && u + v * imageWidth < pixel.Length)

{

mappedPixels[i] = pixel[u + v * imageWidth];

Debug.Log("distance is:" + mappedPixels);

}

}

for (int y = 0; y < depthHeight; ++y)

{

for (int x = 0; x < depthWidth; ++x)

{

Debug.Log(string.Format("print {0}, {1}", x, y));

int pixelIndex = (y * depthWidth) + x;

Debug.Log(pixelIndex);

}

}

// Clean up

projection.Dispose();

image.ReleaseAccess(outBuffer);

}

• 11. Re: How to find 3 or 5 center pixels from depth image

I've never had to use the Unity_CSharp file with R2.  What I have always done is create a new Unity project and then when the new scene is open at the start of the project, run the UnityToolkit file in the RSSDK framework folder with a double-leftclick on it whilst Unity is open.  This imports all the RealSense files and tools into the project.

Although I do develop extensively in Unity, when I was using R2 I always used the Unity Toolkit's tools (where you drag and drop pre-defined scripts into objects and configure menus in the Inspector panel) to create RealSense controls rather than writing my own RealSense scripts from new (I mostly edited the Toolkit scripts to my needs).  So I would be of no help in debugging your script, unfortunately.

If you are interested in using the RealSense tools that the Toolkit installs in Unity, I wrote an extensive range of very detailed step by step guides in the past.  An idnex of them can be found at the link below.

Index of Marty G's RealSense Unity How-To Guides