depth image

[LMeng8]

I am a new learner and I bought a D435 RealSense. I want to know the depth image that saved from rs-save-to-disk.cpp is a depth image or a colored image？If it is a depth image, how to acquire the related depth data?

[MartyG]

The 'save-to-disk' sample is saving a depth image that has had color applied to it to better illustrate the depth information visually. So it is primarily a depth image.

In regard to retrieving depth data from a PNG file that has been created, I did not find much useful information during my research, except for the link below regarding converting a PNG frame into a point-cloud depth image.

http://my-it-notes.com/2014/09/how-to-convert-png-pair-of-rgb-and-depth-frames-into-pointcloud-library-pcd-format/ Convert png RGB and Depth frames into Pointcloud PCD format | Мои IT-заметки

[LMeng8]

Thanks for your reply!

So if I want to acquire an RGB scene image and the corresponding depth information, I need to get its point-cloud depth file. Instead of saving the color image and the depth image like the way in 'save-to-disk .cpp'?

Could I save the depth image which contains depth data by using OpenCV? I tried once but it seems not right.

Here is the code:

# include // Include RealSense Cross Platform API

# include "example.hpp" // Include short list of convenience functions for rendering

# include

using namespace std;

using namespace cv;

// Capture Example demonstrates how to

// capture depth and color video streams and render them to the screen

int main(int argc, char * argv[]) try

{

rs2::log_to_console(RS2_LOG_SEVERITY_ERROR);

// Create a simple OpenGL window for rendering:

window app(1280, 720, "RealSense Capture Example");

// Declare two textures on the GPU, one for color and one for depth

texture depth_image, color_image;

int cframe = 0;

// Declare depth colorizer for pretty visualization of depth data

rs2::colorizer color_map;

// Declare RealSense pipeline, encapsulating the actual device and sensors

rs2::pipeline pipe;

// Start streaming with default recommended configuration

pipe.start();

while(app) // Application still alive?

{

rs2::frameset data = pipe.wait_for_frames(); // Wait for next set of frames from the camera

rs2::frame depth = color_map(data.get_depth_frame()); // Find and colorize the depth data

rs2::frame color = data.get_color_frame(); // Find the color data

rs2::frame depth_frame = data.get_depth_frame();

Mat color_Im(Size(640, 480), CV_8UC3, (void*)color.get_data(), Mat::AUTO_STEP);

Mat depth_Im(Size(640, 480), CV_16UC1, (void*)depth_frame.get_data(), Mat::AUTO_STEP);

// For cameras that don't have the RGB sensor, we'll render infrared frames instead of color

if (!color)

color = data.get_infrared_frame();

// Render depth on to the first half of the screen and color on to the second

depth_image.render(depth, { 0, 0, app.width() / 2, app.height() });

color_image.render(color, { app.width() / 2, 0, app.width() / 2, app.height() });

//image write

stringstream ss;

string str;

ss << (cframe);

str = ss.str();

imwrite("C:\\Users\\mliu6\\Desktop\\test1\\captureColIm\\color" + str + ".png", color_Im);

imwrite("C:\\Users\\mliu6\\Desktop\\test1\\captureDepIm\\depth" + str + ".png", depth_Im * 15);

cframe++;

}

return EXIT_SUCCESS;

}

catch (const rs2::error & e)

{

std::cerr << "RealSense error calling " << e.get_failed_function() << "(" << e.get_failed_args() << "):\n " << e.what() << std::endl;

return EXIT_FAILURE;

}

catch (const std::exception& e)

{

std::cerr << e.what() << std::endl;

return EXIT_FAILURE;

}

Here are the images I saved

[MartyG]

There is certainly the option of saving the camera stream as a 3D point cloud in .ply file format instead of an image file. That .ply file can then be imported into other software packages such as MeshLab, OpenCV, MATLAB or Blender for further editing work.

In regard to your supplied depth image, there does seem to be some very significant disruption. Fluorescent lighting (like the ceiling strip-lights in your image) has been shown to have a negative effect on RealSense depth scans because fluorescent lights flicker at a rate that cannot be seen easily by the human eye and can create noise in the depth image.

There was a user who reduced the disruption by running their stream at 60 FPS to more closely match the frequency of their 60hz fluorescent lights.

[LMeng8]

Thanks for the solution! There is one more question: why the depth image need be colorized? (rs2::frame depth = color_map(data.get_depth_frame()); in 'rs-capture.cpp')

When I delete the color_map function(rs2::frame depth = data.get_depth_frame();), why does D435 not work anymore?

[MartyG]

There are numerous more references to color in the capture.cpp script than the part you mentioned. The easiest way to remove rendering of the color view may be to let the script caculate color but remove line 38 of the script, which renders the color data on the screen.

color_image.render(color, { app.width() / 2, 0, app.width() / 2, app.height() });

If the depth stream then only shows on half of the screen space, I would also edit line 37 above it to display the depth in the whole of the screen, as it is currently set to divide the screen width into two halves for depth and color. So I would change:

depth_image.render(depth, { 0, 0, app.width() / 2, app.height() });

to this:

depth_image.render(depth, { 0, 0, app.width(), app.height() });

[LMeng8]

Maybe I was telling in the wrong way, I do not want to remove the view of color images, I want to figure out why does the depth image has to be colorized.

I used OpenCV to save the RGB image and the corresponding depth image with SR300 and it worked.

But when I use D435, the depth image seems has to be colorized, otherwise the sensor will not work. And I can not save the depth image which I can use Matlab to acquire its depth data directly.

Here are the images I saved by using SR300:

[MartyG]

The discussion on GitHub in the link below explains why the colorizer is used.

https://github.com/IntelRealSense/librealsense/issues/1223 Some question about rs2::colorizer ? · Issue # 1223 · IntelRealSense/librealsense · GitHub

In the RealSense Viewer, you can obtain a greyscale depth image if you set the Color Scheme option's drop-down menu in the 'Depth Visualization' section to 'White to Black' or the 'Black to White' option.

If you need to make the change with scripting, the link below gives a sample script for setting the 'White to Black' colorizer mode.

https://github.com/IntelRealSense/librealsense/issues/1831 Export depth image in grey scale · Issue # 1831 · IntelRealSense/librealsense · GitHub