- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
How to convert depth frame to point cloud using Matlab? I know one way is if we know the camera parameters,we can obtain point cloud .But what are the parameters ?or Is there any other way to do it in Matlab?
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
If you are using Windows, in the most recent 2.16.0 version of the RealSense SDK, a MATLAB wrapper was introduced. This contains point cloud support..
https://github.com/IntelRealSense/librealsense/blob/master/wrappers/matlab/pointcloud.m librealsense/pointcloud.m at master · IntelRealSense/librealsense · GitHub
You can find the front page of the wrapper here:
https://github.com/IntelRealSense/librealsense/tree/master/wrappers/matlab librealsense/wrappers/matlab at master · IntelRealSense/librealsense · GitHub
If you are not using Windows, the discussion below looks at the process from the opposite direction (converting a point cloud to depth) and may provide some useful insights.
https://github.com/IntelRealSense/librealsense/issues/2204 Convert pointcloud to Depth image (3d to 2d) · Issue # 2204 · IntelRealSense/librealsense · GitHub
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,MartyG,
I triesd the above ways,but I'm so sorry that I still don't know how to convert depth frame to point cloud ,the 2.16.0 version does not appear a program about converting depth frame into point clouds.
Best Regards!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have not used the MATLAB wrapper myself. If it works like the SDK's other wrappers though then it would give access via MATLAB to functions in the core SDK such as point cloud creation. The problem tends to be that until example programs or script snippets for a particular function are published for a wrapper then it is hard to know how to write a script for the wrapper that produces the desired result. Such samples emerge over time, whether published by Intel or a member of the community. I understand that this does not help you today though.
I did some further research. On the page linked to below, there is a MATLAB function that can be downloaded that converts a depth image to a point cloud in MATLAB. Scroll down the page to the heading titled 'Depth Image to Point Cloud (MATLAB)' and click on the 'depthToCloud.m' link to launch its download in your browser.
https://rgbd-dataset.cs.washington.edu/software.html RGB-D (Kinect) Object Dataset
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi, Marty
Fortunately, the approach you recommend is feasible, but as I said earlier, converting a depth image to a point cloud requires some camera parameters, such as:
focal length:
I saw two focal lengths in the camera data,I don't know which one I choose,
the other parameter :The physical dimensions of each pixel on the x and y axes of the image plane。That is the red part of the formula.
For clarity, here is the conversion formula:
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Why must you use Matlab? C++ code is here and like this:
/librealsense-2.16.0/examples/pointcloud/rs-pointcloud.cpp
// Declare pointcloud object, for calculating pointclouds and texture mappings rs2::pointcloud pc; // We want the points object to be persistent so we can display the last cloud when a frame drops rs2::points points; // Declare RealSense pipeline, encapsulating the actual device and sensors rs2::pipeline pipe; // Start streaming with default recommended configuration pipe.start(); auto frames = pipe.wait_for_frames(); auto depth = frames.get_depth_frame(); // Ge...- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The D415's focal length is 1.88 mm and the D435's focal length is 1.93 mm.
The Dx / DY question is a harder one. It has been said that it is dependent on sensor size. The rolling shutter of the D415 has a pixel size of 1.4um x 1.4 um and the global shutter of the D435 has a sensor size of 3um x 3 um.
RealSense expert Samontab once gave the following explanation of pixel size:
https://software.intel.com/en-us/forums/realsense/topic/671453# comment-1879445 R200 pixel size
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you very much! This is very useful to me. Because I have been working on matlab, and I am not very familiar with c++, so I want to use matlab. I'll try to use c++.
Thanks again!
Bset reagrds!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,I have run the above program and researched ahout it ,but it only sample one point cloud .Unfortunately,that's not what I'm trying to accomplish!
Thank you!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Nevyn,
A .ply represents a pointcloud format that contains the information from only one frame.
If this is not what you are trying to accomplish, could you please give us more details in order to help you?
Thank you best regards,
Eliza
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Eliza,
I want to get color image and corresponding point cloud video sequence, but the only way is converting the depth image to point cloud.I know the transformation formual ,but don't know the camer paremeters.
Do you have other ways to finish it.
Thanke you,
Nevyn
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Nevyn,
Thank you for the clarifications!
In this case you are interested in knowing the Intrinsic Parameters, you can read about it on https://github.com/IntelRealSense/librealsense/wiki/Projection-in-RealSense-SDK-2.0# intrinsic-camera-parameters Intrinsic Camera Parameters 2. Deprojection.
You might also find https://github.com/IntelRealSense/librealsense/issues/1706 this GitHub thread helpful.
Please let us know if this information helps you with your project!
Thank you and best regards,
Eliza
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Eliza,
When I start the Intel RealSense Viewer use 'record' to recoed a sequence , I get the error message:
Unknown in rs2_create_record_device(device:0000020C63E7EB90, file:C):
Error opening file: C:\Users\\20180912_155849.bag
I don't know how to do it ,can you help me?
Thanks.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Nevyn,
This seems to be a permission error.
If you save a snapshot, do you received the same error?
Thank you,
Eliza
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Eliza,
I have solved the problem,but I still need to look at the previous questions about camera parameters.
Thank you,
Nevyn
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Eliza,
Fortunately, the information is useful, but I am not familiar with C program, do not know how to use the above files.
Please forgive me for always wanting to know the camera parameters in order to MATLAB code. Because I don't do program development,
I just do some research using it.
Thank you !
Best Regards,
Nevyn
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Nevyn,
You can check out https://github.com/IntelRealSense/librealsense/blob/master/wrappers/matlab/video_stream_profile.m Video Stream Profile Matlab example, as it contains a function that gets the intrinsic parameters from the camera.
I hope this information helps you!
Thank you,
Eliza
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Nevyn,
Could you please let us know if the example provided helped you?
Thank you,
Eliza
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Eliza,
Unfortunately, I didn't find he parameters I wanted. and I tried another way, but it didn't work.I still hope to use matlab to convert the depth chart point cloud, I don't know you have a better way?
Thank you,
Nevyn
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Nevyn,
Could you please let us know exactly what are the parameters that you are interested in?
From one of the posts you made on this thread, I understood that you are interested in knowing:
Focal Length: which for D435 is 1.93mm and for D415 is 1.88mm. You can find this information in the https://www.intel.com/content/dam/support/us/en/documents/emerging-technologies/intel-realsense-technology/Intel-RealSense-D400-Series-Datasheet.pdf datasheet, at page number 32.
Physical dimensions of each pixel on the x and y axes of the image plane: The image sensor for D415 has the pixel size 1.4um x 1.4um and the Image sensor for D435 has the pixel size 3um x 3um. You can find this information https://newsroom.intel.com/wp-content/uploads/sites/11/2018/01/realsense-d415-d435-fact-sheet.pdf here at page number 2.
Besides this, what other parameters are you interested to know?
Thank you in advance,
Eliza
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Eliza ,
My goal is to use matlab to turn the depth chart into a point cloud,so I need the parameters of the camera.
But through yesterday's communication, I have implemented this function by rs-convert.Unfortunately, there are also a problem about the program.
When I use this program to convert to a PNG file, there will be inconsistency between the number of depth images and color images.so this problem needs to be solved.
Thank you ,
Best Regards,
Nevyn
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page