1 2 3 4 Previous Next 51 Replies Latest reply on Oct 16, 2018 1:50 AM by MartyG

    convert depth image to point cloud

    Nevyn

      How to convert depth frame to point cloud using Matlab? I know one way is if we know the camera parameters,we can obtain point cloud .But what are the parameters ?or Is there any other way to do it in Matlab?

        • 1. Re: convert depth image to point cloud
          MartyG

          If you are using Windows, in the most recent 2.16.0 version of the RealSense SDK, a MATLAB wrapper was introduced.  This contains point cloud support..

           

          librealsense/pointcloud.m at master · IntelRealSense/librealsense · GitHub

           

          You can find the front page of the wrapper here:

           

          librealsense/wrappers/matlab at master · IntelRealSense/librealsense · GitHub

           

          If you are not using Windows, the discussion below looks at the process from the opposite direction (converting a point cloud to depth) and may provide some useful insights.

           

          Convert pointcloud to Depth image (3d to 2d) · Issue #2204 · IntelRealSense/librealsense · GitHub

          • 2. Re: convert depth image to point cloud
            Nevyn

            Hi,MartyG,

            I triesd the above ways,but I'm so sorry that I still don't know how to convert depth frame to point cloud ,the 2.16.0 version does not appear a program about converting depth frame into point clouds.

             

            Best Regards!

            • 3. Re: convert depth image to point cloud
              MartyG

              I have not used the MATLAB wrapper myself.  If it works like the SDK's other wrappers though then it would give access via MATLAB to functions in the core SDK such as point cloud creation.  The problem tends to be that until example programs or script snippets for a particular function are published for a wrapper then it is hard to know how to write a script for the wrapper that produces the desired result.  Such samples emerge over time, whether published by Intel or a member of the community.  I understand that this does not help you today though.

               

              I did some further research.  On the page linked to below, there is a MATLAB function that can be downloaded that converts a depth image to a point cloud in MATLAB.  Scroll down the page to the heading titled 'Depth Image to Point Cloud (MATLAB)' and click on the 'depthToCloud.m' link to launch its download in your browser.

               

              RGB-D (Kinect) Object Dataset

              1 of 1 people found this helpful
              • 4. Re: convert depth image to point cloud
                Nevyn

                Hi, Marty

                 

                Fortunately, the approach you recommend is feasible, but as I said earlier, converting a depth image to a point cloud requires some camera parameters, such as:

                focal length:

                I saw two focal lengths in the camera data,I don't know which one I choose,

                1.png

                the other parameter :The physical dimensions of each pixel on the x and y axes of the image plane。That is the red part of the formula.

                 

                For clarity, here is the conversion formula:

                2.png

                • 5. Re: convert depth image to point cloud
                  bruce_che

                  Hi,

                      Why must you use Matlab? C++ code is here and like this:

                  /librealsense-2.16.0/examples/pointcloud/rs-pointcloud.cpp

                  // Declare pointcloud object, for calculating pointclouds and texture mappings
                      rs2::pointcloud pc;
                      // We want the points object to be persistent so we can display the last cloud when a frame drops
                      rs2::points points;
                      // Declare RealSense pipeline, encapsulating the actual device and sensors
                      rs2::pipeline pipe;
                      // Start streaming with default recommended configuration
                      pipe.start();
                   
                          auto frames = pipe.wait_for_frames();
                          auto depth = frames.get_depth_frame();
                          // Generate the pointcloud and texture mappings
                          points = pc.calculate(depth);//you get points without color
                          auto color = frames.get_color_frame();
                          // For cameras that don't have RGB sensor, we'll map the pointcloud to infrared instead of color
                          if (!color)
                              color = frames.get_infrared_frame();
                          // Tell pointcloud object to map to this color frame
                          pc.map_to(color);//you get points with color

                   

                  if  you want to export the point out, use " export_to_ply", just find it. or you can code yourself.

                   

                  Hope to help u.

                  • 6. Re: convert depth image to point cloud
                    MartyG

                    The D415's focal length is 1.88 mm and the D435's focal length is 1.93 mm.

                     

                    The Dx / DY question is a harder one.   It has been said that it is dependent on sensor size.  The rolling shutter of the D415 has a pixel size of 1.4um x 1.4 um and the global shutter of the D435 has a sensor size of 3um x 3 um.

                     

                    RealSense expert Samontab once gave the following explanation of pixel size:

                     

                    R200 pixel size

                    • 7. Re: convert depth image to point cloud
                      Nevyn

                      Thank you very much! This is very useful to me. Because I have been working on matlab, and I am not very familiar with c++, so I want to use matlab. I'll try to use c++.

                      Thanks again!

                       

                      Bset reagrds!

                      • 8. Re: convert depth image to point cloud
                        Nevyn

                        Hi,I have run  the above program and researched ahout it ,but it only sample one point cloud .Unfortunately,that's not what I'm trying to accomplish!

                         

                        Thank you!

                        • 9. Re: convert depth image to point cloud
                          Intel Corporation
                          This message was posted on behalf of Intel Corporation

                          Hello Nevyn,

                          A .ply represents a pointcloud format that contains the information from only one frame. 
                          If this is not what you are trying to accomplish, could you please give us more details in order to help you?

                          Thank you best regards,
                          Eliza
                           

                          • 10. Re: convert depth image to point cloud
                            Nevyn

                            Hi Eliza,

                             

                            I want to get color image and corresponding point cloud video sequence, but the only way is converting the depth image to point cloud.I know the transformation  formual ,but don't know the camer paremeters.

                            Do you have other ways to finish it.

                             

                            Thanke you,

                            Nevyn

                            • 11. Re: convert depth image to point cloud
                              Intel Corporation
                              This message was posted on behalf of Intel Corporation

                              Hello Nevyn,

                              Thank you for the clarifications! 

                              In this case you are interested in knowing the Intrinsic Parameters, you can read about it on Intrinsic Camera Parameters 2. Deprojection.

                              You might also find this GitHub thread helpful.

                              Please let us know if this information helps you with your project! 

                              Thank you and best regards,
                              Eliza

                              • 12. Re: convert depth image to point cloud
                                Nevyn

                                Hello Eliza,

                                 

                                When I start the Intel RealSense Viewer  use 'record' to recoed a sequence , I get the error message:

                                 

                                Unknown in rs2_create_record_device(device:0000020C63E7EB90, file:C):

                                Error opening file: C:\Users\\20180912_155849.bag

                                 

                                I don't know how to do it ,can you help me?

                                 

                                Thanks.

                                • 13. Re: convert depth image to point cloud
                                  Intel Corporation
                                  This message was posted on behalf of Intel Corporation

                                  Hello Nevyn,

                                  This seems to be a permission error. 
                                  If you save a snapshot, do you received the same error? 

                                  Thank you,
                                  Eliza

                                  • 14. Re: convert depth image to point cloud
                                    Nevyn

                                    Hello Eliza,

                                     

                                    I have solved the problem,but I still need to look at the previous questions about camera parameters.

                                     

                                    Thank you,

                                    Nevyn

                                    1 2 3 4 Previous Next