6 Replies Latest reply on Oct 3, 2018 2:20 PM by jjbos

    MATLAB RGB point cloud

    jjbos

      I would like to acquire and plot RGB colored point clouds from a D435 in MATLAB.  The desired output is analogous to the rs-pointcloud.cpp example.

       

      I reviewed rs-pointcloud.cpp and the MATLAB wrappers, and I'm able to acquire a point cloud, however I can't find how to properly color and plot the point cloud.  Below is my code, which I hope will be useful to others.

       

      There are two options that I'm considering:

      1) is there a way for MATLAB to consume the realsense.pointcloud() object directly?

      2) Assuming the answer to 1) is no, can I access and use the vertices and textures to reformat the RS point cloud object into a MATLAB point cloud object?

       

      points.get_vertices(); and points.get_texture_coordinates(); seem to get correct values, but I'm not sure how to use them in MATLAB.

       

      Thanks!

       

      MATLAB:

          % Make Pipeline object to manage streaming

          pipe = realsense.pipeline();

         

          % define point cloud object

          pcl_obj = realsense.pointcloud();

         

          % define colorizer to give point cloud color

          colorizer = realsense.colorizer();

         

          % Start streaming on an arbitrary camera with default settings

          profile = pipe.start();

       

       

          % Get frames. We discard the first couple to allow

          % the camera time to settle

          for i = 1:5

              frames = pipe.wait_for_frames();

          end

         

          % Stop streaming

          pipe.stop();

       

          % Select depth frame

          depth = frames.get_depth_frame();  

       

          % get point cloud points without color

          points = pcl_obj.calculate(depth);

         

          % get texture mapping

          color = frames.get_color_frame();

            

          % map point cloud to color

          pcl_obj.map_to(color);  

        

          % get vertices (nx3)

          vertices = points.get_vertices();

         

          % get texture coordinates (nx2)

          tex_coords = points.get_texture_coordinates();

        • 1. Re: MATLAB RGB point cloud
          Intel Corporation
          This message was posted on behalf of Intel Corporation

          Hello jjbos,

          Thank you for our interest in the Intel RealSense D435 camera. 

          Please make sure that you have the latest firmware on your camera and librealsense version 2.16.1. 

          We have modified your code (it follow the export_to_ply python examples).
           

          function pc_example

           % Make Pipeline object to manage streaming

              pipe = realsense.pipeline(); 

              % define point cloud object

              pcl_obj = realsense.pointcloud();

             % Start streaming on an arbitrary camera with default settings

              pipe.start();

              % Get frames. We discard the first couple to allow

              % the camera time to settle

              for i = 1:5

                  frames = pipe.wait_for_frames();

              end

               % Select depth frame

              depth = frames.get_depth_frame();     

              % get color frame

              color = frames.get_color_frame();

              % map point cloud to color

              pcl_obj.map_to(color);  

              % get point cloud points without color

              pnts = pcl_obj.calculate(depth);

              %export ply file

              pnts.export_to_ply('pc_example.ply',color);

           pipe.stop();

          end


          Please let us know if this works with you as well!

          Best regards,
          Eliza
          1 of 1 people found this helpful
          • 2. Re: MATLAB RGB point cloud
            jjbos

            Hi Eliza,

             

            Thank you.  This is helpful, and *almost* what I need.

             

            I would like to visualize the colored point cloud live stream, just like the rs-pointcloud.cpp example... but in MATLAB. 

             

            Is there a way to convert the RealSense point cloud object to something MATLAB can manipulate without having to write and then read an intermediate .ply file?

             

            Thanks,

            -Joe

            • 3. Re: MATLAB RGB point cloud
              Intel Corporation
              This message was posted on behalf of Intel Corporation

              Hello jjbos,

              You check out this post that I believe can help you with your project: https://www.mathworks.com/matlabcentral/answers/7202-processing-on-a-live-webcam-streaming-video. Please note that you might need the Image Acquisition Toolbox. 

              Thank you,
              Eliza

              • 4. Re: MATLAB RGB point cloud
                jjbos

                Hi Eliza,

                 

                Thanks for looking but this does not address the issue.  The link you sent relates to processing 2D images.  Please correct me if you saw something related to point clouds.

                 

                Again, I need to convert the RealSense point cloud object and other data (e.g. pc.vertices, pc.texture, color image, depth image) into a MATLAB point cloud without going through an intermediate .ply file.  MATLAB requires point cloud vertices and RGB values for each vertex.

                 

                It appears I can get the correct vertices from the point cloud object but I don't know how to get the corresponding RGB values to color the point cloud.  Can you help with this?

                 

                Thanks,

                -Joe

                 

                Here is how I get the point cloud vertices (x,y,z values), but again I need the corresponding RGB values:

                 

                % Make Pipeline object to manage streaming

                pipe = realsense.pipeline();

                 

                % define point cloud object

                pcl_obj = realsense.pointcloud();

                 

                % define colorizer to give point cloud color

                colorizer = realsense.colorizer();

                 

                % Start streaming on an arbitrary camera with default settings

                profile = pipe.start();

                 

                % Get frames.  Discard the first several to allow

                % the camera time to settle

                for i = 1:5

                    frames = pipe.wait_for_frames();

                end

                 

                % Select depth frame

                depth = frames.get_depth_frame();

                 

                % get texture mapping

                color = frames.get_color_frame();

                 

                % map point cloud to color

                pcl_obj.map_to(color);

                 

                 

                % get point cloud points without color

                points = pcl_obj.calculate(depth);

                 

                % get vertices (nx3)

                vertices = points.get_vertices();

                 

                % get texture coordinates (nx2)

                tex_coords = points.get_texture_coordinates();

                • 5. Re: MATLAB RGB point cloud
                  Intel Corporation
                  This message was posted on behalf of Intel Corporation

                  Hello jjbos,

                  The RealSense team is working on an example to demonstrate how to do this but I don't have a timeline for when it will be completed. Meanwhile, you can check out this code: https://communities.intel.com/community/tech/realsense/blog/2018/10/03/unofficial-pointcloud-stream-viewer-in-matlab.

                  We need to figure out how to use the pcshow command and to use the same algorithm in the align.cpp example to make the depth and color frames the same size.

                  Regards,
                  Jesus G.
                  Intel Customer Support

                  1 of 1 people found this helpful
                  • 6. Re: MATLAB RGB point cloud
                    jjbos

                    Thank you.  I very much look forward to the colored point cloud example in MATLAB.

                     

                    -Joe