9 Replies Latest reply on Jun 10, 2018 4:17 AM by MartyG

    Depth of a Pixel

    mstfldmr

      Hello,

       

      What is the correct way of reading the depth of individual pixels in C++?

       

      I want to copy depth matrix (without colorizing) to an OpenCV Mat and then make calculations with it.

        • 1. Re: Depth of a Pixel
          MartyG

          Are you using RealSense SDK 2.0 or the older '2016 R2' or '2016 R3' RealSense SDK, please?

          • 2. Re: Depth of a Pixel
            mstfldmr

            I am using RealSense SDK 2.0  on a RS300, which will be replaced with D435

            • 3. Re: Depth of a Pixel
              MartyG

              Here's some C++ code for getting pixel depth in meters in SDK 2.0.

               

              rs2::depth_frame dpt_frame = frame.as<rs2::depth_frame>();

              float pixel_distance_in_meters = dpt_frame.get_distance(x,y);

              • 4. Re: Depth of a Pixel
                mstfldmr

                Thanks.

                 

                is there a way of getting distance values for all pixels at once?

                • 5. Re: Depth of a Pixel
                  MartyG

                  I wonder if a Point Cloud would be a suitable way for you to get all depth points at the same time.

                   

                  librealsense/examples/pointcloud at master · IntelRealSense/librealsense · GitHub

                   

                  You can also get a point cloud in the RealSense Viewer software by clicking on the '3D' option next to '2D' in the top corner of the Viewer program when a stream is playing.

                  • 6. Re: Depth of a Pixel
                    mstfldmr

                    I need the depth of every pixel for calculations in OpenCV. It will be running on an embedded device with Ubuntu, so Realsense Viewer is not an option.

                    • 7. Re: Depth of a Pixel
                      MartyG

                      rs2_deproject_pixel_to_point may do the job.  Deprojection takes a 2D pixel location on a stream's images, as well as a depth, specified in meters, and maps it to a 3D point location within the stream's associated 3D coordinate space.

                       

                      deprojection pixels to 3d point · Issue #1413 · IntelRealSense/librealsense · GitHub

                       

                      You may also be interested in a pre-made depth program for OpenCV by developer UnaNancyOwen.

                       

                      RealSense2Sample/sample/Depth at master · UnaNancyOwen/RealSense2Sample · GitHub

                      • 8. Re: Depth of a Pixel
                        JJR

                        Hi,

                         

                        I have done it this way;

                         

                        Quote

                         

                        float darray [480 * 640];

                         

                        int main()
                        {

                        //Contruct a pipeline which abstracts the device
                        rs2::pipeline pipe;

                        //Create a configuration for configuring the pipeline with a non default profile
                        rs2::config cfg;
                         
                        //Add desired streams to configuration
                        cfg.enable_stream(RS2_STREAM_COLOR, 640, 480, RS2_FORMAT_BGR8, 15);
                        cfg.enable_stream(RS2_STREAM_INFRARED, 640, 480, RS2_FORMAT_Y8, 15);
                        cfg.enable_stream(RS2_STREAM_DEPTH, 640, 480, RS2_FORMAT_Z16, 15);


                        //Instruct pipeline to start streaming with the requested configuration
                        pipe.start(cfg);

                        namedWindow("Colour", WINDOW_AUTOSIZE); namedWindow("IR", WINDOW_AUTOSIZE);


                        while (1) {

                          rs2::frameset frames;
                         
                           //Wait for all configured streams to produce a frame
                           frames = pipe.wait_for_frames();
                         
                           // Try to get a frame of a depth image
                           rs2::depth_frame d = frames.get_depth_frame();

                         

                           // darray contains distance meters of all the x,y
                           for (int j = 0; j < 480; j++)
                           {

                            for (int i = 0; i< 640; i++)
                            {


                             darray[i + (j * 640)]= d.get_distance(i, j);
                            
                            }
                           
                           }


                         


                          //Get each frame
                          rs2::frame color_frame = frames.get_color_frame();
                          rs2::frame ir_frame = frames.first(RS2_STREAM_INFRARED);
                             
                          // Creating OpenCV Matrix from a color image
                          Mat color(Size(640, 480), CV_8UC3, (void*)color_frame.get_data(), Mat::AUTO_STEP);
                          Mat ir(Size(640, 480), CV_8UC1, (void*)ir_frame.get_data(), Mat::AUTO_STEP);
                          

                          // Display in a GUI
                         
                         

                          imshow("Colour", color); imshow("IR", ir);

                          char key = waitKey(30);
                          if (key == 27) {
                           cout << "key pressed" << endl;

                           color.release(); ir.release();

                           break;
                          }

                        }

                        destroyAllWindows();

                        return 0;
                        }

                         

                        Unquote

                        • 9. Re: Depth of a Pixel
                          MartyG

                          Awesome, thanks so much for sharing your script with the community, JJR