7 Replies Latest reply on Oct 13, 2018 12:08 AM by MartyG

    D415 - Basic Code For Saving Depth Stream As Binary File

    IntewellAnyoha

      I am a complete beginner to coding. I can do some things in Matlab and Python but that's about it. All I want to do is record depth with my D415 camera and save the stream as a binary file (little endian) that contains the depth value of every pixel for each frame (all I want is depth, nothing else). So if my resolution is 840x480, I want a file that has 840 X 480 depth values for every frame. Here is what I can do myself. I can implement, the examples on github. I am able to get my linux machine to run:

       

        $ g++ -std=c++11 rs-save-to-disk.cpp -lrealsense2

      from librealsense/examples/save-to-disk at master · IntelRealSense/librealsense · GitHub

      or

        $ gcc depth.c -lrealsense2

      from librealsense/examples/C/depth at master · IntelRealSense/librealsense · GitHub

       

      My understanding of these languages is so small that I cannot simply read the sdk and do what I want. (If I was working in MATLAB I could do that no problem).

       

      There must be code out there SOMEWHERE that accomplishes my very basic task.

       

      My end goal is to be able to record with the camera for hours. I eventually want to model the depth data and do some real time processing (sort of like object recognition). Please help me out, I'd be very grateful. I'm looking for code I can copy and paste and use right away or a step by step (so simple that a child could understand) walk through of how to do this myself. I'd prefer the former. In the meantime I am learning c and c++ from scratch.

       

      Thanks

        • 1. Re: D415 - Basic Code For Saving Depth Stream As Binary File
          MartyG

          A MATLAB wrapper is on SDK 2.0's list of upcoming features.  Hopefully this will make development easier for you, since MATLAB is the environment that you are most comfortable with.     EDIT: the MATLAB wrapper is now available from SDK version 2.16.0 onwards.

           

          In the meantime, there is a Python tutorial in the SDK for displaying the depth stream in the console as ASCII text characters.

           

          librealsense/python-tutorial-1-depth.py at development · IntelRealSense/librealsense · GitHub

           

          You could perhaps add a line to the script that prints the contents to a text file too.  For details of how to do this, google 'python print to text file'.  Looking at the example script, it seems that the output is stored in the variable called 'line' and printed to screen.

           

          Here's an example of a guide to adding code to print to a text file.

           

          Python Print String To Text File - Stack Overflow

          • 2. Re: D415 - Basic Code For Saving Depth Stream As Binary File
            nstalati

            Hello MartyG,

             

            I tried to implement something in MATLAB which is analogous to the python link you pointed to however, I notice that reading depth through the function call depth.get_distance(x,y) is taking quite some time when trying to get depth for the entire field of view. Specifically, I notice that it takes about ~9 seconds to get depth of every pixel in a given frame using the depth call. I am using RealSense d435 with 1280*720 resolution and FPS = 30. The code used is pasted below:

             

            %Make Pipeline object to manage streaming

                pipe = realsense.pipeline();

                % Make Colorizer object to prettify depth output

                colorizer = realsense.colorizer();

                %Start streaming on an arbitrary camera with default settings

                profile = pipe.start();

             

             

                while (true)

                    fs = pipe.wait_for_frames();

                    depth = fs.get_depth_frame();

                    tic

                    for i = 1:720

                        for j  = 1:1280

                            dist(i,j) = depth.get_distance(j,i);

                        end

                    end

                    toc

                    color = colorizer.colorize(depth);

                    data = color.get_data();

                    img = permute(reshape(data',[3,color.get_width(),color.get_height()]),[3 2 1]);

                    imshow(img);

                end

             

            Is there a faster way to do this so that I get depth value for the entire FOV under 1s for example? Maybe, the way it is done on the RealSense viewer perhaps? (which is displaying a video at 30 FPS while giving me depth info when i hover over a pixel)

             

             

            Your help is much appreciated!

             

            Regards,

            Neel

            • 3. Re: D415 - Basic Code For Saving Depth Stream As Binary File
              MartyG

              If you want to display the depth frame in MATLAB, the recently introduced MATLAB wrapper for SDK 2.0 has an example script for doing so on the front page of its documentation.  You'll find it if you go right to the bottom of the page.

               

              librealsense/wrappers/matlab at master · IntelRealSense/librealsense · GitHub

               

              Edit: the wrapper requires Windows 10.  Do you have Windows?

              • 4. Re: D415 - Basic Code For Saving Depth Stream As Binary File
                nstalati

                Thanks for the quick response!

                 

                Yes, I had looked at it earlier however, that example is not giving me physical distance for each pixel in its FOV. The depth example seems to be only displaying one depth frame. I am looking to get a depth value (in m) for each pixel in the FOV of the camera and store it in a matrix and do it repeatedly for every frame. I am currently able to do that but it is taking very long. I am wondering if there is a faster way to do it.

                 

                Edit: Yes I have windows 10

                 

                Regards,

                Neel

                • 5. Re: D415 - Basic Code For Saving Depth Stream As Binary File
                  MartyG

                  What about the Python distance_to_object tutorial that measures depth?

                   

                  librealsense/distance_to_object.ipynb at jupyter · IntelRealSense/librealsense · GitHub

                   

                  Edit: no, it has to involve MATLAB, doesn't it.  Sorry.  Checking again ...

                   

                  Edit 2: I recalled that Dorodnic the SDK Manager gave another user a MATLAB depth script for Pyrealsense today.

                   

                  https://github.com/IntelRealSense/librealsense/issues/2520#issuecomment-429339555

                   

                  In that script, it is setting the colorizer configuration to 'White to Black' instead of the 'Jet' setting.  So you may have to change the setting number to the configuration you want.

                  • 6. Re: D415 - Basic Code For Saving Depth Stream As Binary File
                    nstalati

                    Thanks for all the help!

                     

                    So the links you posted were both for python and I am working in MATLAB, but they were still very helpful! As it turns out, I found the magical line of code I was looking for which can basically give me depth for every pixel in the FOV in a single call!! This makes everything a lot faster. For future reference to others, the following lines of code will provide depth (in mm) in a row vector form, for every pixel in a frame, using a single call.

                     

                            pipe = realsense.pipeline();

                            fs = pipe.wait_for_frames();

                            depth = fs.get_depth_frame();

                            dist = depth.get_data();

                     

                    Cheers,

                    Neel

                    • 7. Re: D415 - Basic Code For Saving Depth Stream As Binary File
                      MartyG

                      When you said you were trying to create something analogous to the Python link, I thought you were using Python and MATLAB together.  Apologies for the confusion. 

                       

                      Awesome news that you found a solution.  Thanks for sharing your code with the community!