Matlab - How to get points from each fram of a .bag file?

[GRedd]

I have run the rosbag_example.m on a recorded .bag file to get a depth frame from it.

In an similar way, can I use the export_to_ply or get_vertices functions to get the point data from each recorded frame? Would also like the timestamp for each frame.

Thanks in advance for any help.

[MartyG]

For others who are reading this post: if you have access to a Windows PC, you may also be interested to know that a MATLAB wrapper was added to RealSense SDK 2.in version(2.16.0. The wrapper contains the rosbag example.(rosbag_example.m) that gpred mentions.

https://github.com/IntelRealSense/librealsense/tree/master/wrappers/matlab librealsense/wrappers/matlab at master · IntelRealSense/librealsense · GitHub

Hi gpred, there are a couple of options that you could explore:

1. RealSense user Leponzo suggested that it was possible to get all the data from a bag file by using ROS and MATLAB's Robotics System Toolbox.

2. Regarding non-MATLAB options: there is a discussion at the moment about converting a bag to a PCD point cloud.

Others in the RealSense community may have further suggestions about how to extract the points from a bag.

[GRedd]

Thanks Marty. Unfortunately I don't have the Robotics Toolbox at the moment.

The rosbag example in Matlab seems to show that the images can be extracted from the .bag easily enough without the Robotics Toolbox, so I think something like the 'export_to_ply' or 'get_vertices' already present in the matlab wrapper released by intel should be able to be used to get the pointcloud without having access to the Robotics toolbox. I just haven't been able to figure it out yet

[MartyG]

Have you seen the pointcloud script in the new wrapper?

https://github.com/IntelRealSense/librealsense/blob/master/wrappers/matlab/pointcloud.m librealsense/pointcloud.m at master · IntelRealSense/librealsense · GitHub

[GRedd]

Yeah, can't get anything to work from it though. Need a few more examples in the Matlab wrapper, their are only two at the moment.

[idata]

Hi gpred.

Could you tell us what specific errors are you getting? Did you try to run the example as it is?If you modified it, could you give us the source code?Regards,Alexandra

[GRedd]

Hi Alexandra,

Yes I ran the rosbag_example and it works perfectly.

I have tried various modifications to try and get the point cloud or save a ply, and these have all failed.

For example, when I put the following line after depth = fs.get_depth_frame();

points = realsense.frame.get_vertices();

I get the error 'No appropriate method, property, or field 'get_vertices' for class 'realsense.depth_frame''

Another example, when I put the following line after depth = fs.get_depth_frame();

The class realsense.frame has no Constant property or Static method named 'export_to_ply'.

I get the error 'The class realsense.frame has no Constant property or Static method named 'export_to_ply'.'

I have tried many other incarnations with similar results and feel as though without any documentation and/or further examples I am going to continue to fail. Any help in exporting to ply from this example would be awesome.

[idata]

Hi gpred,

You can try this code:

function rosbag_example_ply(filename)

% Make Config object to manage pipeline settings

cfg = realsense.config();

validateattributes(filename, {'char','string'}, {'scalartext', 'nonempty'}, '', 'filename', 1);

% Tell pipeline to stream from the given rosbag file

cfg.enable_device_from_file(filename)

% Make Pipeline object to manage streaming

pipe = realsense.pipeline();

% Make Colorizer object to prettify depth output

colorizer = realsense.colorizer();

% define point cloud object

pcl_obj = realsense.pointcloud();

% Start streaming from the rosbag with default settings

profile = pipe.start(cfg);

% Get streaming device's name

dev = profile.get_device();

name = dev.get_info(realsense.camera_info.name);

% Get frames. We discard the first couple to allow

% the camera time to settle

for i = 1:5

fs = pipe.wait_for_frames();

end

% Stop streaming

pipe.stop();

% Select depth frame

depth = fs.get_depth_frame();

% get texture mapping

color = fs.get_color_frame();

% map point cloud to color

pcl_obj.map_to(color);

% get point cloud points without color

pnts = pcl_obj.calculate(depth);

pnts.export_to_ply('pc_example_1.ply', color);

end

The get_vertices and export_to_ply functions are in the points class (pnts in the code above), not the frame class.

Regards,Alexandra

[GRedd]

Thanks for replying to this!

Unfortunately I can't test it as for some reason the realsense Matlab sdk implementation has ceased working. For example when I run realsense.depth_example I get the error,

Undefined variable "realsense" or class "realsense.depth_example".

When I run rosbag_example I get,

Undefined variable "realsense" or class "realsense.config"

Was all working fine before

[idata]

Hi gpred,

The .m file must be in the same folder as the librealsense_mex.lib and librealsense_mex..mexw64 files which is where all the other .m files for the wrapper are located. Also, the function name and filename have to be the same. If you hover your mouse over the function name in the MatLab editor a little pop-up says that. If you name the function, "depth_example_ply" then the file must be named "depth_example_ply.m."Regards,Alexandra

[GRedd]

Yes, I was doing all of that. It turns out the issue was with the Matlab path. The Matlab wrapper folder won't work if I copy it to a new location as suggested in the Intel documentation. It also won't work if I add the +realsense folder to the Matlab paths, as suggested in the Intel documentation. The only way it work for me is if I add the Matlab folder to the path one level up from the +realsense folder in the directory where the sdk installs.

So having figured out that I can now once again run the matlab wrapper functions including your rosbag_example_ply which works!

Thanks very much for your help on this. Any idea how I would get the timestamp of the frame so that I can then include that in the file name that is saved?

[MartyG]

You can get the frame timestamp with the instruction frame_timestamp

https://github.com/IntelRealSense/librealsense/issues/2188 Frame metadata timestamps · Issue # 2188 · IntelRealSense/librealsense · GitHub

Here is an example use of it (highlighted on line 102 of a script).

https://github.com/IntelRealSense/librealsense/blob/master/examples/C/color/rs-color.c# L102 librealsense/rs-color.c at master · IntelRealSense/librealsense · GitHub

[GRedd]

Thanks Marty. We are talking about post processing a rosbag file after capture here, is the code example compatible with this?

[MartyG]

A bag is just a recording of live camera data, and if you load it into a program such as RealSense Viewer that can playback the bag contents then you can do work on the data as though it was a live stream. So I would think that the frame timestamp code would still work with it.

Speaking of doing post-processing on a bag, have you seen Intel's Python tutorial on loading a bag and applying post-processing filters to it?

https://github.com/IntelRealSense/librealsense/blob/jupyter/notebooks/depth_filters.ipynb librealsense/depth_filters.ipynb at jupyter · IntelRealSense/librealsense · GitHub

[GRedd]

In this thread we have just figured out how to post process a rosbag in Matlab. Which I am pretty pleased with! Would be awesome to be able to then also get the timestamp in Matlab. Seems a bit like a backward step to post process in Matlab then switch to c or python just to get a timestamp for a frame. Then one will also need to try and match up the timestamp obtained in c or python (or whatever) to the frame in Matlab.

[MartyG]

The SDK's MATLAB wrapper seems to have support for getting the timestamp. Would it be possible for you to use the MATLAB wrapper?

https://github.com/IntelRealSense/librealsense/blob/master/wrappers/matlab/frame.m# L24 librealsense/frame.m at master · IntelRealSense/librealsense · GitHub

I tried to locate a direct equivalent timestamp-getting function for frames in MATLAB that didn't involve RealSense SDK 2.0 but had no luck.

[JMoha1]

Hello,

I am trying to run the rosbag_example.m using a recorded BAG file. I following all the points that are discussed in this forum, but still I receive the following error:

My command: rosbag_example('File')

Error:

Error using librealsense_mex

Failed to resolve request. Request to enable_device_from_file("File") was invalid, Reason: Failed to create ros reader: Error opening file: File

Error in realsense.pipeline/start (line 30)

        out = realsense.librealsense_mex('rs2::pipeline', 'start', this.objectHandle, config.objectHandle);

Error in rosbag_example (line 14)

  profile = pipe.start(cfg);

I would appreciate if you could offer me a solution.

Regards.

Javad

[MartyG]

The instructions for the example programs say that once the wrapper has been built, you should go to the MATLAB prompt and type realsense.rosbag_example

https://github.com/IntelRealSense/librealsense/tree/master/wrappers/matlab#building-from-source