1 Reply Latest reply on May 8, 2018 1:41 AM by MartyG

    Streaming compressed video data to disk (via FFMpeg and DirectShow)


      We've been working with the RealSense SR300 model for several weeks now, and we hope to start working with the D435 when it arrives. We primarily use the Python interface to the SDK 2.0. However, I'll soon be collecting a sizable data set that will be saved for later processing via computer vision algorithms, and I am not sure how best to approach this. I am somewhat new to computer vision and video processing.


      I see that it is possible to stream to disk in ROS bag format. That's great, but it results in very large data files, and we will be storing/transferring a lot of data. After some discussion, we've decided that lossy video compression would be acceptable. I have successfully transcoded both the RGB and depth streams to disk using FFMpeg, but I did so using custom Python code that manually passes each video frame from the RealSense pipeline to FFMpeg via a pipe. It would be much more straightforward if I could use DirectShow to capture the streams directly (It is a requirement of our project that we must use the Windows platform). This seems to work fine for the RGB stream, but fails for the depth data. The symptom is the same as you would see if viewing the camera through VLC: there is only a solid green video stream. From what I understand, DirectShow is interpreting the depth stream as a color stream because it does not recognize an appropriate pixel format. Is there a workaround for this? Or an alternate solution?


      Although I did not create it, this forum post seems to describe the problem I have:


      In this example, DirectShow seems to support only the yuyv422 pixel format for the depth stream. All of the other entries are listed as "unknown compression type". This is also what I see. For the record, when writing directly via FFMpeg, I used the gray16le pixel format.


      Please let me know if I can provide any additional information that might be useful.