4 Replies Latest reply on May 9, 2018 6:28 AM by tiwoti

    Rounding of depth value SR300 to 1 mm (SDK 2016 R2)

    tiwoti

      Hi there,

       

      Currently I'm trying to read the real world coordinates of a pre recorded ".rssdk" file hence the use of the old SDK.

      However, the depth values I'm getting keep being rounded to 1 mm.

      I've seen multiple topics about this issue and I've tried implementing the recommended solutions.

      Getting precise depth data without QueryVertices()

      Depth Resolution of Real Sense Camera

      Getting float depth data


      However the implementation of the suggestions were not successful. So I'm not sure if my own implementation is wrong or that my pre recorded rssdk file does not contain depth values with a higher accuracy.

      I've tried 3 different implementations in C++ shown below here (with a few checks removed).

       

      A typical output I would get from this script are the following values:

      pos3D[i].x = 0.305 mm

      pos3D[i].y = 145.285 mm

      pos3D[i].z = 520.000 mm

      dPixels[i]    = 520.000 mm

      dPixelsRaw[i]  = 4160

      zconvert     = 520.000 mm

       

       

      int wmain()

      {

           // Filename

           pxcCHAR file[1024] = L"filename.rssdk";

       

           // create the PXCSenseManager inluding the instance

           PXCSenseManager *sm = 0;

           sm = PXCSenseManager::CreateInstance();

       

           // Set file recording or playback

           sm->QueryCaptureManager()->SetFileName(file, false);

       

           // Select the color stream

           sm->EnableStream(PXCCapture::STREAM_TYPE_DEPTH, 640, 480);

       

           // Initialize

           sm->Init();

       

           // Analyze first 10 frames

           for (int i = 0; i < 10; i++) {

                // This function blocks until a color sample is ready

                if (sm->AcquireFrame(true) < PXC_STATUS_NO_ERROR) break;

       

                // Retrieve the depth sample

                PXCCapture::Sample *sample = sm->QuerySample();

       

                PXCCapture::Device *device = sm->QueryCaptureManager()->QueryDevice();

                pxcF32  depthUnit = device->QueryDepthUnit();

       

                // Example 1: Read image data in F32

                PXCImage::ImageData depthImage;

                sample->depth->AcquireAccess(PXCImage::ACCESS_READ, PXCImage::PIXEL_FORMAT_DEPTH_F32, &depthImage);

                float *dPixels;

                dPixels = (float*)depthImage.planes[0];

       

                // Example 2: Projection with QueryVertices in 3D F32

                PXCImage::ImageInfo imgInfo = sample->depth->QueryInfo();

                int depth_width = imgInfo.width;

                int depth_height = imgInfo.height;

                int num_pixels = depth_width * depth_height;

                PXCProjection * projection = device->CreateProjection();

                PXCPoint3DF32 *pos3D = new PXCPoint3DF32[num_pixels];

                sts = projection->QueryVertices(sample->depth, &pos3D[0]);

       

                // Example 3: Raw Depth

                PXCImage::ImageData depthImageRaw;

                sample->depth->AcquireAccess(PXCImage::ACCESS_READ, PXCImage::PIXEL_FORMAT_DEPTH_RAW, &depthImageRaw);

                pxcU16 *dPixelsRaw;

                dPixelsRaw = (pxcU16*)depthImageRaw.planes[0];

       

                // Display values

                for (int i = 0; i < num_pixels; i++)

                {

                     // Only output values to command prompt

                     if (pos3D[i].x > 0)

                     {

                          // Convert the raw value to world coordinates

                          float zconvert = float(dPixelsRaw[i])*(depthUnit / 1000.0f);

                          cout << fixed << setprecision(3) << pos3D[i].x << "   " << pos3D[i].y << "   " << pos3D[i].z << "    " << dPixels[i] << "     " << dPixelsRaw[i] << "    " << zconvert << endl;

                     }

           }

           // Go fetching the next sample

           sm->ReleaseFrame();

           }

      sm->Release();

      }

        • 1. Re: Rounding of depth value SR300 to 1 mm (SDK 2016 R2)
          MartyG

          I'll link into this discussion jb455 the RealSense stream programming expert from the link you saw, 'Getting float depth data'.  Hopefully he can provide useful suggestions for your problem.

          • 2. Re: Rounding of depth value SR300 to 1 mm (SDK 2016 R2)
            jb455

            The only thing I can think of is if the float type you're using is 16-bit instead of 32 and it's truncating the values. But I think the C++ float type is 32 bit so you're probably alright there. Everything else looks OK.

            It's possible the rssdk file doesn't store the full precision, either due to a bug or by design if they wanted to save space - hopefully someone from Intel will be able to confirm. In the meantime, you could try decoding it yourself - this may help - to see if the issue is with recording (not storing full precision) or playback (not reading the file properly).

             

            BTW, the new SDK can record and play back colour/depth streams from the cameras, though I don't think it's compatible with rssdk files (and unlikely to be by the looks of things) so if you need to work on these specific files and new ones can't be recorded with the new SDK you may be SOL.

            • 3. Re: Rounding of depth value SR300 to 1 mm (SDK 2016 R2)
              tiwoti

              Great, thanks for the reply! I will look into it and report my findings.

              The C++ float is indeed 32-bit but for clarity sake I changed the floats to pxcF32 types in my code. The results were not affected by this change.

               

              A part of my recordings can't be recorded again, however I'm still creating some new recordings so if the depth accuracy indeed is being rounded in rssdk files I'll switch to the new SDK.

              Unfortunately, I'm also using the 3D face/landmark detection algorithms from the old SDK, and as far as I know these are currently not available in SDK 2.0.

              I know it's possible with OpenCV in 2D and then performing a projection to 3D  (Noob Question: RealSense 2.0 sdk: does it have face recognition etc? ) but it's a shame there is no native implementation/correction for the 3D data as with the old SDK.

              • 4. Re: Rounding of depth value SR300 to 1 mm (SDK 2016 R2)
                tiwoti

                During a live stream, the code from above indeed returns float values for the depth data as expected. Therefore, it indeed seems to be a recording or playback problem of the rssdk file itself. My first try was to decompress the original file with the RSSDK ClipEditor in case it might help (I didn't expect it would, but who knows), but this had no effect on the precision of the depth values. Decoding the rsssdk file myself might turn out to be a bit too difficult, so I'm first trying to see what happens if I disable the LZO compression for the depth data, to see if I can increase the accuracy for a new recording. At the moment the registry edit to disable the compression didn't work properly (https://software.intel.com/en-us/blogs/2015/10/15/how-to-record-and-playback-streaming-sequences-in-intel-realsense-sdk) but hopefully I can fix this soon and see if this helps.

                 

                Edit - After disabling the LZO compression the depth data is still being stored as rounded numbers. So, as far as I can tell it is not possible to store the float depth values in a rssdk file (or read them properly again).