1 2 Previous Next 26 Replies Latest reply on Apr 27, 2017 9:34 AM by Intel Corporation

    Depth Control Parameters

    jb455

      Hi,

       

      I'm trying to improve the quality of the depth image I get from each of the cameras (particularly the R200 but also SR300). I found in the librealsense source they have the following method (from here, 63-86 with similar for the SR300 in 89-121):

       

      static void rs_apply_depth_control_preset(rs_device * device, int preset)
      
      {
      
         static const rs_option depth_control_options[10] = {
      
        RS_OPTION_R200_DEPTH_CONTROL_ESTIMATE_MEDIAN_DECREMENT,
      
        RS_OPTION_R200_DEPTH_CONTROL_ESTIMATE_MEDIAN_INCREMENT,
      
        RS_OPTION_R200_DEPTH_CONTROL_MEDIAN_THRESHOLD,
      
        RS_OPTION_R200_DEPTH_CONTROL_SCORE_MINIMUM_THRESHOLD,
      
        RS_OPTION_R200_DEPTH_CONTROL_SCORE_MAXIMUM_THRESHOLD,
      
        RS_OPTION_R200_DEPTH_CONTROL_TEXTURE_COUNT_THRESHOLD, 
      
        RS_OPTION_R200_DEPTH_CONTROL_TEXTURE_DIFFERENCE_THRESHOLD,
      
        RS_OPTION_R200_DEPTH_CONTROL_SECOND_PEAK_THRESHOLD,
      
        RS_OPTION_R200_DEPTH_CONTROL_NEIGHBOR_THRESHOLD,
      
        RS_OPTION_R200_DEPTH_CONTROL_LR_THRESHOLD
      
        };
      
         double depth_control_presets[6][10] = {
      
        {5, 5, 192,  1,  512, 6, 24, 27,  7,  24}, /* (DEFAULT) Default settings on chip. Similar to the medium setting and best for outdoors. */
      
        {5, 5,  0,  0, 1023, 0,  0,  0,  0, 2047}, /* (OFF) Disable almost all hardware-based outlier removal */
      
        {5, 5, 115,  1,  512, 6, 18, 25,  3,  24}, /* (LOW) Provide a depthmap with a lower number of outliers removed, which has minimal false negatives. */
      
        {5, 5, 185,  5,  505, 6, 35, 45, 45,  14}, /* (MEDIUM) Provide a depthmap with a medium number of outliers removed, which has balanced approach. */
      
        {5, 5, 175, 24,  430, 6, 48, 47, 24,  12}, /* (OPTIMIZED) Provide a depthmap with a medium/high number of outliers removed. Derived from an optimization function. */
      
        {5, 5, 235, 27,  420, 8, 80, 70, 90,  12}, /* (HIGH) Provide a depthmap with a higher number of outliers removed, which has minimal false positives. */
      
        };
      
         rs_set_device_options(device, depth_control_options, 10, depth_control_presets[preset], 0);
      
      }
      
      

       

      One of the users mentioned that they were getting a noisy depth image from the R200, but when they set it to the HIGH preset, the image is much better. Does anyone know if changing these parameters is possible in the RSSDK, or if it's an extra feature the librealsense guys & gals added? I looked through the Device interface but couldn't find anything similarly-named.

       

      Thanks,

      James

        • 1. Re: Depth Control Parameters
          MartyG

          The SR-300's image quality parameters can be altered with the SetIVCamFilterOption instruction.

           

          Intel® RealSense™ SDK 2016 R2 Documentation

          • 2. Re: Depth Control Parameters
            jb455

            Thanks Marty, I'm playing with that now, will report back my findings. There doesn't seem to be anything similar for the R200 though.

            • 3. Re: Depth Control Parameters
              MartyG

              This forum thread on the R200 image quality may be of use to you if you have not seen it already.  Good luck!

               

              Way to configure R200 gain and other parameters

              • 4. Re: Depth Control Parameters
                jb455

                Ok, so I've tested the SetIVCamFilterOption options at ranges from 15cm to about a metre with my SR300. There doesn't seem to be that much difference between Very Close, Close and Mid range, though perhaps that's not too surprising as they all claim to be good for "up to 2m for SR300". I'm more interested in close ranges so didn't spend much time on Far and Very Far, but in case anyone else is interested, I found that I could get up to about 5 metres if I also set MotionRangeTradeOff to 100 while on Very Far mode.

                 

                Regarding that thread from the old forum: I won't have any control over the lighting conditions present when my app is being used, so I can't rely on extra ambient IR etc. The camera will be hand-held too so it would be difficult to average depth across a number of frames without having to do some object-tracking stuff too which seems a bit much! The librealsense depth control presets sound pretty much ideal for what I need, so I'm hoping there's a way of getting similar functionality with the SDK.

                • 5. Re: Depth Control Parameters
                  Intel Corporation
                  This message was posted on behalf of Intel Corporation

                  Hello jb455,

                  Thank you for interest in the Intel® RealSense™ Technology.

                  In the SDK, there is not a way to control the depth parameters similar to the librealsense method. As MartyG kindly pointed out, the links provide the best way to modify the depth parameters through the SDK (SetIVCamFilterOption and ScenePerception). 

                  Regards,
                  Andres V.

                  • 6. Re: Depth Control Parameters
                    jb455

                    Hi Andre,

                     

                    That's a bit annoying, but thanks for the information.

                     

                    I've been looking at the EnhanceDepth function for another way to improve the depth image. I've managed to get the EnhanceDepth thing itself working, but am struggling to get the created depth image (which has different dimensions to the captured depth image) to map back to the original colour image so I can get depth values for points in the colour image. All the samples I've found show how to create the enhanced image but don't show how to actually use it.

                    I've tried using projection.CreateDepthImageMappedToColor and projection.QueryInvUVMap with the enhanced depth image as arguments, but both just return arrays of zeroes. There is mention in the docs here about a projection instance being encoded in the depth image but I can't figure out how to get it out (I've tried using QueryMetadata and CreateSerializable<PXCMProjection> and a few other things but haven't managed to get anything other than null) so I've been using the standard device.CreateProjection instance I use for the normal captured depth/colour images instead (which just returns zeroes).

                    So: is it possible to map between the enhanced depth image and the standard colour image?

                     

                    Thanks for your help!

                    James

                    • 7. Re: Depth Control Parameters
                      Intel Corporation
                      This message was posted on behalf of Intel Corporation

                      Hello James,

                      Thank you for sharing your investigation and testing process.

                      I’ll need more time to come up with information that you may find relevant.

                      Thank you for your patience.

                      Regards,
                      Andres

                      • 8. Re: Depth Control Parameters
                        MartyG

                        I came across an interesting looking F200 script that merges depth and color together into a single combined image instead of showing them as separate streams.  I don't know if it'd be any use but I thought I'd throw it your way, just in case.

                         

                        https://mtaulty.com/2015/04/16/m_15794/

                        • 9. Re: Depth Control Parameters
                          Intel Corporation
                          This message was posted on behalf of Intel Corporation

                          Hello James,

                           

                          Have you already performed tests setting the Depth Stream Properties in the RealSense SDK to improve the quality of your images?  Here is the link to the instruction set: https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?property_device_pxccapture.html

                           

                          I’ll be waiting for your response.

                           

                          Regards,
                          Andres V.

                          • 10. Re: Depth Control Parameters
                            jb455

                            Hi Andres, thanks for the link.

                             

                            I've been playing with the properties which have public Query and Set methods, but some of the properties listed on that page don't have them. How can I enumerate the Device.Property object to inspect and change values (in C#)?

                             

                            And Marty, thanks, that's actually very useful for something else I'll need to look at soon!

                             

                            Thanks,

                            James

                            • 11. Re: Depth Control Parameters
                              MartyG

                              You're very welcome - glad the script will be useful 

                               

                              Regarding enumeration of Device.Property, have you seen this page?

                               

                              Property (Advanced, +UWP)

                              • 12. Re: Depth Control Parameters
                                jb455

                                Yeah, that's the page Andres linked. Doesn't say how to actually access the properties!

                                • 13. Re: Depth Control Parameters
                                  MartyG

                                  I found a Unity script that uses Device.Property.  Maybe you could find some useful insights by looking through that.

                                   

                                  Edit: never mind, that script was for the old Perceptual Computing Camera.  It's use of Device.Property looks remarkably similar to RealSense's though.

                                   

                                  ********

                                   

                                  using System;

                                  using System.Collections.Generic;

                                  using System.ComponentModel;

                                  using System.Data;

                                  using System.Drawing;

                                  using System.Linq;

                                  using System.Text;

                                  using System.Threading;

                                  using System.Threading.Tasks;

                                  using System.Windows.Forms;

                                   

                                   

                                  namespace Testing2

                                  {

                                      public partial class Form1 : Form

                                      {

                                          static UtilMPipeline pp = new UtilMPipeline();

                                   

                                   

                                          static System.Drawing.Bitmap rgb;

                                          static System.Drawing.Bitmap binary;

                                   

                                   

                                          public Form1()

                                          {

                                              InitializeComponent();

                                   

                                   

                                              Thread camera = new Thread(new ThreadStart(CameraCapture));

                                              camera.Start();

                                          }

                                   

                                   

                                          public void CameraCapture()

                                          {

                                              pp.EnableImage(PXCMImage.ColorFormat.COLOR_FORMAT_DEPTH);

                                              pp.EnableImage(PXCMImage.ColorFormat.COLOR_FORMAT_RGB32);

                                   

                                   

                                              pp.Init();

                                   

                                   

                                              for (;;)

                                              {

                                                  pp.capture.device.SetProperty(PXCMCapture.Device.Property.PROPERTY_DEPTH_SMOOTHING, 0);

                                   

                                   

                                                  if (!pp.AcquireFrame(true)) break;

                                   

                                   

                                                  PXCMImage depth = pp.QueryImage(PXCMImage.ImageType.IMAGE_TYPE_DEPTH);

                                                  PXCMImage color = pp.QueryImage(PXCMImage.ImageType.IMAGE_TYPE_COLOR);

                                   

                                   

                                                  //******Older method*****

                                                  //color.QueryBitmap(pp.QuerySession(), out rgb);

                                                  //depth.QueryBitmap(pp.QuerySession(), out binary);

                                                  //pictureBox1.Image = binary;

                                   

                                   

                                                  //******New method*****

                                                  PXCMImage.ImageData data;

                                                  depth.AcquireAccess(PXCMImage.Access.ACCESS_READ_WRITE, out data);

                                                  pictureBox1.Image = data.ToBitmap(depth.info.width, depth.info.height);

                                                  // The depth image is pointed by the PXCMImage.Data structure.

                                                  depth.ReleaseAccess(ref data);

                                   

                                   

                                                  pp.ReleaseFrame();

                                              }

                                   

                                   

                                              pp.Close();

                                   

                                   

                                              pp.Dispose();

                                          }

                                      }

                                  }

                                  • 14. Re: Depth Control Parameters
                                    jb455

                                    Yeah, that must be from before they added the Query() and Set() methods for some of the properties. I don't have access to a device.SetProperty() method

                                    1 2 Previous Next