5 Replies Latest reply on Jun 15, 2018 9:44 AM by Intel Corporation

    D435 Dept frame output in OpenCV - how to get rid of the flickering  effect?

    JJR

      Hi

       

      I am obtaining depth frame following method;

       

      Quote

      Mat DD;

      rs2::config cfg;

       

        cfg.enable_stream(RS2_STREAM_DEPTH, 640, 480, RS2_FORMAT_Z16, 15);

       

        rs2::frameset frames;

        rs2::frame depth_frame = frames.first(RS2_STREAM_DEPTH);

        Mat depth(Size(640, 480), CV_16SC1, const_cast<void*>(depth_frame.get_data()));

        double min, max;

        minMaxLoc(depth, &min, &max);

        depth.convertTo(DD, CV_8U, 255.0 / (max - min), -min * 255.0 / (max - min));

        imshow("Depth", DD);

       

      Unquote

      There is  a constant flickering of the frame. How can I fix it?

       

      Thanks

        • 1. Re: D435 Dept frame output in OpenCV - how to get rid of the flickering  effect?
          Intel Corporation
          This message was posted on behalf of Intel Corporation

          Hello JJR, 

          The flicker may be due to the scaling in the convertTo statement. Changing the statement to this, stops the flickering:

          depth.convertTo(DD, CV_8U, 1, 0);

          Best Regards, 

          Juan N.

          • 2. Re: D435 Dept frame output in OpenCV - how to get rid of the flickering  effect?
            JJR

            HI Juan,

             

            Thanks for prompt reply. With  depth.convertTo(DD, CV_8U, 1, 0)  the output is a white back ground see below.

            D435 is a excellent  product with wide view angle and compact size and easy programming. I love to use it for our application but unfortunately switching on or off the light in the room effects both IR and Depth frame. Previously we used Kinect and it is not affected by the light. Dose Intel has any plan in the future to develop wide view angle TOF depth sensor with  depth distance of 5 meters?

             

            After I did the change "depth.convertTo(DD, CV_8U, 1, 0)"  see the depth out put on the right of the screen

            Thanks

            • 3. Re: D435 Dept frame output in OpenCV - how to get rid of the flickering  effect?
              Intel Corporation
              This message was posted on behalf of Intel Corporation

              Hello JJR, 

              What is the rest of the code you are using? Here is the the code we used to test your request (modified rs-imshow.cpp sample in the wrappers/opencv directory of librealsense):

              #include <librealsense2/rs.hpp> // Include RealSense Cross Platform API

              #include <opencv2/opencv.hpp>   // Include OpenCV API

              int main(int argc, char * argv[]) try

              {

                  // Declare depth colorizer for pretty visualization of depth data

                  rs2::colorizer color_map;

                  // Declare RealSense pipeline, encapsulating the actual device and sensors

                  rs2::pipeline pipe;

                  // Start streaming with default recommended configuration

                  

                  rs2::config cfg;

                  cfg.enable_stream(RS2_STREAM_DEPTH, 640, 480, RS2_FORMAT_Z16, 30);

                  pipe.start(cfg);

                  using namespace cv;

                  const auto window_name = "Display Image";

                  namedWindow(window_name, WINDOW_AUTOSIZE);

                  

                  while (waitKey(1) < 0 && cvGetWindowHandle(window_name))

                  {

                     Mat DD;

                    

                      rs2::frameset frames = pipe.wait_for_frames();

                      

                      rs2::frame depth_frame = frames.get_depth_frame();

                      Mat depth(Size(640, 480), CV_16SC1, (void*)depth_frame.get_data(), Mat::AUTO_STEP);

                      double min, max;

                      minMaxLoc(depth, &min, &max);

                //      depth.convertTo(DD, CV_8UC1, 255.0 / (max - min), -min * 255.0 / (max - min));

                      depth.convertTo(DD, CV_8U, 1, 0);

                      imshow(window_name, DD);

                  }

                  return EXIT_SUCCESS;

              }

              Regarding your question, at this point, Intel does not have plans to develop a depth camera based on time-of-flight technology. The current IR cameras benefit from ambient IR when lights are on so they can see further with good lighting, especially sunlight. When lights are off, they rely purely on the camera's IR projector so the effective distance is not as far.

              Best Regards, 
              Juan N.
              • 4. Re: D435 Dept frame output in OpenCV - how to get rid of the flickering  effect?
                JJR

                Hi Juan,

                 

                Thank you for that. Using your example with slight modification I was able to get what I want, and further MartyG advice me to turn off the auto exposure which helped in  reducing effect of Turning on and off the light.

                Thanks once again

                Best regards

                Johan

                • 5. Re: D435 Dept frame output in OpenCV - how to get rid of the flickering  effect?
                  Intel Corporation
                  This message was posted on behalf of Intel Corporation

                  Hello JJR, 

                  That is Great! I am glad we were able to help you! Thanks for your feedback it was a pleasure helping you. 

                  Best Regards, 
                  Juan N.