1 Reply Latest reply on Sep 19, 2018 7:50 AM by Intel Corporation

    Problem with the Realsense SDK 2.0 and OpenCV

    Honk5000

      Hello

      I'm not sure if the problem comes from the Intel Realsense library or from OpenCV:

      I have a simple program that just gets a RGB image from the Intel Realsense D435, converts the image to BGR with the OpenCV function cvtColor and shows it with OpenCVs imshow.

       

      This works until I activate the depth camera and the RGB camera of the Intel Realsense D435 at the same time. Then I get some frames where it seems like the conversion of the RGB image was not done by OpenCV. So red appears blue and vice versa. This happens in irregular intervals.

       

      I based the code on the OpenCV imshow example from github.

      I'm using the newest OpenCV version 3.4.3 and Intel Realsense SDK 2.0 version 2.16.0.

       

      I also posted this question on answers.opencv.org because I'm not shure if this is a problem with Intel Realsense or OpenCV.

       

      Here is my code:

       

      #include <librealsense2/rs.hpp> // Include RealSense Cross Platform API
      #include <opencv2/opencv.hpp>   // Include OpenCV API

      int main(int argc, char * argv[]) try
      {
        
      // Declare RealSense pipeline, encapsulating the actual device and sensors
        rs2
      ::pipeline pipe;

        rs2
      ::config cfg;

        
      //depth stream config
        cfg
      .enable_stream(RS2_STREAM_DEPTH, 1280, 720, RS2_FORMAT_Z16, 30);

        
      //colour stream config
        cfg
      .enable_stream(RS2_STREAM_COLOR, 1280, 720, RS2_FORMAT_RGB8, 30);

        
      // Start streaming
        pipe
      .start(cfg);

        
      const auto window_name = "Display Image";
        cv
      ::namedWindow(window_name, cv::WINDOW_AUTOSIZE);

        
      while (cv::waitKey(1) < 0 && cvGetWindowHandle(window_name))
        
      {
        rs2
      ::frameset data = pipe.wait_for_frames(); // Wait for next set of frames from the camera
        rs2
      ::frame colour = data.get_color_frame();

        
      // Query frame size (width and height)
        
      const int w = colour.as<rs2::video_frame>().get_width();
        
      const int h = colour.as<rs2::video_frame>().get_height();

        
      // Create OpenCV matrix of size (w,h) from the colorized depth data
        cv
      ::Mat image(cv::Size(w, h), CV_8UC3, (void*)colour.get_data(), cv::Mat::AUTO_STEP);
        
      //convert from RGB to BGR
        cv
      ::cvtColor(image, image, cv::COLOR_RGB2BGR);

        
      // Update the window with new data
        imshow
      (window_name, image);
        
      }

        
      return EXIT_SUCCESS;
      }
      catch (const rs2::error & e)
      {
        std
      ::cerr << "RealSense error calling " << e.get_failed_function() << "(" << e.get_failed_args() << "):\n " << e.what() << std::endl;
        
      return EXIT_FAILURE;
      }
      catch (const std::exception& e)
      {
        std
      ::cerr << e.what() << std::endl;
        
      return EXIT_FAILURE;
      }

       

       

        • 1. Re: Problem with the Realsense SDK 2.0 and OpenCV
          Intel Corporation
          This message was posted on behalf of Intel Corporation

          Hello Honk5000,

          Thank you for your interest in the Intel RealSense D435 camera. 

          The problem we have noticed is that the source Mat and the destination Mat are the same
          cv::cvtColor(image, image, cv::COLOR_RGB2BGR);

          Change the code as below and it will work: 
           

                 cv::Mat image(cv::Size(w, h), CV_8UC3, (void*)colour.get_data(), cv::Mat::AUTO_STEP);

                 cv::Mat dst; //added this

                  //convert from RGB to BGR

                  cv::cvtColor(image, dst, cv::COLOR_RGB2BGR); //changed destination matrix to dst

                  // Update the window with new data

                  imshow(window_name, dst); //show the new destination matrix

          Best regards,
          Eliza

          1 of 1 people found this helpful