1 2 Previous Next 15 Replies Latest reply on Jun 10, 2018 4:43 AM by MartyG

    D435  USB-powered depth camera

    JJR

      I am interested D435 for an application we developed with Kinect.

      1. Can the above said camera work in a completely dark room?
      2. Will depth information get effected by turning on or off the light in the room? – Will the sudden change of light effect depth data at that time
      3. What are kind frames available for extraction?
      4. Can I extract IR frame ?

       

      Will be using C++ . IDE is MS Visual Studio 2015.

      Your early reply on the above will be much appreciated.

      Thanks
      Johan

        • 1. Re: D435  USB-powered depth camera
          MartyG

          1.  RealSense IR sensors require a certain amount of light in order to generate a good quality image.  If the location is too dark then the image will be mostly or completely black.

           

          I ran a test though with a mostly dark room at 7 am (not completely dark) with the lights off and blinds shut with a D415 camera, and still got a decent IR image.

           

          2.  When the lights were turned on, it was a very smooth, instant transition to the stronger lighting conditions.

           

           

          3.  You can extract RGB color, depth and IR.  There are different formats for each of these main stream types.  The link below has a script with a list of image formats on lines 54 to 74

           

          librealsense/rs_sensor.h at 5e73f7bb906a3cbec8ae43e888f182cc56c18692 · IntelRealSense/librealsense · GitHub

           

          4. If you use the RealSense Viewer tool that comes with SDK 2.0 to generate the infrared stream, you can snap a single frame using the SDK 's record and playback functions.  To capture a frame instead of recording a sequence of frames with the Record button, click on the Pause icon to pause the stream, and then click on the Snapshot button next to it to capture that paused frame as a PNG image.

           

          • 2. Re: D435  USB-powered depth camera
            JJR

            Hi, Thank you for your reply.

             

            I will be using OpenCV libraries and C++ to develop my own application. I would like to know is there a C++ sample code to extract IR frame and depth data.

             

            Thanks once again.

             

            Johan

            • 3. Re: D435  USB-powered depth camera
              MartyG

              The quickest method would likely to be to take the Capture sample (which shows color and depth streams) and alter the code to show the IR stream instead of the color one.

               

              librealsense/examples/capture at master · IntelRealSense/librealsense · GitHub

              1 of 1 people found this helpful
              • 4. Re: D435  USB-powered depth camera
                JJR

                Hi,

                 

                Thank you very much that was very helpful.

                 

                Just to give you an idea of the application I am planning to develop;

                Mavis - YouTube

                I did that with Kinect and Kinect is no more and trying to find a alternative.

                 

                Further what is the recommended specs for a PC to use D435?

                 

                Best Regards

                Johan

                • 5. Re: D435  USB-powered depth camera
                  MartyG

                  That's a very cool application! 

                   

                  The D415 and D435 cameras work with any Intel or ARM processor and only need a USB 3.0 port. 

                  1 of 1 people found this helpful
                  • 6. Re: D435  USB-powered depth camera
                    JJR

                    Thanks

                    • 7. Re: D435  USB-powered depth camera
                      JJR

                      HI,

                       

                      Finally, I received my D435 after a long await.

                       

                      At this time I am using windows 10 to develop my prototype application but eventually move to Ubuntu.

                       

                      I installed the latest SDK 2.12.0. I am using MS Visual studio 2015 for compiling the code.

                       

                      Following code compiles without any problem

                       

                      quote

                      // License: Apache 2.0. See LICENSE file in root directory.
                      // Copyright(c) 2017 Intel Corporation. All Rights Reserved.

                      #include <librealsense2/rs.hpp> // Include RealSense Cross Platform API
                      #include <opencv2/opencv.hpp>   // Include OpenCV API

                      int main(int argc, char * argv[]) try
                      {
                      // Declare depth colorizer for pretty visualization of depth data
                      rs2::colorizer color_map;

                      // Declare RealSense pipeline, encapsulating the actual device and sensors
                      rs2::pipeline pipe;
                      // Start streaming with default recommended configuration
                      pipe.start();

                      using namespace cv;
                      const auto window_name = "Display Image";
                      namedWindow(window_name, WINDOW_AUTOSIZE);

                      while (waitKey(1) < 0 && cvGetWindowHandle(window_name))
                      {
                        rs2::frameset data = pipe.wait_for_frames(); // Wait for next set of frames from the camera
                        rs2::frame depth = color_map(data.get_depth_frame());

                        // Query frame size (width and height)
                        const int w = depth.as<rs2::video_frame>().get_width();
                        const int h = depth.as<rs2::video_frame>().get_height();

                        // Create OpenCV matrix of size (w,h) from the colorized depth data
                        Mat image(Size(w, h), CV_8UC3, (void*)depth.get_data(), Mat::AUTO_STEP);

                        // Update the window with new data
                        imshow(window_name, image);
                      }

                      return EXIT_SUCCESS;
                      }
                      catch (const rs2::error & e)
                      {
                      std::cerr << "RealSense error calling " << e.get_failed_function() << "(" << e.get_failed_args() << "):\n    " << e.what() << std::endl;
                      return EXIT_FAILURE;
                      }
                      catch (const std::exception& e)
                      {
                      std::cerr << e.what() << std::endl;
                      return EXIT_FAILURE;
                      }

                      Unquote

                       

                      But when I run the application I am getting following error;

                       

                      API version mismatch: librealsense.so was compiled with API version 2.9.1 but the application was compiled with 2.12.0! Make sure correct version of the library is installed (make install)

                       

                      Please advice.

                       

                      Thanks

                      Johan

                      • 8. Re: D435  USB-powered depth camera
                        JJR

                        HI,

                         

                        Please ignore my question. I worked it out. Coping the realsense2.dll file to application directory fixed the problem.

                        I would like to how can get non colorizer raw depth frame gray scale and also the depth information of x,y coordinates.

                        In OpenNi2 I was able to save depth info from frame to Unit_16 array. Is it possible to do the same?

                         

                        Thanks

                        Johan

                        • 9. Re: D435  USB-powered depth camera
                          MartyG

                          I researched your question carefully but there are no precedents to refer to for a Windows user - the small number of cases involving this error are all based on the Linux version of the SDK.

                           

                          The solutions in those Linux cases could be summed up as:

                           

                          1.  Compile the 2.12.0 SDK from source code instead of using a pre-made executable, if the pre-built SDK is what you are using.

                           

                          Or

                           

                          2.  Download the old 2.9.1 version of the SDK and try your program with that.

                           

                          Release Intel® RealSense™ SDK 2.0 (build 2.9.1) · IntelRealSense/librealsense · GitHub

                          • 10. Re: D435  USB-powered depth camera
                            JJR

                            Hi,

                             

                            Thanks.

                            But I fixed the problem by copying the realsense2.dll to the root directory of the application.

                            Any idea how can get the depth data to array for me to process it, ie to find depth at  a certain x,y.

                             

                            Thanks

                            Johan

                            • 11. Re: D435  USB-powered depth camera
                              MartyG

                              Awesome news that you found a solution to your problem! 

                               

                              The discussion linked to below may be helpful to you. 

                               

                              Depth of a Pixel

                              • 12. Re: D435  USB-powered depth camera
                                JJR

                                Hi,

                                 

                                Thanks for that link. It was very helpful.  I have some more work to do but so far progress well. What I have achieved so far;

                                I will keep you posted.

                                 

                                Thank you once again.

                                • 13. Re: D435  USB-powered depth camera
                                  MartyG

                                  Looks good - thanks for sharing your progress. 

                                  • 14. Re: D435  USB-powered depth camera
                                    JJR

                                    Hi,

                                     

                                    I progressed further, I was able to get the depth data in meters for each frame and process it.

                                    I have posted the sample code how to do it in previous link you provided, so it may be useful for some one.

                                     

                                    I have one challenge to overcome. I am processing the IR frame and it seems to get effected by turning on the light in the room. Kinect IR frame does not get effected by it.

                                    Any advice on this is welcomed.

                                     

                                    Thanks

                                    Johan

                                    1 2 Previous Next