9 Replies Latest reply on Oct 14, 2018 12:33 AM by MartyG

    How can you find multiple objects using "SR300"?

    NSK

      I want to find several objects. So I actually found a 3D object using "The Metaio toolbox". However, there is a problem with finding multiple objects that is not well recognized. How can you find multiple objects using "SR300"?

        • 1. Re: How can you find multiple objects using "SR300"?
          MartyG

          In the old 2016 R2 SDK, the SR300 could do object tracking, but object recognition was only for the R200 model of camera.

           

          The current RealSense SDK 2.0, which is compatible with the SR300, can do object recognition though with the help of the OpenCV software.  Here's a couple of example programs.

           

          librealsense/wrappers/opencv/dnn at master · IntelRealSense/librealsense · GitHub

           

          GitHub - twMr7/rscvdnn: Test program for OpenCV DNN object detection with RealSense camera

          • 2. Re: How can you find multiple objects using "SR300"?
            NSK

            I'm doing something similar to the example "object tracking." But in my program, it's well recognized for a single object. However, if you have multiple objects to recognize, it is not well recognized.

             

            The parts that represent the objects being tracked are:

            private void UpdateObjectTraking()

                    {

                        // 추적하고 있는 객체를 표시.

                        foreach (int id in targetIds1)

                        {

                            PXCMTracker.TrackingValues trackData;

                            var sts = tracker.QueryTrackingValues(id, out trackData);

                            if (sts < pxcmStatus.PXCM_STATUS_NO_ERROR)

                            {

                                continue;

                            }

                           

                            if (PXCMTracker.IsTracking(trackData.state))

                            {

                                if (trackData.cosID == 1)

                                {

                                    ShowTrackingValue(trackData);  //ShowTrackingValue, it's simply the part where the image is displayed.

                                }

                            }

                        }

             

             

                        foreach (int id in targetIds2)

                        {

                            PXCMTracker.TrackingValues trackData;

                            var sts = tracker.QueryTrackingValues(id, out trackData);

                            if (sts < pxcmStatus.PXCM_STATUS_NO_ERROR)

                            {

                                continue;

                            }

             

             

                            if (PXCMTracker.IsTracking(trackData.state))

                            {

                                if (trackData.cosID == 2)

                                {

                                    ShowTrackingValue(trackData);

                                }

                            }

                            //else targetBottle = false;

                           

                        }

                    }

             

             

             

            Can I get some advice?

            • 3. Re: How can you find multiple objects using "SR300"?
              MartyG

              Using your SR300 camera with the OpenCV software is your best chance of achieving multiple object recognition, unfortunately.  The object recognition mode in the '2016 R2' SDK was classed as being a 'Preview' state, meaning that it was an early-stage prototype that was not developed far.

              • 4. Re: How can you find multiple objects using "SR300"?
                NSK

                So how do we find Multiple targets in the following example? Don't you know about this?

                • 6. Re: How can you find multiple objects using "SR300"?
                  MartyG

                  In that sample program, you provide the program with image files to teach it what to look for, and the program then looks for similar-looking areas in the camera's view and highlights them.

                   

                  So technically, according to the documentation for the program, it can highlight two different objects at the same time if you loaded in two different reference images, and the program can see elements in the camera view that look similar to both of these images.

                   

                  I do not know if it can recognize two similar-looking examples of the same object, though it probably can.  Give it a try. 

                   

                  • 7. Re: How can you find multiple objects using "SR300"?
                    NSK

                    Thank you. We actually configured the program, and we found that the recognition rate has dropped significantly. Is there a way to raise awareness?

                    • 8. Re: How can you find multiple objects using "SR300"?
                      NSK

                      And it's not well recognized by using the background in bright green and then turning it dark brown. Does the background shade affect you?

                      • 9. Re: How can you find multiple objects using "SR300"?
                        MartyG

                        I cannot see anything in the options to affect tracking performance.  It is my experience with older RealSense models such as the F200 and Sr300 though that it is more likely to recognize and track objects that are in the center of the camera's view.   Tracking may be lost with objects that are off to the side of the camera view, because those older moels have a smaller field of view (FOV) than the current 400 Series cameras.

                         

                        Regarding the problems when changing background color: there is a known problem with depth cameras of any type that dark gray or black surfaces are hard for the camera to see, because the laser's light gets absorbed by it.  .Maybe you are experiencing a similar absorption with dark brown.