1 2 Previous Next 21 Replies Latest reply on Sep 14, 2017 1:20 PM by Pototo

    Many SR300, One IR light, While Looking at Same Object (i.e., Use Two SR300, And One IR Light)

    Pototo

      If I have two SR300 looking at the same object, how can I make one realsense (with Laser Power set to Zero) use the IR light of the other SR300? I tried it, but I cannot see the depth image in of the SR300 that has Laser Power equal zero. Is there a way to do this?

        • 1. Re: Many SR300, One IR light, While Looking at Same Object (i.e., Use Two SR300, And One IR Light)
          MartyG

          I am not sure from your description what the goal of your project is.  Are you trying to take scans of an object from different sides of it, with the SR300 cameras arranged around it?

           

          If this is the case then the approach I would use would be to cycle through the cameras with ID numbers.  You would make one camera active and capture an image, then deactivate that camera and make the next camera in the sequence activate and capture an image.  when the last camera is reached, the cycle gets back to the first camera again.  E.g if you had 4 cameras, 1 - 2 - 3 - 4 -1.

           

          Once you have taken a series of captures from different sides, you can stitch the separate images together into a single 360 degree model of the object using 3D modeling software.

           

          If what I am suggesting is different from what you are trying to do with your project, please explain your project more.  Thanks!

          • 2. Re: Many SR300, One IR light, While Looking at Same Object (i.e., Use Two SR300, And One IR Light)
            Pototo

            MartyG,

             

             

            I am recording videos of the same object with two cameras at the same time at 30FPS (two streams per cameras). All cameras are synchronized. I tried deactivating a camera when the other one is running, but the cameras don't react fast enough, and the videos don't look good...and frames are skipped.

            I am running a while loop that captures two frames (one IR, one Depth) from each camera:

             

            while (true) {

                  for(auto dev : devices) {

                    /// Wait for new images, otherwise we may get empty frames

                    if(dev->is_streaming()) {

                      dev->wait_for_frames();

                    }

                    dev->poll_for_frames();

                    ///***************** DEPTH *******************************************

                    image = cv::Mat(cv::Size(width, height), CV_16UC1,

                      (void*)dev->get_frame_data(rs::stream::depth), cv::Mat::AUTO_STEP);

            ...

             

            the image is then either displayed or stored in a video file.

             

            There is interference, because I see a wave of lines on both the IR and Depth videos of each camera.

            I tried turning off the laser of the camera after I was done grabbing a frame, but that made things worst (because it takes time to turn it back on, etc.)

            • 3. Re: Many SR300, One IR light, While Looking at Same Object (i.e., Use Two SR300, And One IR Light)
              MartyG

              Yes, I thought that there may be interference between cameras.  The newer ZR300 model of camera can use multiple cameras without interference, but the SR300 does not have this.  It is not an easy problem to solve.  In previous cases, I have suggested placing some kind of electromagnetic shielding around the cameras to try to isolate their signals from each other.

               

              Electromagnetic shielding - Wikipedia

               

              A cheap example of this is smartphone covers that have an inner 'Faraday' lining (named after the signal-blocking Faraday Cage) to shield a device from wireless hacking or reduce the body's exposure to EM signals.  You can find examples of this on stores such as Amazon for $10 to 20 by searching for 'faraday phone'.

               

              Someone else asked whether they could put an external shutter on the outside of their camera, presumably blocking the laser when the shutter was closed.  A shutter might be able to react faster than turning a camera on and off.

               

              Another approach that someone tried was to deactivate the camera's Sense Manager and then re-activate it.  This has a time delay involved too though regarding the time it takes for the Sense Manager to re-activate.

              • 4. Re: Many SR300, One IR light, While Looking at Same Object (i.e., Use Two SR300, And One IR Light)
                AikenControls

                We have applications with 6+ SR300s running to take snapshots of objects. These are mounted overhead and are arranged lineally or in row/cols to cover large areas.

                 

                Basically we have each SR300 connected to a NUC i3 with a capture application listening for a "Capture" request on a UDP socket. When the master application sends the request the "odd" cameras take the snapshot, power-off the laser (power=0), then sleep for a predetermined time. The "even" cameras at the same time wake up, wait a small time, start the laser and capture.. then go back to sleep.

                 

                By overlapping the times precisely we achieve a less than 100ms capture time. BUT, this would never work for streaming applications! (If you interested look here - nDepth 3D Scan - AikenControls )

                 

                As Marty suggested we have played with external shutters.. and you can do it.. But it's ugly and expensive..

                 

                It would be SO NICE if there was some external blanking signal for the laser.. Leave the projector vibrating but have a way just to externally turn off the laser (quickly).

                Guess we can dream?

                 

                I just wonder what is changing on the D4xx cameras. Does anyone (Intel?) have information to share for those who have products centered around your technologies?

                • 5. Re: Many SR300, One IR light, While Looking at Same Object (i.e., Use Two SR300, And One IR Light)
                  AikenControls

                  Marty,

                   

                  If I understand the "interference" is the IR pattern corruption (for depth) and the IR overlap (for IR capture).. I don't think it is electrical noise/crosstalk. So I don't the shielding would help out much.

                   

                  The IR projector is very slow to start(really)/stop(not so much).. but the power = 0 is relatively fast to enact.  Still TOO SLOW for video, but better than stopping the projector itself..

                  From what we understand (and see) the projector is a resonance type.. so.. it likes to stay running.. or you REALLY pay a time penalty! I believe that the firmware wants to allow time for it to stabilize before taking depth measurements.

                   

                  And btw.. Thanks for being so active on the forum! I wish I had the time to spend to help out more.  We have lots of F200s, SR300s, and a few ZR300s around if ever there is a need for some specific information that we can test quickly.

                   

                  Chris

                  • 6. Re: Many SR300, One IR light, While Looking at Same Object (i.e., Use Two SR300, And One IR Light)
                    MartyG

                    Thanks muchly, Chris.  I know about EM signal interference, but not so much about pattern interference, so it's always good to have expert voices contributing to add to the community's knowledge-base. 

                     

                    I'm reminded of the Ghostbusters and their proton guns ... "Don't cross the streams!" 

                     

                    Back in 2014, a user cut open a camera's USB cable and spliced in a cable that provided an extra 5v of power for increased reliability (this was how it was discovered that mains-powered hubs solve a lot of camera detection / connection problems).  Maybe it would be possible to splice in a circuit that produces resistance (cutting the power to the camera) and then drops the resistance and restores power again.

                     

                    I once designed a mechanism where a cable would have a gap in it with an electrical terminal pad on each end.  In the gap would be a powered spinning rotor that delivers power to the cable terminals when its rotation took the rotor past the contacts and then cut the power when the rotor's angle moved past the contacts.

                     

                    But that would probably turn off the projector too when the power cut out, so maybe not so useful.

                     

                    I wonder .... if the projector uses less power than the laser, maybe the resistor value could drop enough to disable the laser but still provide enough to keep the projector warm.  I'm just designing off the top of my head here (I have a product engineering degree). 

                     

                    There isn't any publicly available information on the D-series cameras yet other than what is on their pre-release promo page.  I guess you'll be buying some to add to your impressive RealSense camera inventory. 

                    • 7. Re: Many SR300, One IR light, While Looking at Same Object (i.e., Use Two SR300, And One IR Light)
                      AikenControls

                      I think I misunderstood what Pototo was asking! (Happens a lot now that I am getting old.. my wife and employees have confirmed this often.. grin)

                       

                      If you turned off the laser from one sensor.. and then used the other only.. maybe that would work for IR capture. In theory you could even use an external IR source.

                       

                      But, for depth, I would seriously doubt you can use the structured light from one sensor for the other. I am almost sure there is a critical relationship between the projector and capture on the ASIC.

                      If, for example, they are on top, and looking at the pattern from the other sensor (on top).. then I bet you are back at some timing issue from the ASICs.. The projector of one is not synchronized with the capture of the other..

                      If the sensors were opposed.. I don't think you have a chance since the principle is to detect the distorted/translated by the object under inspection...The object would simply obscure the pattern completely.

                       

                      Yep.. crossing streams is bad.. but who you gonna call?

                       

                      For "browning-out" the camera.. I doubt that would be reliable in the end. Probably every camera would have it's own "magically" resistance.. and you would probably cook something due to overheating (as you drop the voltage, the current would increase).

                       

                      It would be NICE to have a hardware blanking and trigger signal for the sensors.. It would open up a world of new possibilities.  Especially for commercial applications where more than one camera is needed. Maybe we need to find the hardware guys and buy them a nice dinner? (Hardware guys.. really.. the offer stands).

                       

                      We have already signed up for the notice on the D4XX kits.. And you are right.. we will add to our collection!

                       

                      We have invested a lot in these sensors.. Have developed our own external calibration stand for sensor specific calibrations (for distortion correction of the depth and color images).. We generate a correction file for each sensor.. it is amazing what you can gain..

                       

                      Now..the quest continues (again.. F200, SR300.. now D4xx)!

                       

                      Thanks for guiding the conversations and the efforts.. if we can help out somewhere, feel free to message us. 

                      • 8. Re: Many SR300, One IR light, While Looking at Same Object (i.e., Use Two SR300, And One IR Light)
                        MartyG

                        Thanks too for your kind words. 

                         

                        One more idea I had ... the HTC Vive's sensors use a rracking box called Lighthouse where the laser is spun round at 1000+ RPM.  Maybe that is adaptable in this case ... if The RealSense camera were spun round fast on a turntable then its laser would always be on but it would be pointing away from the object for half of the time (potentially stopping the streams from crossing during those periods).

                         

                        Spinning a camera on a turntable like this would be physically tricky because of the potential for the cable getting wound up.  A way around this may be to have a hole in the center of the turntable's base and thread the lead through that, so the cable is spinning round on the spot instead of winding round the turntable's stand.

                         

                        There is also the matter of whether the camera's FPS could cope with such rotation.  The 90 FPS on the new D-series cameras might be better suited.

                         

                        If the scanning area was a white box surrounding the object then it might reduce the potential for capturing bad data when the camera faces away from the object, as the depth would read as '0' when it could only see the white surface, as there'd be nothing for ir to lock onto.

                        • 9. Re: Many SR300, One IR light, While Looking at Same Object (i.e., Use Two SR300, And One IR Light)
                          AikenControls

                          Interesting.. The cable would be a trouble.. but there are sliprings.. but rated for USB 3.0? that is the sneaky part. (see here - https://www.adafruit.com/product/736?gclid=CjwKCAjww9_MBRAWEiwAlaMJZsqOObasCBP7DVvSy2BEY92ARoHhhDcpS66yZGUivFpPMOsGxodmE… )

                           

                          Early on there was a "Shake and Sense" approach that vibrated the sensors with the Primesense sensors (Kinect V1)..

                          Check this out.. Shake n Sense Makes Kinects Work Together!

                           

                          We never tried it.. It looked sort of brutal.. but maybe it is worth an investigation?

                          • 10. Re: Many SR300, One IR light, While Looking at Same Object (i.e., Use Two SR300, And One IR Light)
                            MartyG

                            By googling for 'slip ring usb 3.0' I saw a couple that claimed to work with USB 3.  Example:

                             

                            USB slip ring / capsule / modular - USB/power/signal capsule slip ring - JINPAT Electronics Co., Ltd.

                             

                            Yeah, I believe they changed the projection method on Kinect 2?  Might be worth testing it with an F200 though just in case it breaks it!  The Japanese build shock absorbers under their buildings to dampen earthquake motion.

                            • 11. Re: Many SR300, One IR light, While Looking at Same Object (i.e., Use Two SR300, And One IR Light)
                              AikenControls

                              I have bookmarked the slip ring.. that is nice.. 1GB Ethernet as well! Could come in handy one day.

                               

                              The F200 and the Primesense were really close.. We have the Kinect V2.. but we abandon it earlier on since it was big and required external power. It was really hard to repackage. And they did change the approach. It is a "Time of Flight" technology, not structured light.

                               

                              For F200 and SR300 it is still structured light..But I am not sure if this approach could work. It was magic even for the Primesense sensor.

                              For the D400s it's anyone's guess since we don't know as yet the technology inside.

                               

                              I really wish Intel had a DEVELOPERS PROGRAM for Realsense. A real one.. under NDA if needed.. and not free. We would pay for it.

                              That way we could have a way to get a view of what's coming and be prepared, know timelines, get attention if there is a real tech trouble.. etc..

                               

                              Thanks again!

                              • 12. Re: Many SR300, One IR light, While Looking at Same Object (i.e., Use Two SR300, And One IR Light)
                                MartyG

                                Chris, just wanted to add regarding your wish for advanced info that a good place to follow for this is Intel's press Newsroom.  The D-cameras were first mentioned there last August, for example, when they were known as the R400.

                                 

                                Intel Newsroom | Intel Official News and Information

                                • 13. Re: Many SR300, One IR light, While Looking at Same Object (i.e., Use Two SR300, And One IR Light)
                                  Pototo

                                  Do you have any sample videos and images of ZR300 cameras looking at the same object and displaying the depth images?

                                  • 14. Re: Many SR300, One IR light, While Looking at Same Object (i.e., Use Two SR300, And One IR Light)
                                    AikenControls

                                    Pototo.. I don't think we do. These are only support under Linux we haven't played with them much as yet.

                                    I will check with my developers and see if anyone has played with them at home.

                                    1 2 Previous Next