1 2 Previous Next 26 Replies Latest reply on Oct 11, 2018 9:33 AM by Intel Corporation

    D435 VSYNC Pin

    jakeware

      Is it true that external synchronization mode is only supported by the D415?  We noticed that there is a 1.8V pulse that appears on the VSYNC pin at the camera framerate.  Can we rely on this as a synchronization mechanism for external devices for frames captured on the D435?

       

      If the pulse on the VSYNC pin is tied to the image capture in a reliable and repeatable way, when does it occur relative to image capture?  For example, is it at the beginning of the frame capture, at the end, or something else?

       

      Does changing the RS2_OPTION_INTER_CAM_SYNC_MODE option affect the behavior of the VSYNC pin in default vs master mode?

       

      Regards,

      -Jake

        • 1. Re: D435 VSYNC Pin
          MartyG

          The short version is that the official position remains that sync is only supported for D415 at present.

           

          Here's the long version, if you want to get into the technicalities of the subject ... 

           

          Here are the pin-out details from the 400 Series data sheet document for the external sync pins

          .I guess you are referring to pin 5, the ZVSYNC?

           

          Earlier this year before the sync function was supported, another user who tried to get useful output from the pins said: "We tried to read out pin 1 from the external sensors sync connector (GVSYNC0) while enabling the "Output Trigger" option in the RealSense viewer but we could only measure a very noisy signal containing shutter noise, and not a clear trigger signal (e.g. 5V TTL)".  They were told by an Intel support representative that support in the firmware was required for it to work.

           

          The experiences of the above user suggest that you might be able to get some kind of noisy raw output from a pin but it may not be usable without the firmware support for sync on D435.

           

          The multi-camera sync white paper says that whilst you can have a master and slave setup with the cameras, "Alternatively an external signal generator can also be used as the master trigger with all cameras set to slave mode.  When applying an external sync pulse, the HW SYNC input requires a 100 microsecond positive pulse at the nominal camera frame rate, 33.33 ms for a 30Hz frame rate for example".

           

          "Inputs are high impedance, 1.8V CMOS voltage levels.  However, it is important to make sure to use a high resolution signal generator.  The frequency of the signal generator needs to exactly match the sensor frame rate.  For example, if the sensor is set up as 30 fps, the real frame rate may be 30.015 fps.  You may need to use an oscilloscope to measure the real frame and configure the signal generator to the same frequency.  For this reason, it may be better to just using one additional camera as master sync signal generator"

           

          At the start of 2018, Intel did a demo of four D435s linked together with hardware sync cable to produce 'volumetric capture'.  They initiated the start and stop of capture using a commercial grade flash.

           

          Volumetric Capture @ Sundance using Intel RealSense Depth Cameras

           

          The principle of using the flash as a trigger was probably similar to this:

           

          http://www.dslreports.com/forum/r31259855-Syncing-a-camera-and-a-pulsed-flash-light-using-a-TTL-signal

          • 2. Re: D435 VSYNC Pin
            jakeware

            Thanks, Marty.  You are right that I am referring to pin 5 of the External Sync Sensor Connector and that I want to get technical.  Thanks for providing the additional details and all of your posts on this and other topics related to the realsense cameras.  I have read and benefited from many of them.  Let me add some more information to clarify.

             

            I'm relatively certain the "other user" you are referring to created this github issue.  Similar to what you said, they claimed that pin 1 was noisy and unusable, but pin 5 seemed to have a usable pulse on it.  We confirmed this and have setup a synchronization scheme for another sensor based on it.  Note that this is true despite the fact that external synchronization isn't technically supported on the D435.  For example, our experience is that checking sensors[DEPTH].supports(rs2_option::RS2_OPTION_INTER_CAM_SYNC_MODE) in the ROS driver returns false.  I'm guessing the same would be true of setting it with the method outlined in the multiple camera synchronization white paper you referred to.  However, the fact remains that there is a 1.8V pulse on pin 5 of the External Sync Sensor Connector that occurs every frame.  If nothing else, we just want to know if that pulse is actually tied to a specific event associated with frame capture.  Any info on this would be much appreciated.

             

            As an aside, we are using the D435 on a multi-rotor drone, so we would prefer to stick with its global shutter cameras as opposed to switching to the rolling shutter cameras of the D415.  We will probably pick up a D415 just to investigate this situation, though.

             

            Thanks again,

            -Jake

            • 3. Re: D435 VSYNC Pin
              MartyG

              Thanks for your kind words.  

               

              Whilst I could not find information of a relationship between pin 5 and frame generation, I wonder if such a pulse would manifest somehow in the frame metadata that is generated every frame.

               

              librealsense/frame_metadata.md at master · IntelRealSense/librealsense · GitHub

              • 4. Re: D435 VSYNC Pin
                jakeware

                Marty,

                 

                You are right that the frame metadata has useful information in it.  We modified the ROS driver to extract and publish it so we could use the realsense's internal timestamps, exposure time, and frame counter.  However, it isn't clear how we might use any of that information to figure out if the sync pulse is related to a particular event during frame capture.

                 

                Regards,

                -Jake

                • 5. Re: D435 VSYNC Pin
                  MartyG

                  I believe that a slave camera checks for sync, and it produces unsynced frames if it does not detect a sync signal. so there is apparently checking going on.

                   

                  RealSense Group CTO Anders said recently: "When you set a camera to slave it will still acquire frames at the right frame rate even if there is no sync input signal.  It tries to sync but if there is no signal that comes in at the right time, it gives up and triggers itself."

                  • 6. Re: D435 VSYNC Pin
                    SMM77

                    Do we have any indication for when this may be officially supported for D435? Thanks!

                    • 7. Re: D435 VSYNC Pin
                      MartyG

                      I have no knowledge of when D435 will be able to initiate sync in the same way D415 does, though apparently D435s can respond to an externally generated sync pulse.

                       

                      In the absence of an official sync solution for D435, the next easiest approach may be to use a D415 as master sync signal generator and have all the slaves be D435s.

                      • 8. Re: D435 VSYNC Pin
                        SMM77

                        Marty, what version of the Product Data Sheet did you reference above? I'm looking at July 2018 and they're a little bit different. Is there a newer version?

                         

                        Looks like you were looking at January 2018, actually (update).

                         

                         

                        Thanks!

                        • 9. Re: D435 VSYNC Pin
                          MartyG

                          Yes, there were some changes across editions.  In the March 2018 edition onward, for example, the section relating to adding a tracking module component to the camera module was removed because the tracking module was discontinued by Intel, and they decided not to enable use of the 50 pin tracking module connector for custom components.

                           

                          Customizations and additions to Intel® RealSense™ D400 Series Modules are not supported

                           

                          There were also separate editions published, one at the camera's launch that covered only the D415 and D435 USB cameras, and a larger one that covered all the modules in the 400 Series family.  The pin-out details quoted above can be found on page 46 of the most recent July 2018 edition of the larger publication that covers all modules.

                           

                          https://www.intel.com/content/dam/support/us/en/documents/emerging-technologies/intel-realsense-technology/Intel-RealSen…

                           

                          However, I checked the layout of the information and you were correct - the images were taken from page 56 of the January 2018 edition.  I could confirm this because the details are split across 2 pages in the January edition, and all on one page in the July one, and I remember copy-and-pasting from the two pages. 

                           

                          There are some descriptive differences in the most recent July 2018 pin-out.  The main difference is that the chart no longer refers to the pins providing external sensor interrupts for the timestamp.

                           

                          Here's January's one:

                          And here's the updated July one:

                          • 10. Re: D435 VSYNC Pin
                            SMM77

                            Marty, with a sync cable, is it possible to trigger the synchronization to be ON in the Realsense viewer? Or is custom software needed?

                            • 11. Re: D435 VSYNC Pin
                              MartyG

                              I do not have two 400 Series cameras so I cannot test sync myself.  I would speculate though that once you add more than one camera to the RealSense Viewer via the Add Source option then the Sync button would un-ghost and be able to be  used to trigger sync, with whatever of the cable-linked cameras that is set to Master being the one that provides the sync pulse.  The master camera should be a D415.  My understanding is that a D435 could be a slave but cannot currently act as the master signal generator, and that an all-D435 setup needs an external trigger signal

                               

                              You should also be using at least SDK 2.13.0 (when sync was added to the SDK) and firmware 5.9.14 (when sync was added to firmware).  You need sync support in both the SDK and firmware for it to work.

                               

                              Of course, you should also have the sync cabling connected up before initiating sync. 

                               

                               

                              I believe that when hardware sync is working correctly, you should no longer get the red System Time message (which is software sync) when clicking the info button on a stream's toolbar.

                               

                              • 12. Re: D435 VSYNC Pin
                                MartyG

                                You may also be interested in a couple of multi-camera scripts that have been made available.

                                 

                                RealSense multi-camera access and control scripts

                                • 13. Re: D435 VSYNC Pin
                                  srizvi

                                  Hi jakeware,

                                   

                                  Were you able to get your questions answered, or did you need further assistance?


                                  Please let us know!

                                   

                                  Regards,

                                  Intel Customer Support

                                  • 14. Re: D435 VSYNC Pin
                                    jakeware

                                    srizvi, thanks for following up on this thread.  My questions has not yet been answered.  For clarity, I will restate the the most important and relevant subset of my questions below.

                                     

                                    When you enable the output trigger on the D435, there is a 1.8V pulse that appears at the camera framerate on pin 5 (VSYNC) of the External Sensor Sync Connector pin.  Is this pulse on the VSYNC pin tied to image capture in a reliable and repeatable way?  If so, when does it occur relative to image capture?  For example, is it at the beginning of the frame capture, at the end, or something else?

                                     

                                    Thanks again for your help,

                                    -Jake

                                    1 2 Previous Next