Items with no label

D435 VSYNC Pin

JWare
Novice
6,277 Views

Is it true that external synchronization mode is only supported by the D415? We noticed that there is a 1.8V pulse that appears on the VSYNC pin at the camera framerate. Can we rely on this as a synchronization mechanism for external devices for frames captured on the D435?

If the pulse on the VSYNC pin is tied to the image capture in a reliable and repeatable way, when does it occur relative to image capture? For example, is it at the beginning of the frame capture, at the end, or something else?

Does changing the RS2_OPTION_INTER_CAM_SYNC_MODE option affect the behavior of the VSYNC pin in default vs master mode?

Regards,

-Jake

42 Replies
MartyG
Honored Contributor III
1,615 Views

Here's the long version, if you want to get into the technicalities of the subject ...

Here are the pin-out details from the 400 Series data sheet document for the external sync pins

.I guess you are referring to pin 5, the ZVSYNC?

Earlier this year before the sync function was supported, another user who tried to get useful output from the pins said: "We tried to read out pin 1 from the external sensors sync connector (GVSYNC0) while enabling the "Output Trigger" option in the RealSense viewer but we could only measure a very noisy signal containing shutter noise, and not a clear trigger signal (e.g. 5V TTL)". They were told by an Intel support representative that support in the firmware was required for it to work.

The experiences of the above user suggest that you might be able to get some kind of noisy raw output from a pin but it may not be usable without the firmware support for sync on D435.

The multi-camera sync white paper says that whilst you can have a master and slave setup with the cameras, "Alternatively an external signal generator can also be used as the master trigger with all cameras set to slave mode. When applying an external sync pulse, the HW SYNC input requires a 100 microsecond positive pulse at the nominal camera frame rate, 33.33 ms for a 30Hz frame rate for example".

"Inputs are high impedance, 1.8V CMOS voltage levels. However, it is important to make sure to use a high resolution signal generator. The frequency of the signal generator needs to exactly match the sensor frame rate. For example, if the sensor is set up as 30 fps, the real frame rate may be 30.015 fps. You may need to use an oscilloscope to measure the real frame and configure the signal generator to the same frequency. For this reason, it may be better to just using one additional camera as master sync signal generator"

At the start of 2018, Intel did a demo of four D435s linked together with hardware sync cable to produce 'volumetric capture'. They initiated the start and stop of capture using a commercial grade flash.

https://realsense.intel.com/intel-realsense-volumetric-capture/ Volumetric Capture @ Sundance using Intel RealSense Depth Cameras

The principle of using the flash as a trigger was probably similar to this:

http://www.dslreports.com/forum/r31259855-Syncing-a-camera-and-a-pulsed-flash-light-using-a-TTL-signal http://www.dslreports.com/forum/r31259855-Syncing-a-camera-and-a-pulsed-flash-light-using-a-TTL-signal

0 Kudos
JWare
Novice
1,615 Views

Thanks, Marty. You are right that I am referring to pin 5 of the External Sync Sensor Connector and that I want to get technical. Thanks for providing the additional details and all of your posts on this and other topics related to the realsense cameras. I have read and benefited from many of them. Let me add some more information to clarify.

I'm relatively certain the "other user" you are referring to created https://github.com/IntelRealSense/librealsense/issues/1212 this github issue. Similar to what you said, they claimed that pin 1 was noisy and unusable, but pin 5 seemed to have a usable pulse on it. We confirmed this and have setup a synchronization scheme for another sensor based on it. Note that this is true despite the fact that external synchronization isn't technically supported on the D435. For example, our experience is that checking sensors[DEPTH].supports(rs2_option::RS2_OPTION_INTER_CAM_SYNC_MODE) in the ROS driver returns false. I'm guessing the same would be true of setting it with the method outlined in the https://www.intel.com/content/dam/support/us/en/documents/emerging-technologies/intel-realsense-technology/RealSense_Multiple_Camera_WhitePaper.pdf multiple camera synchronization white paper you referred to. However, the fact remains that there is a 1.8V pulse on pin 5 of the External Sync Sensor Connector that occurs every frame. If nothing else, we just want to know if that pulse is actually tied to a specific event associated with frame capture. Any info on this would be much appreciated.

As an aside, we are using the D435 on a multi-rotor drone, so we would prefer to stick with its global shutter cameras as opposed to switching to the rolling shutter cameras of the D415. We will probably pick up a D415 just to investigate this situation, though.

Thanks again,

-Jake

0 Kudos
MartyG
Honored Contributor III
1,615 Views

Thanks for your kind words.

Whilst I could not find information of a relationship between pin 5 and frame generation, I wonder if such a pulse would manifest somehow in the frame metadata that is generated every frame.

https://github.com/IntelRealSense/librealsense/blob/master/doc/frame_metadata.md librealsense/frame_metadata.md at master · IntelRealSense/librealsense · GitHub

0 Kudos
JWare
Novice
1,615 Views

Marty,

You are right that the frame metadata has useful information in it. We modified the ROS driver to extract and publish it so we could use the realsense's internal timestamps, exposure time, and frame counter. However, it isn't clear how we might use any of that information to figure out if the sync pulse is related to a particular event during frame capture.

Regards,

-Jake

0 Kudos
MartyG
Honored Contributor III
1,615 Views

I believe that a slave camera checks for sync, and it produces unsynced frames if it does not detect a sync signal. so there is apparently checking going on.

RealSense Group CTO Anders said recently: "When you set a camera to slave it will still acquire frames at the right frame rate even if there is no sync input signal. It tries to sync but if there is no signal that comes in at the right time, it gives up and triggers itself."

0 Kudos
idata
Employee
1,615 Views

Do we have any indication for when this may be officially supported for D435? Thanks!

0 Kudos
MartyG
Honored Contributor III
1,615 Views

I have no knowledge of when D435 will be able to initiate sync in the same way D415 does, though apparently D435s can respond to an externally generated sync pulse.

In the absence of an official sync solution for D435, the next easiest approach may be to use a D415 as master sync signal generator and have all the slaves be D435s.

0 Kudos
idata
Employee
1,615 Views

Marty, what version of the Product Data Sheet did you reference above? I'm looking at July 2018 and they're a little bit different. Is there a newer version?

Looks like you were looking at January 2018, actually (update).

Thanks!

0 Kudos
MartyG
Honored Contributor III
1,615 Views

Yes, there were some changes across editions. In the March 2018 edition onward, for example, the section relating to adding a tracking module component to the camera module was removed because the tracking module was discontinued by Intel, and they decided not to enable use of the 50 pin tracking module connector for custom components.

There were also separate editions published, one at the camera's launch that covered only the D415 and D435 USB cameras, and a larger one that covered all the modules in the 400 Series family. The pin-out details quoted above can be found on page 46 of the most recent July 2018 edition of the larger publication that covers all modules.

https://www.intel.com/content/dam/support/us/en/documents/emerging-technologies/intel-realsense-technology/Intel-RealSense-D400-Series-Datasheet.pdf https://www.intel.com/content/dam/support/us/en/documents/emerging-technologies/intel-realsense-technology/Intel-RealSen

However, I checked the layout of the information and you were correct - the images were taken from page 56 of the January 2018 edition. I could confirm this because the details are split across 2 pages in the January edition, and all on one page in the July one, and I remember copy-and-pasting from the two pages.

There are some descriptive differences in the most recent July 2018 pin-out. The main difference is that the chart no longer refers to the pins providing external sensor interrupts for the timestamp.

Here's January's one:

And here's the updated July one:

0 Kudos
idata
Employee
1,615 Views

Marty, with a sync cable, is it possible to trigger the synchronization to be ON in the Realsense viewer? Or is custom software needed?

0 Kudos
MartyG
Honored Contributor III
1,615 Views

I do not have two 400 Series cameras so I cannot test sync myself. I would speculate though that once you add more than one camera to the RealSense Viewer via the Add Source option then the Sync button would un-ghost and be able to be used to trigger sync, with whatever of the cable-linked cameras that is set to Master being the one that provides the sync pulse. The master camera should be a D415. My understanding is that a D435 could be a slave but cannot currently act as the master signal generator, and that an all-D435 setup needs an external trigger signal

You should also be using at least SDK 2.13.0 (when sync was added to the SDK) and firmware 5.9.14 (when sync was added to firmware). You need sync support in both the SDK and firmware for it to work.

Of course, you should also have the sync cabling connected up before initiating sync.

I believe that when hardware sync is working correctly, you should no longer get the red System Time message (which is software sync) when clicking the info button on a stream's toolbar.

0 Kudos
MartyG
Honored Contributor III
1,615 Views

You may also be interested in a couple of multi-camera scripts that have been made available.

0 Kudos
idata
Employee
1,399 Views

Hi mzahana,

 

 

Currently, the RealSense ROS wrapper does not support external triggering for multicamera sync. The engineers are working on this capability. We do not have a timeline for when it will be available.

 

 

Regards,

 

Alexandra
0 Kudos
Wgao6
Novice
1,399 Views

Hi Marty,

When D435 works in master mode, what are signals on Pin 4 (External Projector, laser PWM) and Pin 5 (ZVSYNC) relative to the actual optical output of the projector? is the following correct?

what we see is only projector optical output, but we do not see any signal on pin 5 and pin 4.

when we set to default mode, we do see the Pin 5 signal following the VCSEL optical PWM output.

Our objective is to connect the cameras as follows and synchronize with our external projector

0 Kudos
MartyG
Honored Contributor III
1,399 Views

I believe Pin 5 is the 'sync trigger' pin and so contains the most usable data of all the pins.

I don't think that the projector actually has much to do with sync. Sync is usually a color or depth sync (D435 is depth sync only). What the projector - whether the internal RealSense one or an external projector - does is project a random dot pattern onto objects that can add texture to a scene to aid the camera in depth-analyzing the room. It is not vital to the sync process though.

The July version of the data sheet indicates that Pin 4 has a dual role. It is used for laser PWM, and also an external sensor interrupt for time-stamping.

0 Kudos
Wgao6
Novice
1,399 Views

Marty,

Thanks.

We do not use the Color camera. basically it is D430.

For multiple camera synchronization, how do we do external laser projector?

Could you give us an hardware diagram of 3 cameras and 3 external pattern projectors?

Thanks

Wen

0 Kudos
MartyG
Honored Contributor III
1,399 Views

As far as I know, you don't need any special wiring to use external projectors in a multiple camera setup, and should just point the projectors towards the area where you think additional texture may be needed (such as plain objects like walls and doors). The more dots you project into the area, the more texture it will have for the camera to analyze. You can also move external projectors around and shake them without affecting the scene much.

So I do not have any diagrams of an external projector in a multi-cam setup. The link below to an illustrated use of an external LED projector with a single camera may be of interest to you though.

0 Kudos
Wgao6
Novice
1,399 Views

so master/slave mode multiple camera situation.

the synchronization between master camera and slave camera is through hardware or software (my guess the pin 5 is used in this case)?

What does the pin 5 waveform looks like relative to the internal laser output ?

0 Kudos
MartyG
Honored Contributor III
1,399 Views

Once you have connected the cameras together with custom-built sync cabling, you can control the sync process with scripting. The link below has some example scripts that allow individual cameras in a multi cam setup to be accessed.

https://github.com/IntelRealSense/librealsense/issues/2219# issuecomment-412348293 Access & control any devices in a multicam setup · Issue # 2219 · IntelRealSense/librealsense · GitHub

If your sync cables are going to be 3 meters or longer, you should build electro static discharge (ESD) components into the cable, or the camera counters can reset if an ESD charge occurs. The multi camera white paper has an electronics diagram for this.

An oscilloscope could likely give you a visual reading of what pin 5 is doing.

0 Kudos
Wgao6
Novice
1,399 Views

So when I have the following setting, what are the Pin 4 and Pin 5 waveforms supposed to be?

Camera A: Master mode, output trigger OFF, Laser Emitter: Enabled

Camera B: Slave mode, Output Trigger OFF, Laser emitter: enabled

Connection between them is Pin 5 (ZVsync) and Pin 9 (GND)

0 Kudos
Reply