D415/435; SDK 2.14; Setting focus to a specific region of interest

[MRobi5]

I have a D435 mounted on its side with the sensor side at an angle about 2m above the floor and am using the depth sensor to trigger photographs when an object comes within a certain range of a specified point within the depth image. I need the camera to be on its side because I want a portrait image (due to the height of some of the objects I want to photograph combined with the proximity to where the camera is mounted).

Currently, I am getting pictures via the SDK, but the main region of interest is what would normally be the left half of the image if the camera was mounted normally (landscape) however I often find (especially for smaller objects) that the object of interest is slightly out of focus but the background is in perfect focus. This is of course further complicated by the fact that the object itself is often moving.

I thought that I would be able to use the "region_of_interest" API calls to tell the sensor that I am only/more interested in a specific area, however though they appear supported in the realsense-viewer.exe, it appears that they are not supported by the "RGB Camera" sensor (https://github.com/IntelRealSense/librealsense/issues/2104# issuecomment-407409405 according to the SDK team here) so that idea is ruled out.

Is there another way that I can tell the camera that I want the focus within the captured RGB image to be concentrated on a particular part of the overall image?

[MartyG]

What a RealSense robotics expert called McCool has done in the past to get the field of view that they wanted is to set up multiple cameras and overlap their FOV. By doing this using old-model R200 and ZR300 RealSense cameras, they could get a 150 degree FOV if the cameras were arranged horizontally, or a 130 degree FOV if the cameras were stacked vertically.

As the 400 Series cameras have a considerably larger FOV than those older models (D435 being the largest), you could potentially get quite a large FOV by arranging the cameras in a way that provides a vertical portrait orientation.

With the 400 Series cameras, it is possible to set up a hardware sync between all the cameras, joined by a custom-made cable, so that all the cameras are recording frames in sync. Intel have written a multiple camera paper on the subject which explains how to set it up.

https://realsense.intel.com/wp-content/uploads/sites/63/Multiple_Camera_WhitePaper_rev1.1.pdf https://realsense.intel.com/wp-content/uploads/sites/63/Multiple_Camera_WhitePaper_rev1.1.pdf

Having the cameras vertically stacked, in their usual orientation instead of side-on and positioned so their FOVs overlap, may help to scan the tall objects that you want to photograph. Overlapping the FOV can even be beneficial for depth accuracy - the IR Projector component can project a random pattern of dots onto a scene to add texture to it, so the more cameras you have looking at the same area, the more dots are in the scene and so the greater the texture for the camera to lock onto.

[MRobi5]

Thanks for the suggestion; not sure how well that would fit because though some of the items are tall many of them are only about 1m wide so I could end up with the same scenario where I only care about the middle of the image and not the sides. I will however take a look at the paper and see if it offers anything else I had not thought of.

M.

[MartyG]

I wonder if another possibility would be Object Detection - training the camera to recognize particular objects on an RGB image and ignore the rest of the image.

Here is an example that uses the 400 Series camera and the OpenCV vision software to achieve this:

https://github.com/twMr7/rscvdnn GitHub - twMr7/rscvdnn: Test program for OpenCV DNN object detection with RealSense camera

This could be complicated by the camera being side-on though unless you train the software to view how objects should look when side-on.

[Sahira_Intel]

Hi martin.robins,

Were you able to try the solutions mentioned above to correct your problem?

Please let us know if we can assist you further.

Regards,

Intel Customer Support