This message was posted on behalf of Intel Corporation
Thanks for reaching out!
Please correct me if I'm wrong. You are able to map depth using the RGB image using QueryUVMap (https://software.intel.com/sites/landingpage/realsense/camera-sdk/v2016r3/documentation/html/queryuvmap_pxcprojection.html) but now you would like to do it using the IR (right or left) image, right? If not, then I'm confused and I would like you to explain me your goal again.
According to https://software.intel.com/sites/landingpage/realsense/camera-sdk/v2016r3/documentation/html/doc_essential_coordinates_mapping.html, which is where you got the lines of code above, this is the only way in which the SDK maps depth. Unfortunately the SDK has this limitation, therefore it seems that the only way you will be able to do this mapping will be between RGB and depth.
You might want to check if librealsense (https://github.com/IntelRealSense/librealsense) has this option available, perhaps it has an option that fits your needs better.
I hope this information helps you,
Using the Raw Streams sample, it looks like the left IR image is aligned with the depth image (if not exactly then very close) - can you use that fact? You may be out of luck with the right image though unless you can use the calibration parameters somehow to get a map from left to right?
Thanks a lot for your answer, I am so sorry it took me so long to reply.
You are right, I need to the mapping from the IR left or right camera to the depth image, but the SDK does not have that option.
I have also tried with the librealsense without luck, but I think I found a solution.
Your suggestion was a nice attempt, but actually the depth image is located exactly between the two infrared cameras, so is not possible to take advantage of that fact. Nevertheless, you gave me a great idea to solve my problem.
The previous answers gave me an idea,
at the beginning I needed to find out how to map a specific point detected on one of the infrared cameras to the depth image. So, my final solution was to subtract the sector of interest (detected in the IR Left camera, which is the nearest to the color camera) from the RGB image and from the result doing the mapping to depth.
The gap exists, but is not too much.