12 Replies Latest reply on Feb 24, 2017 8:32 AM by MartyG

    CreateDepthImageMappedToColor Alternative

    andakkino

      Hello,

       

      I can see from this discussion Memory leak persists in CreateDepthImageMappedToColor of the latest SDK  and from my code that the memory leakege problem of CreateDepthImageMappedToColor persist.

      I need an alternative way to map the depth  image to the color image of the camera. Do you know an alternative solution? Have you found some code (possibly easy to understand)

       

      I hope to find help here,

      Thanks for your time and assistance.

        • 1. Re: CreateDepthImageMappedToColor Alternative
          jb455

          MartyG brought this to my attention the other day: https://mtaulty.com/2015/04/16/m_15794/

           

          Lines 131-183 create an image similar to CreateDepthImageMappedToColor, though it may be a little slower than the native method.

          • 2. Re: CreateDepthImageMappedToColor Alternative
            Intel Corporation
            This message was posted on behalf of Intel Corporation

            Hello andakkino,

            I was wondering if you had the chance to check the suggestion provided by MartyG.

            If you have any other question, don’t hesitate to contact us.

            Regards,
            Andres V.

            • 3. Re: CreateDepthImageMappedToColor Alternative
              andakkino

              Hello,

               

              Unfortunately I have not solved my problem yet but I'm investigation on my own code (not strictly related to the SDK, in particular I begin to think that  I'm creating a wrong texture with depth data collected from the camera and so I don't obtain the final desired result).

               

              I will let you know if I'll need more help from you.

              Obviously if I'll find a working solution I will let you know (so I will help other members of the community with their projects).

               

              In the meantime, thank you for the support.

              • 4. Re: CreateDepthImageMappedToColor Alternative
                Intel Corporation
                This message was posted on behalf of Intel Corporation

                Hello andakkino,

                In case you need further assistance, don’t hesitate to contact us.

                Thank you for considering sharing your experience with the community, we really appreciate it.

                Regards,
                Andres

                • 5. Re: CreateDepthImageMappedToColor Alternative
                  andakkino

                  I'm here again,

                  I still have my problem, I try to explain it at my best and I'm sorry for my bad English.

                   

                  Due to the differences in a camera's physical location, lens size, and field of view, color and depth coordinates do not map 1:1 to each other.

                  So the depth image is "translated" with respect to the color image.

                   

                  the CreateDepthImageMappedToColor solves my problem because it generates a new depth image with the same resolution of the color image and with the modified coordinates. Unfortunately, due to the memory leakage problem I can't use this function.

                   

                  I need at least, to correct the coordinate mismatch between the depth and color image. Is there any function in the SDK that can do this?

                   

                  Jb455, thanks for the link. I studied that code, but I have some doubts:

                  why he creates a UInt32* colorPtr pointing to colorData.planes[0]  [line 138] but then he sets every element of the vector to 0x22FFFFFF; [line 146]

                   

                  then at line 170 he does the |= 0xCC000000.

                   

                  I don't understand clearly what the author of the code wants to do. I was thinking that his purpose was to obtain a new deph image with coordinates adapted from the color image. But at this point I think I'm wrong.

                   

                  ps: I'm programming using C++;

                  • 6. Re: CreateDepthImageMappedToColor Alternative
                    jb455

                    What do you want to do with it? Create a hybrid depth/colour image to view or get the depth values for each (valid) pixel in the colour image? Also, what language are you using? I outlined a method to do the latter using C# in this thread: Getting float depth data. The code I linked to before creates an image where only pixels with valid depth values are coloured.

                    • 7. Re: CreateDepthImageMappedToColor Alternative
                      MartyG

                      Also, this person used Projection to calibrate their camera.

                       

                      [RealSense SDK] Calibrate Color and Depth Stream | Putu Yuwono Kusmawan

                      • 8. Re: CreateDepthImageMappedToColor Alternative
                        andakkino

                        Jb455:

                         

                        I'm using C++;

                         

                        What I want is to have two images: One color image and one depth image. But I need that color and depth coordinates are mapped 1:1 to each other. So for instance if I were able to create a mesh using the "depth image" I could texturing it using the "color image".

                         

                        If I use the CreateDepthImageMappedToColor function then I'm able to do what I described above.

                        For this reason I'm in search of an alternative way.

                         

                        In other words:

                        -I have a color image

                        -I have a depth image

                        -In the best scenario I would like to have a new depth image where the coordinates mismatch is correct, So the new depth image is spatially aligned with the color image.

                         

                        MartyG:

                        I saw that, unfortunately that person uses CreateDepthImageMappedToColor function that is affected by the memory leakage problem.

                         

                        Thanks again for your help, I really appreciate that.

                        • 9. Re: CreateDepthImageMappedToColor Alternative
                          MartyG

                          Intel staffer David Lu posted some C++ calibration code a couple of years ago.  I apologize if it's the function with the memory leak (JB455 is the resident color / depth programming expert on this forum - thanks for all you do in this field to help people, JB!)

                           

                          The script was found here: R200 Calibration Data

                           

                           

                          PXCCalibration *calib = m_pProjection->QueryInstance<PXCCalibration>();

                           

                          PXCCalibration::StreamCalibration calibData={};

                           

                          PXCCalibration::StreamTransform calibTrans = {};

                           

                          m_sts[0] = (STATUS)calib->QueryStreamProjectionParameters(PXCCapture::StreamType::STREAM_TYPE_COLOR, &calibData, &calibTrans);

                           

                          m_sts[0] = (STATUS)calib->QueryStreamProjectionParametersEx(PXCCapture::StreamType::STREAM_TYPE_COLOR, PXCCapture::Device::StreamOption::STREAM_OPTION_ANY,&calibData, &calibTrans);

                          • 10. Re: CreateDepthImageMappedToColor Alternative
                            jb455

                            Ok, yeah, the 'Correct Answer' here will give you what you want (an array of depth values mapped to the colour image) once you translate it. So the nth item in the MappedPixels array will contain the depth value for the nth pixel in the colour image (scanning left to right, top to bottom).

                            • 11. Re: CreateDepthImageMappedToColor Alternative
                              andakkino

                              Ok Guys, I followed your suggestions and now something works!. I post my code here, sorry if it is not optimized but it seems to work (obviusly this is only the code related to this topic, for everyone who will use this code remember also to release the access to the rgb image (I did it but few lines of code later).

                               

                              I'm very grateful for your help

                               

                              rgb_image = sample->color;
                                rgb_image->AcquireAccess(PXCImage::ACCESS_READ, Intel::RealSense::Image::PIXEL_FORMAT_RGB24, &data_camera_rgb);
                                int rgb_width = sample->color->QueryInfo().width;
                                int rgb_height = sample->color->QueryInfo().height;
                              
                                //depth_image = sample->depth;
                                //creating new depth image mapped to Color image
                                if (projection != NULL) {
                              
                                depth_image = sample->depth;
                                PXCImage::ImageData ddata; //creo un oggetto image data
                                depth_image->AcquireAccess(PXCImage::ACCESS_READ, Intel::RealSense::Image::PIXEL_FORMAT_DEPTH, &ddata); //accedo all'immagine, in mdoalita depth f32
                                int dwidth = depth_image->QueryInfo().width; //larghezza dell'immagine di depth
                                int dheight = depth_image->QueryInfo().height; // altezza dell'immagine di depth
                              
                              
                                pointer = (unsigned short*)ddata.planes[0];
                                for (int i = 0; i < dwidth *dheight; i++) {
                              
                              
                                dPixels[i] = (unsigned short) pointer[i];
                                }
                              
                              
                                depth_image->ReleaseAccess(&ddata);  //release access
                                projection->QueryInvUVMap(depth_image, &invuvmap[0]); //right syntax: pxcStatus QueryInvUVMap(PXCImage *depth, PXCPointF32 *inv_uvmap); 
                              
                                for (int i = 0; i < rgb_width *rgb_height; i++) {
                                int u = (int)(invuvmap[i].x * dwidth);
                                int v = (int)(invuvmap[i].y * dheight);
                                if ((u >= 0) && (v >= 0) && (u + v * dwidth < dwidth * dheight)) {
                                mappedPixels[i] = dPixels[u + v * dwidth];
                                }
                                else {
                                mappedPixels[i] = 0;
                                }
                                }
                                }
                              
                              1 of 1 people found this helpful
                              • 12. Re: CreateDepthImageMappedToColor Alternative
                                MartyG

                                It's great that you found a solution!  Thanks so much for sharing this valuable information for the community, andakkino.  Please come to the forum again if you need further help.