9 Replies Latest reply on Apr 27, 2017 2:21 PM by Intel Corporation

    QueryInvUVMap sometimes throwing DATA_UNAVAILABLE

    jb455

      Hi,

       

      I'm using the InvUVMap to create a colour/depth hybrid image while streaming the camera (and also for mapping the camera coordinates to the colour image). Most of the time this is fine, but occasionally (once every few hundred frames on average) QueryInvUVMap returns PXCM_STATUS_DATA_UNAVAILABLE and the outputted array is full of zeroes.

      I've found that if I refresh my projection instance using device.CreateProjection() (like in the snippet below) that often fixes it for that frame, but sometimes it needs 2 or 3 refreshes and once I left it running for a while and when I came back it was stuck in the loop having been through it over 15000 times!

      So: what actually causes this error? And what's the proper way of fixing it? It's not a huge deal if it happens during streaming as the single frame just gets dropped, but if I'm unlucky enough that it ("it" being that it never gets the UV map after a few attempts) happens on the final frame after the user has pressed 'stop', that means I won't be able to map the camera coordinates which are needed for the main functionality of our app: big problem!

       

      Thanks,

      James

       

       

      pxcmStatus st = pxcmStatus.PXCM_STATUS_DATA_UNAVAILABLE;
      int it = 0;
      while (st.IsError())
      {     
           it++;
           st = projection.QueryInvUVMap(depth, invUVMap);
           if(st.IsError())
           {
                //QueryInvUVMap sometimes throws a "DATA_UNAVAILABLE" status. Most of the time, refreshing the projection instance fixes it, but sometimes it needs a few attempts (sometimes after 100s of attempts it still doesn't work)
                projection = device.CreateProjection();
           }
           if (it > 3)
                break;
      }
      
        • 1. Re: QueryInvUVMap sometimes throwing DATA_UNAVAILABLE
          jb455

          Just tried it with the SDK debugger on, these are the errors I get in sdk_app.log, in case it's any use:

           

          
          2017-03-09 11:06:43,856 rssdk.libpxccore (T-0x00002224) C:\Builds\2160\SW\src\_studio\core\core\src\coreaccel_manager.cpp:     475 ImageInfo, PXCImage::ImageData>::InternalAcquireAccess<>      ERROR - Format conversion [DEPTH(invalid),DEPTH(NATIVE)(invalid),1073741826(invalid)] -> DEPTH conversion failed
          2017-03-09 11:06:43,856 rssdk.libpxccore (T-0x00002224) C:\Builds\2160\SW\src\_studio\core\core\src\coreaccel_manager.cpp:     475 ImageInfo, PXCImage::ImageData>::InternalAcquireAccess<>      ERROR - Format conversion [DEPTH(invalid),DEPTH(NATIVE)(invalid),1073741826(invalid)] -> DEPTH conversion failed
          2017-03-09 11:06:43,858 rssdk.libpxcprojection (T-0x00002224)      projection_ivcam.cpp:     621             QueryInvUVMap      ERROR - Error PXC_STATUS_DATA_UNAVAILABLE (Data not available for MW model or processing) calling depth->AcquireAccess(PXCImage::ACCESS_READ, PXC_COLOR_FORMAT_UVMAP, &data_uvmap)
          2017-03-09 11:06:43,858 rssdk.libpxcprojection (T-0x00002224)      projection_ivcam.cpp:     621             QueryInvUVMap      ERROR - Error PXC_STATUS_DATA_UNAVAILABLE (Data not available for MW model or processing) calling depth->AcquireAccess(PXCImage::ACCESS_READ, PXC_COLOR_FORMAT_UVMAP, &data_uvmap)
          
          • 2. Re: QueryInvUVMap sometimes throwing DATA_UNAVAILABLE
            jb455

            I've noticed another issue too. Sometimes, when I call AcquireAccess on either the colour or depth image it'll return PXCM_STATUS_PARAM_UNSUPPORTED and any attempt to do anything with the ImageData results in an unhandled exception (unhandled by the SDK), despite the given parameters being unchanged from the other frames when it is successful. I found this thread on the old forum with the same issue but it was never resolved.

             

            Interestingly, it looks like it might be related to the above: the "data unavailable" issue only ever occurs in the frame after this "param unsupported" issue pops up (my AcquireAccess calls are later in the loop than QueryInvUVMap so it goes back around to the next frame when InvUV errors). However, I don't get "data unavailable" after every "param unsupported", only every second or third time. I can't find any errors in the SDK logs correlating with this new issue.

            • 3. Re: QueryInvUVMap sometimes throwing DATA_UNAVAILABLE
              Intel Corporation
              This message was posted on behalf of Intel Corporation

              Hi James,

              Thanks for reaching out.

              I'm not sure why you are getting this issue. I was searching about this function in the SDK, and I found the projection sample: https://software.intel.com/sites/landingpage/realsense/camera-sdk/v2016r3/documentation/html/index.html?sample_projection_cpp.html, I recommend you to check it, maybe you can fix this issue using the source code as reference.

              I hope you find this helpful.

              Have a nice day.

              Regards,
              Leonardo R.

              • 4. Re: QueryInvUVMap sometimes throwing DATA_UNAVAILABLE
                jb455

                Thanks, using ACCESS_WRITE instead of ACCESS_READ_WRITE in my colourImage.AcquireAccess() call seems to help - doesn't stop it completely but looks like it has reduced the frequency of the issue. I would still like to stop it completely though so any further advice you can offer would be great! I'm using C# by the way (the projection sample is only available in C++, not sure if that makes any difference)

                • 5. Re: QueryInvUVMap sometimes throwing DATA_UNAVAILABLE
                  Intel Corporation
                  This message was posted on behalf of Intel Corporation

                  Hi James,

                  Well, that's weird. Could you please send me the code that you are using? It's to replicate the issue, and analyze it for a better assistance.

                  I will be waiting for your reply.

                  Have a nice weekend.

                  Regards,
                  Leonardo R.

                  • 6. Re: QueryInvUVMap sometimes throwing DATA_UNAVAILABLE
                    Intel Corporation
                    This message was posted on behalf of Intel Corporation

                    Hi James,

                    Do you still need help with your issue? Can you send us your code?

                    We will be waiting for your response.

                    Regards,
                    Leonardo R.

                    • 7. Re: QueryInvUVMap sometimes throwing DATA_UNAVAILABLE
                      jb455

                      Hi Leonardo,

                       

                      Sorry, I got pulled away onto something else for a few days.

                       

                      I've been experimenting with it some more, and found:

                      • I was getting a memory issue where GarbageCollect was being called every frame. I traced this to the fact that I was calling "var invUVMap = new PXCMPointF32[cwidth * cheight]" each frame, so it was constantly destroying and creating over 2 million objects. I now just initialise the array before streaming and overwrite it with every QueryInvUVMap call, which has fixed the memory issue and also seems to have reduced the frequency of DATA_UNAVAILABLE (DA) and PARAM_UNSUPPORTED (PU).
                      • Calling CreateProjection when QueryInvUVMap fails doesn't seem to help - just calling QueryInvUVMap again does the trick when it is fixable.
                      • If I had PU in the previous frame when I get DU, calling QueryInvUVMap again always fixes it first time. Sometimes though I get DU without getting PU before, and in this case it never fixes itself (at least, not within the 3000-odd loops I let it run)
                      • Putting ACCESS_READ instead of ACCESS_WRITE in image.AcquireAccess doesn't seem to change anything - I can still write to the image buffer (via pointers) either way. Should this be the case?

                       

                      The errors appear much less frequently than before - on average maybe 5000 frames between errors, but occasionally I'll get it happening a few times within 100 frames. At the moment it's infrequent enough that it's not a huge problem, but I would like to get rid of it completely (particularly the QueryInvUVMap issue) to minimise the chance of any bigger problems occurring.

                       

                      The relevant code is below (I do much more in each frame but these issues only started appearing when I added this stuff).

                       

                      projection = device.CreateProjection();
                      var invUVMap = new PXCMPointF32[cwidth * cheight];
                      mappedPixels2 = new float[cwidth * cheight];
                      
                      
                      while (!stop)
                      {
                      
                        pxcmStatus s = pSenseManager.AcquireFrame(true);
                      
                      
                        PXCMCapture.Sample photo = pSenseManager.QuerySample();
                        PXCMImage image = photo.color, depth = photo.depth;
                      
                        pxcmStatus st = pxcmStatus.PXCM_STATUS_DATA_UNAVAILABLE;
                        int it = 0;
                        while (st.IsError())
                        {
                        st = projection.QueryInvUVMap(depth, invUVMap);
                        //QueryInvUVMap sometimes throws a "DATA_UNAVAILABLE" status. Sometimes calling it again fixes it, but sometimes it needs a few attempts (sometimes after 100s of attempts it still doesn't work)
                        it++;
                        if (it > (stop ? 6 : 3))//if this is the last frame, give it a few more chances to get the UV map as it's really needed then!
                        break;
                        }
                      
                        pSenseManager.ReleaseFrame();
                      
                        PXCMImage.ImageData cData, dData;
                        var sts = image.AcquireAccess(PXCMImage.Access.ACCESS_WRITE, PXCMImage.PixelFormat.PIXEL_FORMAT_RGB32, out cData);
                        var sts2 = depth.AcquireAccess(PXCMImage.Access.ACCESS_READ, PXCMImage.PixelFormat.PIXEL_FORMAT_DEPTH, out dData);
                      
                      
                        if (sts.IsSuccessful() && sts2.IsSuccessful() && invUVtask != null)
                        {
                        //Generate colour/depth hybrid image using pointers to access & modify the image buffers
                      
                        image.ReleaseAccess(cData);
                        depth.ReleaseAccess(dData);
                      
                        //render image
                        }
                        else
                        {
                        //The AcquireFrame call sometimes fails - this else is to be able to catch when it does while debugging.
                        //Doesn't cause a crash, just drops this frame.
                        image.ReleaseAccess(cData);
                        depth.ReleaseAccess(dData);
                        }
                        image?.Dispose();
                        depth?.Dispose();
                      }
                      
                      • 8. Re: QueryInvUVMap sometimes throwing DATA_UNAVAILABLE
                        Intel Corporation
                        This message was posted on behalf of Intel Corporation

                        Hi James,

                        Thank you for the information provided.

                        We are going to investigate more about this, and we will get back to you when we have updates.

                        Have a nice day.

                        Regards,
                        Leonardo R.
                         

                        • 9. Re: QueryInvUVMap sometimes throwing DATA_UNAVAILABLE
                          Intel Corporation
                          This message was posted on behalf of Intel Corporation

                          Hi James,

                          We couldn't replicate the issue, so unfortunately we don't have more suggestions to help you with this. The only information available related to this is the SDK documentation that I gave you before, and this helped you to decrease the error.

                          We encourage you to post any solution that you have found to help the community.

                          Regards,
                          Leonardo R.