QueryInvUVMap sometimes throwing DATA_UNAVAILABLE

[jb455]

Hi,

I'm using the InvUVMap to create a colour/depth hybrid image while streaming the camera (and also for mapping the camera coordinates to the colour image). Most of the time this is fine, but occasionally (once every few hundred frames on average) QueryInvUVMap returns PXCM_STATUS_DATA_UNAVAILABLE and the outputted array is full of zeroes.

I've found that if I refresh my projection instance using device.CreateProjection() (like in the snippet below) that often fixes it for that frame, but sometimes it needs 2 or 3 refreshes and once I left it running for a while and when I came back it was stuck in the loop having been through it over 15000 times!

So: what actually causes this error? And what's the proper way of fixing it? It's not a huge deal if it happens during streaming as the single frame just gets dropped, but if I'm unlucky enough that it ("it" being that it never gets the UV map after a few attempts) happens on the final frame after the user has pressed 'stop', that means I won't be able to map the camera coordinates which are needed for the main functionality of our app: big problem!

Thanks,

James

pxcmStatus st = pxcmStatus.PXCM_STATUS_DATA_UNAVAILABLE;

int it = 0;

while (st.IsError())

{

it++;

st = projection.QueryInvUVMap(depth, invUVMap);

if(st.IsError())

{

//QueryInvUVMap sometimes throws a "DATA_UNAVAILABLE" status. Most of the time, refreshing the projection instance fixes it, but sometimes it needs a few attempts (sometimes after 100s of attempts it still doesn't work)

projection = device.CreateProjection();

}

if (it > 3)

break;

}

[jb455]

Just tried it with the SDK debugger on, these are the errors I get in sdk_app.log, in case it's any use:

2017-03-09 11:06:43,856 rssdk.libpxccore (T-0x00002224) C:\Builds\2160\SW\src\_studio\core\core\src\coreaccel_manager.cpp: 475 ImageInfo, PXCImage::ImageData>::InternalAcquireAccess<> ERROR - Format conversion [DEPTH(invalid),DEPTH(NATIVE)(invalid),1073741826(invalid)] -> DEPTH conversion failed

2017-03-09 11:06:43,856 rssdk.libpxccore (T-0x00002224) C:\Builds\2160\SW\src\_studio\core\core\src\coreaccel_manager.cpp: 475 ImageInfo, PXCImage::ImageData>::InternalAcquireAccess<> ERROR - Format conversion [DEPTH(invalid),DEPTH(NATIVE)(invalid),1073741826(invalid)] -> DEPTH conversion failed

2017-03-09 11:06:43,858 rssdk.libpxcprojection (T-0x00002224) projection_ivcam.cpp: 621 QueryInvUVMap ERROR - Error PXC_STATUS_DATA_UNAVAILABLE (Data not available for MW model or processing) calling depth->AcquireAccess(PXCImage::ACCESS_READ, PXC_COLOR_FORMAT_UVMAP, &data_uvmap)

2017-03-09 11:06:43,858 rssdk.libpxcprojection (T-0x00002224) projection_ivcam.cpp: 621 QueryInvUVMap ERROR - Error PXC_STATUS_DATA_UNAVAILABLE (Data not available for MW model or processing) calling depth->AcquireAccess(PXCImage::ACCESS_READ, PXC_COLOR_FORMAT_UVMAP, &data_uvmap)

[jb455]

I've noticed another issue too. Sometimes, when I call AcquireAccess on either the colour or depth image it'll return PXCM_STATUS_PARAM_UNSUPPORTED and any attempt to do anything with the ImageData results in an unhandled exception (unhandled by the SDK), despite the given parameters being unchanged from the other frames when it is successful. I found https://software.intel.com/en-us/forums/realsense/topic/600562 this thread on the old forum with the same issue but it was never resolved.

Interestingly, it looks like it might be related to the above: the "data unavailable" issue only ever occurs in the frame after this "param unsupported" issue pops up (my AcquireAccess calls are later in the loop than QueryInvUVMap so it goes back around to the next frame when InvUV errors). However, I don't get "data unavailable" after every "param unsupported", only every second or third time. I can't find any errors in the SDK logs correlating with this new issue.

[idata]

Hi James,

Thanks for reaching out.I'm not sure why you are getting this issue. I was searching about this function in the SDK, and I found the projection sample: https://software.intel.com/sites/landingpage/realsense/camera-sdk/v2016r3/documentation/html/index.html?sample_projection_cpp.html, I recommend you to check it, maybe you can fix this issue using the source code as reference.I hope you find this helpful.Have a nice day.Regards,Leonardo R.

[jb455]

Thanks, using ACCESS_WRITE instead of ACCESS_READ_WRITE in my colourImage.AcquireAccess() call seems to help - doesn't stop it completely but looks like it has reduced the frequency of the issue. I would still like to stop it completely though so any further advice you can offer would be great! I'm using C# by the way (the projection sample is only available in C++, not sure if that makes any difference)

[idata]

Hi James,

Well, that's weird. Could you please send me the code that you are using? It's to replicate the issue, and analyze it for a better assistance.I will be waiting for your reply.Have a nice weekend.Regards,Leonardo R.

[idata]

Hi James,

Do you still need help with your issue? Can you send us your code?We will be waiting for your response.Regards,Leonardo R.

[jb455]

Hi Leonardo,

Sorry, I got pulled away onto something else for a few days.

I've been experimenting with it some more, and found:

• I was getting a memory issue where GarbageCollect was being called every frame. I traced this to the fact that I was calling "var invUVMap = new PXCMPointF32[cwidth * cheight]" each frame, so it was constantly destroying and creating over 2 million objects. I now just initialise the array before streaming and overwrite it with every QueryInvUVMap call, which has fixed the memory issue and also seems to have reduced the frequency of DATA_UNAVAILABLE (DA) and PARAM_UNSUPPORTED (PU).
• Calling CreateProjection when QueryInvUVMap fails doesn't seem to help - just calling QueryInvUVMap again does the trick when it is fixable.
• If I had PU in the previous frame when I get DU, calling QueryInvUVMap again always fixes it first time. Sometimes though I get DU without getting PU before, and in this case it never fixes itself (at least, not within the 3000-odd loops I let it run)
• Putting ACCESS_READ instead of ACCESS_WRITE in image.AcquireAccess doesn't seem to change anything - I can still write to the image buffer (via pointers) either way. Should this be the case?

The errors appear much less frequently than before - on average maybe 5000 frames between errors, but occasionally I'll get it happening a few times within 100 frames. At the moment it's infrequent enough that it's not a huge problem, but I would like to get rid of it completely (particularly the QueryInvUVMap issue) to minimise the chance of any bigger problems occurring.

The relevant code is below (I do much more in each frame but these issues only started appearing when I added this stuff).

projection = device.CreateProjection();

var invUVMap = new PXCMPointF32[cwidth * cheight];

mappedPixels2 = new float[cwidth * cheight];

while (!stop)

{

pxcmStatus s = pSenseManager.AcquireFrame(true);

PXCMCapture.Sample photo = pSenseManager.QuerySample();

PXCMImage image = photo.color, depth = photo.depth;

pxcmStatus st = pxcmStatus.PXCM_STATUS_DATA_UNAVAILABLE;

int it = 0;

while (st.IsError())

{

st = projection.QueryInvUVMap(depth, invUVMap);

//QueryInvUVMap sometimes throws a "DATA_UNAVAILABLE" status. Sometimes calling it again fixes it, but sometimes it needs a few attempts (sometimes after 100s of attempts it still doesn't work)

it++;

if (it > (stop ? 6 : 3))//if this is the last frame, give it a few more chances to get the UV map as it's really needed then!

break;

}

pSenseManager.ReleaseFrame();

PXCMImage.ImageData cData, dData;

var sts = image.AcquireAccess(PXCMImage.Access.ACCESS_WRITE, PXCMImage.PixelFormat.PIXEL_FORMAT_RGB32, out cData);

var sts2 = depth.AcquireAccess(PXCMImage.Access.ACCESS_READ, PXCMImage.PixelFormat.PIXEL_FORMAT_DEPTH, out dData);

if (sts.IsSuccessful() && sts2.IsSuccessful() && invUVtask != null)

{

//Generate colour/depth hybrid image using pointers to access & modify the image buffers

image.ReleaseAccess(cData);

depth.ReleaseAccess(dData);

//render image

}

else

{

//The AcquireFrame call sometimes fails - this else is to be able to catch when it does while debugging.

//Doesn't cause a crash, just drops this frame.

image.ReleaseAccess(cData);

depth.ReleaseAccess(dData);

}

image?.Dispose();

depth?.Dispose();

}

[idata]

Hi James,

Thank you for the information provided.We are going to investigate more about this, and we will get back to you when we have updates.Have a nice day.Regards,Leonardo R.

[idata]

Hi James,

We couldn't replicate the issue, so unfortunately we don't have more suggestions to help you with this. The only information available related to this is the SDK documentation that I gave you before, and this helped you to decrease the error.We encourage you to post any solution that you have found to help the community.Regards,Leonardo R.