queryNumJoints() only returns 0

[MMich26]

Hi,

I am currently working with the Person Tracking Module using an R200. The personTracking module works successfully, and I can successfully query features like centerMass. However, querying skeleton joints has not been working. To start, I queried the number of joints that my module was currently configured to track.

PXCPersonTrackingData::PersonTracking* personTracking = personData->QueryTracking();PXCPersonTrackingData::PersonJoints * ptj = personData->QuerySkeletonJoints();int numJoints = ptj->QueryNumJoints();printf("numJoints: %d ", numJoints);The above code outputs 0, no matter which tracking area is selected (AREA_FULL_BODY, AREA_FULL_BODY_ROUGH, AREA_UPPER_BODY, and AREA_UPPER_BODY_ROUGH). Has anyone gotten queryNumJoints to successfully return the number of joints being tracked?

Michael

[idata]

Hello Michael,

Thanks for reaching out!May I ask, which SDK version are you using? Also, have you already tried running the example found in https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?doc_pt_tracking_joints.html. If so, how did it go? Does it also returns 0?Let me know.-Peter.

[MMich26]

Hi Peter,

I am using the 2016 R2 SDK. I tried your link, and am still only getting the function to return 0. I used the example you provided, and ended up with the following code:

/* Enabling skeleton joint tracking */PXCPersonTrackingConfiguration* personTrackingConfig = pp->QueryPersonTracking()->QueryConfiguration();personTrackingConfig->SetTrackedAngles(PXCPersonTrackingConfiguration::TrackingAngles::TRACKING_ANGLES_ALL);PXCPersonTrackingConfiguration::SkeletonJointsConfiguration* skeletonJoints = personTrackingConfig->QuerySkeletonJoints();skeletonJoints->Enable();printf("is jointTracking Enabled?: %d\n", skeletonJoints->IsEnabled());

...

int numPeople = personModule->QueryOutput()->QueryNumberOfPeople();/* Found a person */if (numPeople != 0) {printf("Number of people: %d\n", numPeople);PXCPersonTrackingData::Person* personData = personModule->QueryOutput()->QueryPersonData(PXCPersonTrackingData::ACCESS_ORDER_BY_ID, 0);assert(personData != NULL);PXCPersonTrackingData::PersonJoints* ptj = personData->QuerySkeletonJoints();int numJoints = ptj->QueryNumJoints();printf("numJoints: %d ", numJoints);}

It states that joint tracking is enabled, but continues to loop through and print that there were 0 joints found.

Michael

[MMich26]

Ok, so I've been experimented with the Person Tracking example included in the SDK example library. I attached a console to the GUI, and have been able to reverse engineer some important details. The example, for starters, has working skeleton tracking with 6 points. I can explain what I have found a little more if anyone needs it, but for now I'll just list the joints I was able to uncover:

JOINT_HEAD

JOINT_LEFT_SHOULDER

JOINT_RIGHT_SHOULDER

JOINT_LEFT_HAND

JOINT_RIGHT_HAND

JOINT_SPINE_MID

While this is nowhere close to the number of points mentioned in the documentation, it should work for my purposes. The example code has a working queryNumJoints function--it returns 6. It also has a working queryJoints function...something that I have so far been unable to port into my own code. In the picture below, you can see that I have been able to access the 6 skeleton joints and their respective members inside the example code.

So, it is nice to see that there is a working evidence of skeleton joints tracking. However, I cannot figure out how to enable it in my own programs and the online documentation frankly sucks. I cannot figure out the right combination/ordering of method required to enable skeletonJoint tracking. The example code bring up another function, setTrackingMode(), that was not even mentioned in the SDK documentation and seems to be necessary to function. Is there anyone on your end that can explain or provide a working piece of code to track these 6 functions and that doesn't come cluttered with a massive GUI?

Thanks,

Michael

[idata]

Hi Michael,

I understand, this is a strange behavior. Please let me investigate about this and if I'm able to find something useful, I'll make sure to share it with you.Nevertheless, I would like to point out that setTrackingMode is actually found on the SDK documentation. It can be found in https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?settrackingmode_pxcfaceconfiguration.html.-Peter.

[MMich26]

OK, thanks for the help!

[idata]

Hi Michael,

Thank you for your patience, I understand this could be frustrating. We will try to get a simplified code snippet that works for tracking joints from RealSense developers but I cannot promise if and when we will be able to get the code. I will get back to you once I have an update.-Peter.

[SCoop2]

I am actually working on the skeleton tracking via the Intel Euclid Developer Kit but have so far been unsuccessfull in obtaining the 6 points mentioned in the documentation. I downloaded a skeleton launch file from this site as well but only shows person tracking when I run itit https://github.com/IntelRealSense/realsense_samples_ros/tree/kinetic-devel/realsense_ros_person/launch realsense_samples_ros/realsense_ros_person/launch at kinetic-devel · IntelRealSense/realsense_samples_ros · GitHub

Could you please explain to me if it's possible how you obtained the 6 body tracking points? Or any method that you suggest to do so?

Thanks.

[MMich26]

I'm not sure how relevant my research will be to yours, I was using the R200 and R2 of the sdk...I know it's all changed considerably since then. If you would like to review the workaround code that I had come up with, feel free to check it out here: https://github.com/michael5486/Senior-Design GitHub - michael5486/Senior-Design: All our files and code for the ATV

I used a provided sample program from the sdk examples to reverse engineer a working skeleton tracking format for use in my own code. I'd recommend beginning with that. Confirm skeleton tracking actually works, and then adapt it to your needs.

[SCoop2]

michael5486

Thanks for your response! I will be sure to check out your code. As a matter of fact I did try to reverse engineering a realsense sample code, I took the pointing gesture detection sample (pt_tutorial_3) and tried to create a skeleton tracking module using the commands of body tracking here https://software.intel.com/sites/products/realsense/person/developer_guide.html Intel® RealSense™ for Linux - Person Library: Developer's Guide

However, perhaps because of my lack of language in C++, I have not managed to do much....If by any chance anybody had the code that comprised the commands in this link I'd much appreciate it.

[STiru]

Hello Michael and Peter, I am also having this same issue. The example in the SDK is able to identify six of the joints. First, I must say I'm disappointed in the marketing and documentation which led me to believe that the R200 device had greater joint tracking capabilities. The device is impressive nonetheless.

With that aside, QueryNumJoints() returns a value of 0 in my program as well. While perusing the SDK source files, I noticed the variable joints is both a member variable of the PersonTrackingRenderer2D class as well as a variable defined separately as:

if (!joints)

{

joints = new PXCPersonTrackingData::PersonJoints::SkeletonPoint[personJoints->QueryNumJoints()];

}

personJoints->QueryJoints(joints);

This variable can be used exclusively to plot the location of the known joints within the DrawSkeleton method of the aforementioned class. Perhaps, Peter, you can clear up this ambiguity to help us out. Furthermore, there is some level of redundancy in accessing joint position data. Do you have suggestions on the best way to access this information?

Thank you,

Sragvi

[MartyG]

I remembered that a couple of people on the old RealSense forum had this returning-zero issue, so I dug out the links as a reference source.

https://software.intel.com/en-us/forums/realsense/topic/685380 RealSense R200: How resolve call to QueryJoints (C# ) causing exception?

https://software.intel.com/en-us/forums/realsense/topic/607652?language=es Skeleton Joint Data Available?

https://software.intel.com/en-us/forums/realsense/topic/624543 Issue with SkeletonPoint

[idata]

Skeleton tracking will work only when tracking mode is set to Interactive (It is not mentioned in the documentation ) and also you must enable skeleton tracking (by default skeleton tracking is disabled)

here is my code (tested with R200 and SDK R2)

int wmain(int argc,WCHAR* argv[])

{

PXCSenseManager *sm = PXCSenseManager::CreateInstance();

PXCPersonTrackingData::PersonJoints::SkeletonPoint* joints;

sm->EnablePersonTracking();

PXCPersonTrackingModule *tracker = sm->QueryPersonTracking();

sm->Init();

PXCPersonTrackingConfiguration *config = tracker->QueryConfiguration();

PXCPersonTrackingData *PersonData = tracker->QueryOutput();

config->QuerySkeletonJoints()->Enable();

config->QueryTracking()->SetTrackingMode(PXCPersonTrackingConfiguration::TrackingConfiguration::TRACKING_MODE_INTERACTIVE);

while (sm->AcquireFrame(true) >= PXC_STATUS_NO_ERROR)

{

pxcI32 npersons = PersonData->QueryNumberOfPeople();

for (pxcI32 i = 0; i < npersons; i++)

{

PXCPersonTrackingData::Person *Person = PersonData->QueryPersonData(PXCPersonTrackingData::ACCESS_ORDER_BY_ID, i);

PXCPersonTrackingData::PersonTracking *ptt = Person->QueryTracking();

PXCPersonTrackingData::PersonJoints *PersonJoint = Person->QuerySkeletonJoints();

pxcI32 numJoints = PersonJoint->QueryNumJoints();

joints = new PXCPersonTrackingData::PersonJoints::SkeletonPoint[numJoints];

PersonJoint->QueryJoints(joints);

wprintf(L"no.of joints : %d", numJoints);

}

wprintf(L"\nno.of person : %d\n", npersons);

sm->ReleaseFrame();

}

sm->Release();

return 0;

}

[MartyG]

Thanks so much for the info, Kamalraj

[YLi77]

I found that there is not SetTrackingMode function in C# . Neither in C# project nor in Unity project. Any idea?

[MartyG]

Whilst there is not a SetTrackingMode function in the RealSense online documentation pages for C# UWP, you can click on the C# tab to get code in ordinary C# . UWP stands for Universal Windows Platform. So you only need to use the UWP version instead of ordinary C# if you want to develop a single universal Windows app for all types of Microsoft-supported devices, including phone, Surface tablet, HoloLens headset and Xbox console.

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?settrackingmode_pxchandconfiguration.html Intel® RealSense™ SDK 2016 R2 Documentation

[YLi77]

Hi, MartyG, thanks for your reply. I need to make me more clear. I can not find the SetTrackingMode function in the SDK. Such as :

ptc.QueryTracking().Enable();

ptc.QueryTracking().SetTrackingMode();

There will be error: PXCMPersonTrackingConfiguration.TrackingConfiguration does not contain a definition for "SetTrackingMode".

I install the RealSense SDK 2016 R2, and import it into Unity 3D, as https://software.intel.com/en-us/blogs/2015/02/13/add-realsense-to-unity 8 Steps to Add Intel RealSense Unity Toolkit to your Project | Intel® Software

[MartyG]

That guide is a couple of years old and the SDK has gone through about 5 different versions since then. I suspect though that your error may be related to Unity, if you have followed that guide. Although the procedure for copying RealSense DLL files into a Unity project was correct at the time, things changed when Unity 5 was introduced.

Up until Unity 4 Pro, Unity was based on 32-bit code. After Unity 5, Unity switched to 64-bit code and so the 32-bit RealSense library files placed in the Plugins and Plugins. Managed folders by the Unity Toolkit were no longer compatible. You need to ensure that these folders have the 64-bit versions of the RealSense files, as you cannot rely on the Unity Toolkit to do it properly automatically.

Go to the Plugins and Plugins.Managed folders in your project and delete their contents (libpxccpp2c.dll and libpxcclr.unity.dll).

Then go to C:\Program Files (x86)\Intel\RSSDK\bin\x64 (which is the location for the 64-bit version of files) and from this directory, copy libpxccpp2c.dll into Plugins and libpxcclr.unity.dll into Plugins.Managed. The filenames are the same, they are just the 64-bit version of the files that Unity 5 requires.

The rest of the Unity Toolkit files do not need to be changed.

Please comment here again if you still have problems.

[YLi77]

Hi, MartyG. After changed as you said, there is still no definition for "SetTrackingMode", and even more, there are no definition for "QuerySkeletonJoints" in PXCMPersonTrackingData.Person and no definition for "QueryJoints" in PXCMPersonTrackingData.PersonJoints.

[MartyG]

1. What camera are you using please? F200, R200 or SR-300?

2. Are you using the R2 version of the RealSense SDK or version R3? The newest R3 version and the Unity Toolkit files are not currently working with Unity, as it is very different in structure to the previous R2 version. If you want to develop in Unity with RealSense, you should use R2.

[YLi77]

1. I use R200.

2. I use version R2.