Items with no label
3335 Discussions

intel realsense sr300 object tracking

LMart13
Novice
3,186 Views

how can i track the same target identifier of an object every time?

it seems not returns the same trackvalue of an certain object

0 Kudos
46 Replies
MartyG
Honored Contributor III
728 Views

Are you using 2D or 3d object tracking, please? And are you using the RealSense SDK in Windows or the RealSense SDK For Linux?

In the RealSense SDK for Windows, the object tracking documentation says that you will get more reliable results if you use 3D tracking for objects whose position is moving. For stationary objects such as a table or an object sitting still on the table, 2D tracking may suffice.

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?doc_ot_object_tracking.html Intel® RealSense™ SDK 2016 R2 Documentation

0 Kudos
LMart13
Novice
728 Views

I tried both,neither 2D nor 3d object tracking stable. And i use the RealSense SDK in Windows.

i want to calculate the rotate angle of my object,so i want the same point.but when i rotate my object ,the trackvalue become werid

0 Kudos
MartyG
Honored Contributor III
728 Views

There was an object tracking script for the F200 (the SR300's predecessor camera) that calls QueryAllTrackingValues. By doing so, it provides information on both the rotation and position of the tracked object.

https://mtaulty.com/2015/04/28/m_15824/ https://mtaulty.com/2015/04/28/m_15824/

0 Kudos
LMart13
Novice
728 Views

QueryAllTrackingValues returns trackvalues[5]. how can i fix one identifier,so i can calculate the rotation angles relative to previous identifier.then i can calculate the correct rotation angles of the object.

if everytime i use QueryAllTrackingValues returns different identifiers,i don't know the exact rotation angles of the object relative to the previous pose when the object rotated.

is there any functions can return the assigned identifier,no matter how i move or rotate the object.so that i can keep the same pose all the time.it's important when i want to pick the object when the object pose changed,

the accurate angles should be returned.:(MartyG

0 Kudos
MartyG
Honored Contributor III
728 Views

QueryTrackingValues is supposed to return the values of a particular identifier (cosID).

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?querytrackingvalues_pxctracker.html Intel® RealSense™ SDK 2016 R2 Documentation

0 Kudos
LMart13
Novice
728 Views

the questions is how to align a cosID,returned IDs are random.

0 Kudos
LMart13
Novice
728 Views

how can i ensure the same trackvalue of the object?MartyG,

please help me.

0 Kudos
LMart13
Novice
728 Views

let me describe my application scenarios for short:

i use my gripper to pick some object.and first of first.i should tell my gripper the initial pose and the final pose of the object(the angle and the position).

so the tracked thing should be the same identifier on the object,if not i can't calculate the exact position and the angles.these can be useful when my gripper want to grab the object.

if the tracked object rotated and moved,then my gripper should rotate the same angles and move the same distance relative to the initial pose.rather than the random rotation and position returned from

QueryAllTrackingValues or QueryTrackingValues,

do i need to mark the markdID on my trakced object.but the align map worked not so well.it affected by the light or other unknown reasons,i dont know.

0 Kudos
MartyG
Honored Contributor III
728 Views

I wonder if another way to achieve your project goal would be in the RealSense implementation in the Unity game engine. The RealSense SDK provides a script called TrackingAction that can be set to object tracking, to change the position and rotation of a virtual object based on the tracking of a real-world object.

LMart13
Novice
728 Views

please give me a link of this TrackingAction software

0 Kudos
MartyG
Honored Contributor III
728 Views

Unity can be downloaded from here. Once you have done that, I can explain the further steps for setting up RealSense in it.

https://store.unity.com/download?ref=personal Unity - Download

0 Kudos
LMart13
Novice
728 Views

Unity sees like a game related develop tool.Hope it will useful

0 Kudos
MartyG
Honored Contributor III
728 Views

Yes, Unity is a game creation engine. It is very powerful, and I use it for my own game project.

You can learn more about using RealSense with it by reading my large range of how-to guides for it. Good luck when you get to try it on the 6th.

https://software.intel.com/en-us/forums/realsense/topic/676139 Index of Marty G's RealSense Unity How-To Guides

0 Kudos
LMart13
Novice
728 Views

sr300 can't be used to track certain object .if the result is random,it cant be used to do anything.

0 Kudos
MartyG
Honored Contributor III
728 Views

Although Unity is primarily used as a game creation engine, it does not mean that you have to make games with it. You can use it to make serious projects too. You can think of it as a visual object programming environment.

It is true that some objects are harder to track than others. For this reason, RealSense has a 3D tracking mode called Edge Based Tracking that may be able to follow objects that other tracking modes cannot. What kinds of objects have you found cannot be detected by the tracking, please?

In RealSense's 3D model scanning feature, objects with surface curvatures that are difficult to detect can be made more easily detectable if you place other objects in the background, as it helps the camera to lock onto the object you are trying to scan. Maybe you could see if this principle works with object tracking too.

0 Kudos
LMart13
Novice
728 Views

I dont know how to align the cosID of the object with Unity.Your samples are all about virtual object.

0 Kudos
LMart13
Novice
726 Views

how can i use Unity to do with physical object

0 Kudos
MartyG
Honored Contributor III
726 Views

The Trackingaction script in Unity is placed inside a virtual object. It then tracks something in the real world such as a hand, face or object and changes the position and / or rotation of the virtual object in Unity based on those motions.

If I am reading your questions correctly, I do not think you want to move a virtual object. Instead, you want to know what the rotation and position of the real world object is? My logic in suggesting Unity is that by reading the rotation and position values of the virtual object, you should get an approximate reading of the changes in the real-world object.

If you want to track the real-world position coordinates of an object's position relative to the camera then you could use the SDK's ProjectCameraToDepth instruction without Unity. This would not give you the rotation value though.

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?projectcameratodepth_pxcprojection.html Intel® RealSense™ SDK 2016 R2 Documentation

0 Kudos
LMart13
Novice
726 Views

the values(pos & angles)of the virtual object related to the real world object,are 1:1 ? or need some mathematical formulas ? approximate values are not tolerated.

 

if i want to get the changes of object rotation,do i need to launch the Unity engine or just reference some libs to get Unity work,so that i can get the datas i want.

game engines are all resource consuming.this is the real reason i dont wan to use it in my programme.

i tried ProjectCameraToDepth before my current project,it was not so stable.memory leak error returned occasionally.since the depth camera and the color camera locate two different places,

i have to take it consideration.ProjectCameraToColor is a better choice,and the color coordinates are absolutely,which means stable.this is my experience for couple months work.

0 Kudos
MartyG
Honored Contributor III
726 Views

Tracking is unlikely to be precisely 1:1 in Unity, so this may not be the best choice for you.

Memory leaks were a problem for a long time with ProjectCameraToDepth and ProjectCameraToColor. The solution found was that at the end of the script, you need to do a Release() instruction to release the memory. Then the leak does not occur.

Example end of script:

// Resume next frame processing

sm.ReleaseFrame();

}

// Clean up

sm.Dispose();

0 Kudos
Reply