Items with no label
3335 Discussions

[Realsense SDK] Merging Scan & Align Object

AHoan1
Novice
10,376 Views

Dear,

I'm working on Realsense SDK and SR300,

1 - How to merge two PXCM3DScan or create multi PXCM3DScan in a PXCMSenseManager;

2 - Having two parts of an object, how to auto align?

3 - In PXCM3DScan, can you show me how to Reconstruct 2 PXCM3DScan to a file?

4 - Having *rssdk files of a model, how to merge?

Thanks!

71 Replies
idata
Employee
1,677 Views

Hello An.Ho,

 

 

Thanks for posting in the Intel RealSense community.

 

 

The RealSense SDK for Windows does not allow for multi-camera capture.

 

 

I'll recommend you to check the following community threads for more information:

 

 

How to use multiple SR300 looking at the same object

 

https://communities.intel.com/message/484501# 484501 https://communities.intel.com/message/484501# 484501

 

 

Multiple Camera and 3D Point Cloud Problems

 

https://communities.intel.com/message/464565# 464565 https://communities.intel.com/message/464565# 464565

 

 

Streaming from two cameras

 

https://communities.intel.com/message/460597# 460597 https://communities.intel.com/message/460597# 460597

 

 

There is no API in the SDK to achieve what you are planning, it is recommended for you to develop your own custom solution.

 

 

Hope this information helps to clarify your question.

 

 

Best Regards,

 

Juan N.
AHoan1
Novice
1,677 Views

Dear Juan N.,

Thank for your help, but this is unhelpful for me.

I have a new question.

How can using sm.Enable3DScan() and sm.EnableStreams(desc) in a PXCMSenseManager.

PXCMSenseManager can not init both at a time.

/************** thi is my code *************/

PXCMSenseManager sm = PXCMSenseManager.CreateInstance();

pxcmStatus result = sm.Enable3DScan();

PXCMSenseManager sm = PXCMSenseManager.CreateInstance();

pxcmStatus result = sm.Enable3DScan();

sm.QueryCaptureManager().SetFileName(file, false);

// Select the color stream

PXCMCapture.StreamType st = PXCMCapture.StreamType.STREAM_TYPE_DEPTH;

PXCMVideoModule.DataDesc desc = new PXCMVideoModule.DataDesc();

desc.streams[st].frameRate.min = desc.streams[st].frameRate.max = 30;

desc.streams[st].sizeMin.height = desc.streams[st].sizeMax.height = 480;

desc.streams[st].sizeMin.width = desc.streams[st].sizeMax.width = 640;

desc.streams[st].options = PXCMCapture.Device.StreamOption.STREAM_OPTION_ANY;

result = sm.EnableStreams(desc);

// Initialize and Record 300 frames

PXCM3DScan scan = null;

if (sm.Init() >= pxcmStatus.PXCM_STATUS_NO_ERROR){

}else{

//show message "PXCM_STATUS_ITEM_UNAVAILABLE"

}

Best Regards.

0 Kudos
idata
Employee
1,677 Views

Hello An.Ho,

 

 

Thanks for your update.

 

 

The DF_3DScan.cs sample demonstrate how to do this in the function StreamColorDepth beginning on line 229.

 

 

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/sample_3d_scan_cs.html

 

 

Best Regards,

 

Juan N.
0 Kudos
AHoan1
Novice
1,677 Views

Dear Juan. N

Thank you so much for your support,

I know how to scan with depth stream. But I want a sample How to scan and get model 3D form playback video rssdk instead of the camera.

Looking forward to your reply.

Best regards,

An. Ho

0 Kudos
AHoan1
Novice
1,677 Views

my code:

 

sm.QueryCaptureManager().SetFileName(file, false);

sm.EnableScan();

sm.Init() >> unavailable

0 Kudos
idata
Employee
1,677 Views

Hello An.Ho,

 

 

Thanks for your patience.

 

 

Your best reference for implementation of scanning using a .rssdk file is the DF_3Dscan.cs sample at C:\Program Files (x86)\Intel\RSSDK\framework\CSharp\DF_3DScan.cs. To create the file .rssdk file:

 

 

Select Mode->Record.

 

Click "Start Camera"

 

Click "Start Scanning"

 

Stop Scanning

 

Save .rssdk file.

 

 

After you have completed the scan and saved the file you can play it back. You can do the following steps with another .rssdk file that was recorded with the same camera model.

 

 

Select Mode->Playback.

 

Browse to the file you would like to play back.

 

Click "Start Camera"

 

Click "Start Scanning"

 

 

You can use clip_editor.exe, found at C:\Program Files (x86)\Intel\RSSDK\bin\win32\clip_editor.exe to edit and manipulate .rssdk files. See instructions here: https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?doc_tools_tool_clip_editor.html.

 

 

Hope this helps to clarify your question.

 

 

Best Regards,

 

Juan N.
0 Kudos
AHoan1
Novice
1,677 Views

Dear,

Thank for your support.

Now, I'm working on Intel Realsense viewer v2.10.0 and Realsense D415, But when I export 3D model to PLY format then I cannot open exported file.

Please refer to the attached file.

Best regards,

An.Ho

0 Kudos
AHoan1
Novice
1,677 Views

Can you show me how to export a ply file with the realsense D415.

If there is any source code c# , c++. Please share with me.

Thank you so much.

0 Kudos
MartyG
Honored Contributor III
1,677 Views

The RealSense Viewer tool that comes with the RealSense SDK 2.0 has an 'Export 3D model to ply' button. To find it:

(1) Click on the '3D' button'

(2) Click on the ply export button. Both are marked on the image below.

0 Kudos
AHoan1
Novice
1,677 Views

Dear MartyG,

That's exactly what I did but the *.ply file is the error. - Oh, man. Why?

I have some question as below:

1. How to get a model 3D from D415, is there any source code for the sample?

2. If I can recording and get model from a stereo module video with D415?

I'm looking forward to receiving your reply.

0 Kudos
MartyG
Honored Contributor III
1,677 Views

As far as I know, the RealSense Viewer is the best option for creating a 3D model scan easily with the D415 and D435 cameras. This is because there is not a separate model scanning program in this SDK like there was in previous RealSense SDKs, and model scanning using software platforms such as OpenCV has very little support - the same process as the Viewer has to be followed, creating a Point Cloud and then converting it into a mesh.

I tried exporting a PLY myself with my D415 using the RealSense Viewer and was able to do so successfully. Could you check two things for me please:

(1) Is your RealSense Viewer maximized in fullscreen? If you have the window shrunk then the ply controls can become unusable because the toolbar buttons overlap over the top of each other.

(2) At the side of the Viewer, does it say 'Intel RealSense 415' or 'Intel RealSense 415 (USB2)'. If it says USB2, this can mean that the camera is operating in USB 2.0 mode instead of full USB 3.0 (even if the camera is plugged into the USB 3.0 port) and the camera is not functioning to its full capability.

0 Kudos
MartyG
Honored Contributor III
1,677 Views

I was able to successfully import the PLY file into the free 3D modeling software Blender. So it appears to be the 3D Builder app that has an incompatibility with PLY files generated by the RealSense Viewer.

You can download Blender from here:

https://www.blender.org/download/ https://www.blender.org/download/

Once Blender is installed and run, you can import a PLY by going to File > Import > Stanford (.ply). Then the imported ply data appears in Blender.

0 Kudos
AHoan1
Novice
1,677 Views

I did it.

Thank you very much, but it does not show model 3D.

I think this is a 3d point cloud of a frame and it is very noisy

How to get 3D body of a person?

0 Kudos
MartyG
Honored Contributor III
1,677 Views

Yes, i believe this is point cloud data. The next step is to convert it into a mesh.

It could be done with the MeshLab program, as described in this guide.

https://www.andreasjakl.com/capturing-3d-point-cloud-intel-realsense-converting-mesh-meshlab/ https://www.andreasjakl.com/capturing-3d-point-cloud-intel-realsense-converting-mesh-meshlab/

If you want to stick with Blender, a good starting point for learning how to do so may be this tutorial YouTube video. The video is 7 years old but you may be able to apply the principles to the current version of Blender.

https://www.youtube.com/watch?v=wM52RaM6sTo From Cloud to Mesh using Blender - YouTube

0 Kudos
AHoan1
Novice
1,677 Views

Dear MartyG,

How can I use camera D415 with Realsense SDK R2 2016 to scan model?

- Using infrared instead of stereo?

Best regards,

An.Ho

0 Kudos
MartyG
Honored Contributor III
1,677 Views

The D415 does not work with the '2016 R2' SDK, unfortunately, as it requires SDK 2.0. I did actually try the D415 with the R2 SDK's 3DScan 3D scanning program just now, and confirmed that the D415 could not be detected by the program.

0 Kudos
AHoan1
Novice
1,677 Views

Thank for your support,

However, There is any sample source code for scanning with D415?

0 Kudos
MartyG
Honored Contributor III
1,677 Views

There is not a sample program yet for scanning with SDK 2.0. The method described earlier, of making a point cloud and turning it into a mesh, is the best known method so far for making a model scan with this SDK.

An alternative approach, which nobody has tried as far as I know, may be to try to get a commercial 3D scanning software package to work with the D415. Preferably, software that is free or has a 30 day free trail so that you can try them with the camera without risking wasting your money.

For example, ReconstructMe seems to have a flexible approach to detecting RealSense cameras, with the ability to configure options in sensor configuration files and a Librealsense start instruction (SDK 2.0 is a next-generation Librealsense, though not exactly the same as the original Librealsense).

http://reconstructme.net/2017/03/13/reconstructme-sdk-support-for-intel-realsense/ ReconstructMe SDK support for Intel RealSense | ReconstructMe

Another popular 3D scanning package with RealSense compatibility (at least, for older RealSense models) is ItSeez3D.

https://realsenseapp.intel.com/apps/itseez3d/ itSeez3D - Intel RealSense

0 Kudos
AHoan1
Novice
1,529 Views

I really do not understand, why the realsense D415 was created but there is not any sample source publish to scanning?

0 Kudos
Reply