SR300 3D scan in java

[CTian4]

I can find java library libpxcclr.java.jar In SDK intel_rs_sdk_offline_package_10.0.26.0396.exe, and I can see API in website https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?pxcsensemanager.html https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/d...

but there are some different between the API and the jar. Such as the PXCM3DScan.java、PXCMSenseManager.java and so on.

Any way ,I try to copy the java code which is used to achieve the 3Dscan, but when I run the sample code and save the scan file with function Reconstruct, it will be report an error PXCM_STATUS_ITEM_UNAVAILABLE. Is there any body can tell me how to slove this problem? How can I get the correct jar? May be it can not be achieved in java? Help,please.

[idata]

Hello 与叶寻,

Thanks for reaching out! Could you please share with us the code that you are using? We would like to see if we can replicate this behavior or if we can find any issue in the code. Also, please keep in mind that some of the Java functions may or may not work. So, it is recommended to use C in those cases. We'll be waiting for your response.-Peter.

[CTian4]

Hello Peter,

The following code is my test code

public void scan() {

// Create a SenseManager instance

boolean bAutoExpAndWBChanged = false;

boolean bAutoExposureEnabled = true;

boolean bAutoWhiteBalanceEnabled = true;

PXCMSenseManager sm = PXCMSenseManager.CreateInstance();

pxcmStatus sts = null;

PXCMCapture.DeviceInfo dinfo2 = new PXCMCapture.DeviceInfo();

sm.QueryCaptureManager().FilterByDeviceInfo(dinfo2);

// Enable 3D Scanning

sts = sm.Enable3DScan();

System.out.println("Enable3DScan result: " + sts);

// use SetConfiguration

// Initialize the pipeline

sts = sm.Init();

PrintConnectedDevices();

// Get a 3D scanning instance here for configuration

PXCM3DScan scan = sm.Query3DScan();

// pxcmStatus result =null;

if (scan == null)

System.out.println("3DScan module not found.");

PXCMCapture.Device device = sm.QueryCaptureManager().QueryDevice();

PXCMCapture.Device.StreamProfileSet profiles = new PXCMCapture.Device.StreamProfileSet();

if (device == null) {

System.out.println("==ts add==Please link the camera===");

scan_completed = true;

timer.cancel();

return;

}

DrawFrame c_df = new DrawFrame(cWidth, cHeight);

JFrame cframe = new JFrame("Intel(R) RealSense(TM) SDK - Scan Stream");

Listener listener = new Listener();

cframe.addWindowListener(listener);

cframe.setSize(cWidth, cHeight);

cframe.add(c_df);

cframe.setVisible(true);

PXCMImage image = null;

if (sts == pxcmStatus.PXCM_STATUS_NO_ERROR) {

boolean res = false;

// Stream data

while (!scan_completed) {

res = sm.AcquireFrame(true).isSuccessful();

// Preview the scanning results

image = scan.AcquirePreviewImage();

PXCMImage.ImageData cData = new PXCMImage.ImageData();

sts = image.AcquireAccess(PXCMImage.Access.ACCESS_READ,

PXCMImage.PixelFormat.PIXEL_FORMAT_RGB32, cData);

if (sts.compareTo(pxcmStatus.PXCM_STATUS_NO_ERROR) < 0) {

System.out

.println("Failed to AcquireAccess of color image data");

System.exit(3);

}

int cBuff[] = new int[cData.pitches[0] / 4 * cHeight];

cData.ToIntArray(0, cBuff);

c_df.image.setRGB(0, 0, cWidth, cHeight, cBuff, 0,

cData.pitches[0] / 4);

c_df.repaint();

image.ReleaseAccess(cData);

if (sts.compareTo(pxcmStatus.PXCM_STATUS_NO_ERROR) < 0) {

System.out

.println("Failed to ReleaseAccess of color image data");

System.exit(3);

}

// If scanning is completed, break out.

if (scan_completed)

break;

// Resume next frame processing

sm.ReleaseFrame();

}

System.out.println("====3==scan===scan_completed===="

+ scan_completed + "/" + res);

}

// Save the scanned object

// scan.Reconstruct(PXCM3DScan.FileFormat.STL,

// "C:\\mywork\\myobject.stl");

sts = scan.Reconstruct(PXCM3DScan.FileFormat.STL,

"E:\\ts\\Exclipse\\3DScan\\test\\myobject.stl");

System.out.println("====3==scan===save sts====="

+ sts);

// Clean up

sm.close();

if (image != null)

image.close();

cframe.dispose();

if (bAutoExpAndWBChanged) {

device.SetColorAutoExposure(bAutoExposureEnabled);

device.SetColorAutoWhiteBalance(bAutoWhiteBalanceEnabled);

}

device.close();

device = null;

}

and the log is alaways

Enable3DScan result: PXCM_STATUS_NO_ERROR

Device name: Intel(R) RealSense(TM) 3D Camera SR300

Found 1 devices

===============TimerTask================

====3==scan===scan_completed====true/true

====3==scan===save sts=====PXCM_STATUS_ITEM_UNAVAILABLE

Thanks,

Carol

[idata]

Hi 与叶寻,

Let me run some tests with your code. I'll try to get back to you as soon as possible.-Peter.

[idata]

Hi Carol,

I'd like to ask you if you've been able to use any of the SDK sample codes. You can find the sample codes in https://software.intel.com/en-us/articles/intel-realsense-sdk-code-samples/. This is to confirm that everything in the setup is working. Please run one of these examples and let us know how it goes.-Peter.

[CTian4]

Dear Peter,

I have test the sample code in the webside you gave, every project will get an error "System.NullReferenceException". In project begin after get session by "PXCMSession session = PXCMSession.CreateInstance();" it will return null and next go will get the error. But when I use the Intel RealSense SDK Sample Browser R2 and run the sample program "3D Scan(C# )" which source file is "DF_3DScan.cs", it worked very well. And the "3D Scan(C# )" program also use "PXCMSession session = PXCMSession.CreateInstance();" to get a PXCMSession example. Finally I found the lib "libpxcclr.cs.dll" and "libpxccpp2c.dll" are different between the two programs, I changed the two libraries in sample code with which in program "3D Scan(C# )" then the sample code was working properly except project "Emotion" and "Emotion-Callback". I thought may be my camera upgraded the firmware before caused this issue, any way my camera is working good.

May I ask the test situation about my code ? Thanks!

Carol

[idata]

Hello Carol,

Thank you for confirming that the examples worked. We are still working on the tests.Meanwhile, I'd like to mention that the error message you are receiving "ITEM_UNAVAILABLE" occurs when the camera is not being detected. Could you please confirm that the camera is being detected in Device Manager with entries for RGB, Depth, and Virtual Driver? If the camera is indeed detected, could you please share a picture of the Device Manager where this is shown?Also, could you please share a screenshot of one of the examples running? You can find the actual samples in the SDK installation at C:\Program Files (x86)\Intel\RSSDK\bin\x64.We will be waiting for your response.-Peter.

[CTian4]

Hi Peter,

The camera can be detected in Device Manager with entries for RGB,Depth,and Vritual Driver,following is the picture

My code also can get a scan screen and I get a picture for you

"3D Scan(C# )" which source file is "DF_3DScan.cs", installation package is "DF_3DScan.cs.exe" its screenshot is

and I can generate scan files *.obj or *.html from this program. Thanks!

Carol.

[idata]

Hi Carol,

Thank you for confirming this information. It is very strange that you are receiving the "ITEM_UNAVAILABLE" error, even more if you consider that the camera is indeed being recognized and it is functional as it can be seen in the screenshots you posted above.I would like to verify if this behavior only happens when using Java. I mean, it could be an issue with the JDK installation or something related. Could you please test a Java sample code? If any of them is functional please post a screenshot of the sample running. The Java samples can be found in C:\Program Files (x86)\Intel\RSSDK\framework\Java.Let us know.-Peter.

[CTian4]

Hi Peter

All three sample codes had been test OK, they all functional, the following is the screenshot

java_DF_CsmeraViewer:

java_DF_FaceTracking:

java_FF_HandsViewer:

And I also test my code with run.bat then get same error as use eclipse. The entire scan process seems to have not started, there is no green box appears just like a camera screen. There is no sense of kind of sticky in scanning process like C# sample code when we move fast then get a blurred screen. It's just like I didn't made scan start, and I cann't find any configration about java scan like "Configuration" it cann't be modified in java library.

By the way what about the test result of my code in your side? Does it work? Or get the same error? I am looking forward to your reply。

Thanks，

Carol

[idata]

Hi Carol,

Thank you for providing us this information. We are still working on this. Please let me investigate what might be happening. I'll try to contact you back as soon as possible.-Peter.

[idata]

Hi Carol,

I've an update on your case.As you know, we have been testing your code, please correct me if I'm wrong but this is not the entire code, right? If so, could you please provide us the entire code? This code snippet has given us some trouble to compile. Also, I noticed that the file location you're trying to save the scan into is "E:\\ts\\Exclipse\\3DScan\\test\\myobject.stl". Please check that this is the correct path because it seems there is a type-o. Shouldn't "Exclipse" be "Eclipse?" We believe the "ITEM_UNAVAILABLE" error is happening because of this command: sts = scan.Reconstruct(PXCM3DScan.FileFormat.STL, "E:\\ts\\Exclipse\\3DScan\\test\\myobject.stl"); The sts object is not being created. I think it's because of the type-o.I mean, since the Java examples are working for you, the issue is most likely found in the code.Let us know.-Peter.

[CTian4]

Hi Peter

The following is the etire code,

package scan;

import intel.rssdk.PXCM3DScan;

import intel.rssdk.PXCMCapture;

import intel.rssdk.PXCMImage;

import intel.rssdk.PXCMSenseManager;

import intel.rssdk.PXCMSession;

import intel.rssdk.pxcmStatus;

import java.awt.Component;

import java.awt.Graphics;

import java.awt.Graphics2D;

import java.awt.event.WindowAdapter;

import java.awt.event.WindowEvent;

import java.awt.image.BufferedImage;

import java.util.EnumSet;

import java.util.Timer;

import java.util.TimerTask;

import javax.swing.JFrame;

public class MyClassTest{

private PXCMCapture.Device device;

private static boolean scan_completed = false;

static int dWidth, dHeight;

static int cWidth = 640;

static int cHeight = 480;

private static Timer timer;

static MyClassTest mc;

public static void main(String[] args) {

System.out.println("==============" + System.getProperty("os.arch"));

System.out.println(System.getProperty("java.library.path "));

mc = new MyClassTest();

scan_completed = false;

mc.timer = new Timer();

mc.timer.schedule(tt, 20000);

mc.scan();

}

static TimerTask tt = new TimerTask() {

@Override

public void run() {

System.out.println("===============TimerTask================");

scan_completed = true;

}

};

public void scan() {

// Create a SenseManager instance

boolean bAutoExpAndWBChanged = false;

boolean bAutoExposureEnabled = true;

boolean bAutoWhiteBalanceEnabled = true;

PXCMSenseManager sm = PXCMSenseManager.CreateInstance();

pxcmStatus sts = null;

PXCMCapture.DeviceInfo dinfo2 = new PXCMCapture.DeviceInfo();

sm.QueryCaptureManager().FilterByDeviceInfo(dinfo2);

// Enable 3D Scanning

sts = sm.Enable3DScan();

System.out.println("Enable3DScan result: " + sts);

// use SetConfiguration

// Initialize the pipeline

sts = sm.Init();

PrintConnectedDevices();

// Get a 3D scanning instance here for configuration

PXCM3DScan scan = sm.Query3DScan();

// pxcmStatus result =null;

if (scan == null)

System.out.println("3DScan module not found.");

PXCMCapture.Device device = sm.QueryCaptureManager().QueryDevice();

PXCMCapture.Device.StreamProfileSet profiles = new PXCMCapture.Device.StreamProfileSet();

if (device == null) {

System.out.println("==ts add==Please link the camera===");

scan_completed = true;

timer.cancel();

return;

}

DrawFrame c_df = new DrawFrame(cWidth, cHeight);

JFrame cframe = new JFrame("Intel(R) RealSense(TM) SDK - Scan Stream");

Listener listener = new Listener();

cframe.addWindowListener(listener);

cframe.setSize(cWidth, cHeight);

cframe.add(c_df);

cframe.setVisible(true);

PXCMImage image = null;

if (sts == pxcmStatus.PXCM_STATUS_NO_ERROR) {

boolean res = false;

// Stream data

while (!scan_completed) {

res = sm.AcquireFrame(true).isSuccessful();

// Preview the scanning results

image = scan.AcquirePreviewImage();

PXCMImage.ImageData cData = new PXCMImage.ImageData();

sts = image.AcquireAccess(PXCMImage.Access.ACCESS_READ,

PXCMImage.PixelFormat.PIXEL_FORMAT_RGB32, cData);

if (sts.compareTo(pxcmStatus.PXCM_STATUS_NO_ERROR) < 0) {

System.out

.println("Failed to AcquireAccess of color image data");

System.exit(3);

}

int cBuff[] = new int[cData.pitches[0] / 4 * cHeight];

cData.ToIntArray(0, cBuff);

c_df.image.setRGB(0, 0, cWidth, cHeight, cBuff, 0,

cData.pitches[0] / 4);

c_df.repaint();

image.ReleaseAccess(cData);

if (sts.compareTo(pxcmStatus.PXCM_STATUS_NO_ERROR) < 0) {

System.out

.println("Failed to ReleaseAccess of color image data");

System.exit(3);

}

// If scanning is completed, break out.

if (scan_completed)

break;

// Resume next frame processing

sm.ReleaseFrame();

}

System.out.println("====3==scan===scan_completed===="

+ scan_completed + "/" + res);

}

// Save the scanned object

// scan.Reconstruct(PXCM3DScan.FileFormat.STL,

// "C:\\mywork\\myobject.stl");

sts = scan.Reconstruct(PXCM3DScan.FileFormat.STL,

"E:\\ts\\myobject.stl");

System.out.println("====3==scan===save sts=====" + sts);

// Clean up

sm.close();

if (image != null)

image.close();

cframe.dispose();

if (bAutoExpAndWBChanged) {

device.SetColorAutoExposure(bAutoExposureEnabled);

device.SetColorAutoWhiteBalance(bAutoWhiteBalanceEnabled);

}

device.close();

device = null;

}

public static void PrintConnectedDevices() {

PXCMSession session = PXCMSession.CreateInstance();

PXCMSession.ImplDesc desc = new PXCMSession.ImplDesc();

PXCMSession.ImplDesc outDesc = new PXCMSession.ImplDesc();

desc.group = EnumSet.of(PXCMSession.ImplGroup.IMPL_GROUP_SENSOR);

desc.subgroup = EnumSet

.of(PXCMSession.ImplSubgroup.IMPL_SUBGROUP_VIDEO_CAPTURE);

int numDevices = 0;

for (int i = 0;; i++) {

if (session.QueryImpl(desc, i, outDesc).isError())

break;

PXCMCapture capture = new PXCMCapture();

if (session.CreateImpl(outDesc, capture).isError())

continue;

for ...

[idata]

Hello Carol,

Thank you for providing us this information, it will be of much help. Please let us run some tests and analyze the data you provided us, we'll try to get back to you as soon as possible. -Peter.

[idata]

Hello Carol,

We have been able to reproduce this issue, yet we have not been able to find out why it is happening.However, I must point out that 3D Object Scan with SR300 is in Beta mode in 2016R2 and that Java support has been deprecated as of 2016R3. Therefore, there will be no new code updates for Java.We have contacted the SDK developers and we are trying to get a conclusive response for this issue but unfortunately it may take a few weeks.-Peter.

[CTian4]

Hello Peter,

Thank you very much. I will waiting for your response. If in any case will not work, I will change the way a coding. Thanks!

Carol

[idata]

Hi Carol,

I'm glad to be of help!Please note that an update for this case might take a while, so, my best suggestion for this situation is that you try using a different language, specifically one that is still supported on the latest SDK.If you have any further doubts, please don't hesitate to come back to the community, we'll try to help you in any way we can.-Peter.

[CTian4]

Hi Peter

OK, I understand, thanks!

Carol