1 2 Previous Next 17 Replies Latest reply on Jun 28, 2017 7:52 AM by laugre

    realsense SR300 R3 SDK+ Unity3d5.5.2f1 Bug!

    terryYu

      Hi everyone,

       

      This sample can be used in Editor, and can be released using exe.

       

      C:\Program Files (x86)\Intel\RSSDK\sample\core\RawStreams.unity\   ========>(Only Core)

       

      However, if you use Core+Face,

      Can be used in the Editor, but the release of exe can not work.

      -------------------------------------------------------------------------------------

      c# script =>start()

       

            faceModule = FaceModule.Activate(sm);    
              if (faceModule == null)
              {
                  Debug.LogError("FaceModule Initialization Failed");

              }
              //faceModule.FrameProcessed += FaceModule_FrameProcessed;

              FaceConfiguration moduleConfiguration = faceModule.CreateActiveConfiguration();
              if (moduleConfiguration == null)
              {
                  Debug.LogError("FaceConfiguration Initialization Failed");

              }
              moduleConfiguration.TrackingMode = TrackingModeType.FACE_MODE_COLOR;
              moduleConfiguration.Strategy = TrackingStrategyType.STRATEGY_RIGHT_TO_LEFT;
              moduleConfiguration.Detection.maxTrackedFaces = 1
              moduleConfiguration.Landmarks.maxTrackedFaces = 1

              moduleConfiguration.Detection.isEnabled = true;
              moduleConfiguration.Landmarks.isEnabled = true;

              moduleConfiguration.Pose.isEnabled = false;

                ...........

      -------------------------------------------------------------------------------------

      Log:

       

      UnloadTime: 0.882964 ms

      Fallback handler could not load library D:/Unity3d Project/RealSense Unity Project/Main_Data/Mono/libpxcface_c

      Fallback handler could not load library D:/Unity3d Project/RealSense Unity Project/Main_Data/Mono/.\libpxcface_c

      Fallback handler could not load library D:/Unity3d Project/RealSense Unity Project/Main_Data/Mono/libpxcface_c

      Fallback handler could not load library D:/Unity3d Project/RealSense Unity Project/Main_Data/Mono/libpxcface_c

      Fallback handler could not load library D:/Unity3d Project/RealSense Unity Project/Main_Data/Mono/.\libpxcface_c

      Fallback handler could not load library D:/Unity3d Project/RealSense Unity Project/Main_Data/Mono/libpxcface_c

      Fallback handler could not load library D:/Unity3d Project/RealSense Unity Project/Main_Data/Mono/libpxcface_c

      Fallback handler could not load library D:/Unity3d Project/RealSense Unity Project/Main_Data/Mono/.\libpxcface_c

      Fallback handler could not load library D:/Unity3d Project/RealSense Unity Project/Main_Data/Mono/libpxcface_c

      DllNotFoundException: libpxcface_c

        at (wrapper managed-to-native) Intel.RealSense.Face.FaceConfiguration:FaceConfiguration_SetTrackingMode (intptr,Intel.RealSense.Face.TrackingModeType)

        at Intel.RealSense.Face.FaceConfiguration.set_TrackingMode (TrackingModeType value) [0x00000] in <filename unknown>:0

        at RawStreamsController.Start () [0x00000] in <filename unknown>:0

      -------------------------------------------------------------------------------------

      In addition, I found no copy Face related DLL in runtime.  (D:\Unity3d Project\RealSense Unity Project\Main_Data\Plugins\runtime)

      I manually copy the DLL here also can not work. (C:\Program Files (x86)\Intel\RSSDK\runtime\face)

        • 1. Re: realsense SR300 R3 SDK+ Unity3d5.5.2f1 Bug!
          MartyG

          As a first step, please check through the steps in my guide to installing R3 in a Unity project to ensure that you have done each of the steps.  Thanks!

           

          A Guide To Setting up RealSense SDK 2016 R3 In Unity

          • 2. Re: realsense SR300 R3 SDK+ Unity3d5.5.2f1 Bug!
            terryYu

            MartyG

             

            C:\Program Files (x86)\Intel\RSSDK\doc\PDF\sdkdevguide.pdf

            yes.I checked it, and I didn't use R2 sdk.

             

            I submitted the bugDemo project on github. Please view. Thanks..

             

            GitHub - yty/RawStreams.unityBug: RawStreams.unityBug

             

            Run exe. can't work.

            Editor Run Work.

            • 3. Re: realsense SR300 R3 SDK+ Unity3d5.5.2f1 Bug!
              terryYu

              1. editor play

              only realsense.core =  work ok...

              realsense.core + realsense.face = work ok ....

               

              EditorPlay.jpg

               

              2. exe play

              only realsense.core =  work ok...

              realsense.core + realsense.face = work error ....

               

              RawStreams.unityBug/output_log.txt at master · yty/RawStreams.unityBug · GitHub

               

              ExePlay.jpg

              • 4. Re: realsense SR300 R3 SDK+ Unity3d5.5.2f1 Bug!
                MartyG

                It is night time in my time zone sadly, so I will try to look at your github on Monday morning.  Good luck!

                • 5. Re: realsense SR300 R3 SDK+ Unity3d5.5.2f1 Bug!
                  terryYu

                  The problem is more specific.  
                  Use the following functions will have a problem, there may be more..

                   

                  //moduleConfiguration.TrackingMode = TrackingModeType.FACE_MODE_COLOR;  //add => have bug

                  //Status applyChangesStatus = moduleConfiguration.ApplyChanges(); //add => have bug

                  //moduleConfiguration.EnableAllAlerts(); //add => have bug

                  • 6. Re: realsense SR300 R3 SDK+ Unity3d5.5.2f1 Bug!
                    MartyG

                    It is a difficult issue to diagnose.  When a full Build and Run test is done in Unity (what you call an exe run), Unity accesses some functions that it does not use in the editor run.  So a program may run fine in the editor but have breakages in the full Build and Run mode.

                     

                    Logic would suggest that if your project works in Build and Run with the Core (Essentials) module only but fails when the optional Face module is also installed, then there is something in the Face module that Unity does not like when in Build and Run mode.

                     

                    In my own R3 Unity installation, I never encountered a file called Intel.RealSense.Face.unity.dll.  When I imported the Face module, the files in the project were:

                     

                    Intel.RealSense.unity.dll

                    libpxcccore.dll

                    libpxcface.dll

                     

                    As far as I know, these three DLL files should be all that your project needs for it to use the Face features, if you installed R3 using the steps in my guide (import the Core module into your project first and then import the Face module). .

                     

                    But as I said earlier, if your Core + Face program runs fine in the editor but not in Build and Run, then that suggests that Unity is finding a conflict in the Face module when it accesses whatever extra features it uses to generate the Build and Run mode.

                    • 7. Re: realsense SR300 R3 SDK+ Unity3d5.5.2f1 Bug!
                      laugre

                      Hi

                      I get the same problem

                      Fallback handler could not load library D:/Blue Yeti/laugre/new_unity_projects/bin/paleosite/un_air_de_famille_Data/Mono/libpxcface_c

                      Fallback handler could not load library D:/Blue Yeti/laugre/new_unity_projects/bin/paleosite/un_air_de_famille_Data/Mono/.\libpxcface_c

                      Fallback handler could not load library D:/Blue Yeti/laugre/new_unity_projects/bin/paleosite/un_air_de_famille_Data/Mono/libpxcface_c

                      Fallback handler could not load library D:/Blue Yeti/laugre/new_unity_projects/bin/paleosite/un_air_de_famille_Data/Mono/libpxcface_c

                      Fallback handler could not load library D:/Blue Yeti/laugre/new_unity_projects/bin/paleosite/un_air_de_famille_Data/Mono/.\libpxcface_c

                      Fallback handler could not load library D:/Blue Yeti/laugre/new_unity_projects/bin/paleosite/un_air_de_famille_Data/Mono/libpxcface_c

                      DllNotFoundException: libpxcface_c

                        at (wrapper managed-to-native) Intel.RealSense.Face.FaceConfiguration:FaceConfiguration_EnableAllAlerts (intptr)

                        at Intel.RealSense.Face.FaceConfiguration.EnableAllAlerts () [0x00000] in <filename unknown>:0

                        at RealsenseController.Start () [0x00000] in <filename unknown>:0

                       

                      I tried in both 32 and 64 bits build but that's the same.

                      I checked in my unity binary Plugin folder and I have libpxccore_c.dll and libpxcface_idc.dll but that doesn't work.

                      Did you find a solution ?

                       

                      Thank's

                      • 8. Re: realsense SR300 R3 SDK+ Unity3d5.5.2f1 Bug!
                        laugre

                        Hi again,

                         

                        Still get the same error with a call to ApplyChanges().

                         

                        Fallback handler could not load library D:/Blue Yeti/laugre/new_unity_projects/bin/paleositeUnity5.6.0/un_air_de_famille_Data/Mono/libpxcface_c

                        Fallback handler could not load library D:/Blue Yeti/laugre/new_unity_projects/bin/paleositeUnity5.6.0/un_air_de_famille_Data/Mono/.\libpxcface_c

                        Fallback handler could not load library D:/Blue Yeti/laugre/new_unity_projects/bin/paleositeUnity5.6.0/un_air_de_famille_Data/Mono/libpxcface_c

                        DllNotFoundException: libpxcface_c

                          at (wrapper managed-to-native) Intel.RealSense.Face.FaceConfiguration:FaceConfiguration_ApplyChanges (intptr,Intel.RealSense.Face.AllFaceConfigurations)

                          at Intel.RealSense.Face.FaceConfiguration.ApplyChanges () [0x00000] in <filename unknown>:0

                          at RealsenseController.Start () [0x00000] in <filename unknown>:0

                         

                        What is strange is that if I comment calls to EnableAllAlerts(), ApplyChanges() the built executable work well and I can use FaceModule data functions like faceDatas.QueryFaceByIndex(0)

                        but I don't think face config is applied.

                        Does the unity wrapper implement all SDK features ?

                        Any idea ?

                        Here is my code :

                         

                        // Use this for initialization

                            void Start()

                            {

                                /* Create SenseManager Instance */

                                sm = SenseManager.CreateInstance();

                         

                         

                                /* Create a SampleReader Instance */

                                sampleReader = SampleReader.Activate(sm);

                         

                         

                                /* Enable Color & Depth Stream */

                                sampleReader.EnableStream(StreamType.STREAM_TYPE_COLOR, colorWidth, colorHeight, colorFPS);

                         

                         

                                /* Subscribe to sample arrived event */

                                sampleReader.SampleArrived += SampleArrived;

                         

                         

                                faceModule = FaceModule.Activate(sm);

                                faceDatas = faceModule.CreateOutput();

                                faceConfig = faceModule.CreateActiveConfiguration();

                                //faceConfig.EnableAllAlerts();

                                faceConfig.Detection.isEnabled = true;

                                faceConfig.Landmarks.isEnabled = true;

                                faceConfig.Pose.isEnabled = true;

                                //faceConfig.TrackingMode = TrackingModeType.FACE_MODE_COLOR;

                                faceConfig.Strategy = Intel.RealSense.Face.TrackingStrategyType.STRATEGY_CLOSEST_TO_FARTHEST;

                                faceConfig.Detection.maxTrackedFaces = 1;

                                faceConfig.Landmarks.maxTrackedFaces = 1;

                                faceConfig.Pose.maxTrackedFaces = 1;

                                //faceConfig.ApplyChanges();

                                faceModule.FrameProcessed += FaceModule_FrameProcessed;

                         

                         

                                // Initialize pipeline

                                sm.Init();

                         

                         

                                // Create NativeTexturePlugin to render Texture2D natively

                                texPlugin = NativeTexturePlugin.Activate();

                         

                         

                                // Update material's Texture2D with enabled image size.

                                RGBMaterial.mainTexture = new Texture2D(colorWidth, colorHeight, TextureFormat.BGRA32, false);

                         

                         

                                // Flip the image

                                //RGBMaterial.mainTextureScale = new Vector2(-1, -1);

                         

                         

                                // Retrieve native Texture2D Pointer

                                colorTex2DPtr = RGBMaterial.mainTexture.GetNativeTexturePtr();

                         

                         

                                // Start Streaming

                                sm.StreamFrames(false);

                         

                         

                            }

                         

                         

                            void SampleArrived(object sender, SampleArrivedEventArgs args)

                            {

                                if (cameraPaused) return;

                                if (args.sample.Color != null) texPlugin.UpdateTextureNative(args.sample.Color, colorTex2DPtr);

                            }

                         

                         

                            private void FaceModule_FrameProcessed(object sender, FrameProcessedEventArgs args)

                            {

                                if (cameraPaused) return;

                                faceDatas.Update();

                         

                         

                                Face face = faceDatas.QueryFaceByIndex(0);

                                if (face != null)

                                {

                                    ldata = face.Landmarks;

                                    if (ldata != null)

                                    {

                                        points = ldata.Points;

                                    }

                         

                         

                                    PoseData posDatas = face.Pose;

                                    if (posDatas != null)

                                    {

                                        headPos.Set(-posDatas.HeadPosition.headCenter.x / 1000, posDatas.HeadPosition.headCenter.y / 1000, posDatas.HeadPosition.headCenter.z / 1000);

                                        headRot = Quaternion.Euler(-posDatas.Angles.pitch, -posDatas.Angles.yaw, posDatas.Angles.roll);

                                    }

                                }

                            }

                        • 9. Re: realsense SR300 R3 SDK+ Unity3d5.5.2f1 Bug!
                          MartyG

                          I'm afraid I do not yet have enough knowledge about using the R3 SDK with Unity to be able to identify your project's error, as I have only had time to spend a couple of days with R3 before I had to return to the R2 SDK, which my own full-time project is based in.

                           

                          I did some research in general on Unity's "Fallback handler could not load library" error though.  A lot of people seemed to have experienced it in non-RealSense projects.  People reported that their project still ran despite the errors - it was just annoying to have the debug console fill up with the messages.

                           

                          A frequent cause of the error seemed to be if a Unity project was started on one PC and then transferred to another PC and the user tried to continue developing the project on that machine. Have you moved your own project between machines?

                           

                          For more information on this subject, you can google for 'fallback handler could not load library unity'

                           

                          A safe way to transfer Unity content (objects, scripts, etc) from an existing project into a fresh project file on a new machine, so you do not have to start from the beginning, is to package your existing project's content into an 'Asset Package'.   You can then import the created package into a new project file on another PC.

                           

                          Unity - Manual: Exporting Packages

                          • 10. Re: realsense SR300 R3 SDK+ Unity3d5.5.2f1 Bug!
                            laugre

                            Hi MartyG

                            Thx for the search and links.

                            I reproduced the problem with the R3 SDK unity sample Rawstream.unity project by adding these lines of code

                                  faceModule = FaceModule.Activate(sm);

                                    faceDatas = faceModule.CreateOutput();

                                    faceConfig = faceModule.CreateActiveConfiguration();

                                    faceConfig.EnableAllAlerts();

                                    faceConfig.Detection.isEnabled = true;

                                    faceConfig.Landmarks.isEnabled = true;

                                    faceConfig.Pose.isEnabled = true;

                                    faceConfig.TrackingMode = TrackingModeType.FACE_MODE_COLOR;

                                    faceConfig.Strategy = Intel.RealSense.Face.TrackingStrategyType.STRATEGY_CLOSEST_TO_FARTHEST;

                                    faceConfig.Detection.maxTrackedFaces = 1;

                                    faceConfig.Landmarks.maxTrackedFaces = 1;

                                    faceConfig.Pose.maxTrackedFaces = 1;

                                    faceConfig.ApplyChanges();

                                    faceModule.FrameProcessed += FaceModule_FrameProcessed;

                             

                            error was :

                            Fallback handler could not load library D:/Blue Yeti/laugre/new_unity_projects/bin/rawstream/test_Data/Mono/libpxcface_c

                            Fallback handler could not load library D:/Blue Yeti/laugre/new_unity_projects/bin/rawstream/test_Data/Mono/.\libpxcface_c

                            Fallback handler could not load library D:/Blue Yeti/laugre/new_unity_projects/bin/rawstream/test_Data/Mono/libpxcface_c

                            DllNotFoundException: libpxcface_c

                              at (wrapper managed-to-native) Intel.RealSense.Face.FaceConfiguration:FaceConfiguration_ApplyChanges (intptr,Intel.RealSense.Face.AllFaceConfigurations)

                              at Intel.RealSense.Face.FaceConfiguration.ApplyChanges () [0x00000] in <filename unknown>:0

                              at RawStreamsController.Start () [0x00000] in <filename unknown>:0

                             

                            Some dll seemed to be not copied in binary directory.

                            But I finaly got the built executable working by adding face module dlls in binary Plugin directory manually like this :

                            Capture.PNG

                            1 of 1 people found this helpful
                            • 11. Re: realsense SR300 R3 SDK+ Unity3d5.5.2f1 Bug!
                              MartyG

                              Thanks so much for the information on how you resolved your problem.  It is sure to be useful to others who encounter this error.  Best of luck with the rest of your project, and please come back to the forum any time if you need further assistance. 

                              • 12. Re: realsense SR300 R3 SDK+ Unity3d5.5.2f1 Bug!
                                laugre

                                I'm facing with a crash I related to in this thread SR300 + R3 SDK + Unity 3D 5.6.0 Crash !!!
                                I'm wondering if R2 SDK is better in stability with Unity3D.

                                Regarding your samples you use it a lot MartyG.

                                 

                                Do you know if with R2SDK I can :

                                - get a 1920x1080 color image

                                - get head position and head rotation

                                - get face landmarks

                                • 13. Re: realsense SR300 R3 SDK+ Unity3d5.5.2f1 Bug!
                                  MartyG

                                  I'm not sure I can say whether R2 is more stable than R3 with Unity, as I have only used R3 with Unity for a couple of days.  I think the best way to put it would be that the RealSense tools are far more proven with R2, whereas with R3 you do not have a Unity Toolkit and so have to write your own RealSense scripts.

                                   

                                  I recently used Unity with 1920x1080 in the 'Intel Level Up' game development contest and my RealSense F200 camera worked fine with hand and face tracking at that resolution.  I have not used it for color streaming at that resolution though, so can't offer insight on how a color stream performs under that resolution setting.

                                  • 14. Re: realsense SR300 R3 SDK+ Unity3d5.5.2f1 Bug!
                                    laugre

                                    ok thx for informations

                                    ...and would you have a sample on how to use face tracking landmarks with R2 plz ?

                                    1 2 Previous Next