4 Replies Latest reply on Jun 2, 2017 1:23 PM by CamilaGL Branched to a new discussion.

    Get landmarks ofan external JPG image

    MQMQ

      Hi,

       

      I'm wondering if I can get the landmarks of an external image from a dataset using the SDK Intel algorithms. Or if I can convert this one to an 'Intel Image'.

       

      I mean, I want to extract the landmarks from a JPG face but for that I think I need to convert this file into an PXCImage. I don't know if I right with this supposition, hope someone can help.

       

      Thanks in advance

        • 1. Re: Get landmarks ofan external JPG image
          Intel Corporation
          This message was posted on behalf of Intel Corporation

          Hi MQMQ,

           

          Thanks for your interest in the Intel® Realsense™ Platform.

           

          At the moment I only have knowledge of getting the landmark features from a face configuration Intel® RealSense™ SDK 2016 R2 Documentation . Looking at this thread Can I get the landmarks from an external image? , it seems that MartyG recommended you the PXCVideoModule function, please let us investigate if this function is no longer part of the Realsense API.

           

          We appreciate your patience, have a nice day!

           

          Best Regards,
          -Jose P.

          • 2. Re: Get landmarks ofan external JPG image
            Intel Corporation
            This message was posted on behalf of Intel Corporation

            Hi MQMQ,

            We appreciate your patience.

            Unfortunately, extracting landmarks from a JPG image is an unsupported use case. It may or may not work, we would like encourage to try to achieve this and share your results with the community. It will be really helpful for other users looking to get the landmark from an external JPG image.

            Best Regards,
            -Jose P.

            • 3. Re: Get landmarks ofan external JPG image
              MQMQ

              Thank you, I haven't been able to do it yet, but if I find something, sure I post it here.

              • 4. Re: Get landmarks ofan external JPG image
                CamilaGL

                Hello.
                I've been working towards the same functionality, and I've achieved it - to a certain point.

                Currently using C# and Intel(R) RealSense(TM) SDK 10.0.26.396, and using Columbia Gaze Data Set for analysis.

                 

                The code that I wrote gets the landmarks and face detection for a single jpg image but, after that, the program always exits with one of the following errors:

                • The program '[15956] DF_FaceTracking.cs.vshost.exe' has exited with code -1073740940 (0xc0000374), or
                • An unhandled exception of type 'System.AccessViolationException' occurred in libpxcclr.cs.dll

                 

                This is the relevant excerpt:

                 

                public void ExternalPipeline2()
                        {
                            PXCMSenseManager pp = m_form.Session.CreateSenseManager();
                            if (pp == null)
                            {
                                throw new Exception("PXCMSenseManager null");
                            }
                
                            PXCMCaptureManager captureMgr = pp.captureManager;
                            if (captureMgr == null)
                            {
                                throw new Exception("PXCMCaptureManager null");
                            }
                
                            // Set Module            
                            pp.EnableFace();
                            PXCMFaceModule faceModule = pp.QueryFace();
                            if (faceModule == null)
                            {
                                Debug.Assert(faceModule != null);
                                return;
                            }
                
                            //Up until here, it's just the same as the FaceTracking sample.
                            PXCMFaceConfiguration moduleConfiguration = faceModule.CreateActiveConfiguration();
                            if (moduleConfiguration == null)
                            {
                                Debug.Assert(moduleConfiguration != null);
                                return;
                            }
                
                            moduleConfiguration.SetTrackingMode(PXCMFaceConfiguration.TrackingModeType.FACE_MODE_COLOR);
                            moduleConfiguration.strategy = PXCMFaceConfiguration.TrackingStrategyType.STRATEGY_RIGHT_TO_LEFT;
                
                            moduleConfiguration.detection.maxTrackedFaces = 1;
                            moduleConfiguration.landmarks.maxTrackedFaces = 1;
                            moduleConfiguration.pose.maxTrackedFaces = 1;
                            moduleConfiguration.detection.isEnabled = true;
                            moduleConfiguration.landmarks.isEnabled = true;
                            moduleConfiguration.pose.isEnabled = false;
                
                
                            pxcmStatus applyChangesStatus = moduleConfiguration.ApplyChanges();
                            if (applyChangesStatus < pxcmStatus.PXCM_STATUS_NO_ERROR || pp.Init() < pxcmStatus.PXCM_STATUS_NO_ERROR)
                            {
                                Console.WriteLine("Init Failed");
                                m_form.UpdateStatus("Init Failed", MainForm.Label.StatusLabel);
                            }
                            else
                            {
                                // Read from file
                                string sourceDir = @"C:\Directory\"; //There's only one image in the source folder
                                string[] fileEntries = Directory.GetFiles(sourceDir, "*.jpg");
                
                                PXCMVideoModule videoModule = faceModule.QueryInstance<PXCMVideoModule>();
                               
                               foreach (string fileEntry in fileEntries)
                                {
                                    Bitmap b = (Bitmap)ResizeImage(fileEntry); //I'm resizing it to a max of 1920x1280
                                   
                                    w = b.Width;
                                    h = b.Height;
                                    PXCMImage.ImageInfo iinfo = new PXCMImage.ImageInfo()
                                    {
                                        width = b.Width,
                                        height = b.Height,
                                        format = PXCMImage.PixelFormat.PIXEL_FORMAT_RGB24 //All images I'm working with are rgb24
                                    };
                
                                    // Lock bits to mass edit
                                    BitmapData bdata = b.LockBits(new Rectangle(0, 0, b.Width, b.Height),
                                               ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);
                
                                    // Get the address of the first line.
                                    IntPtr ptr = bdata.Scan0;
                
                                    // Declare an array to hold the bytes of the bitmap.
                                    int bytes = b.Width * b.Height;
                                    byte[] rgbValues = new byte[bytes];
                
                                    // Copy the RGB values into the array.
                                    System.Runtime.InteropServices.Marshal.Copy(ptr, rgbValues, 0, bytes);
                
                                    PXCMImage.ImageData idata = new PXCMImage.ImageData();
                                    idata.format = PXCMImage.PixelFormat.PIXEL_FORMAT_RGB24; //This should match the info object, I reckon
                                    idata.planes[0] = ptr; //This is the critical bit. This is the actual data.
                                    idata.pitches[0] = bdata.Stride; // ALIGN64(iinfo.width * 4);
                
                                    // Create image with data and info
                                    PXCMImage img = m_form.Session.CreateImage(iinfo, idata);
                                    b.UnlockBits(bdata);
                
                                    //Create sample with image
                                    PXCMCapture.Sample sample = new PXCMCapture.Sample();
                                    sample.color = img;
                                    
                                    PXCMSyncPoint syn;
                                    pxcmStatus status;
                                    PXCMVideoModule.DataDesc inputs = new PXCMVideoModule.DataDesc();
                                    status = videoModule.QueryCaptureProfile(0, out inputs);
                
                                    // Send sample 
                                    status = videoModule.ProcessImageAsync(sample, out syn);
                                    if (status < pxcmStatus.PXCM_STATUS_NO_ERROR)
                                    { // handle fail
                                    }
                                    else
                                    {
                                        // Synchronize to process image sent to VideoModule
                                        syn.Synchronize();
                                        using (PXCMFaceData moduleOutput = faceModule.CreateOutput())
                                        {
                                            // Face processing is done at this point, you can query face data
                                            Debug.Assert(moduleOutput != null);
                                            moduleOutput.Update();
                                            int cf = moduleOutput.QueryNumberOfDetectedFaces();
                                            DisplayOnBitmap(sample.color); //Same method as in FaceTracking Sample
                                            Save(moduleOutput, fileEntry);// Save to file
                                        }
                                    syn.Dispose();
                                    }
                                sample.ReleaseImages();
                                pp.ReleaseFrame();
                                }
                                videoModule.Dispose();
                                faceModule.Dispose();
                                moduleConfiguration.Dispose();
                            }
                            pp.Close();
                            pp.Dispose();
                        }
                

                 

                There's another thread (https://software.intel.com/en-us/forums/realsense/topic/606341#comment-1857656 ) that helped a lot.
                I hope this is helpful to you.

                 

                I would really appreciate any suggestions or comments on why those errors occur after this method ends.

                Thank you.