1 2 Previous Next 26 Replies Latest reply on Jan 31, 2018 2:57 PM by Intel Corporation

    D415 in Advanced Mode

    BjarneG

      Hello community! My first post here so please be gentle... 

       

      I cannot find any documentation or explanation of the "Advanced Mode" settings for the D415 camera. Does this exist somewhere?

       

      In particular, I need to remove (or at least significantly reduce) speckles of "unknown depth" in the depth-stream returned from the camera. It would be nice to achieve this by tweaking settings but I could also implement some post-procesing - any hints in this direction would be most welcome!

       

      Regards,

      - Bjarne

        • 1. Re: D415 in Advanced Mode
          MartyG

          Welcome to the community! 

           

          The documentation for the 400 Series Advanced Mode is here:

           

          librealsense/rs400_advanced_mode.md at master · IntelRealSense/librealsense · GitHub

          • 2. Re: D415 in Advanced Mode
            BjarneG

            Thankyou, and thankyou.

             

            The documentation you link provides good basic info, as well as explaining why rs-sensor-control do not display advanced-mode params when cmake'ing it like I did.

             

            However, in order to avoid randomly permute 100+ params, some description of the algorithm and heuristics / thresholds in the ASIC would be useful. Browsing the forum I found this paper , also containing interesting information, but although it mentions the D400-series it focus on R200 which do not have the advanced-mode params.

             

            Does anyone know a paper or article describing how the D400 actually calculates depth? In particular - what heuristics are used to determine if a pixel-depth is unknown?

             

            I'll play around over the next few days and if I discover anything definite, I'll mention it here. Feel free to do the same!

            • 3. Re: D415 in Advanced Mode
              MartyG

              The full data sheet document for the 400 Series cameras may be of use to you.

               

              https://software.intel.com/sites/default/files/Intel_RealSense_Depth_Cam_D400_Series_Datasheet.pdf

               

              Pages 11 and 12 explain how the 400 Series cameras calculate depth,

              • 4. Re: D415 in Advanced Mode
                BjarneG

                Thanks Marty.

                • 5. Re: D415 in Advanced Mode
                  BjarneG

                  I`m building v2.9.0 from github with -DBUILD_RS400_EXTRAS=true and -DBUILD_EXAMPLES=true.

                   

                  The docs mention a program called  rs400-advanced-mode-sample but I cannot find any trace of it after building, nor do I find any source-file with a similar name. Have anyone successfully built and used rs400-advanced-mode-sample?

                  • 6. Re: D415 in Advanced Mode
                    MartyG

                    If you are using cmake, it suggests you use the following instruction, which has "cmake .." on the front of it.  Have you done this, please?

                     

                    cmake .. -DBUILD_EXAMPLES=true -DBUILD_RS400_EXTRAS=true

                    • 7. Re: D415 in Advanced Mode
                      BjarneG

                      Hi Marty.

                       

                      I do the equivalent of what you suggest, but with cmake-gui (I want to control some other settings as well). Running cmake from cmd-line on a clean build-directory gives me the following

                       

                      begreen@intel-sdk:~/git/librealsense/new-build$ cmake .. -DBUILD_EXAMPLES=true -DBUILD_RS400_EXTRAS=true

                      -- The C compiler identification is GNU 5.4.0

                      -- The CXX compiler identification is GNU 5.4.0

                      -- Check for working C compiler: /usr/bin/cc

                      -- Check for working C compiler: /usr/bin/cc -- works

                      -- Detecting C compiler ABI info

                      -- Detecting C compiler ABI info - done

                      -- Detecting C compile features

                      -- Detecting C compile features - done

                      -- Check for working CXX compiler: /usr/bin/c++

                      -- Check for working CXX compiler: /usr/bin/c++ -- works

                      -- Detecting CXX compiler ABI info

                      -- Detecting CXX compiler ABI info - done

                      -- Detecting CXX compile features

                      -- Detecting CXX compile features - done

                      -- Info: REALSENSE_VERSION_STRING=2.9.0

                      -- Performing Test COMPILER_SUPPORTS_CXX11

                      -- Performing Test COMPILER_SUPPORTS_CXX11 - Success

                      -- Performing Test COMPILER_SUPPORTS_CXX0X

                      -- Performing Test COMPILER_SUPPORTS_CXX0X - Success

                      -- Try OpenMP C flag = [-fopenmp]

                      -- Performing Test OpenMP_FLAG_DETECTED

                      -- Performing Test OpenMP_FLAG_DETECTED - Success

                      -- Try OpenMP CXX flag = [-fopenmp]

                      -- Performing Test OpenMP_FLAG_DETECTED

                      -- Performing Test OpenMP_FLAG_DETECTED - Success

                      -- Found OpenMP: -fopenmp 

                      -- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.1")

                      -- Looking for pthread.h

                      -- Looking for pthread.h - found

                      -- Looking for pthread_create

                      -- Looking for pthread_create - found

                      -- Found Threads: TRUE 

                      -- Found OpenGL: /usr/lib/x86_64-linux-gnu/libGL.so 

                      -- Checking for module 'gtk+-3.0'

                      --   Found gtk+-3.0, version 3.18.9

                      -- Configuring done

                      -- Generating done

                      CMake Warning:

                        Manually-specified variables were not used by the project:

                       

                          BUILD_RS400_EXTRAS

                       

                       

                      -- Build files have been written to: /home/begreen/git/librealsense/new-build

                      begreen@intel-sdk:~/git/librealsense/new-build$

                       

                      Note the comment near end about BUILD_RS400_EXTRAS not used, and that  REALSENSE_VERSION_STRING==2.9.0

                       

                      After building it (make -j8), I do the following:

                       

                      begreen@intel-sdk:~/git/librealsense/new-build$ find . -name *rs400* -print

                      begreen@intel-sdk:~/git/librealsense/new-build$

                       

                      Note that no filename containing the string "rs400" exists. Have you been able to sucessfully build rs400-advanced-mode-sample ?

                      • 8. Re: D415 in Advanced Mode
                        MartyG

                        I don't have a 400 Series camera unfortunately, so haven't been able to do a build with it in SDK 2.0 myself  yet.

                         

                        I note that you are using BUILD_RS400_EXTRAS in your cmake, but the warning message says BUILD_RS400_EXTRAS (i.e missing the D on the front of BUILD).  I'm not sure why it would be losing the D, but perhaps that is why it cannot find it if it is looking for BUILD instead of DBUILD?

                         

                        I note though that you were not able to find any file reference containing the term 'RS400'.

                         

                        I also note that the docs say that "You will need one of the following compilers (or newer) for JSON support:

                         

                        GCC 4.9

                        Clang 3.4

                        Microsoft Visual C++ 2015 / Build Tools 14.0.25123.0

                         

                        Are you using one of these?

                        • 9. Re: D415 in Advanced Mode
                          BjarneG

                          Hi Marty. Thanks for your relentless efforts to help me out here.

                           

                          The "-D" is just the way you tell cmake to "define" a property, hence "BUILD_RS400_EXTRAS" (without  -D) is a named property defined to be true.

                           

                          I am using GCC 5.4 so this should not be an issue (although I don't really look for JSON support at this point)

                           

                          begreen@intel-sdk:~/git/librealsense/new-build$ gcc --version

                          gcc (Ubuntu 5.4.0-6ubuntu1~16.04.5) 5.4.0 20160609

                          Copyright (C) 2015 Free Software Foundation, Inc.

                          This is free software; see the source for copying conditions.  There is NO

                          warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

                           

                          You don't need a 400-camera to build the SDK I believe. It would be interesting to know your results if you decide to try it out.

                           

                          • 10. Re: D415 in Advanced Mode
                            MartyG

                            I develop primarily in the Unity game engine and so do not have any kind of conventional C# / C++ development environment such as Clang or Visual Studio set up on my PC, unfortunately.

                             

                            I am not an Intel employee and so I have taken the resarch into your issue as far as I can, as I do not have access to the RealSense developer team for advice.  Hopefully one of the Intel support agents can take up your case and find a solution with the greater resources they have available.  I apologize that I was not able to resolve your issue myself and wish you the best of luck! 

                            • 11. Re: D415 in Advanced Mode
                              BjarneG

                              Your efforts are greatly appreciated Marty. Thanks!

                               

                              I also hope someone with closer access to the dev-team can respond to this...

                              • 12. Re: D415 in Advanced Mode
                                BjarneG

                                Mmm....  quick update on this. Perhaps someone from Intel would like to comment / clarify? I'm currently on the "master" branch...

                                 

                                The files named "rs400_advanced..." apparently have been renamed to "rs_advanced_mode....". Moreover,  the rs400-advanced-mode-sample  I have been trying to find seems to exist in source-form under ./build/CMakeFiles/realsense2.dir/src/ds5/  but is no compiled to an executable by the build-process.

                                 

                                The information here should (at least) be updated with the right names and info, and the build process should be fixed. (There are also some missing includes in several sourcefiles on master at the time of writing, but this will probably be fixed quickly as it's pretty obvious to anyone trying to compile.)

                                • 13. Re: D415 in Advanced Mode
                                  BjarneG

                                  I sense that you are trying to tell me something, but don't really understand what....

                                  • 14. Re: D415 in Advanced Mode
                                    Intel Corporation
                                    This message was posted on behalf of Intel Corporation

                                    Hello BjarneG,

                                    The Advanced Mode functionality has been built into the RealSense Viewer tool as shown in the attached screenshot. I will share your feedback regarding the status of the documentation with the RealSense team.

                                    Disregard the posts by tester123. We are investigating them.

                                    Regards,
                                    Jesus G.
                                    Intel Customer Support

                                    1 2 Previous Next