5 Replies Latest reply on May 7, 2018 3:47 AM by Intel Corporation

    optical-flow does not work due to image quality

    vivitter

      Hi all,

       

      I am trying to set up optical flow to my Aero RTF Drone.

      I have set up my drone according to these information.

       

      * aero-optical-flow/README.md at master · intel-aero/aero-optical-flow · GitHub

      * https://docs.px4.io/en/flight_controller/intel_aero.html

       

      Versions:

       

      root@intel-aero:~# aero-get-version.py
      
      
      BIOS_VERSION = Aero-01.00.13
      OS_VERSION = Poky Aero (Intel Aero Linux Distro) 1.6.1 (pyro)"
      AIRMAP_VERSION = 1.8
      FPGA_VERSION = 0xc2
      AeroFC firmware version = 1.6.5
      

       

      Lidar:

      I have connected Lidar Lite v3.

       

      Status:

      After an optical-flow process is executed, quality value is getting lower.

      After 10 or 20 seconds, the quality is about 0 ~ 10.

       

      Trial-an-error:

      At first, I suppose that the texture of the floor is not enough.

      So, I modified optical-flow source codes and added these functions.

      • Get actual images before inserting to an optical-flow function
      • Logging additional parameters, for examples, gain and exposure.

       

      Then, I realized that gain and exposure values are changing drastically.

      According to these parameters, actual image gradually turns to white within 10 seconds.

       

      Initial Status

      logs

      [Fri Apr 13 17:27:57.330 2018] [Debug] =TEST= gain 0 exposure 850
      [Fri Apr 13 17:27:57.400 2018] [Debug] =TEST= msg.quality: 176

       

      Image

      000_cropped.jpg

       

      After a few seconds

      logs

      [Fri Apr 13 17:28:09.204 2018] [Debug] =TEST= gain 127 exposure 1727
      [Fri Apr 13 17:28:09.328 2018] [Debug] =TEST= msg.quality: 176
      

       

      Image

      030_cropped.jpg

       

      After a few more seconds

      logs

      [Fri Apr 13 17:29:20.566 2018] [Debug] =TEST= gain 1 exposure 1
      [Fri Apr 13 17:29:20.595 2018] [Debug] =TEST= msg.quality: 0
      
      

       

      Image

      096_cropped.jpg

       

      What should I do to run optical-flow properly?

      Thank you very much for your attention.

        • 1. Re: optical-flow does not work due to image quality
          Intel Corporation
          This message was posted on behalf of Intel Corporation

          Hello Kohei,

          Thank you for your interest in the Intel Aero drone. 

          Can you please clarify as to what program(s) are you using to test?
          Is it one of the test functions in the instructions, your own code, or code from somewhere else? That can help bring some understanding.

          Regards,
          Octavian
          • 2. Re: optical-flow does not work due to image quality
            vivitter

            Dear Octavian,

             

            I have modified mainloop.cpp in this repository.

            aero-optical-flow/mainloop.cpp at master · intel-aero/aero-optical-flow · GitHub

             

            To save image data, I have added a block between l.320 and l.322.

            Please check below.

             

            cv::Mat cropped;
            // Copy the data into new matrix -> cropped_image.data can not be used in calcFlow()...
            cropped_image.copyTo(cropped);
            cropped_image.release();
            
            // ========== Added this block to save images.
            // Save image data just before executing calcFlow()
            static int cnt_cropped = 0;
            static int cnt_sec = 0;
            if(cnt_sec==30){
                 if(cnt_cropped <1000){
                      std::ostringstream oss;
                      oss << std::setfill('0') << std::setw(3) << cnt_cropped++;
                      imwrite(oss.str() + "_cropped.jpg", cropped);
                 }
                 cnt_sec=0;
            }
            // ========== END
            
            int flow_quality = _optical_flow->calcFlow(cropped.data, (uint32_t)img_time_us, dt_us, flow_x_ang, flow_y_ang);
            

             

            Besides, I have added these lines to check exposure, gain and quality.

             

            After L.228

            DEBUG("=TEST= gain %u exposure %u", (uint16_t)gain, (uint16_t)exposure);
            

             

            After L. 271

            DEBUG("=TEST= msg.quality: %i", msg.quality);
            

             

             

            Make new source on the Aero Compute Board.

            git clone https://github.com/intel-aero/aero-optical-flow.git
            # Update mainloop.cpp file
            cd aero-optical-flow
            git submodule update --init --recursive
            mkdir build
            cd build
            cmake ..
            make
            

             

            Execute it

             

            cd aero-optical-flow/build
            ./aero-optical-flow -c /dev/video2
            

             

            Thank you for your support.

             

            Best regards,

            Kohei

            • 3. Re: optical-flow does not work due to image quality
              Intel Corporation
              This message was posted on behalf of Intel Corporation

              Hello Kohei,

              Unfortunately, we can't help you with debugging the code as that's beyond our scope.

              Regards,
              Octavian

              • 4. Re: optical-flow does not work due to image quality
                vivitter

                Dear Octavian,

                 

                Originally, we got the same result when I have executed your original code.

                Quality value decreased drastically.

                 

                To check the cause, I have added these lines I have copied.

                Then, I have noticed that optical flow process cannot obtain proper quality image.

                 

                I have provided this code according to your request.

                I have never requested you to debug it.

                I would like to just know why your original code does not work properly in my drone.

                 

                I would like to discriminate the cause of the issue.

                Could you give me any suggestion?

                 

                Best regards,

                Kohei

                • 5. Re: optical-flow does not work due to image quality
                  Intel Corporation
                  This message was posted on behalf of Intel Corporation

                  Hi vivitter,


                  Can you please tell me if you have made any modifications/changes when seeing the initial issue or trying the guide?
                  It would also be very helpful if you could turn on debug mode and provide us the log information if you still see it with the guide. 


                  Thank you in advance, 
                  Casandra