8 Replies Latest reply on Feb 10, 2015 10:21 PM by dylanTachyon

    OpenCV controlled Robot Arm

    dylanTachyon

      Hello,

       

       

      This is a project I was thinking about doing once I get better with the Edison and OpenCV. I would like to get some opinions on also how difficult ( I assume it is by no means easy)  it would be, and if there are any resources or already done projects online I can follow. However, to control a robot arm with computer vision, could it be done partially in the Arduino language and the computer vision part in python-OpenCV? If, how does one go about interfacing the outputs of an OpenCV program with an Arduino program?

       

      Thank you.

        • 1. Re: OpenCV controlled Robot Arm
          CMata_Intel

          Hi dylanTachyon

           

          When you say computer vision do you mean cameras, sensors, both, or what?

          Something about OpenCV  and USB-camera

           

          Also, I'm not sure if the combination of Arduino IDE and OpenCV-Python is required, I think you could be able to do this using just one of them. But, if you want to use the Arduino and OpenCV, maybe creating a sketch in the IDE and a script that will be running at the same time in the board.

          Now in the Arduino sketch you could get information using UART's, SPI, I2C. It depends on what are you going to use, which interfaces uses the robot-arm and how you are going to develop the "computer vision" part to control it. Maybe you could develop the control section with OpenCV and send interrupts to the pins in order to be checked in the sketch, just some ideas.

          I haven't done something like this but it sounds great and as you may know will get a lot of help in here if you needed.

          Have you already have something structured, components, more ideas about it?

           

          Take a look at this project (with Galileo): Control robotic arm with gloves, Android Phone and Arduino

           

          Regards;

          CMata

          • 2. Re: OpenCV controlled Robot Arm
            dylanTachyon

            Hi CMata,

             

            Thank you for the advice. Sorry I wasn't more specific, I meant OpenCV and a USB camera. Well I started off here. This site led me to a 3D printed robot arm that I already printed. The arm works with micro servos. The project in the link I posted sounded interesting, but I have been playing with OpenCV so much that I wanted to use it with the arm somehow. However, I thought maybe that some of the code used in that project could play a part somehow (as a core for controller the arm), being tweaked of course. In addition,I though possibly starting with something simpler, liking getting it to just rotate from the base and follow a hand. And so far, with OpenCV, I've train a couple of ok classifiers and then have been trying to combine them with the OpenCV face recognition class, which I am still playing around with. However, since I am a beginner I was looking for possibly a tutorial to start off with. In addition, the UART's, SPI, and I2C you mentioned, are those Arduino libraries?

             

            -Thanks,

            Dylan

            • 3. Re: OpenCV controlled Robot Arm
              CMata_Intel

              Hi dylanTachyon

               

              The interfaces that I mentioned: SPI, I2C, UART are protocols to send data to another device. If you are going to use a USB webcam probably you won't need them. All the libraries that the Arduino IDE (version downloaded from here) has libraries for all of those protocols (SPI, Wire, Serial). In order to check all the available libraries, go to:

                C:\arduino-1.5.3-Intel.1.0.4\hardware\arduino\edison\libraries

                C:\arduino-1.5.3-Intel.1.0.4\hardware\arduino\edison\cores

              There are libraries that will let you use servos too.

               

              Now, the example that you want to use: TrainableRoboticArm/Trainable_Arm.ino at master · vectrasoft/TrainableRoboticArm · GitHub

              Was made for the Arduino board and using the Arduino IDE for those boards, so the libraries are different. They use Servo.h and EEPROM.h, the Arduino IDE for the Edison board has libraries with those names but they may not be the same so you might need to do some changes in the libraries in order to obtain the same result. Or it may work at the first try.

               

              Regards;

              CMata

              • 4. Re: OpenCV controlled Robot Arm
                dylanTachyon

                Thank you. In addition, if I were to do a project involving computer vision (USB webcam) with the arm (hopefully as I get better), you mentioned earlier that I would not have to use OpenCV-python and Arduino. Out of curiosity, why is that? And, alternatively, if I were to use both running on the same board, how could data, such as pin interrupts, be communicated between OpenCV-python and Arduino. I also like that project you posted earlier as it gave me the idea of possibly using less degrees of freedom to start off with.  The project in the link I posted uses an arm with possibly too many degrees of freedom to deal with for a first computer-vision controlled arm project.

                 

                -Thanks again,

                Dylan

                • 5. Re: OpenCV controlled Robot Arm
                  CMata_Intel

                  Hi Dylan!

                   

                  I'm sorry for the delay in my response. What I meant is that It's not neccesary to use a lot of scripts/sketches at the same time, let's say that you are going to run a script in the board using OpenCV and at the same time you are going to run a sketch made in the Arduino environment, I haven't done a project like the one you have in mind but I think you could use OpenCV without the sketch of the Arduino environment. I know that this means not using the .ino file from: Arduino Controlled Trainable Robotic Arm « Adafruit Industries – Makers, hackers, artists, designers and engineers! but you could do it. Just ideas, but I think that having the .ino file from the Adafruit's link it will be a good start and after you will see if you need another script/sketch to control other things. I hope this answers your question.

                   

                  Kind Regards;

                  CMata

                  1 of 1 people found this helpful
                  • 6. Re: OpenCV controlled Robot Arm
                    dylanTachyon

                    No problem. Thank you. Yeah I'm probably gonna start with the Adafruit .ino file like you said. And regarding the OpenCV project, I assume it is possible to set up flags or arguments for an arduino script when ran from the terminal, so that when the arduino script runs after the OpenCV one, it can accept the outputs of the OpenCV script? If, is there an arduino library you know of that allows this?

                     

                    Thank you for your help,

                    Dylan

                    • 7. Re: OpenCV controlled Robot Arm
                      CMata_Intel

                      Hi Dylan;

                       

                      This is a nice example, the Arduino-script uses the lectures from the Serial port so the OpenCV routine send bytes to the port and the Arduino-script reads the bytes and works according to the lectures. This could be a good start.

                      http://aleksandarkrstikj.com/tracking-a-ball-and-rotating-camera-with-opencv-and-arduino/ 

                       

                      Regards;

                      CMata

                      • 8. Re: OpenCV controlled Robot Arm
                        dylanTachyon

                        Awesome! Thank you so much for your help. I'm sure you'll see more of my posts on the forum about my progress with the project once I start to run into issues, but first I'm going to do the tutorial you posted and the Adafruit arm.

                         

                        Thanks again,

                        Dylan