Items with no label
3335 Discussions

Mapping - localization - Autonomous driving

GCair
Beginner
1,858 Views

Good afternoon,

I'm new with cameras and this type of software implementation. I need some advices about my project application.

I need to build an autonomous device able to navigate autonomously in an indoor context. With the Realsense cameras (D400 Series) is it possible to understand the exact location of the camera with respect to a predefined map of the environment? if yes, is there any implemented library or some documntation where I can start? Obviously if there are other ways for the localization application let me know. Thank you very much for your help.

Regards,

Giovanni Cairoli

0 Kudos
7 Replies
MartyG
Honored Contributor III
413 Views

The RealSense 400 Series cameras were designed with applications such as robotics and autonomous vehicles in mind. The D435 model is especially suited to vehicles, as it has a fast 'global shutter' that handles motion better than the D415 model.

Indoor navigation with mapping can be performed with a system called SLAM. My recommendation for a good pre-made SLAM system to use with the 400 Series cameras is called ORB-SLAM2.

https://github.com/raulmur/ORB_SLAM2 GitHub - raulmur/ORB_SLAM2: Real-Time SLAM for Monocular, Stereo and RGB-D Cameras, with Loop Detection and Relocalizati…

Also, here is a paper written about ORB-SLAM2.

https://arxiv.org/pdf/1610.06475.pdf https://arxiv.org/pdf/1610.06475.pdf

You may be interested in the guide linked to below, which provides instructions and links to resources for constructing a RealSense-equipped mobile robot.

https://software.intel.com/en-us/articles/build-an-autonomous-mobile-robot-with-the-intel-realsense-camera-ros-and-sawr?wapkw=slam+vision+library Build an Autonomous Mobile Robot with the Intel® RealSense™ Camera, ROS*, and SAWR | Intel® Software

0 Kudos
GCair
Beginner
413 Views

Thank you very much MartyG, I will read everything. I need some time to understand if it is feasible.

So you guarantee that it is possible to localize your cameras into a shopping centre map. Just using few cameras?

Thank you, It is a good starting point.

Kind Regards,

Giovanni

0 Kudos
MartyG
Honored Contributor III
413 Views

I am trying to visualize your project. I am guessing it is a mobile robot that has a fixed map of a shopping mall in it (it should not need to update the map in real-time), and the robot uses the map to navigate? I can imagine the concept of such a robot because I remember a case from 2016 about a security robot patrolling a mall that allegedly knocked over a child.

https://money.cnn.com/2016/07/14/technology/robot-stanford-mall/index.html https://money.cnn.com/2016/07/14/technology/robot-stanford-mall/index.html

Like an autonomous car, you may therefore need some form of real-time obstacle mapping if there are going to be shoppers in the path of the robot.

The more cameras that you use, the wider the field of view that the cameras will have. Four RealSense cameras could provide a 180 degree view, and eight cameras could provide a 360 degree view (360 degrees has been done with 6 cameras though). The more cameras you have, the better the data is, because there are fewer blind-spots in the vision.

0 Kudos
GCair
Beginner
413 Views

Yes MartyG, in may application I have a fixed map related to the shopping mall. At the same time I need to be able to undestand where the robot is with respect to the map (in order to run the navigation algorithm and go from point A to point B) and detect the moving obstacle around it (implementing the obstacle avoidance algorithm).

I understood the issue about te number of cameras. Thank you.

Kind Regards,

Giovanni

0 Kudos
MartyG
Honored Contributor III
413 Views

It does sound like ORB-SLAM2 would meet your needs. At the bottom of the ORB-SLAM2 page, it says about its Localization mode: "This mode can be used when you have a good map of your working area. In this mode the Local Mapping and Loop Closing are deactivated. The system localizes the camera in the map (which is no longer updated), using relocalization if needed".

So basically, I believe that you would create the map of the mall by having the robot do patrols of it in SLAM mode (where it creates and updates a map in real-time) and then once your robot has a good map, you switch it to Localization mode, where the camera is localized on the map but the map itself is fixed.

0 Kudos
GCair
Beginner
413 Views

Hi MartyG,

Yes it sounds great, thank you for your support.

Kind regards,

Giovanni

0 Kudos
MartyG
Honored Contributor III
413 Views

You are very welcome. Feel free to return to this forum with further questions at any time as your project develops.

0 Kudos
Reply