How to compile and run ORB-SLAM3? [ORB-SLAM3 运行示例]

# Resources - Official Github: https://github.com/UZ-SLAMLab/ORB_SLAM3 - My clone version: https://github.com/yubaoliu/ORB_SLAM3 - Youtube demo: https://youtu.be/WfY6mCTnFGg ORB-SLAM3 is the first real-time SLAM library able to perform Visual, Visual-Inertial and Multi-Map SLAM with monocular, stereo and RGB-D cameras, using pin-hole and fisheye lens models. In all sensor configurations, ORB-SLAM3 is as robust as the best systems available in the literature, and significantly more accurate. # Build Project - Install Dependences ```sh sudo apt-get install build-essential sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev ``` - Install OPenCV3: https://docs.opencv.org/3.1.0/d7/d9f/tutorial_linux_install.html ```sh git clone https://github.com/opencv/opencv.git git chekcout 3.3.1 mkdir build cd build cmake -D CMAKE_BUILD_TYPE=Release -D CMAKE_INSTALL_PREFIX=/usr/local .. make -j3 make install ``` - Install Eigen3 ```sh sudo apt install libeigen3-dev ``` - Install Pangolin ```sh git clone https://github.com/stevenlovegrove/Pangolin.git cd Pangolin mkdir build cd build cmake .. -DCMAKE_BUILD_TYPE=Release make -j 3 make install ``` - Build ORB SLAM3 ```sh ./build.sh ``` # Get EuRoc Dataset ```sh wget -c http://robotics.ethz.ch/~asl-datasets/ijrr_euroc_mav_dataset/machine_hall/MH_01_easy/MH_01_easy.zip ``` Unzip to **MH01** folder. # Run sample Replace **pathDatasetEuroc** to you dataset path containing **MH01**. **mono_euroc**: ```sh ./Examples/Monocular/mono_euroc ./Vocabulary/ORBvoc.txt ./Examples/Monocular/EuRoC.yaml "$pathDatasetEuroc"/MH01 ./Examples/Monocular/EuRoC_TimeStamps/MH01.txt dataset-MH01_mono ``` **mono_inertial_euroc**: ```sh ./Examples/Monocular-Inertial/mono_inertial_euroc ./Vocabulary/ORBvoc.txt ./Examples/Monocular-Inertial/EuRoC.yaml "$pathDatasetEuroc"/MH01 ./Examples/Monocular-Inertial/EuRoC_TimeStamps/MH01.txt dataset-MH01_monoi ``` # Run MONO, Stereo and RGB-D without IMU Cannot run Mono, Stereo, RGBD without IMU? errors: ORB-SLAM3 seems not work well with an RGB-D mode ```sh ORB Extractor Parameters: Number of Features: 1000 Scale Levels: 8 Scale Factor: 1.2 Initial Fast Threshold: 20 Minimum Fast Threshold: 7 Depth Threshold (Close/Far Points): 3.07156 ``` Refer issues: 1. https://github.com/UZ-SLAMLab/ORB_SLAM3/issues/8 1. https://github.com/UZ-SLAMLab/ORB_SLAM3/issues/3 Solution: Refer: 1. [fix mono&stereo&RGB_D mode on Kitti&TUM RGBD](https://github.com/DreamWaterFound/ORB_SLAM3/commit/e8c3395915af5e733df6a909c0d70ebdfc7a0559) 1. [update kittiXX.yaml](https://github.com/DreamWaterFound/ORB_SLAM3/commit/5a07ca06212a3d49fb1a77893b001d48f22aabf9) ## How to modify? Add this to **yaml** for MONO/RGBD mode of TUM and KITI Dataset: ```sh Camera.type: "PinHole" ``` Stereo mode: Examples/Stereo/KITTI00-02.yaml: ```sh Camera.type: "PinHole" # Opencv format LEFT.height: 1241 LEFT.width: 376 LEFT.D: !!opencv-matrix rows: 1 cols: 5 dt: d data: [0.0, 0.0, 0.0, 0.0, 0.0] LEFT.K: !!opencv-matrix rows: 3 cols: 3 dt: d data: [718.856, 0.0, 607.1928, 0.0, 718.856, 185.2157, 0.0, 0.0, 1.0] LEFT.R: !!opencv-matrix rows: 3 cols: 3 dt: d data: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0] LEFT.P: !!opencv-matrix rows: 3 cols: 4 dt: d data: [7.188560000000e+02, 0.000000000000e+00, 6.071928000000e+02, 0.000000000000e+00, 0.000000000000e+00, 7.188560000000e+02, 1.852157000000e+02, 0.000000000000e+00, 0.000000000000e+00, 0.000000000000e+00, 1.000000000000e+00, 0.000000000000e+00] RIGHT.height: 1241 RIGHT.width: 376 RIGHT.D: !!opencv-matrix rows: 1 cols: 5 dt: d data: [0.0, 0.0, 0.0, 0.0, 0.0] RIGHT.K: !!opencv-matrix rows: 3 cols: 3 dt: d data: [718.856, 0.0, 607.1928, 0.0, 718.856, 185.2157, 0.0, 0.0, 1.0] RIGHT.R: !!opencv-matrix rows: 3 cols: 3 dt: d data: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0] RIGHT.P: !!opencv-matrix rows: 3 cols: 4 dt: d data: [7.188560000000e+02, 0.000000000000e+00, 6.071928000000e+02, -3.861448000000e+02, 0.000000000000e+00, 7.188560000000e+02, 1.852157000000e+02, 0.000000000000e+00, 0.000000000000e+00, 0.000000000000e+00, 1.000000000000e+00, 0.000000000000e+00] ``` Examples/Stereo/KITTI03.yaml: ```sh Camera.type: "PinHole" # Opencv format LEFT.height: 1241 LEFT.width: 376 LEFT.D: !!opencv-matrix rows: 1 cols: 5 dt: d data: [0.0, 0.0, 0.0, 0.0, 0.0] LEFT.K: !!opencv-matrix rows: 3 cols: 3 dt: d data: [721.5377, 0.0, 609.5593, 0.0, 721.5377, 172.854, 0.0, 0.0, 1.0] LEFT.R: !!opencv-matrix rows: 3 cols: 3 dt: d data: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0] LEFT.P: !!opencv-matrix rows: 3 cols: 4 dt: d data: [7.215377000000e+02, 0.000000000000e+00, 6.095593000000e+02, 0.000000000000e+00, 0.000000000000e+00, 7.215377000000e+02, 1.728540000000e+02, 0.000000000000e+00, 0.000000000000e+00, 0.000000000000e+00, 1.000000000000e+00, 0.000000000000e+00] RIGHT.height: 1241 RIGHT.width: 376 RIGHT.D: !!opencv-matrix rows: 1 cols: 5 dt: d data: [0.0, 0.0, 0.0, 0.0, 0.0] RIGHT.K: !!opencv-matrix rows: 3 cols: 3 dt: d data: [721.5377, 0.0, 609.5593, 0.0, 721.5377, 172.854, 0.0, 0.0, 1.0] RIGHT.R: !!opencv-matrix rows: 3 cols: 3 dt: d data: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0] RIGHT.P: !!opencv-matrix rows: 3 cols: 4 dt: d data: [7.215377000000e+02, 0.000000000000e+00, 6.095593000000e+02, -3.875744000000e+02, 0.000000000000e+00, 7.215377000000e+02, 1.728540000000e+02, 0.000000000000e+00, 0.000000000000e+00, 0.000000000000e+00, 1.000000000000e+00, 0.000000000000e+00] ``` Examples/Stereo/KITTI04-12.yaml: ```sh Camera.type: "PinHole" # Opencv format LEFT.height: 1241 LEFT.width: 376 LEFT.D: !!opencv-matrix rows: 1 cols: 5 dt: d data: [0.0, 0.0, 0.0, 0.0, 0.0] LEFT.K: !!opencv-matrix rows: 3 cols: 3 dt: d data: [707.0912, 0.0, 601.8873, 0.0, 707.0912, 183.1104, 0.0, 0.0, 1.0] LEFT.R: !!opencv-matrix rows: 3 cols: 3 dt: d data: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0] LEFT.P: !!opencv-matrix rows: 3 cols: 4 dt: d data: [7.070912000000e+02, 0.000000000000e+00, 6.018873000000e+02, 0.000000000000e+00, 0.000000000000e+00, 7.070912000000e+02, 1.831104000000e+02, 0.000000000000e+00, 0.000000000000e+00, 0.000000000000e+00, 1.000000000000e+00, 0.000000000000e+00] RIGHT.height: 1241 RIGHT.width: 376 RIGHT.D: !!opencv-matrix rows: 1 cols: 5 dt: d data: [0.0, 0.0, 0.0, 0.0, 0.0] RIGHT.K: !!opencv-matrix rows: 3 cols: 3 dt: d data: [707.0912, 0.0, 601.8873, 0.0, 707.0912, 183.1104, 0.0, 0.0, 1.0] RIGHT.R: !!opencv-matrix rows: 3 cols: 3 dt: d data: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0] RIGHT.P: !!opencv-matrix rows: 3 cols: 4 dt: d data: [7.070912000000e+02, 0.000000000000e+00, 6.018873000000e+02, -3.798145000000e+02, 0.000000000000e+00, 7.070912000000e+02, 1.831104000000e+02, 0.000000000000e+00, 0.000000000000e+00, 0.000000000000e+00, 1.000000000000e+00, 0.000000000000e+00] ``` include/Frame.h: ```cpp // Constructor for RGB-D cameras. -Frame(const cv::Mat &imGray, const cv::Mat &imDepth, const double &timeStamp, ORBextractor* extractor,ORBVocabulary* voc, cv::Mat &K, cv::Mat &distCoef, const float &bf, const float &thDepth, Frame* pPrevF = static_cast(NULL), const IMU::Calib &ImuCalib = IMU::Calib()); +Frame(const cv::Mat &imGray, const cv::Mat &imDepth, const double &timeStamp, ORBextractor* extractor,ORBVocabulary* voc, cv::Mat &K, cv::Mat &distCoef, const float &bf, const float &thDepth, GeometricCamera* pCamera, Frame* pPrevF = static_cast(NULL), const IMU::Calib &ImuCalib = IMU::Calib()); ``` src/Frame.cc: ```cpp -Frame::Frame(const cv::Mat &imGray, const cv::Mat &imDepth, const double &timeStamp, ORBextractor* extractor,ORBVocabulary* voc, cv::Mat &K, cv::Mat &distCoef, const float &bf, const float &thDepth, Frame* pPrevF, const IMU::Calib &ImuCalib) +Frame::Frame(const cv::Mat &imGray, const cv::Mat &imDepth, const double &timeStamp, ORBextractor* extractor,ORBVocabulary* voc, cv::Mat &K, cv::Mat &distCoef, const float &bf, const float &thDepth, GeometricCamera* pCamera, Frame* pPrevF, const IMU::Calib &ImuCalib) :mpcpi(NULL),mpORBvocabulary(voc),mpORBextractorLeft(extractor),mpORBextractorRight(static_cast(NULL)), mTimeStamp(timeStamp), mK(K.clone()),mDistCoef(distCoef.clone()), mbf(bf), mThDepth(thDepth), mImuCalib(ImuCalib), mpImuPreintegrated(NULL), mpPrevFrame(pPrevF), mpImuPreintegratedFrame(NULL), mpReferenceKF(static_cast(NULL)), mbImuPreintegrated(false), mpCamera2(nullptr), mTimeStereoMatch(0), mTimeORB_Ext(0) mpCamera(pCamera), mpCamera2(nullptr), mTimeStereoMatch(0), mTimeORB_Ext(0) ``` src/Tracking.cc: ```cpp - mCurrentFrame = Frame(mImGray,imDepth,timestamp,mpORBextractorLeft,mpORBVocabulary,mK,mDistCoef,mbf,mThDepth); + mCurrentFrame = Frame(mImGray,imDepth,timestamp,mpORBextractorLeft,mpORBVocabulary,mK,mDistCoef,mbf,mThDepth, mpCamera); ``` --- # Possible Errors ## GeometricCamera.h Issue: When run ./build_ros.sh ```sh #include "GeometricCamera.h" not found ``` Solution: Add ``${PROJECT_SOURCE_DIR}/../../../include/CameraModels`` to CMakeLists.txt: ```sh include_directories( ${PROJECT_SOURCE_DIR} ${PROJECT_SOURCE_DIR}/../../../ ${PROJECT_SOURCE_DIR}/../../../include ${PROJECT_SOURCE_DIR}/../../../include/CameraModels ${Pangolin_INCLUDE_DIRS} ) ``` --- When run ./build_ros.sh ```sh ros_stereo.cc:40:18: error: 'ORB_SLAM2' has not been declared ImageGrabber(ORB_SLAM2::System* pSLAM):mpSLAM(pSLAM){} ^~~~~~~~~ /root/catkin_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/ros_stereo.cc:40:35: error: expected ')' before '*' token ImageGrabber(ORB_SLAM2::System* pSLAM):mpSLAM(pSLAM){} ^ /root/catkin_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/ros_stereo.cc:44:5: error: 'ORB_SLAM2' does not name a type ORB_SLAM2::System* mpSLAM; ^~~~~~~~~ /root/catkin_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/ros_stereo.cc: In function 'int main(int, char**)': /root/catkin_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/ros_stereo.cc:62:5: error: 'ORB_SLAM2' has not been declared ORB_SLAM2::System SLAM(argv[1],argv[2],ORB_SLAM2::System::STEREO,true); ^~~~~~~~~ /root/catkin_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/ros_stereo.cc:64:23: error: 'SLAM' was not declared in this scope ImageGrabber igb(&SLAM); ^~~~ /root/catkin_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/ros_stereo.cc:64:23: note: suggested alternative: 'SNAN' ImageGrabber igb(&SLAM); ^~~~ SNAN /root/catkin_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/ros_rgbd.cc:40:18: error: 'ORB_SLAM2' has not been declared ImageGrabber(ORB_SLAM2::System* pSLAM):mpSLAM(pSLAM){} ^~~~~~~~~ /root/catkin_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/ros_rgbd.cc:40:35: error: expected ')' before '*' token ImageGrabber(ORB_SLAM2::System* pSLAM):mpSLAM(pSLAM){} ^ /root/catkin_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/ros_rgbd.cc:44:5: error: 'ORB_SLAM2' does not name a type ORB_SLAM2::System* mpSLAM; ^~~~~~~~~ /root/catkin_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/ros_rgbd.cc: In function 'int main(int, char**)': /root/catkin_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/ros_rgbd.cc:60:5: error: 'ORB_SLAM2' has not been declared ORB_SLAM2::System SLAM(argv[1],argv[2],ORB_SLAM2::System::RGBD,true); ``` It seems the ROS version has not yet compeleted. (Maybe) Solution: - change **ORB_SLAM2** to **ORB_SLAM3** for all the src files under Example/ROS ## undefined reference to symbol When build ROS wrapper: ```sh [ 44%] Linking CXX executable ../Stereo /usr/bin/ld: CMakeFiles/RGBD.dir/src/ros_rgbd.cc.o: undefined reference to symbol '_ZN5boost6system15system_categoryEv' /usr/lib/x86_64-linux-gnu/libboost_system.so: error adding symbols: DSO missing from command line collect2: error: ld returned 1 exit status CMakeFiles/RGBD.dir/build.make:227: recipe for target '../RGBD' failed make[2]: *** [../RGBD] Error 1 CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/RGBD.dir/all' failed make[1]: *** [CMakeFiles/RGBD.dir/all] Error 2 make[1]: *** Waiting for unfinished jobs.... /usr/bin/ld: CMakeFiles/Stereo.dir/src/ros_stereo.cc.o: undefined reference to symbol '_ZN5boost6system15system_categoryEv' /usr/lib/x86_64-linux-gnu/libboost_system.so: error adding symbols: DSO missing from command line collect2: error: ld returned 1 exit status CMakeFiles/Stereo.dir/build.make:227: recipe for target '../Stereo' failed make[2]: *** [../Stereo] Error 1 CMakeFiles/Makefile2:104: recipe for target 'CMakeFiles/Stereo.dir/all' failed make[1]: *** [CMakeFiles/Stereo.dir/all] Error 2 Makefile:129: recipe for target 'all' failed make: *** [all] Error 2 ``` Solution: Config Boost library: ```sh find_package(Boost REQUIRED COMPONENTS system) ${Boost_INCLUDE_DIRS}$ ${Boost_LIBRARIES} ``` Refer: https://cmake.org/cmake/help/v3.15/module/FindBoost.html # References - Forster, C., Carlone, L., Dellaert, F., & Scaramuzza, D. (n.d.). On-Manifold Preintegration for Real-Time Visual-Inertial Odometry. Retrieved September 5, 2019, from https://youtu.be/CsJkci5lfco - Elvira, R., Tardós, J. D., & Montiel, J. M. M. (2019). ORBSLAM-Atlas: a robust and accurate multi-map system. - Campos, C., Elvira, R., Rodríguez, J. J. G., Montiel, J. M. M., & Tardós, J. D. (2020). ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM. http://arxiv.org/abs/2007.11898