Raspbian Buster ROS RealSense

This page documents problems encountered while trying to install ROS and Intel RealSense onto Raspbian Buster.

I am doing this just a few months after the release of the Raspberry Pi 4. The Raspbian release is Buster. The ROS version I am installing is Kinetic Kame Desktop.

Before You Start

I am using a 64GB microSD card, UHS-I U3. Raspbian Buster Desktop image was flashed to it. SSH was enabled before first boot. I went through the steps to expand the filesystem to fill the whole card, and the other "usual" rasp-config stuff.

I added 1GB of swap space, this is required to build OpenCV and some other packages, otherwise there's a really mysterious crash that happens when compiling those source files.

I recommend you use apt-get to install a few dependencies before you start:

sudo apt-get update && sudo apt-get upgrade
sudo apt-get install build-essential cmake pkg-config
sudo apt-get install libjpeg-dev libtiff5-dev libjasper-dev libpng12-dev
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
sudo apt-get install libxvidcore-dev libx264-dev
sudo apt-get install libgtk2.0-dev libgtk-3-dev
sudo apt-get install libatlas-base-dev gfortran
sudo apt-get install python2.7-dev python3-dev

Installing everything will take HOURS, make sure your power supply and SSH connection will not be interrupted. Read a book or watch a movie, or in my case, remotely access your home network so you can work while you do this installation. Also, make sure your Raspberry Pi is sufficiently cooled, it will become very hot, even with heatsinks attached, it can hurt a finger.

Installing ROS

I followed instructions from http://wiki.ros.org/ROSberryPi/Installing%20ROS%20Kinetic%20on%20the%20Raspberry%20Pi

Different people will encounter different problems, most likely due to the rapid pace of version changes. I did not encounter some of the problems mentioned on that page, and I did encounter some ones that were not mentioned on that page.

Problem: "CMake may have trouble finding FindEigen3.cmake"

I did not encounter this problem

Problem: error: invalid conversion from 'const char*' to 'char*'

Full error text is

/home/pi/ros_catkin_ws/src/opencv3/modules/python/src2/cv2.cpp: In function ‘bool pyopencv_to(PyObject*, T&, const char*) [with T = cv::String; PyObject = _object]’:
/home/pi/ros_catkin_ws/src/opencv3/modules/python/src2/cv2.cpp:885:34: error: invalid conversion from ‘const char*’ to ‘char*’ [-fpermissive]
     char* str = PyString_AsString(obj);

Apparently this is due to some changes in Python 3.7, and the fix is to find the file cv2.cpp and edit the offending line from

     char* str = PyString_AsString(obj);

to

     const char* str = PyString_AsString(obj);

Citation: https://github.com/opencv/opencv/issues/14856#issuecomment-504416696

Problem: fatal error: boost/tr1/unordered_set.hpp: No such file or directory

Full error text is

In file included from /home/pi/ros_catkin_ws/src/rospack/src/rospack.cpp:28:
/home/pi/ros_catkin_ws/src/rospack/include/rospack/rospack.h:108:10: fatal error: boost/tr1/unordered_set.hpp: No such file or directory
 #include <boost/tr1/unordered_set.hpp>
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [CMakeFiles/rospack.dir/build.make:63: CMakeFiles/rospack.dir/src/rospack.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
In file included from /home/pi/ros_catkin_ws/src/rospack/src/rospack_backcompat.cpp:29:
/home/pi/ros_catkin_ws/src/rospack/include/rospack/rospack.h:108:10: fatal error: boost/tr1/unordered_set.hpp: No such file or directory
 #include <boost/tr1/unordered_set.hpp>
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [CMakeFiles/rospack.dir/build.make:76: CMakeFiles/rospack.dir/src/rospack_backcompat.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:371: CMakeFiles/rospack.dir/all] Error 2
make: *** [Makefile:141: all] Error 2

This was mentioned on the install instruction page. "On Raspbian Buster the compilation may fail with "'boost/tr1/unordered_set.hpp' file not found". This is because rospack version used in Kinetic is dependent on boost 1.58. To fix this error try installing boost 1.58 manually."

I followed these instructions to install boost 1.58: http://osdevlab.blogspot.com/2016/02/how-to-install-latest-boost-library-on.html , but note: the page here says 1.60, but I changed that to 1.58

Problem: /usr/bin/ld: cannot find -l-lpthread

Full error text:

[ 94%] Compiling generated code for qt_gui_cpp_sip Python bindings...
make[3]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.
/usr/bin/ld: cannot find -l-lpthread
collect2: error: ld returned 1 exit status
make[3]: *** [Makefile:38: "/home/pi/ros_catkin_ws/devel_isolated/qt_gui_cpp/lib/python2.7/dist-packages/qt_gui_cpp/libqt_gui_cpp_sip".so] Error 1
make[2]: *** [src/qt_gui_cpp_sip/CMakeFiles/libqt_gui_cpp_sip.dir/build.make:61: /home/pi/ros_catkin_ws/devel_isolated/qt_gui_cpp/lib/python2.7/dist-packages/qt_gui_cpp/libqt_gui_cpp_sip.so] Error 2
make[1]: *** [CMakeFiles/Makefile2:380: src/qt_gui_cpp_sip/CMakeFiles/libqt_gui_cpp_sip.dir/all] Error 2
make: *** [Makefile:141: all] Error 2

This one was nasty... I can't actually explain it, but I found a working fix

Citation: https://aur.archlinux.org/packages/ros-melodic-qt-gui-cpp/ , user hansbonini commented on 2019-05-31 02:34 "Run the package install for the first time, restart the compilation, add this line before cmake"

sed -i -e 's/\-l\-lpthread//g' sip/qt_gui_cpp_sip/Makefile

Note: depending on what directory you are in, you may need to change the path to the correct Makefile in the command above

Note: I did not put that line before cmake, instead I simply ran that command from the terminal

Problem: logWarn or logError not declared in scope

Full error text, is very similar to:

/home/pi/ros_catkin_ws/src/geometry2/tf2/src/buffer_core.cpp: In member function ‘bool tf2::BufferCore::setTransform(const TransformStamped&, const string&, bool)’:
/home/pi/ros_catkin_ws/src/geometry2/tf2/src/buffer_core.cpp:221:5: error: ‘logError’ was not declared in this scope
     logError("TF_SELF_TRANSFORM: Ignoring transform from authority \"%s\" with frame_id and child_frame_id  \"%s\" because they are the same",  authority.c_str(), stripped.child_frame_id.c_str());
     ^~~~~~~~
/home/pi/ros_catkin_ws/src/geometry2/tf2/src/buffer_core.cpp:221:5: note: suggested alternative: ‘strerror’
     logError("TF_SELF_TRANSFORM: Ignoring transform from authority \"%s\" with frame_id and child_frame_id  \"%s\" because they are the same",  authority.c_str(), stripped.child_frame_id.c_str());
     ^~~~~~~~
     strerror

or

/home/pi/ros_catkin_ws/src/geometry2/tf2/src/buffer_core.cpp: In member function ‘bool tf2::BufferCore::warnFrameId(const char*, const string&) const’:
/home/pi/ros_catkin_ws/src/geometry2/tf2/src/buffer_core.cpp:126:5: error: ‘logWarn’ was not declared in this scope
     logWarn("%s",ss.str().c_str());
     ^~~~~~~
/home/pi/ros_catkin_ws/src/geometry2/tf2/src/buffer_core.cpp:134:5: error: ‘logWarn’ was not declared in this scope
     logWarn("%s",ss.str().c_str());
     ^~~~~~~

This is another code revision incompatibility, the internet says that logError became CONSOLE_BRIDGE_logError. The fix is going into the source file buffer_core.cpp and add in a macro to redefine logWarn and logError:

#include "tf2/buffer_core.h"
#include "tf2/time_cache.h"
#include "tf2/exceptions.h"
#include "tf2_msgs/TF2Error.h"

#include <assert.h>
#include <console_bridge/console.h>
#include "tf2/LinearMath/Transform.h"
#include <boost/foreach.hpp>

#ifndef logError // added by frank26080115
#define logError CONSOLE_BRIDGE_logError
#endif

#ifndef logWarn // added by frank26080115
#define logWarn CONSOLE_BRIDGE_logWarn
#endif


namespace tf2
{

...

rest of the code

Citation: https://github.com/ros/console_bridge/issues/56 and https://github.com/ros/ros-overlay/issues/509

I am unsure if my method is the best but it worked.

Installing librealsense

Follow instructions from https://github.com/IntelRealSense/librealsense/blob/master/doc/installation_raspbian.md

DO NOT follow instructions from https://github.com/IntelRealSense/librealsense/blob/master/doc/RaspberryPi3.md or https://github.com/IntelRealSense/librealsense/blob/development/doc/installation.md , these instructions are missing the -DFORCE_LIBUVC=true option during cmake, which is actually quite important

Problem: undefined reference to __atomic_store_8

Full error text is

/usr/bin/ld: ../../librealsense2.so.2.25.0: undefined reference to '__atomic_load_8'
/usr/bin/ld: ../../librealsense2.so.2.25.0: undefined reference to '__atomic_store_8'
/usr/bin/ld: ../../librealsense2.so.2.25.0: undefined reference to '__atomic_fetch_add_8'
collect2: error: ld returned 1 exit status
make[2]: *** [examples/hello-realsense/CMakeFiles/rs-hello-realsense.dir/build.make:91: examples/hello-realsense/rs-hello-realsense] Error 1
make[1]: *** [CMakeFiles/Makefile2:455: examples/hello-realsense/CMakeFiles/rs-hello-realsense.dir/all] Error 2
/usr/bin/ld: ../../librealsense2.so.2.25.0: undefined reference to '__atomic_load_8'
/usr/bin/ld: ../../librealsense2.so.2.25.0: undefined reference to '__atomic_store_8'
/usr/bin/ld: ../../librealsense2.so.2.25.0: undefined reference to '__atomic_fetch_add_8'
collect2: error: ld returned 1 exit status
make[2]: *** [examples/software-device/CMakeFiles/rs-software-device.dir/build.make:91: examples/software-device/rs-software-device] Error 1
make[1]: *** [CMakeFiles/Makefile2:511: examples/software-device/CMakeFiles/rs-software-device.dir/all] Error 2
make: *** [Makefile:130: all] Error 2

I don't fully understand the problem, but the internet offers a solution

Citation: https://github.com/IntelRealSense/librealsense/issues/4565#issuecomment-518359584

Find the file librealsense/CMakeList.txt and put this line near the top

set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -latomic")

Problem: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ...

Full error text is

src/wrapper.c: In function ‘__Pyx_ExceptionSave’:
src/wrapper.c:11474:21: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘exc_type’; did you mean ‘curexc_type’?
     *type = tstate->exc_type;
                     ^~~~~~~~
                     curexc_type
src/wrapper.c:11475:22: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘exc_value’; did you mean ‘curexc_value’?
     *value = tstate->exc_value;
                      ^~~~~~~~~
                      curexc_value
src/wrapper.c:11476:19: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
     *tb = tstate->exc_traceback;
                   ^~~~~~~~~~~~~
                   curexc_traceback
src/wrapper.c: In function ‘__Pyx_ExceptionReset’:
src/wrapper.c:11488:24: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘exc_type’; did you mean ‘curexc_type’?
     tmp_type = tstate->exc_type;
                        ^~~~~~~~
                        curexc_type
...
...
error: command 'gcc' failed with exit status 1

Another Python 3.7 incompatibility problem.

Citation: https://github.com/mcfletch/pyopengl/issues/11

What I did was: git clone the pyopengl repo, go into pyopengl/accelerate/src and find those files mentioned (wrapper.c, vbo.c, formathandler.c and numpy_formathandler.c), and perform the replacements (exc_type, exc_value, exc_traceback to curexc_type, curexc_value, curexc_traceback respectively)

Patching uvcvideo

The problem shows up if you check dmesg

...
[20391.573663] uvcvideo: Unknown video format 00000050-0000-0010-8000-00aa00389b71
...
...
[20391.581523] uvcvideo: Unknown video format 36315752-1a66-a242-9065-d01814a8ef8a
...

Here's the story in a nutshell: uvcvideo is a kernel module that essentially makes webcams plug-and-play. RealSense cameras have some special video streams, such as depth and IR image that have special video formats, but some software we are using still want it to be plug-and-play. (if you run "lsusb -v" you can actually see those GUIDs)

Intel's repo has both patch files and scripts to apply the patch to your kernel, but these won't work on Raspbian. Raspbian Buster, already has half of these already in its copy of uvcvideo so the patch file won't work anyways.

Here is where we need to download the kernel source code and patch it ourselves by editing the source files directly.

Start by gathering the source code and getting some tools

sudo apt-get install bison flex bc
sudo apt-get install raspberrypi-kernel-headers
mkdir ~/uvcvideo_patch
cd ~/uvcvideo_patch
git clone https://github.com/raspberrypi/linux.git
cd linux
uname -r
git status

at this point, "uname -r" shows you what kernel version you are on, "git status" should show you what branch you are on. If the two doesn't match, use "git checkout" to fetch the appropriate branch.

Take a look at https://github.com/IntelRealSense/librealsense/blob/master/scripts/realsense-camera-formats.patch , there are 4 files you are editing: "drivers/media/usb/uvc/Makefile", "drivers/media/usb/uvc/uvc_driver.c", "drivers/media/usb/uvc/uvcvideo.h", "include/uapi/linux/videodev2.h" . Edit those files appropriately according to the additions in the patch file, but without adding duplicates.

After saving the source files, continue with the commands

cd ~/uvcvideo_patch
cp  /usr/src/linux-headers-`uname -r`/Module.symvers .
make scripts oldconfig modules_prepare

At this point, you may be prompted to answer some questions, use the default answer if you don't know what to do

cd drivers/media/usb/uvc
cp ~/uvcvideo_patch/linux/Module.symvers .
make -C ~/uvcvideo_patch/linux M=~/uvcvideo_patch/linux/drivers/media/usb/uvc modules

If you've edited the source files wrong, then this will tell you where.

sudo modprobe -r uvcvideo
sudo rm /lib/modules/`uname -r`/kernel/drivers/media/usb/uvc/uvcvideo.ko
sudo cp ~/uvcvideo_patch/linux/drivers/media/usb/uvc/uvcvideo.ko /lib/modules/`uname -r`/kernel/drivers/media/usb/uvc/uvcvideo.ko

Reboot!

Installing ROS Package: realsense-ros

Here's the package, https://github.com/IntelRealSense/realsense-ros/ , it has to be installed through catkin. The repo's readme says that the instructions are in .travis.yml but those instructions are meant for Ubuntu and apt-get, some of those won't work in our case on Raspbian.

Start from these commands:

mkdir -p ~/catkin_ws/src
cd ~/catkin_ws/src/
mv * ~/catkin_ws/src/realsense/
cd ~/catkin_ws/src/
git clone https://github.com/IntelRealSense/realsense-ros.git
cd realsense-ros/
git checkout 'git tag | sort -V | grep -P "^\d+\.\d+\.\d+" | tail -1'
cd ..
catkin_init_workspace
cd ..

at this point, you should be inside ~/catkin_ws , if you ran "catkin_make clean" right now, you will run into an error saying ddynamic_reconfigure is missing, so we will install it

cd src
git clone https://github.com/pal-robotics/ddynamic_reconfigure.git
cd ddynamic_reconfigure
git checkout 'git tag | sort -V | grep -P "^\d+\.\d+\.\d+" | tail -1'
cd ../..

at this point, you should be inside ~/catkin_ws again, and you should be able to proceed with the rest of the installation

catkin_make clean
catkin_make -DCATKIN_ENABLE_TESTING=False -DCMAKE_BUILD_TYPE=Release
catkin_make install
echo "source ~/catkin_ws/devel/setup.bash" >> ~/.bashrc
source ~/.bashrc

Installing RTAB-Map

rtabmap has to be installed separately, from source code, see instructions from https://github.com/introlab/rtabmap/wiki/Installation#raspberrypi

The list of dependancies shown on the instructions are slightly out of date (libpcl-dev will pull in libvtk7-qt-dev, which conflicts with the instructions, which will install libvtk6-qt-dev), also, you should already have OpenCV by now. I advise you install in the following order:

sudo apt-get install libpcl-dev
sudo apt-get install libopenni2-dev libsqlite3-dev

Make sure you use "git checkout" to grab the right version tag. You will need to install rtabmap_ros later, and the versions must match!

cd ~
git clone https://github.com/introlab/rtabmap.git
cd rtabmap
git checkout 0.19.3-kinetic
cd build
cmake ..
make -j1
sudo make install

Installing rtabmap_ros

This is a ROS package install, from source code, so it involves using git to clone rtabmap_ros into catkin_ws/src first. Make sure to use "git checkout" on the right branch (in this case, I used the release tag "0.19.3-kinetic").

cd ~/catkin_ws/src
git clone https://github.com/introlab/rtabmap_ros.git
cd rtabmap_ros
git checkout 0.19.3-kinetic
cd ~/catkin_ws
catkin_make -j1

OK you will probably be hammered with a bunch of dependancy errors, CMake will look for other packages that you don't have yet.

First, some of them you can get with apt-get:

sudo apt-get install libbullet-dev

Then, the ROS packages you are missing can be git-cloned into ~/catkin_ws/src before you run catkin_make again, for example

cd ~/catkin_ws
git clone --branch <branchnameortagname> <giturl> src/<reponame>
catkin_make -j1

You'll do the above sequence of commands a bunch of times, for each of the package you are missing. When you are told what package you are missing, visit the URL

http://wiki.ros.org/<packagename>

and then choose the ROS version by clicking on "kinetic", then it will show you which github URL to go to and which branch you should use.

How do you know which branch or tag? Check what is available. Look for a branch like "kinetic" or "kinetic-devel" (think this way, if there's a melodic version already, then kinetic-devel should be pretty stable). If that doesn't exist, then you have to look for a branch with the name of a previous ROS release, such as "indigo". Remember, you can look for older versions but not newer versions.

Example Problem

Error text is

-- Could not find the required component 'pcl_ros'. The following CMake error indicates that you either need to install the package with the same name or change your environment so that it can be found.
CMake Error at /opt/ros/kinetic/share/catkin/cmake/catkinConfig.cmake:83 (find_package):
  Could not find a package configuration file provided by "pcl_ros" with any
  of the following names:

    pcl_rosConfig.cmake
    pcl_ros-config.cmake

I will visit http://wiki.ros.org/pcl_ros , choose the ROS version by clicking on "kinetic", and it shows me the github URL is https://github.com/ros-perception/perception_pcl and the branch is "kinetic-devel"

I can then run the commands

cd ~/catkin_ws
git clone --branch kinetic-devel https://github.com/ros-perception/perception_pcl.git src/perception_pcl
catkin_make -j1

For your reference, these are the directories inside my catkin_ws/src

ddynamic_reconfigure
geometry2
navigation_msgs
pcl_conversions
pcl_msgs
perception_pcl
realsense-ros
rtabmap_ros
vision_opencv

Problem: logWarn or logError not declared in scope

This problem may come up again because we are building geometry2 from source code. The solution is already described on this page (see above).