Assessment 03 - Tele-operation of a mobile robot using ROSWeb Tools

Thiago Lima Souto - Mat.: 19044686

Professor Dr. Khalid

Masters of Engineering - Mechatronics

Massey University - 2nd semester 2019 - Auckland

Course: 282.758 – Simulation, Modelling and Optimisation Assessment: Assessment 02 - Simulation, Modeling and OptimizationROS based SLAM and Navigation for a Gazebo simulated robot

Assignment 03

Robot Web Tools is a collection of open-source modules and tools for building web-based robot apps. While ROS works great for applications on the robot another layer is needed to connect external devices and applications. rosbridge is both a JSON spec for interacting with ROS and a transport layer, providing a WebSocket for clients to communicate over.

In this assignment a mobile robot will be tele-operated using Robot Web Tools. The robot will need to be capable of avoiding obstacles and move towards a target goal autonomously. For indoor robots a map may be available to navigate the robot from point to point. For the outdoor robots GPS location will be used for navigation from point to point.

Due date: 01/11/2019 by 23:59PM.

Report Requirements:

1. Not more than 20 pages in total including references and figures.
2. A block diagram of the overall system with corresponding description in not more than 3 pages.
3. A brief discussion on networking in ROS and snapshot of the robot setup.
4. A brief discussion on the Robot Web Tools and any packages used for the tele-operation of the robot.
5. Setup and describe robot control over local area network (LAN) using ROS on all machines. The robot must be able to avoid obstacles and move to a target on its own. The remote machine will serve as a user console for monitoring and control of the robot through vision based tele-operation.
6. Add a new widget with a JoyStick to ROSWeb (as described on this webpage). The new widget publishes cmd_vel to control the robot. You can use any available JavaScript code for the JoyStick 1 . [5% marks]
7. Submit the following for this assignment.

a. Full report in word or pdf format.

b. A short video (1-2 min) of the robot console and the actual robot operating through remote commands.

c. ROS catkin package with all necessary launch files. Any JavaScript or other code. The code will be tested and marked.

Introduction

On this assignment a simulated model was created using ROS, Gazebo and RVIZ where the P3dx robot package together with movebase, gmapping, rosbridge suite and teleop_twist_keyboard were used and launched by the Mylaunch package. Also a package to send goals to the robot was built.

Two computers were used to tele-operate the robot via ROS Network, one was a IMAC running Ubuntu 19.04 on an Oracle VM VirtualBox and the other one was a dual boot notebook with Ubuntu 19.04 and Windows 10. The tele-operation of the robot was made in two ways, the first using the teleop_twist_keyboard package been executed on the IMAC which controlled the robot on the Notebook and the second using a custom package that sent a goal from the IMAC to be executed on the Notebook. ROS Melodic Morenia was used in both computers.

A ROSWeb server was also created with a new joystick gadget using javascript.

Overall system

As can be seen on the block diagram of the overall system below, the teleop_twist_keyboard and the /move_base (via simple_goal) subscribe to the /cmd_vel topic, /gasebo execute the /cmd_vel and move the robot around or to the goal established, then /slam_gmapping reads the /scan data and update the map and the costmap.

the /rosbridge_websocket connects the robot the the server and the /rosapi provides service calls for getting meta-information related to ROS like topic lists as well as interacting with the Parameter Server.

Packages used (inside the /src folder):

Block diagram of the overall system:

Networking in ROS

ROS is designed for distributed computing so we can connect multiple devices via a common network, the ROS network.

The way It works is because all the devices share the same modem they form a local network, this network is therefore referred to as the local ROS network. On this particular application all connected components have the same public IP address, which is the modem's IP address.

As an Example we can connect a robot to a local ROS network together with many other devices as shown below, all the components publish and/or subscribe to this robot, and ROS will manages the flow of data, ensuring proper robot operations.

Example of a ROS Network:

For security purposes every external threat from outside the network is blocked on the modem.

To give demonstrate some practical applications of a local ROS network three examples will be shown next. A simple communication via a talker and listener nodes, a tele-operation using the teleop-twist_keyboard package and a tele-operation using a custom package that sends a goal via the LAN.

Considerations on running ROS across multiple machines

In a ROS network, Its nodes, with few exceptions don't care where It runs in the network. To run ROS systems in multiple machine some aspects have to be taken in consideration:

• Only one master is needed

• All nodes shall be configured to use the same master, with the command ROS_MASTER_URI

• Bi-directional connectivity between all pairs of machines, on all ports must be complete

• Each machine must advertise itself by a name that all other machines can resolve

Talker / listener across two machines

Setting up the Master

First the names by which you would address the machines has to be defined, for that we execute SSH on the master computer as shown below:

thiago@notebook:~$ ssh thiago@192.168.1.188
thiago@192.168.1.188's password: 

Welcome to Ubuntu 18.04.3 LTS (GNU/Linux 4.18.0-15-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

 * Kata Containers are now fully integrated in Charmed Kubernetes 1.16!
   Yes, charms take the Krazy out of K8s Kata Kluster Konstruction.

     https://ubuntu.com/kubernetes/docs/release-notes

 * Canonical Livepatch is available for installation.
   - Reduce system reboots and improve kernel security. Activate at:
     https://ubuntu.com/livepatch

66 packages can be updated.
5 updates are security updates.

Your Hardware Enablement Stack (HWE) is supported until April 2023.
Last login: Sun Oct 20 08:52:46 2019 from 192.168.1.188

Then export ROS_MASTER_URI using the ip address of the master as shown below:

thiago@notebook:~$ export ROS_MASTER_URI=http://192.168.1.188:11311

After that the listener node can be executed after run roscore

thiago@notebook:~$ rosrun rospy_tutorials listener.py

After configuring the other machine this computer will receive the talker msgs as shown below:

[INFO] [1571526855.533602]: /listener_4932_1571526754445I heard hello world 1571526855.52
[INFO] [1571526855.633147]: /listener_4932_1571526754445I heard hello world 1571526855.62
[INFO] [1571526855.736313]: /listener_4932_1571526754445I heard hello world 1571526855.72
[INFO] [1571526855.831768]: /listener_4932_1571526754445I heard hello world 1571526855.82
[INFO] [1571526855.932375]: /listener_4932_1571526754445I heard hello world 1571526855.92
[INFO] [1571526856.031923]: /listener_4932_1571526754445I heard hello world 1571526856.02
[INFO] [1571526856.132456]: /listener_4932_1571526754445I heard hello world 1571526856.12

Setting up the other machine

On the other machine ROS has to be installed and then run roscore. Also SSH has to be installed with the command sudo apt-get install openshh-server if the machine doesn't have it.

Then the same command executed on the master machine has to be executed on this machine.

thiago@mac:~$ export ROS_MASTER_URI=http://192.168.1.188:11311

Then the talker node can be executed as follow:

thiago@mac:~$ rosrun rospy_tutorials talker.py

The result can be seen below:

[INFO] [1571526855.517345]: hello world 1571526855.52
[INFO] [1571526855.618535]: hello world 1571526855.62
[INFO] [1571526855.718575]: hello world 1571526855.72
[INFO] [1571526855.817612]: hello world 1571526855.82
[INFO] [1571526855.917668]: hello world 1571526855.92
[INFO] [1571526856.018036]: hello world 1571526856.02
[INFO] [1571526856.118623]: hello world 1571526856.12

A Photograph of the computers communicating is shown below:

Remote Tele-operation over Local Area Network (LAN)

Now that the two machines are on the same network we can execute packages on the secondary machine to be executed on the master.

A very important aspect to be considered is that all the dependencies installed on the Master has to be installed on the secondary machine. Some simple packages like the teleop_twist_keyboard that doesn't have many dependencies can run with no problems, but more advanced packages that uses, for example, geometry messages needs all dependencies and libraries to run.

A small video showing the following applications in action is submitted together with this assignment.

Teleop_twist_keyboard

On this application the Simulated Robot, on the notebook, is controlled by the IMAC.

The connection between the secondary machine and the master was made on the same way as the first application, via export ROS_MASTER_URI=http://192.168.1.188:11311 and by SSH.

Some photographs of the application running on both machines is shown below:

Sending a Goal via LAN

On this application all the dependencies had to be installed prior to the normal operation of the package including all the libraries. A custom package called sending simple goals was created to send a goal to the simulated robot on the master, this package was executed on the secondary machine.

An obstacles (Gazebo box) was positioned in front of the robot so that he can avoid obstacles and move to the target on It's own.

An illustration of the result can be find below:

Robot Web Tools

Robot Web Tools is a interface between ROS and Web Application, It allows the application to execute a variety of tasks in robots running middleware. They use mainly HTML, Javascript and Typescript as web technology.

Each element is marked with its unique local IP addresses within the local ROS network, see figure below. These are the IP addresses used to run ROS on multiple local computers. The ROS design allows computation capabilities to be relocated at run-time to match the available resources.

The Rosbridge Suite allows any client to subscribe or publish to ROS topics by sending JSON files, with this suite the clients can also call ROS services along with other features.

By installing the Rosbridge Suite we are installing all the packages included in the suite including rosbridge_library, rosbridge_server and rosapi. The rosbridge_library receives the JSON strings and controls the ROS services accordingly. The rosbridge_server has the WebSocket server that exposes the rosbridge_library and the rosapi provides service calls as well as interaction with the Parameters Server. The rosapi and the rosbridge_websocket can be seen on the Block diagram presented on the beginning of this report.

There are some issues with the cloud based solution. This is because connecting cloud-based solutions to ROS involves the development of a interface between ROS, the cloud-based solution and ROS Bridge, which can be a massive task in case of complex robots. Although for less complex systems it is a very good solution that can solve many problems and with the advance of webservices is very likely to develop in a fast pace.

Implementing the ROSWeb server

To start It's necessary to instal the following:

sudo apt-get install ros-melodic-rosbridge-suite

rosdep update
rosdep install rosbridge_server
rosdep install web_video_server

After the installation the rosbridge package is added to the Robot Model from assignment 01, also inside the catkin_ws/src the git page and catkin_make the packages are added as can be seen below:

git clone -b develop https://github.com/RobotWebTools/rosbridge_suite.git
cd ..
source devel/setup.bash
catkin_make
roslaunch rosbridge_server rosbridge_websocket.launch
rosrun web_video_server web_video_server

After that the launch file is executed with just the p3dx with the laser and gazebo, the following topics can be seen on rostopic list:

thiago@thiago-GS75-Stealth-8SF:~/catkin_ws$ rostopic list
/base_pose_ground_truth
/client_count
/clock
/cmd_vel
/connected_clients
/gazebo/link_states
/gazebo/model_states
/gazebo/parameter_descriptions
/gazebo/parameter_updates
/gazebo/set_link_state
/gazebo/set_model_state
/joint_states
/odom
/rosout
/rosout_agg
/scan
/tf
/tf_static

The connected_clients is a new topic

Using the working demo and executing the same topic list by using the Widget #param_viewer this is the result:

On this one notice that the topics and the types are displayed and that the /connected_clients topic is a rosbridge_msgs/ConnectedClients topic type.

Data from the laser can be accessed as shown in the picture below, on a refresh rate configured on the widget as well as the topic to be published, in this case /scan. The Gazebo model has the same data as the webservice.

Creating the joystick Widget

The ROSWeb allows us to implement new Widgets and It can be done using JavaScript or TypeScript.

In the following example a javaScript Joystick was implemented on the new Widget, the Figure below shows the joystick working inside the Widget.

The next step would have been to connect it with ROS.

Conclusions

By discussing ROS Networks and ROS web tools was clear the distributed computing nature of ROS and It's limitations, especially on complex robots through webserver applications.

The Robot was successfully operate over LAN in two tele-operation tasks, one using the teleop_twist_keyboard package, and another one with a custom package sending a goal. A application where one computer execute a talker node and the other computer listen to the talker node was also accomplished. The robot was capable of avoiding an obstacle and moved to the goal on its own, while he updated the maps.

A video was submitted showing the two tele-operation activities between the IMAC and the notebook.

The ROS web server had a favorable result in regards the connection via local host and was able to execute all the common widget that comes with the ROSWeb package. The joystick gadget was successfully created and was operational, although the ROS link was not successful, a final working version will be submitted after the assignment.

References

https://github.com/EESC-LabRoM/rosweb

http://robotwebtools.org/

http://wiki.ros.org/ROS/Tutorials/MultipleRemoteMachines

https://github.com/RobotWebTools/rosbridge_suite

https://github.com/yoannmoinet/nipplejs

"Establishing remote networks for ROS applications via Port Forwarding: A detailed tutorial" - Hajjaj, S. S. H. and Sahari, K. S. M., International Journal of Advanced Robotic Systems - 2017

research References

http://wiki.ros.org/actionlib

https://www.youtube.com/watch?v=XOSX3Z6qkok

https://servicerobots.co.nz/rosweb/#

https://github.com/EESC-LabRoM/rosweb

http://robotwebtools.org/

https://github.com/RobotWebTools/rosbridge_suite

http://wiki.ros.org/ROS/Tutorials/MultipleMachines

http://wiki.ros.org/ROS/Tutorials/MultipleRemoteMachines

http://wiki.ros.org/ROS/NetworkSetup

http://wiki.ros.org/actionlib_tutorials/Tutorials/SimpleActionServer%28ExecuteCallbackMethod%29

https://servicerobots.co.nz/rosweb/

https://www.youtube.com/watch?v=7kjJ3eJrZKo

http://wiki.ros.org/actionlib_tutorials/Tutorials

http://wiki.ros.org/actionlib_tutorials/Tutorials/RunningServerAndClientWithNodes

http://wiki.ros.org/actionlib_tutorials/Tutorials/SimpleActionClient

http://wiki.ros.org/actionlib/DetailedDescription

http://wiki.ros.org/actionlib

For the joystick:

Git:

https://github.com/yoannmoinet/nipplejs

Page:

https://github.com/yoannmoinet/nipplejs

Code Pen:

https://codepen.io/YoannM/pen/gapmMG

Teleop:

https://github.com/ros-teleop/teleop_twist_keyboard/blob/master/teleop_twist_keyboard.py

Ros creating Widget:

https://eesc-labrom.github.io/roswebpage/creating_widgets.html