Developing Web Interfaces For ROS Robots #4 – Streaming robot’s camera on the web page

Developing Web Interfaces For ROS Robots #4 – Streaming robot’s camera on the web page

Hello ROS Developers,

This is the 4th of 4 posts of the series Developing web interfaces for ROS Robots.

In this post, we are going to stream the images of the robot’s camera on the webpage.

 


1 – Loading a new JavaScript library

Before we go to the camera streaming server or the code to open its connection, we need to import the library that is in charge of making everything possible. That is the MJpeg library.

Let’s import it in our <head> section:

    ...

    <script src="https://cdn.jsdelivr.net/npm/eventemitter2@5.0.1/lib/eventemitter2.min.js">
    </script>
    <script type="text/javascript" src="https://static.robotwebtools.org/mjpegcanvasjs/current/mjpegcanvas.min.js">
    </script>

</head>

 


2 – Adding the camera viewer element to the page

Now for the webpage, we need a new element there which will define where the camera images are going to be shown:

Let’s use the div row we have created for the buttons (in the previous post) to embed the camera. It must be replaced by the following code:

        <div class="row">
            <div class="col-md-6 row">
                <div class="col-md-12 text-center">
                    <h5>Commands</h5>
                </div>

                <!-- 1st row -->
                <div class="col-md-12 text-center">
                    <button @click="forward" :disabled="loading || !connected" class="btn btn-primary">Go straight</button>
                    <br><br>
                </div>

                <!-- 2nd row -->
                <div class="col-md-4 text-center">
                    <button @click="turnLeft" :disabled="loading || !connected" class="btn btn-primary">Turn left</button>
                </div>
                <div class="col-md-4 text-center">
                    <br>
                    <button @click="stop" :disabled="loading || !connected" class="btn btn-danger">Stop</button>
                    <br>
                    <br>
                </div>
                <div class="col-md-4 text-center">
                    <button @click="turnRight" :disabled="loading || !connected" class="btn btn-primary">Turn right</button>
                </div>

                <!-- 3rd row -->
                <div class="col-md-12 text-center">
                    <button @click="backward" :disabled="loading || !connected" class="btn btn-primary">Go straight</button>
                </div>
            </div>

            <div class="col-md-6">
                <div id="mjpeg"></div>
            </div>
        </div>

Yes, the camera code is only the final part:

    ...
    <div class="col-md-6">
        <div id="mjpeg"></div>
    </div>
</div>

3 – Webserver for image streaming

The same way we need start rosbridge server, we need to start a specific ros process to stream the images:

rosrun web_video_server web_video_server _port:=11315

We are specifying the port at the end because of ROSDS, it provides us a specific address to this port. If you are working local, you can access it at locahost:<port>. The port number is shown in the terminal you run it.

 


4 – Adjusting JavaScript code

In our main.js file, let’s add the camera connector. It’s basically a way to make the pre-defined element (in our HTML) to receive the images.

  • Create a new method:
        setCamera: function() {
            console.log('set camera method')
            this.cameraViewer = new MJPEGCANVAS.Viewer({
                divID: 'mjpeg',
                host: '54.167.21.209',
                width: 640,
                height: 480,
                topic: '/camera/rgb/image_raw',
                port: 11315,
            })
        },

Notice that you have to change the attributes host, topic and port accordling to your configuration.

The attributes width and height are going to be applied to the size of the element.

  • Call the method whenever the rosbridge server is connected (inside the callback)
            this.ros.on('connection', () => {
                this.logs.unshift((new Date()).toTimeString() + ' - Connected!')
                this.connected = true
                this.loading = false
                this.setCamera()
            })

Pay attention to the IP we are using in the code! You must have a different IP than the one used in this post. If you are using ROSDS, get yout public IP executing the following command:

public_ip

If you are on your local computer, it must be:

localhost

5 – Testing the webpage

At this point you must have something similar to the image below:

We have used a different world this time to have some obstacles that we can identify in the camera. You can switch between many other simulations to see different results on the camera!


ROSJect created along with the post:

http://www.rosject.io/l/ddd21cc/

[ROS Mini Challenge] #7 – make a robot follow another robot

[ROS Mini Challenge] #7 – make a robot follow another robot

In this post, we will see how to make a robot follow another robot. We’ll make the iRobot follow the big turtle all around the world when it moves, using ROS TF broadcaster and listener nodes.

PS: This ROS project is part of our ROS Mini Challenge series, which gives you an opportunity to win an amazing ROS Developers T-shirt! This challenge is already solved. For updates on future challenges, please stay tuned to our Twitter channel.

Step 1: Grab a copy of the ROS Project containing the code for the challenge

Click here to get your own copy of the project. If you don’t have an account on the ROS Development Studio, you will need to create one. Once you create an account or log in, we will copy the project to your workspace. That done, open your ROSject using the Open button. This might take a few moments, please be patient.

You should now see a notebook with detailed instructions about the challenge. This post includes a summary of these instructions as well as the solution to the challenge.

PS: Please ignore the Claim your Prize! section because…well…you are late the party 🙂

Step 2: Start the Simulation and get the robots moving

  1. Click on the Simulations menu and then Choose launch file . In the dialog that appears, select rotw7.launch under turtle_tf_3d package. Then click the Launch button. You should see a Gazebo window popup showing the simulation.
  2. Get the robots moving. Pick a Shell from the Tools menu and run the following commands:
user:~$ source ~/catkin_ws/devel/setup.bash
user:~$ roslaunch rotw7_pkg irobot_follow_turtle.launch

At this point, you should already see the iRobot moving towards the big turtle.

Nothing happened? Heck, we gotta fix this! Let’s do that in the next section.

Step 3: Let’s find the problem

So the robots didn’t move as we expected. And we had this error message:

[INFO] [1580892397.791963, 77.216000]: Retrieveing Model indexes
[INFO] [1580892397.860043, 77.241000]: Robot Name=irobot, is NOT in model_state, trying again

The error message above says it cannot find the model name specified in the code, so let’s check that up. Fire up the IDE from the Tools menu and browse to the directory catkin_ws/src/rotw7_pkg/scripts. We have two Python scripts in there:

  • turtle_tf_broadcaster.py
  • turtle_tf_listener.py

The robot model names are specified on line 19 of turtle_tf_broadcaster.py:, in the publisher_of_tf function:

robot_name_list = ["irobot","turtle"]

Let’s check if we can find these robots in the simulation, using a Gazebo service:

user:~$ rosservice call /gazebo/get_world_properties "{}"
sim_time: 424.862
model_names: [ground_plane, coke_can, turtle1, turtle2]
rendering_enabled: True
success: True
status_message: "GetWorldProperties: got properties"

So we see that the names we specified are not in the simulation! The robots we need are turtle2 and turtle1.

Also, on line 19 of turtle_tf_listener.py, the code is publishing the “cmd_vel” (the topic that moves the robot) of the following robot:

turtle_vel = rospy.Publisher('/cmd_vel', geometry_msgs.msg.Twist,queue_size=1)

But, which of the turtles is the follower, and what is the correct topic for its “cmd_vel”? We have a hint from the launch file irobot_follow_turtle.launch:

<?xml version="1.0" encoding="UTF-8"?>
<launch>
    <include file="$(find rotw7_pkg)/launch/run_turtle_tf_broadcaster.launch"/>
    <include file="$(find rotw7_pkg)/launch/run_turtle_tf_listener.launch">
        <arg name="model_to_be_followed_name" value="turtle1" />
        <arg name="follower_model_name" value="turtle2" />
    </include>
</launch>

So the follower is turtle2. Now, let’s check what it’s “cmd_vel” topic is. It’s specified as /cmd_vel in the code, but is this true? Let’s check the list of topics:

user:~$ rostopic list
#...
/turtle1/cmd_vel
/turtle2/cmd_vel

Probably, it’s /turtle2/cmd_vel. How do we know? Let’s publish to both /cmd_vel and /turtle2/cmd_vel and see which works.

ser:~$ rostopic pub /cmd_vel
Display all 152 possibilities? (y or n)
user:~$ rostopic pub /turtle2/cmd_vel geometry_msgs/Twist "linear:
  x: 0.2
  y: 0.0
  z: 0.0
angular:
  x: 0.0
  y: 0.0
  z: 0.0"
publishing and latching message. Press ctrl-C to terminate
^Cuser:~$ rostopic pub /turtle2/cmd_vel geometry_msgs/Twist "linr:r
  x: 0.0
  y: 0.0
  z: 0.0
angular:
  x: 0.0
  y: 0.0
  z: 0.0"
publishing and latching message. Press ctrl-C to terminate

Publishing to /turtle2/cmd_vel works. /cmd_vel didn’t work.

Step 4: Let’s fix the problem

We saw the problems in Step 3, now let’s implement the fix and test again.

On line 19 of turtle_tf_broadcaster.py, change the list to reflect the real turtle names:

robot_name_list = ["turtle1","turtle2"]

Also, on line 19 of turtle_tf_listener.py, change /cmd_velto /turtle2/cmd_vel:

turtle_vel = rospy.Publisher('/turtle2/cmd_vel', geometry_msgs.msg.Twist,queue_size=1)

Now rerun the commands to move the robots:

user:~$ source ~/catkin_ws/devel/setup.bash
user:~$ roslaunch rotw7_pkg irobot_follow_turtle.launch

You should now see the iRobot moving towards the big turtle. Now you can start moving the Turtle using the keyboard. Pick another Shell from the Tools menu and run the following command:

user:~$ roslaunch turtle_tf_3d turtle_keyboard_move.launch

Move the big turtle around with the keyboard, and you should see that the iRobot follows it. Done, that’s an example of how to make a robot follow another robot.

Extra: Video of this post

We made a video showing how we solved this challenge and made the iRobot follow another robot. If you prefer “sights and sounds” to “black and white”, here you go:

Related Resources

Feedback

Did you like this post? Do you have any questions about the explanations? Whatever the case, please leave a comment on the comments section below, so we can interact and learn from each other.

If you want to learn about other ROS or ROS2 topics, please let us know in the comments area and we will do a video or post about it.

ROS Mini Challenge #6 – detect the position of a robot using ROS Service

ROS Mini Challenge #6 – detect the position of a robot using ROS Service

 

 

 

 

 

 

 

 

What we are going to learn

Learn how to get the position and orientation of a robot, when someone asks through ROS Services.

List of resources used in this post

 

Opening the ROSject

Once you clicked in the ROSject link provided (https://buff.ly/2Pd5msM) you will get a copy of the ROSject. You can then click Open to have it open.

Where to find the code

Once you open the ROsject, in the ~/catkin_ws folder, you will find all the ROS packages associated with this challenge. You can have a look at it using the IDE, for instance.

To see the full code, open the IDE by going to the top menu and select Tools->IDE

Code Editor (IDE) - ROSDS

Code Editor (IDE) – ROSDS

Launching the Simulation

Go to the top menu and select Simulations. On the menu that appears, click on the Choose launch file… button.

Choose lunch file to open simulation in ROSDS

Choose lunch file to open simulation in ROSDS

 

Now, from the launch files list, select the launch file named rotw5.launch from the rosbot_gazebo package.

ROS Mini Challenged #6 – launch file

 

Finally, click on the Launch button to start the simulation. In the end, you should get something like this:

Husky Simulation in ROSDS

 

The problem to solve

As you can see, this ROSject contains 1 package inside its catkin_ws workspace: rotw6_pkg. This package contains a couple of Python scripts (get_pose_service.py and get_pose_client.py), which contains a Service Server and a Client, respectively.

So, the main purpose of this Service is to be able to get the Pose (position and orientation) of the Husky robot when called. In order to get the Pose of the robot, all you have to do is the following:

First, make sure you source your workspace so that ROS can find your package:
source ~/catkin_ws/devel/setup.bash

 

Now, start your Service Server with the following command:
rosrun rotw6_pkg get_pose_service.py

 

Finally, just launch your Service Client with the following command:

rosrun rotw6_pkg get_pose_client.py

 

Now, in the Shell where you started the Service Server, you should see the following:

Get robot pose service in ROSDS

NOTE: Here, it’s VERY IMPORTANT to note that it’s only retrieving the position and orientation of the robot, without any additional data.

Solving the ROS Mini Challenge

Ok, so… where’s the problem? If you have tried to reproduce the steps described above you have already seen that it DOES NOT WORK. When you run the Service Client, you are getting some errors that aren’t supposed to be there. So… what’s going on?

When we launch the service client, we get something like the error below:

user:~$ rosrun rotw6_pkg get_pose_client.py
Traceback (most recent call last):
  File "/home/user/catkin_ws/src/rotw6_pkg/src/get_pose_client.py", line 11, in <module>
    result = get_pose_client(get_pose_request_object)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 435, in __call__
    return self.call(*args, **kwds)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 495, in call
    service_uri = self._get_service_uri(request)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 444, in _get_service_uri
    raise TypeError("request object is not a valid request message instance")
TypeError: request object is not a valid request message instanc

The original content of the get_pose_client.py in the rotw6_pkg package is:

#! /usr/bin/env python

import rospy
from std_srvs.srv import Empty

rospy.init_node('rotw6_client')
rospy.wait_for_service('/get_pose_service')
get_pose_client = rospy.ServiceProxy('/get_pose_service', Empty)
get_pose_request_object = Empty()

result = get_pose_client(get_pose_request_object)

The problem is on the line get_pose_request_object = Empty(). We need to use EmptyRequest instead. The correct client, then, would be:

#! /usr/bin/env python

import rospy
from std_srvs.srv import Empty, EmptyRequest


rospy.init_node('rotw6_client')
rospy.wait_for_service('/get_pose_service')
get_pose_client = rospy.ServiceProxy('/get_pose_service', Empty)
get_pose_request_object = EmptyRequest()

result = get_pose_client(get_pose_request_object)

If we now run the client again, we should see no errors:

rosrun rotw6_pkg get_pose_client.py

and in the output of the server, we have:

$ rosrun rotw6_pkg get_pose_service.py
Robot Pose:
header:  seq: 86596  stamp:    secs: 3032    nsecs: 830000000
  frame_id: "odom"
child_frame_id: "base_link"
pose:
  pose:
    position:
      x: -0.0712665171853
      y: -0.00030466544589
      z: 0.0
    orientation:
      x: 0.0
      y: 0.0      z: -0.493612905474      w: 0.869681723132  covariance: [86.67158253691281, -6.76509699953108e-05, 0.0, 0.0, 0.0, -0.010918187001924307, -6.765096999530603e-05, 86.68358224257844, 0.0, 0.0, 0.0, -0.18378969144765284, 0.0, 0.0, 4.997917968019755e-07
, -4.557105661285207e-23, 3.4006027312434455e-20, 0.0, 0.0, 0.0, -4.5571056612852066e-23, 4.99583853513
7798e-07, 2.1597386386459286e-32, 0.0, 0.0, 0.0, 3.4006027312434443e-20, -6.711057929105504e-33, 4.995838535137798e-07, 0.0, -0.010918187001923645, -0.18378969144767304, 0.0, 0.0, 0.0, 103.95483332789887]
twist:
  twist:
    linear:
      x: -1.972482338e-05
      y: 0.0
      z: 0.0
    angular:
      x: 0.0
      y: 0.0
      z: 0.0292241612329
  covariance: [0.0008855670954413368, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0008855670954413368, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 4.996877927028334e-07, -7.07772753671664e-33, 5.268413186594352e-30, 0.0, 0.0, 0.0, -7.077727536716642e-33, 4.987546680559569e-07, 1.3419297548549267e-41, 0.0, 0.0, 0.0, 5.268413186594355e-30, -4.178170793934322e-42, 4.987546680559569e-07, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 9.592381167731944e-07]

as you can see, we have too much data in the output. We just need to show the position and orientation. The reason why we have so much data is that in the server code (get_pose_service.py) we print the message directly. The original server code is:

#! /usr/bin/env python

import rospy
from std_srvs.srv import Empty, EmptyResponse
from geometry_msgs.msg import PoseWithCovarianceStamped, Pose
from nav_msgs.msg import Odometry

robot_pose = Pose()

def service_callback(request):
    print "Robot Pose:"
    print robot_pose
    return EmptyResponse()

def sub_callback(msg):
    global robot_pose
    robot_pose = msg


rospy.init_node('rotw6_service')
my_service = rospy.Service('/get_pose_service', Empty , service_callback)
sub_pose = rospy.Subscriber('/odometry/filtered', Odometry, sub_callback)
rospy.spin()

The message type is Odometry. With rosmsg show Odometry  we can see this message type has a lot of fields:

$ rosmsg show Odometry
[nav_msgs/Odometry]:
std_msgs/Header header
  uint32 seq
  time stamp
  string frame_id
string child_frame_id
geometry_msgs/PoseWithCovariance pose
  geometry_msgs/Pose pose
    geometry_msgs/Point position
      float64 x
      float64 y
      float64 z
    geometry_msgs/Quaternion orientation
      float64 x
      float64 y
      float64 z
      float64 w
  float64[36] covariance
geometry_msgs/TwistWithCovariance twist
  geometry_msgs/Twist twist
    geometry_msgs/Vector3 linear
      float64 x
      float64 y
      float64 z
    geometry_msgs/Vector3 angular
      float64 x
      float64 y
      float64 z
  float64[36] covariance

In the message structure above, we are only interested in the position and orientation. To achieve that, in the server we can just change the sub_callback function from:

def sub_callback(msg):
    global robot_pose
    robot_pose = msg

to:

def sub_callback(msg):
    global robot_pose
    robot_pose = msg.pose.pose

as easy as that. Pretty easy, right?

If for any reason you are still struggling to understand or memorize the steps reproduced here, I highly recommend you taking our ROS courses at www.robotigniteacademy.com/en/ .

Youtube video

So this is the post for today. Remember that we have the live version of this post on YouTube. If you liked the content, please consider subscribing to our youtube channel. We are publishing new content ~every day.

Keep pushing your ROS Learning.

 

Developing Web Interfaces For ROS Robots #3 – Building a web joystick to control the robot

Developing Web Interfaces For ROS Robots #3 – Building a web joystick to control the robot

Hello ROS Developers,

This is the 3rd of 4 posts of the series Developing web interfaces for ROS Robots.

In this post, we are going to create a “joystick” to send velocity commands to a robot from the webpage.


1 – Creating the HTML

First we are going to create the buttons that the user will click on to move the robot.

Just after the logs section, let’s add the following:

                    ...
                    ...
	            <h3>Log messages</h3>
                    <div>
                        <p v-for="log of logs">{{ log }}</p>
                    </div>
		</div>
	</div>

        <hr>

        <div class="row">
            <div class="col-md-12 text-center">
                <h5>Commands</h5>
            </div>

            <!-- 1st row -->
            <div class="col-md-12 text-center">
                <button class="btn btn-primary">Go forward</button>
                <br><br>
            </div>

            <!-- 2nd row -->
            <div class="col-md-4 text-center">
                <button class="btn btn-primary">Turn left</button>
            </div>
            <div class="col-md-4 text-center">
                <button class="btn btn-danger">Stop</button>
                <br><br>
            </div>
            <div class="col-md-4 text-center">
                <button class="btn btn-primary">Turn right</button>
            </div>

            <!-- 3rd row -->
            <div class="col-md-12 text-center">
                <button class="btn btn-primary">Go backward</button>
            </div>
        </div>

This must provide an interface like below:


2 – Code to send the commands

Now, in our main.js file, let’s create some new methods to perform the actions we have just idealized. Inside the methods attribute we already have 2 functions. Let’s create the ones below:

        setTopic: function() {
            this.topic = new ROSLIB.Topic({
                ros: this.ros,
                name: '/cmd_vel',
                messageType: 'geometry_msgs/Twist'
            })
        },
        forward: function() {
            this.message = new ROSLIB.Message({
                linear: { x: 1, y: 0, z: 0, },
                angular: { x: 0, y: 0, z: 0, },
            })
            this.setTopic()
            this.topic.publish(this.message)
        },
        stop: function() {
            this.message = new ROSLIB.Message({
                linear: { x: 0, y: 0, z: 0, },
                angular: { x: 0, y: 0, z: 0, },
            })
            this.setTopic()
            this.topic.publish(this.message)
        },
        backward: function() {
            this.message = new ROSLIB.Message({
                linear: { x: -1, y: 0, z: 0, },
                angular: { x: 0, y: 0, z: 0, },
            })
            this.setTopic()
            this.topic.publish(this.message)
        },
        turnLeft: function() {
            this.message = new ROSLIB.Message({
                linear: { x: 0.5, y: 0, z: 0, },
                angular: { x: 0, y: 0, z: 0.5, },
            })
            this.setTopic()
            this.topic.publish(this.message)
        },
        turnRight: function() {
            this.message = new ROSLIB.Message({
                linear: { x: 0.5, y: 0, z: 0, },
                angular: { x: 0, y: 0, z: -0.5, },
            })
            this.setTopic()
            this.topic.publish(this.message)
        },

We have defined a common method setTopic that is used by the others: forwardbackward, stop, turnLeft and turnRight.

The task inside of each one is always the same. Define a message to send, set the topic and publish the message.

The only thing that changes from one to another are the values of the Twist message.


3 – Integrating HTML and JavaScript

Now that we have the elements in our page and the methods that perform the actions, let’s make them work together.

In order to do so, we need to assign the methods to the “click” events of each button. It goes like this:

            <!-- 1st row -->
            <div class="col-md-12 text-center">
                <button @click="forward" :disabled="loading || !connected" class="btn btn-primary">Go forward</button>
                <br><br>
            </div>

            <!-- 2nd row -->
            <div class="col-md-4 text-center">
                <button @click="turnLeft" :disabled="loading || !connected" class="btn btn-primary">Turn left</button>
            </div>
            <div class="col-md-4 text-center">
                <button @click="stop" :disabled="loading || !connected" class="btn btn-danger">Stop</button>
                <br><br>
            </div>
            <div class="col-md-4 text-center">
                <button @click="turnRight" :disabled="loading || !connected" class="btn btn-primary">Turn right</button>
            </div>

            <!-- 3rd row -->
            <div class="col-md-12 text-center">
                <button @click="backward" :disabled="loading || !connected" class="btn btn-primary">Go backward</button>
            </div>

Notice that we are not only defining the @click attribute, but also the “:disabled”. It means that the button will be disabled if one of the conditions are true: Loading or not connected.


4 – Final result

At this point, you must have the following result with your webpage:


ROSJect created along with the post:

http://www.rosject.io/l/ddbab4e/

[ROS Mini Challenge] #5 – make a robot detect and avoid an obstacle

[ROS Mini Challenge] #5 – make a robot detect and avoid an obstacle

In this post, we will see how to make a robot detect and avoid an obstacle in its front. We will move the robot forward until it detects there’s an obstacle (the wall) closer than 1 meter. Then, we will stop the robot so that it does not collide with the wall.

PS: This ROS project is part of our ROS Mini Challenge series, which gives you an opportunity to win an amazing ROS Developers T-shirt! This challenge is already solved. For updates on future challenges, please stay tuned to our Twitter channel.

Step 1: Grab a copy of the ROS Project that contains the code for the challenge

Click here to get your own copy of the project. If you don’t have an account on the ROS Development Studio, you will need to create one. Once you create an account or log in, we will copy the project to your workspace. That done, open your ROSject using the Open button. This might take a few moments, please be patient.

You should now see a notebook with detailed instructions about the challenge. This post includes a summary of these instructions as well as the solution to the challenge.

PS: Please ignore the Claim your Prize! section because…well…you are late the party 🙂

Step 2: Start the Simulation and get the robot moving

  1. Click on the Simulations menu and then Choose launch file . In the dialog that appears, select rotw5.launch under rosbot_gazebo package. Then click the Launch button. You should see a Gazebo window popup showing the simulation: a ROSbot in front of a wall.
  2. Get the robot moving. Pick a Shell from the Tools menu and run the following commands:
user:~$ source ~/catkin_ws/devel/setup.bash
user:~$ rosrun rotw5_pkg detect_wall.py

Yay, the robot moved. But, wait, it didn’t detect and avoid the obstacle – it crashed into the wall! We don’t want that, so let’s fix it in the next section!

Step 3: Identify the problem, man!

So the robot didn’t stop as we expected. As the challenge hinted, something must be wrong with the code, especially the part that should detect the wall and stop the robot. Let’s see.

Fire up the IDE from the Tools menu and browse to catkin_ws/src/rotw5_pkg/src/detect_wall.py

#! /usr/bin/env python

import rospy
from sensor_msgs.msg import LaserScan
from geometry_msgs.msg import Twist

def callback(msg):

  #If the distance to an obstacle in front of the robot is bigger than 1 meter, the robot will move forward
  if msg.ranges[360] > 1:
      move.linear.x = 0.5
      move.angular.z = 0.0

  #If the distance to an obstacle in front of the robot is smaller than 1 meter, the robot will stop
  if msg.ranges[360] < 1:
      move.linear.x = 0.0
      move.angular.z = 0.0

  pub.publish(move)

rospy.init_node('rotw5_node')
sub = rospy.Subscriber('/scan', LaserScan, callback) #We subscribe to the laser's topic
pub = rospy.Publisher('/cmd_vel', Twist)
move = Twist()

rospy.spin()

Now, this is the part that should stop the robot. It says that if there’s an obstacle less than 1 meter away, we should set both linear and angular velocities to zero (stopping the robot)

  #If the distance to an obstacle in front of the robot is smaller than 1 meter, the robot will stop
  if msg.ranges[360] < 1:
      move.linear.x = 0.0
      move.angular.z = 0.0

This is not being executed obviously because the condition msg.ranges[360] < 1 is never True! Let’s test this assumption:

  • In the IDE, add the following lines to the detect_wall.py, after the comment below. Then save.
  #If the distance to an obstacle in front of the robot is smaller than 1 meter, the robot will stop
  print "Number of readings: ", print len(msg.ranges)
  print "Reading at position 360:", msg.ranges[360]
  • From the Shell: Press Ctrl + C to end the current program. Then stop the robot by publishing zero velocities to the /cmd_vel topic. Press Ctrl + C again and then call the Gazebo service that resets the world.
user:~$ rostopic pub /cmd_vel geometry_msgs/Twist "linear:
  x: 0.0
  y: 0.0
  z: 0.0
angular:
  x: 0.0
  y: 0.0
  z: 0.0"
publishing and latching message. Press ctrl-C to terminate
user:~$ rosservice call /gazebo/reset_world "{}"
  • Run the program again and observe the output:
user:~$ rosrun rotw5_pkg detect_wall.py
Number of readings:  720
Reading at position 360: inf
# ...
Number of readings:  720
Reading at position 360: 11.7952680588

You should notice that several readings at 360 are printed, and none of them is less than 1. Whew!

Step 4: Now let’s solve the problem

We saw from the previous section that our problem was the condition msg.ranges[360] < 1, but how did we arrive at this condition?

  • We knew the laser sensor has 720 scan points stored in the ranges array, as seen in the output of our print statement in Step 3.
  • We assumed that the laser had a range of 180 degrees, covering the front of the robot only, so that position 360 is at the front-middle of the robot. But this is false, and we can confirm this by running the following command:
user:~$ rostopic echo /scan -n1
header:
  seq: 49846
  stamp:
    secs: 639
    nsecs:  47000000
  frame_id: "laser"
angle_min: -3.14159011841
angle_max: 3.14159011841
angle_increment: 0.00873877573758
time_increment: 0.0

The command prints out a single message from the /scan topic. We can deduce the range from the angle_max and angle_min values: angle_max - angle_min = 2pi = 360 deg!  Buff! So position 360/720 of the scan messages could be somewhere at the back of the robot. No wonder it didn’t detect the wall.

Now, we have to figure out which position is the front-middle of the robot. Depending on how the sensor was installed, this could be any position. For this robot, the sensor was installed with position “0″ at the front-center, so the readings of positions “0″ or 720 could represent the front-center.  Let’s use the position “0″  and test again.

  • In the IDE, change the front-middle position from 360 to “0″:
#! /usr/bin/env python

import rospy
from sensor_msgs.msg import LaserScan
from geometry_msgs.msg import Twist

def callback(msg):

#If the distance to an obstacle in front of the robot is bigger than 1 meter, the robot will move forward
  if msg.ranges[0] > 1:
      move.linear.x = 0.5
      move.angular.z = 0.0

#If the distance to an obstacle in front of the robot is smaller than 1 meter, the robot will stop
  print "Number of ranges: ", len(msg.ranges)
  print "Reading at position 0:", msg.ranges[0]
  if msg.ranges[0] < 1:
      move.linear.x = 0.0
      move.angular.z = 0.0

  pub.publish(move)

rospy.init_node('rotw5_node')
sub = rospy.Subscriber('/scan', LaserScan, callback) #We subscribe to the laser's topic
pub = rospy.Publisher('/cmd_vel', Twist)
move = Twist()

rospy.spin()
  • From the Shell: Press Ctrl + C to end the current program. Then stop the robot by publishing zero velocities to the /cmd_vel topic. Press Ctrl + C again and then call the Gazebo service that resets the world.
  • Run the program again and observe the output and see if the robot detects and avoids the obstacle:
user:~$ rosrun rotw5_pkg detect_wall.py
Number of ranges:  720
Reading at position 0: 2.49150013924
Number of ranges:  720
#...
Number of ranges:  720
Reading at position 0: 1.92249274254
#...
Number of ranges:  720
Reading at position 0: 0.991922438145

Yay, finally we are able to make the robot detect and avoid the obstacle!

 

Extra: Video of this post

We made a video showing how this challenge was solved. If you prefer “sights and sounds” to “black and white”, here you go:

Related Resources

Feedback

Did you like this post? Do you have questions about what was explained? Whatever the case, please leave a comment on the comments section below, so we can interact and learn from each other.

If you want to learn about other ROS or ROS2 topics, please let us know in the comments area and we will do a video or post about it.

ROS Mini Challenge #4 – Controlling a drone with ROS Actions

ROS Mini Challenge #4 – Controlling a drone with ROS Actions

 

 

 

 

 

 

 

 

 

What we are going to learn

Learn how to create a ROS Action message and compile it for C++, to control a drone.

 

List of resources used in this post

Opening the ROSject

Once you clicked in the link to get a copy of the ROSject,

When you click in the ROSject Link (www.rosject.io/l/e80ec70/), you will get a copy of it. You can then download it if you want to use it on our local computer, or you can just click Open to have it opened in ROSDS.
Once it’s open, you can find all the code associated with this challenge in the ~/catkin_ws folder. To see the full code, open the IDE by going to the top menu and select Tools->IDE.

Code Editor (IDE) - ROSDS

Code Editor (IDE) – ROSDS

Launching the simulation

Go to the top menu and select Simulations. On the menu that appears, click on the Choose launch file… button.

Choose lunch file to open simulation in ROSDS

Choose lunch file to open simulation in ROSDS

Now, from the launch files list, select the launch file named rotw4.launch from the drone_construct package.
ROS Mini Challenge #4 - Drone - ROSDS

ROS Mini Challenge #4 – Drone – ROSDS

Finally, click on the Launch button to start the simulation. In the end, you should get something like this:

Drone simulation in ROSDS

Drone simulation in ROSDS

The problem to solve

As you can see, this ROSject contains 1 package inside its catkin_ws workspace: rotw4_code. The purpose of this packages is to allow you to control the Drone (Take Off or Land) using an Action Server.

So, the main purpose of this ROSject is to be able to send a Goal to this Action Server, specifying a keyword in the goal (either “UP” or “DOWN”), and the Drone receives and executes this goal. The steps in order to achieve this are the following:

First, you will need to compile the Action Server and the Action message used. For this, execute the following command:

cd ~/catkin_ws/
catkin_make
source ~/catkin_ws/devel/setup.bash

 

Once, the compilation has finished successfully, you will start the Action Server by executing the following command:

rosrun rotw4_code rotw4_action_node

 

Once the Action Server is up and running, you can send goals to the Action with the following commands.

In order to make the Drone TAKE OFF:

rostopic pub /rotw4_action/goal rotw4_code/ActionMsgActionGoal "header:
  seq: 0
  stamp:
    secs: 0
    nsecs: 0
  frame_id: ''
goal_id:
  stamp:
    secs: 0
    nsecs: 0
  id: ''
goal:
  goal: 'UP'"

You should see the Drone taking off, like in the following image:

Drone taking off (moving up) in ROSDS

Drone taking off (moving up) in ROSDS

And in order to make the Drone LAND, you could use the following command:

rostopic pub /rotw4_action/goal rotw4_code/ActionMsgActionGoal "header:
  seq: 0
  stamp:
    secs: 0
    nsecs: 0
  frame_id: ''
goal_id:
  stamp:
    secs: 0
    nsecs: 0
  id: ''
goal:
  goal: 'DOWN'"

You should see the Drone landing:

Drone landing (moving down) in ROSDS

Drone landing (moving down) in ROSDS

Also, you will see in the Action Server a message indicating that the action was successfully executed.

Action Server succeeded in ROSDS

Action Server succeeded in ROSDS

Ok, so… where’s the problem? If you have tried to reproduce the steps described above you have already seen that it DOES NOT WORK. When you follow the pipeline you can see that even the 1st step, which is to compile the package, it does not work (though it is not the only error here). But Why? Let’s find it in the section below.

Solving the ROS Mini Challenge

According to the instructions, we need to have a look at the rotw4_code package and try to find there the errors.

If you looked carefully, after running the first commands, you had an error:

cd ~/catkin_ws/
catkin_make

The error was:

Building CXX object rotw4_code/CMakeFiles/rotw4_action_node.dir/src/rotw4_action.cpp.o
/home/user/catkin_ws/src/rotw4_code/src/rotw4_action.cpp:3:34: fatal error: rotw4_code/ActionMsg.h: No such file or directory
compilation terminated.
rotw4_code/CMakeFiles/rotw4_action_node.dir/build.make:62: recipe for target 'rotw4_code/CMakeFiles/rotw4_action_node.dir/src/rotw4_action.cpp.o' failed
make[2]: *** [rotw4_code/CMakeFiles/rotw4_action_node.dir/src/rotw4_action.cpp.o] Error 1
CMakeFiles/Makefile2:1909: recipe for target 'rotw4_code/CMakeFiles/rotw4_action_node.dir/all' failed
make[1]: *** [rotw4_code/CMakeFiles/rotw4_action_node.dir/all] Error 2
Makefile:138: recipe for target 'all' failed
make: *** [all] Error 2
Invoking "make -j2 -l2" failed

The problem is that it can’t find the ActionMsg.h, so, which tell us that our ~/catkin_ws/src/rotw4_code/action/ActionMsg.action is not being compiled. Let’s then open the CMakeLists.txt file located in the rotw4_code package, to see where is the error.

If we look at the line 153 of the CMakeLists.txt file, we find the following:

add_executable(rotw4_action_node src/rotw4_action.cpp)
target_link_libraries(rotw4_action_node
${catkin_LIBRARIES}
)

we didn’t add the dependencies. To solve this compilation part, let’s add the line below right after line 153:

add_dependencies(rotw4_action_node ${rotw4_action_node_EXPORTED_TARGETS}
so, in the end, it will look like:
add_executable(rotw4_action_node src/rotw4_action.cpp)
add_dependencies(rotw4_action_node ${rotw4_action_node_EXPORTED_TARGETS} ${catkin_EXPORTED_TARGETS})
target_link_libraries(rotw4_action_node
${catkin_LIBRARIES}
)

Let’s now try to compile our package again:

cd ~/catkin_ws/
catkin_make

Now we have another error. Although it looks similar, at least now the messages were compiled:

[ 97%] Building CXX object rotw4_code/CMakeFiles/rotw4_action_node.dir/src/rotw4_action.cpp.o
/home/user/catkin_ws/src/rotw4_code/src/rotw4_action.cpp:3:34: fatal error: rotw4_code/ActionMsg.h: No such file or directory
compilation terminated.

We can confirm the message was generated with find ~/catkin_ws/devel/ -name ActionMsg*.h:

$ find ~/catkin_ws/devel/ -name ActionMsg*.h
/home/user/catkin_ws/devel/include/rotw4_code/ActionMsgActionResult.h
/home/user/catkin_ws/devel/include/rotw4_code/ActionMsgActionFeedback.h
/home/user/catkin_ws/devel/include/rotw4_code/ActionMsgAction.h
/home/user/catkin_ws/devel/include/rotw4_code/ActionMsgFeedback.h
/home/user/catkin_ws/devel/include/rotw4_code/ActionMsgGoal.h
/home/user/catkin_ws/devel/include/rotw4_code/ActionMsgActionGoal.h
/home/user/catkin_ws/devel/include/rotw4_code/ActionMsgResult.h

As we can see, we have no file named ActionMsg.h, which is imported by rotw4_code/src/rotw4_action.cpp. Let’s then fix rotw4_action.cpp on line 3 and import ActionMsgAction.h. Now let’s try to compile again:

cd ~/catkin_ws/
catkin_make

Now we have different errors:

Scanning dependencies of target rotw4_action_node
[ 97%] Building CXX object rotw4_code/CMakeFiles/rotw4_action_node.dir/src/rotw4_action.cpp.o
/home/user/catkin_ws/src/rotw4_code/src/rotw4_action.cpp:11:33: error: 'ActionMsg' is not a member of 'rotw4_code'
   actionlib::SimpleActionServer<rotw4_code::ActionMsg> as_;
                                 ^
/home/user/catkin_ws/src/rotw4_code/src/rotw4_action.cpp:11:33: error: 'ActionMsg' is not a member of 'rotw4_code'
/home/user/catkin_ws/src/rotw4_code/src/rotw4_action.cpp:11:54: error: template argument 1 is invalid
   actionlib::SimpleActionServer<rotw4_code::ActionMsg> as_;
                                                      ^
/home/user/catkin_ws/src/rotw4_code/src/rotw4_action.cpp: In constructor 'ActionClass::ActionClass(std::__cxx11::string)':
/home/user/catkin_ws/src/rotw4_code/src/rotw4_action.cpp:29:26: error: expression list treated as compound expression in mem-initializer [-fpermissive]
         action_name_(name) {
                          ^
/home/user/catkin_ws/src/rotw4_code/src/rotw4_action.cpp:30:9: error: request for member 'start' in '((ActionClass*)this)->ActionClass::as_', which is of non-class type 'int'
     as_.start();
         ^
/home/user/catkin_ws/src/rotw4_code/src/rotw4_action.cpp: In member function 'void ActionClass::executeCB(const ActionMsgGoalConstPtr&)':
/home/user/catkin_ws/src/rotw4_code/src/rotw4_action.cpp:43:13: error: request for member 'isPreemptRequested' in '((ActionClass*)this)->ActionClass::as_', which is of non-class type 'int'
     if (as_.isPreemptRequested() || !ros::ok()) {
             ^
/home/user/catkin_ws/src/rotw4_code/src/rotw4_action.cpp:46:11: error: request for member 'setPreempted' in '((ActionClass*)this)->ActionClass::as_', which is of non-class type 'int'
       as_.setPreempted();
           ^
/home/user/catkin_ws/src/rotw4_code/src/rotw4_action.cpp:62:9: error: request for member 'publishFeedback' in '((ActionClass*)this)->ActionClass::as_', which is of non-class type 'int'
     as_.publishFeedback(feedback_);
         ^
/home/user/catkin_ws/src/rotw4_code/src/rotw4_action.cpp:68:11: error: request for member 'setSucceeded' in '((ActionClass*)this)->ActionClass::as_', which is of non-class type 'int'
       as_.setSucceeded(result_);

We have a bunch of errors, so, let’s look at the first one first, which says:

rotw4_action.cpp:11:33: error: 'ActionMsg' is not a member of 'rotw4_code' actionlib::SimpleActionServer<rotw4_code::ActionMsg> as_;

The problem is almost the same as before. Instead of using ActionMsgAction, we used ActionMsg. Let’s fix that on line 11, and try to compile again:

cd ~/catkin_ws/
catkin_make

Congratulations, now everything compiled as expected:

Scanning dependencies of target rotw4_action_node
[ 97%] Building CXX object rotw4_code/CMakeFiles/rotw4_action_node.dir/src/rotw4_action.cpp.o
[100%] Linking CXX executable /home/user/catkin_ws/devel/lib/rotw4_code/rotw4_action_node
[100%] Built target rotw4_action_node
Scanning dependencies of target rotw4_code_generate_messages
[100%] Built target rotw4_code_generate_messages

Now, let’s run our action server:

rosrun rotw4_code rotw4_action_node

Now we could try to make the robot take off by running the following command in a different web shell:

rostopic pub /rotw4_action/goal rotw4_code/ActionMsgActionGoal "header:
  seq: 0
  stamp:
    secs: 0
    nsecs: 0
  frame_id: ''
goal_id:
  stamp:
    secs: 0
    nsecs: 0
  id: ''
goal:
  goal: 'UP'"

The only problem is that the robot doesn’t move as expected yet, although the action server says succeeded.

rotw4_action: Succeeded

If we look the file rotw4_action.cpp, in the executeCB method, everything seems to be correct.

void executeCB(const rotw4_code::ActionMsgGoalConstPtr &goal) {

    std::string upDown = goal->goal;
    takeoff_pub_ = nh_.advertise<std_msgs::Empty>("/drone/takeoff", 1000);
    land_pub_ = nh_.advertise<std_msgs::Empty>("/drone/land", 1000);

    // check that preempt has not been requested by the client
    if (as_.isPreemptRequested() || !ros::ok()) {
      ROS_INFO("%s: Preempted", action_name_.c_str());
      // set the action state to preempted
      as_.setPreempted();
      success_ = false;
    }

    if (upDown == "UP") {
      takeoff_pub_.publish(takeoff_msg_);
      feedback_.feedback = "Taking Off Drone...";
    }

    if (upDown == "DOWN") {
      land_pub_.publish(land_msg_);
      feedback_.feedback = "Landing Drone...";
    }

    // feedback_.feedback = i;
    // publish the feedback
    as_.publishFeedback(feedback_);

    if (success_) {

      ROS_INFO("%s: Succeeded", action_name_.c_str());
      // set the action state to succeeded
      as_.setSucceeded(result_);
    }
  }

The problem is that we are instantiating the topic publishers and publishing on them straight away, and the topics need some time after being defined to start being used. To solve this, let’s add a while loop to publish at least 4 times, once a second:

void executeCB(const rotw4_code::ActionMsgGoalConstPtr &goal) {

    std::string upDown = goal->goal;
    takeoff_pub_ = nh_.advertise<std_msgs::Empty>("/drone/takeoff", 1000);
    land_pub_ = nh_.advertise<std_msgs::Empty>("/drone/land", 1000);
    ros::Rate rate(1);

    // check that preempt has not been requested by the client
    if (as_.isPreemptRequested() || !ros::ok()) {
      ROS_INFO("%s: Preempted", action_name_.c_str());
      // set the action state to preempted
      as_.setPreempted();
      success_ = false;
    }

    if (upDown == "UP") {
      int i = 0;
      while (i < 4) {
        takeoff_pub_.publish(takeoff_msg_);
        feedback_.feedback = "Taking Off Drone...";
        i++;
        rate.sleep();
      }
    }

    if (upDown == "DOWN") {
      land_pub_.publish(land_msg_);
      feedback_.feedback = "Landing Drone...";
    }

    // feedback_.feedback = i;
    // publish the feedback
    as_.publishFeedback(feedback_);

    if (success_) {

      ROS_INFO("%s: Succeeded", action_name_.c_str());
      // set the action state to succeeded
      as_.setSucceeded(result_);
    }
  }

If we now compile the code again

cd ~/catkin_ws/
catkin_make

run the node:

rosrun rotw4_code rotw4_action_node

and publish the message to take off again:

rostopic pub /rotw4_action/goal rotw4_code/ActionMsgActionGoal "header:
  seq: 0
  stamp:
    secs: 0
    nsecs: 0
  frame_id: ''
goal_id:
  stamp:
    secs: 0
    nsecs: 0
  id: ''
goal:
  goal: 'UP'"

now the robot should work as expected.

Youtube video

So this is the post for today. Remember that we have the live version of this post on YouTube. If you liked the content, please consider subscribing to our youtube channel. We are publishing new content ~every day.

Keep pushing your ROS Learning.

Pin It on Pinterest