Got a suggestion for the next steps to take of this project? We would love to hear them in the comments bellow :).
Part 2
Step 1. Environment setup
We’ll start by explaining how to create the gym environment for training. You can create a file called monoped_env.py under the my_hopper_training/src directory with the following content
#!/usr/bin/env python
'''
By Miguel Angel Rodriguez <duckfrost@theconstructsim.com>
Visit our website at ec2-54-246-60-98.eu-west-1.compute.amazonaws.com
'''
import gym
import rospy
import numpy as np
import time
from gym import utils, spaces
from geometry_msgs.msg import Pose
from gym.utils import seeding
from gym.envs.registration import register
from gazebo_connection import GazeboConnection
from joint_publisher import JointPub
from monoped_state import MonopedState
from controllers_connection import ControllersConnection
#register the training environment in the gym as an available one
reg = register(
id='Monoped-v0',
entry_point='monoped_env:MonopedEnv',
timestep_limit=50,
)
class MonopedEnv(gym.Env):
def __init__(self):
# We assume that a ROS node has already been created
# before initialising the environment
# gets training parameters from param server
self.desired_pose = Pose()
self.desired_pose.position.x = rospy.get_param("/desired_pose/x")
self.desired_pose.position.y = rospy.get_param("/desired_pose/y")
self.desired_pose.position.z = rospy.get_param("/desired_pose/z")
self.running_step = rospy.get_param("/running_step")
self.max_incl = rospy.get_param("/max_incl")
self.max_height = rospy.get_param("/max_height")
self.min_height = rospy.get_param("/min_height")
self.joint_increment_value = rospy.get_param("/joint_increment_value")
self.done_reward = rospy.get_param("/done_reward")
self.alive_reward = rospy.get_param("/alive_reward")
self.desired_force = rospy.get_param("/desired_force")
self.desired_yaw = rospy.get_param("/desired_yaw")
self.weight_r1 = rospy.get_param("/weight_r1")
self.weight_r2 = rospy.get_param("/weight_r2")
self.weight_r3 = rospy.get_param("/weight_r3")
self.weight_r4 = rospy.get_param("/weight_r4")
self.weight_r5 = rospy.get_param("/weight_r5")
# stablishes connection with simulator
self.gazebo = GazeboConnection()
self.controllers_object = ControllersConnection(namespace="monoped")
self.monoped_state_object = MonopedState( max_height=self.max_height,
min_height=self.min_height,
abs_max_roll=self.max_incl,
abs_max_pitch=self.max_incl,
joint_increment_value=self.joint_increment_value,
done_reward=self.done_reward,
alive_reward=self.alive_reward,
desired_force=self.desired_force,
desired_yaw=self.desired_yaw,
weight_r1=self.weight_r1,
weight_r2=self.weight_r2,
weight_r3=self.weight_r3,
weight_r4=self.weight_r4,
weight_r5=self.weight_r5
)
self.monoped_state_object.set_desired_world_point(self.desired_pose.position.x,
self.desired_pose.position.y,
self.desired_pose.position.z)
self.monoped_joint_pubisher_object = JointPub()
"""
For this version, we consider 6 actions
1-2) Increment/Decrement haa_joint
3-4) Increment/Decrement hfe_joint
5-6) Increment/Decrement kfe_joint
"""
self.action_space = spaces.Discrete(6)
self.reward_range = (-np.inf, np.inf)
self._seed()
# A function to initialize the random generator
def _seed(self, seed=None):
self.np_random, seed = seeding.np_random(seed)
return [seed]
# Resets the state of the environment and returns an initial observation.
def _reset(self):
# 0st: We pause the Simulator
rospy.logdebug("Pausing SIM...")
self.gazebo.pauseSim()
# 1st: resets the simulation to initial values
rospy.logdebug("Reset SIM...")
self.gazebo.resetSim()
# 2nd: We Set the gravity to 0.0 so that we dont fall when reseting joints
# It also UNPAUSES the simulation
rospy.logdebug("Remove Gravity...")
self.gazebo.change_gravity(0.0, 0.0, 0.0)
# EXTRA: Reset JoinStateControlers because sim reset doesnt reset TFs, generating time problems
rospy.logdebug("reset_monoped_joint_controllers...")
self.controllers_object.reset_monoped_joint_controllers()
# 3rd: resets the robot to initial conditions
rospy.logdebug("set_init_pose...")
self.monoped_joint_pubisher_object.set_init_pose()
# 5th: Check all subscribers work.
# Get the state of the Robot defined by its RPY orientation, distance from
# desired point, contact force and JointState of the three joints
rospy.logdebug("check_all_systems_ready...")
self.monoped_state_object.check_all_systems_ready()
rospy.logdebug("get_observations...")
observation = self.monoped_state_object.get_observations()
# 6th: We restore the gravity to original
rospy.logdebug("Restore Gravity...")
self.gazebo.change_gravity(0.0, 0.0, -9.81)
# 7th: pauses simulation
rospy.logdebug("Pause SIM...")
self.gazebo.pauseSim()
# Get the State Discrete Stringuified version of the observations
state = self.get_state(observation)
return state
def _step(self, action):
# Given the action selected by the learning algorithm,
# we perform the corresponding movement of the robot
# 1st, decide which action corresponsd to which joint is incremented
next_action_position = self.monoped_state_object.get_action_to_position(action)
# We move it to that pos
self.gazebo.unpauseSim()
self.monoped_joint_pubisher_object.move_joints(next_action_position)
# Then we send the command to the robot and let it go
# for running_step seconds
time.sleep(self.running_step)
self.gazebo.pauseSim()
# We now process the latest data saved in the class state to calculate
# the state and the rewards. This way we guarantee that they work
# with the same exact data.
# Generate State based on observations
observation = self.monoped_state_object.get_observations()
# finally we get an evaluation based on what happened in the sim
reward,done = self.monoped_state_object.process_data()
# Get the State Discrete Stringuified version of the observations
state = self.get_state(observation)
return state, reward, done, {}
def get_state(self, observation):
"""
We retrieve the Stringuified-Discrete version of the given observation
:return: state
"""
return self.monoped_state_object.get_state_as_string(observation)
This script creates the gym environment with the following parts:
1.Register the training environment
You will always need to do this step to inform the gym package that a new training environment is created.
2.Load the desired pose
Parameters related to the training is loaded at this step.
3.Connect with the gazebo
In order to connect the gym environment with gazebo simulation, we create a script called gazebo_connection.py in the same directory with the following content
#!/usr/bin/env python
import rospy
from std_srvs.srv import Empty
from gazebo_msgs.msg import ODEPhysics
from gazebo_msgs.srv import SetPhysicsProperties, SetPhysicsPropertiesRequest
from std_msgs.msg import Float64
from geometry_msgs.msg import Vector3
class GazeboConnection():
def __init__(self):
self.unpause = rospy.ServiceProxy('/gazebo/unpause_physics', Empty)
self.pause = rospy.ServiceProxy('/gazebo/pause_physics', Empty)
self.reset_proxy = rospy.ServiceProxy('/gazebo/reset_simulation', Empty)
# Setup the Gravity Controle system
service_name = '/gazebo/set_physics_properties'
rospy.logdebug("Waiting for service " + str(service_name))
rospy.wait_for_service(service_name)
rospy.logdebug("Service Found " + str(service_name))
self.set_physics = rospy.ServiceProxy(service_name, SetPhysicsProperties)
self.init_values()
# We always pause the simulation, important for legged robots learning
self.pauseSim()
def pauseSim(self):
rospy.wait_for_service('/gazebo/pause_physics')
try:
self.pause()
except rospy.ServiceException, e:
print ("/gazebo/pause_physics service call failed")
def unpauseSim(self):
rospy.wait_for_service('/gazebo/unpause_physics')
try:
self.unpause()
except rospy.ServiceException, e:
print ("/gazebo/unpause_physics service call failed")
def resetSim(self):
rospy.wait_for_service('/gazebo/reset_simulation')
try:
self.reset_proxy()
except rospy.ServiceException, e:
print ("/gazebo/reset_simulation service call failed")
def resetWorld(self):
rospy.wait_for_service('/gazebo/reset_world')
try:
self.reset_proxy()
except rospy.ServiceException, e:
print ("/gazebo/reset_world service call failed")
def init_values(self):
rospy.wait_for_service('/gazebo/reset_simulation')
try:
# reset_proxy.call()
self.reset_proxy()
except rospy.ServiceException, e:
print ("/gazebo/reset_simulation service call failed")
self._time_step = Float64(0.001)
self._max_update_rate = Float64(1000.0)
self._gravity = Vector3()
self._gravity.x = 0.0
self._gravity.y = 0.0
self._gravity.z = 0.0
self._ode_config = ODEPhysics()
self._ode_config.auto_disable_bodies = False
self._ode_config.sor_pgs_precon_iters = 0
self._ode_config.sor_pgs_iters = 50
self._ode_config.sor_pgs_w = 1.3
self._ode_config.sor_pgs_rms_error_tol = 0.0
self._ode_config.contact_surface_layer = 0.001
self._ode_config.contact_max_correcting_vel = 0.0
self._ode_config.cfm = 0.0
self._ode_config.erp = 0.2
self._ode_config.max_contacts = 20
self.update_gravity_call()
def update_gravity_call(self):
self.pauseSim()
set_physics_request = SetPhysicsPropertiesRequest()
set_physics_request.time_step = self._time_step.data
set_physics_request.max_update_rate = self._max_update_rate.data
set_physics_request.gravity = self._gravity
set_physics_request.ode_config = self._ode_config
rospy.logdebug(str(set_physics_request.gravity))
result = self.set_physics(set_physics_request)
rospy.logdebug("Gravity Update Result==" + str(result.success) + ",message==" + str(result.status_message))
self.unpauseSim()
def change_gravity(self, x, y, z):
self._gravity.x = x
self._gravity.y = y
self._gravity.z = z
self.update_gravity_call()
This script is responsible to reset, pause and unpause the simulation for the training script.
4.Connect with the controller
Since TF doesn’t like that we reset the environment and it will generate some problems. We have to manually reset the controller with the controller_connection.py script. We won’t go into detail here. If you want to learn more about the controller, please check our controller 101 course.
#!/usr/bin/env python
'''
By Miguel Angel Rodriguez <duckfrost@theconstructsim.com>
Visit our website at ec2-54-246-60-98.eu-west-1.compute.amazonaws.com
'''
import gym
import rospy
import numpy as np
import time
from gym import utils, spaces
from geometry_msgs.msg import Pose
from gym.utils import seeding
from gym.envs.registration import register
from gazebo_connection import GazeboConnection
from joint_publisher import JointPub
from monoped_state import MonopedState
from controllers_connection import ControllersConnection
#register the training environment in the gym as an available one
reg = register(
id='Monoped-v0',
entry_point='monoped_env:MonopedEnv',
timestep_limit=50,
)
class MonopedEnv(gym.Env):
def __init__(self):
# We assume that a ROS node has already been created
# before initialising the environment
# gets training parameters from param server
self.desired_pose = Pose()
self.desired_pose.position.x = rospy.get_param("/desired_pose/x")
self.desired_pose.position.y = rospy.get_param("/desired_pose/y")
self.desired_pose.position.z = rospy.get_param("/desired_pose/z")
self.running_step = rospy.get_param("/running_step")
self.max_incl = rospy.get_param("/max_incl")
self.max_height = rospy.get_param("/max_height")
self.min_height = rospy.get_param("/min_height")
self.joint_increment_value = rospy.get_param("/joint_increment_value")
self.done_reward = rospy.get_param("/done_reward")
self.alive_reward = rospy.get_param("/alive_reward")
self.desired_force = rospy.get_param("/desired_force")
self.desired_yaw = rospy.get_param("/desired_yaw")
self.weight_r1 = rospy.get_param("/weight_r1")
self.weight_r2 = rospy.get_param("/weight_r2")
self.weight_r3 = rospy.get_param("/weight_r3")
self.weight_r4 = rospy.get_param("/weight_r4")
self.weight_r5 = rospy.get_param("/weight_r5")
# stablishes connection with simulator
self.gazebo = GazeboConnection()
self.controllers_object = ControllersConnection(namespace="monoped")
self.monoped_state_object = MonopedState( max_height=self.max_height,
min_height=self.min_height,
abs_max_roll=self.max_incl,
abs_max_pitch=self.max_incl,
joint_increment_value=self.joint_increment_value,
done_reward=self.done_reward,
alive_reward=self.alive_reward,
desired_force=self.desired_force,
desired_yaw=self.desired_yaw,
weight_r1=self.weight_r1,
weight_r2=self.weight_r2,
weight_r3=self.weight_r3,
weight_r4=self.weight_r4,
weight_r5=self.weight_r5
)
self.monoped_state_object.set_desired_world_point(self.desired_pose.position.x,
self.desired_pose.position.y,
self.desired_pose.position.z)
self.monoped_joint_pubisher_object = JointPub()
"""
For this version, we consider 6 actions
1-2) Increment/Decrement haa_joint
3-4) Increment/Decrement hfe_joint
5-6) Increment/Decrement kfe_joint
"""
self.action_space = spaces.Discrete(6)
self.reward_range = (-np.inf, np.inf)
self._seed()
# A function to initialize the random generator
def _seed(self, seed=None):
self.np_random, seed = seeding.np_random(seed)
return [seed]
# Resets the state of the environment and returns an initial observation.
def _reset(self):
# 0st: We pause the Simulator
rospy.logdebug("Pausing SIM...")
self.gazebo.pauseSim()
# 1st: resets the simulation to initial values
rospy.logdebug("Reset SIM...")
self.gazebo.resetSim()
# 2nd: We Set the gravity to 0.0 so that we dont fall when reseting joints
# It also UNPAUSES the simulation
rospy.logdebug("Remove Gravity...")
self.gazebo.change_gravity(0.0, 0.0, 0.0)
# EXTRA: Reset JoinStateControlers because sim reset doesnt reset TFs, generating time problems
rospy.logdebug("reset_monoped_joint_controllers...")
self.controllers_object.reset_monoped_joint_controllers()
# 3rd: resets the robot to initial conditions
rospy.logdebug("set_init_pose...")
self.monoped_joint_pubisher_object.set_init_pose()
# 5th: Check all subscribers work.
# Get the state of the Robot defined by its RPY orientation, distance from
# desired point, contact force and JointState of the three joints
rospy.logdebug("check_all_systems_ready...")
self.monoped_state_object.check_all_systems_ready()
rospy.logdebug("get_observations...")
observation = self.monoped_state_object.get_observations()
# 6th: We restore the gravity to original
rospy.logdebug("Restore Gravity...")
self.gazebo.change_gravity(0.0, 0.0, -9.81)
# 7th: pauses simulation
rospy.logdebug("Pause SIM...")
self.gazebo.pauseSim()
# Get the State Discrete Stringuified version of the observations
state = self.get_state(observation)
return state
def _step(self, action):
# Given the action selected by the learning algorithm,
# we perform the corresponding movement of the robot
# 1st, decide which action corresponsd to which joint is incremented
next_action_position = self.monoped_state_object.get_action_to_position(action)
# We move it to that pos
self.gazebo.unpauseSim()
self.monoped_joint_pubisher_object.move_joints(next_action_position)
# Then we send the command to the robot and let it go
# for running_step seconds
time.sleep(self.running_step)
self.gazebo.pauseSim()
# We now process the latest data saved in the class state to calculate
# the state and the rewards. This way we guarantee that they work
# with the same exact data.
# Generate State based on observations
observation = self.monoped_state_object.get_observations()
# finally we get an evaluation based on what happened in the sim
reward,done = self.monoped_state_object.process_data()
# Get the State Discrete Stringuified version of the observations
state = self.get_state(observation)
return state, reward, done, {}
def get_state(self, observation):
"""
We retrieve the Stringuified-Discrete version of the given observation
:return: state
"""
return self.monoped_state_object.get_state_as_string(observation)
5.Generate state object
The state object is then generated with the current state with the following monoped_state.py script
#!/usr/bin/env python
'''
By Miguel Angel Rodriguez <duckfrost@theconstructsim.com>
Visit our website at ec2-54-246-60-98.eu-west-1.compute.amazonaws.com
'''
import gym
import rospy
import numpy as np
import time
from gym import utils, spaces
from geometry_msgs.msg import Pose
from gym.utils import seeding
from gym.envs.registration import register
from gazebo_connection import GazeboConnection
from joint_publisher import JointPub
from monoped_state import MonopedState
from controllers_connection import ControllersConnection
#register the training environment in the gym as an available one
reg = register(
id='Monoped-v0',
entry_point='monoped_env:MonopedEnv',
timestep_limit=50,
)
class MonopedEnv(gym.Env):
def __init__(self):
# We assume that a ROS node has already been created
# before initialising the environment
# gets training parameters from param server
self.desired_pose = Pose()
self.desired_pose.position.x = rospy.get_param("/desired_pose/x")
self.desired_pose.position.y = rospy.get_param("/desired_pose/y")
self.desired_pose.position.z = rospy.get_param("/desired_pose/z")
self.running_step = rospy.get_param("/running_step")
self.max_incl = rospy.get_param("/max_incl")
self.max_height = rospy.get_param("/max_height")
self.min_height = rospy.get_param("/min_height")
self.joint_increment_value = rospy.get_param("/joint_increment_value")
self.done_reward = rospy.get_param("/done_reward")
self.alive_reward = rospy.get_param("/alive_reward")
self.desired_force = rospy.get_param("/desired_force")
self.desired_yaw = rospy.get_param("/desired_yaw")
self.weight_r1 = rospy.get_param("/weight_r1")
self.weight_r2 = rospy.get_param("/weight_r2")
self.weight_r3 = rospy.get_param("/weight_r3")
self.weight_r4 = rospy.get_param("/weight_r4")
self.weight_r5 = rospy.get_param("/weight_r5")
# stablishes connection with simulator
self.gazebo = GazeboConnection()
self.controllers_object = ControllersConnection(namespace="monoped")
self.monoped_state_object = MonopedState( max_height=self.max_height,
min_height=self.min_height,
abs_max_roll=self.max_incl,
abs_max_pitch=self.max_incl,
joint_increment_value=self.joint_increment_value,
done_reward=self.done_reward,
alive_reward=self.alive_reward,
desired_force=self.desired_force,
desired_yaw=self.desired_yaw,
weight_r1=self.weight_r1,
weight_r2=self.weight_r2,
weight_r3=self.weight_r3,
weight_r4=self.weight_r4,
weight_r5=self.weight_r5
)
self.monoped_state_object.set_desired_world_point(self.desired_pose.position.x,
self.desired_pose.position.y,
self.desired_pose.position.z)
self.monoped_joint_pubisher_object = JointPub()
"""
For this version, we consider 6 actions
1-2) Increment/Decrement haa_joint
3-4) Increment/Decrement hfe_joint
5-6) Increment/Decrement kfe_joint
"""
self.action_space = spaces.Discrete(6)
self.reward_range = (-np.inf, np.inf)
self._seed()
# A function to initialize the random generator
def _seed(self, seed=None):
self.np_random, seed = seeding.np_random(seed)
return [seed]
# Resets the state of the environment and returns an initial observation.
def _reset(self):
# 0st: We pause the Simulator
rospy.logdebug("Pausing SIM...")
self.gazebo.pauseSim()
# 1st: resets the simulation to initial values
rospy.logdebug("Reset SIM...")
self.gazebo.resetSim()
# 2nd: We Set the gravity to 0.0 so that we dont fall when reseting joints
# It also UNPAUSES the simulation
rospy.logdebug("Remove Gravity...")
self.gazebo.change_gravity(0.0, 0.0, 0.0)
# EXTRA: Reset JoinStateControlers because sim reset doesnt reset TFs, generating time problems
rospy.logdebug("reset_monoped_joint_controllers...")
self.controllers_object.reset_monoped_joint_controllers()
# 3rd: resets the robot to initial conditions
rospy.logdebug("set_init_pose...")
self.monoped_joint_pubisher_object.set_init_pose()
# 5th: Check all subscribers work.
# Get the state of the Robot defined by its RPY orientation, distance from
# desired point, contact force and JointState of the three joints
rospy.logdebug("check_all_systems_ready...")
self.monoped_state_object.check_all_systems_ready()
rospy.logdebug("get_observations...")
observation = self.monoped_state_object.get_observations()
# 6th: We restore the gravity to original
rospy.logdebug("Restore Gravity...")
self.gazebo.change_gravity(0.0, 0.0, -9.81)
# 7th: pauses simulation
rospy.logdebug("Pause SIM...")
self.gazebo.pauseSim()
# Get the State Discrete Stringuified version of the observations
state = self.get_state(observation)
return state
def _step(self, action):
# Given the action selected by the learning algorithm,
# we perform the corresponding movement of the robot
# 1st, decide which action corresponsd to which joint is incremented
next_action_position = self.monoped_state_object.get_action_to_position(action)
# We move it to that pos
self.gazebo.unpauseSim()
self.monoped_joint_pubisher_object.move_joints(next_action_position)
# Then we send the command to the robot and let it go
# for running_step seconds
time.sleep(self.running_step)
self.gazebo.pauseSim()
# We now process the latest data saved in the class state to calculate
# the state and the rewards. This way we guarantee that they work
# with the same exact data.
# Generate State based on observations
observation = self.monoped_state_object.get_observations()
# finally we get an evaluation based on what happened in the sim
reward,done = self.monoped_state_object.process_data()
# Get the State Discrete Stringuified version of the observations
state = self.get_state(observation)
return state, reward, done, {}
def get_state(self, observation):
"""
We retrieve the Stringuified-Discrete version of the given observation
:return: state
"""
return self.monoped_state_object.get_state_as_string(observation)
6.Reset
Setup the environment back to its initial observation. This includes the steps like pause the simulation, reset the controller and etc. as we described before.
7.Step
In this part, an action will be decided based on the current state and then be published with the following joint_publisher.py script. After that, the reward is calculated based on the new observation.
#!/usr/bin/env python
import rospy
import math
from std_msgs.msg import String
from std_msgs.msg import Float64
class JointPub(object):
def __init__(self):
self.publishers_array = []
self._haa_joint_pub = rospy.Publisher('/monoped/haa_joint_position_controller/command', Float64, queue_size=1)
self._hfe_joint_pub = rospy.Publisher('/monoped/hfe_joint_position_controller/command', Float64, queue_size=1)
self._kfe_joint_pub = rospy.Publisher('/monoped/kfe_joint_position_controller/command', Float64, queue_size=1)
self.publishers_array.append(self._haa_joint_pub)
self.publishers_array.append(self._hfe_joint_pub)
self.publishers_array.append(self._kfe_joint_pub)
self.init_pos = [0.0,0.0,0.0]
def set_init_pose(self):
"""
Sets joints to initial position [0,0,0]
:return:
"""
self.check_publishers_connection()
self.move_joints(self.init_pos)
def check_publishers_connection(self):
"""
Checks that all the publishers are working
:return:
"""
rate = rospy.Rate(10) # 10hz
while (self._haa_joint_pub.get_num_connections() == 0):
rospy.logdebug("No susbribers to _haa_joint_pub yet so we wait and try again")
try:
rate.sleep()
except rospy.ROSInterruptException:
# This is to avoid error when world is rested, time when backwards.
pass
rospy.logdebug("_haa_joint_pub Publisher Connected")
while (self._hfe_joint_pub.get_num_connections() == 0):
rospy.logdebug("No susbribers to _hfe_joint_pub yet so we wait and try again")
try:
rate.sleep()
except rospy.ROSInterruptException:
# This is to avoid error when world is rested, time when backwards.
pass
rospy.logdebug("_hfe_joint_pub Publisher Connected")
while (self._kfe_joint_pub.get_num_connections() == 0):
rospy.logdebug("No susbribers to _kfe_joint_pub yet so we wait and try again")
try:
rate.sleep()
except rospy.ROSInterruptException:
# This is to avoid error when world is rested, time when backwards.
pass
rospy.logdebug("_kfe_joint_pub Publisher Connected")
rospy.logdebug("All Publishers READY")
def joint_mono_des_callback(self, msg):
rospy.logdebug(str(msg.joint_state.position))
self.move_joints(msg.joint_state.position)
def move_joints(self, joints_array):
i = 0
for publisher_object in self.publishers_array:
joint_value = Float64()
joint_value.data = joints_array[i]
rospy.logdebug("JointsPos>>"+str(joint_value))
publisher_object.publish(joint_value)
i += 1
def start_loop(self, rate_value = 2.0):
rospy.logdebug("Start Loop")
pos1 = [0.0,0.0,1.6]
pos2 = [0.0,0.0,-1.6]
position = "pos1"
rate = rospy.Rate(rate_value)
while not rospy.is_shutdown():
if position == "pos1":
self.move_joints(pos1)
position = "pos2"
else:
self.move_joints(pos2)
position = "pos1"
rate.sleep()
def start_sinus_loop(self, rate_value = 2.0):
rospy.logdebug("Start Loop")
w = 0.0
x = 2.0*math.sin(w)
#pos_x = [0.0,0.0,x]
#pos_x = [x, 0.0, 0.0]
pos_x = [0.0, x, 0.0]
rate = rospy.Rate(rate_value)
while not rospy.is_shutdown():
self.move_joints(pos_x)
w += 0.05
x = 2.0 * math.sin(w)
#pos_x = [0.0, 0.0, x]
#pos_x = [x, 0.0, 0.0]
pos_x = [0.0, x, 0.0]
rate.sleep()
if __name__=="__main__":
rospy.init_node('joint_publisher_node')
joint_publisher = JointPub()
rate_value = 50.0
#joint_publisher.start_loop(rate_value)
joint_publisher.start_sinus_loop(rate_value)
Step 2. Training
Now you have all the code you need to start training. Let’s run the simulation first. Go to Simulations -> Select launch file -> my_legged_robot_sims->main.launch to launch the hopper robot simulation in Gazebo.
Then you can run the training with
roslaunch my_hopper_training main.launch
The algorithm is working and start to train the robot to perform the task we want, however, it still requires lots of tuning to work properly.
You can have multiple instances and train robots with different parameters in parallel with our paid program in ROSDS. Please check it here if you are interested.
In this video we are going to set up in our robotic manipulator a new URDF element: Transmissions. After that, we are able to integrate ROS and gazebo simulation using ROS controllers. We’ll use a simplified model of the robot to see how it works. At the end of the video we’ll be able to send joint position commands to the robot publishing to some ROS topics.
Find a complete course about ROS controllers and URDF
Robot Ignite Academy:
https://goo.gl/93gmxz
In this series, we are going to show you how to build a hopper robot in ROS and make it learn to hop using reinforcement learning algorithm. The hopper robot simulation has been built in the last post. In case you didn’t follow it, you can find the post here.
Part 1
Use OpenAI to make a Hopper robot learn in Gazebo simulator, using ROS Development Studio. We will use Qlearning and Gym for that.
Step 1. Create a training package
Let’s create a package for training
cd ~/simulation_ws/src/loco_motion
catkin_create_pkg my_hopper_training rospy
Then we create a launch file called main.launch inside the my_hopper_training/launch directory with the following content
<!--
Date of creation: 5/II/2018
Application created by: Miguel Angel Rodriguez <duckfrost@theconstructsim.com>
The Construct https://www.theconstruct.ai
License LGPLV3 << Basically means you can do whatever you want with this!
-->
<launch>
<!-- Load the parameters for the algorithm -->
<rosparam command="load" file="$(find my_hopper_training)/config/qlearn_params.yaml" />
<!-- Launch the training system -->
<node pkg="my_hopper_training" name="monoped_gym" type="start_training_v2.py" output="screen"/>
</launch>
To implement reinforcement learning, we’ll use an algorithm called q-learn. We’ll save the parameters for the q-learn algorithm as qlearn_params.yaml under the my_hopper_training/config directory with the following content
# Algortihm Parameters
alpha: 0.1
gamma: 0.8
epsilon: 0.9
epsilon_discount: 0.999 # 1098 eps to reach 0.1
nepisodes: 100000
nsteps: 1000
# Environment Parameters
desired_pose:
x: 0.0
y: 0.0
z: 1.0
desired_force: 7.08 # In Newtons, normal contact force when stanting still with 9.81 gravity
desired_yaw: 0.0 # Desired yaw in radians for the hopper to stay
max_height: 3.0 # in meters
min_height: 0.5 # in meters
max_incl: 1.57 # in rads
running_step: 0.001 # in seconds
joint_increment_value: 0.05 # in radians
done_reward: -1000.0 # reward
alive_reward: 100.0 # reward
weight_r1: 1.0 # Weight for joint positions ( joints in the zero is perfect )
weight_r2: 0.0 # Weight for joint efforts ( no efforts is perfect )
weight_r3: 1.0 # Weight for contact force similar to desired ( weight of monoped )
weight_r4: 1.0 # Weight for orientation ( vertical is perfect )
weight_r5: 1.0 # Weight for distance from desired point ( on the point is perfect )
In this post, we’ll focus on explaining the training script. Let’s create it under the my_hopper_training_src directory and call it start_training_v2.py with the following content
#!/usr/bin/env python
'''
Original Training code made by Ricardo Tellez <rtellez@theconstructsim.com>
Moded by Miguel Angel Rodriguez <duckfrost@theconstructsim.com>
Visit our website at ec2-54-246-60-98.eu-west-1.compute.amazonaws.com
'''
import gym
import time
import numpy
import random
import qlearn
from gym import wrappers
from std_msgs.msg import Float64
# ROS packages required
import rospy
import rospkg
# import our training environment
import monoped_env
if __name__ == '__main__':
rospy.init_node('monoped_gym', anonymous=True, log_level=rospy.INFO)
# Create the Gym environment
env = gym.make('Monoped-v0')
rospy.logdebug ( "Gym environment done")
reward_pub = rospy.Publisher('/monoped/reward', Float64, queue_size=1)
episode_reward_pub = rospy.Publisher('/monoped/episode_reward', Float64, queue_size=1)
# Set the logging system
rospack = rospkg.RosPack()
pkg_path = rospack.get_path('my_hopper_training')
outdir = pkg_path + '/training_results'
env = wrappers.Monitor(env, outdir, force=True)
rospy.logdebug("Monitor Wrapper started")
last_time_steps = numpy.ndarray(0)
# Loads parameters from the ROS param server
# Parameters are stored in a yaml file inside the config directory
# They are loaded at runtime by the launch file
Alpha = rospy.get_param("/alpha")
Epsilon = rospy.get_param("/epsilon")
Gamma = rospy.get_param("/gamma")
epsilon_discount = rospy.get_param("/epsilon_discount")
nepisodes = rospy.get_param("/nepisodes")
nsteps = rospy.get_param("/nsteps")
# Initialises the algorithm that we are going to use for learning
qlearn = qlearn.QLearn(actions=range(env.action_space.n),
alpha=Alpha, gamma=Gamma, epsilon=Epsilon)
initial_epsilon = qlearn.epsilon
start_time = time.time()
highest_reward = 0
# Starts the main training loop: the one about the episodes to do
for x in range(nepisodes):
rospy.loginfo ("STARTING Episode #"+str(x))
cumulated_reward = 0
cumulated_reward_msg = Float64()
episode_reward_msg = Float64()
done = False
if qlearn.epsilon > 0.05:
qlearn.epsilon *= epsilon_discount
# Initialize the environment and get first state of the robot
rospy.logdebug("env.reset...")
# Now We return directly the stringuified observations called state
state = env.reset()
rospy.logdebug("env.get_state...==>"+str(state))
# for each episode, we test the robot for nsteps
for i in range(nsteps):
# Pick an action based on the current state
action = qlearn.chooseAction(state)
# Execute the action in the environment and get feedback
rospy.logdebug("###################### Start Step...["+str(i)+"]")
rospy.logdebug("haa+,haa-,hfe+,hfe-,kfe+,kfe- >> [0,1,2,3,4,5]")
rospy.logdebug("Action to Perform >> "+str(action))
nextState, reward, done, info = env.step(action)
rospy.logdebug("END Step...")
rospy.logdebug("Reward ==> " + str(reward))
cumulated_reward += reward
if highest_reward < cumulated_reward:
highest_reward = cumulated_reward
rospy.logdebug("env.get_state...[distance_from_desired_point,base_roll,base_pitch,base_yaw,contact_force,joint_states_haa,joint_states_hfe,joint_states_kfe]==>" + str(nextState))
# Make the algorithm learn based on the results
qlearn.learn(state, action, reward, nextState)
# We publish the cumulated reward
cumulated_reward_msg.data = cumulated_reward
reward_pub.publish(cumulated_reward_msg)
if not(done):
state = nextState
else:
rospy.logdebug ("DONE")
last_time_steps = numpy.append(last_time_steps, [int(i + 1)])
break
rospy.logdebug("###################### END Step...["+str(i)+"]")
m, s = divmod(int(time.time() - start_time), 60)
h, m = divmod(m, 60)
episode_reward_msg.data = cumulated_reward
episode_reward_pub.publish(episode_reward_msg)
rospy.loginfo( ("EP: "+str(x+1)+" - [alpha: "+str(round(qlearn.alpha,2))+" - gamma: "+str(round(qlearn.gamma,2))+" - epsilon: "+str(round(qlearn.epsilon,2))+"] - Reward: "+str(cumulated_reward)+" Time: %d:%02d:%02d" % (h, m, s)))
rospy.loginfo ( ("\n|"+str(nepisodes)+"|"+str(qlearn.alpha)+"|"+str(qlearn.gamma)+"|"+str(initial_epsilon)+"*"+str(epsilon_discount)+"|"+str(highest_reward)+"| PICTURE |"))
l = last_time_steps.tolist()
l.sort()
rospy.loginfo("Overall score: {:0.2f}".format(last_time_steps.mean()))
rospy.loginfo("Best 100 score: {:0.2f}".format(reduce(lambda x, y: x + y, l[-100:]) / len(l[-100:])))
env.close()
We won’t go into detail to explain the q-learn algorithm. You can find a tutorial here if you are interested. You can simply copy and paste the following code into a file called qlearn.py and put it under the my_hopper_training/src directory
'''
Q-learning approach for different RL problems
as part of the basic series on reinforcement learning @
Robotics needs software engineers and software developers. A lot of them. This article is for software developers who have never considered programming for robots. In this article, we discuss the following:
Reasons why you should become a robotics developer
What that entails for you
How to start
What is a robotics developer?
Robotics developers are people who use their programming skills to program robots.
Robotics developers do not need to develop new path-planning algorithms. They don’t need to develop a new control paradigm. They don’t need to invent a new object recognition system. That is the job of the roboticists and AI at the labs. Robotics developers need to know which algorithms exist, how to use them, when, and how to integrate them into the complete robotics application. Hence, the global robot always does what it is expected to do.
That is the role of the robotics developer.
However, programming robots differ from programming an app or a web page. The program’s hardware changes continuously (because the robot moves in the world) and often experiences unexpected problems. That makes the programs more complex than apps.
Robotic programs interact with hardware constantly to take action and perceive the result of that action. That goes beyond mere user interaction because now, the program considers the user interaction plus the robot’s interaction with the world all that in a continuously changing environment.
Why become a robotics developer?
The first question you may have is why you should consider becoming a robotics developer. I have the following answers:
Robot programming has a big future. The number of robots used in the world is increasing exponentially.
Forecast of the robotics market in the world for the next years (source Statista)
I’m talking about industrial robots, and robots targeting commercial and consumer spaces. I’m talking about robots that interact with humans and help them with tasks. I’m talking about service robots.
Source: Robotics Tomorrow
Even if the industrial and military robot sectors are growing, service robotics is growing even faster.
More big companies are entering the robotics game. Companies like Dyson and Tesla, are creating robots. Additionally, Google’s spin-off Intrinsic bought the Robot Operating System (ROS) team of developers to release Flowstate, their own software product for programming robots. That is a trend.
Not enough engineers…yet. Due to the increasing demand for robots, many companies now have a robotics division. I know that because every week, more companies join our academy (Robot Ignite Academy). They want to train their engineers to program robots. These companies do not have robotics divisions, but are considering creating one and cannot find properly trained people that know how to program robots.
What is starting to happen in robotics already happened with data engineers and deep learning just a few years ago. Today, every company wants a machine learning engineer, hence, you can find machine learning engineers everywhere. The machine learning space is now too crowded.
However, that is not the case for robotics programming. There are very few robotics developers in the world. Now is the best time to jump onto the robotics wagon and be one of the first in this field.
The demand for robotics engineers is very high. You only have to look at the robotics-worldwide mailing list to see how many job offers relating to robotics programmers are posted daily. The offers include various jobs: you can work for companies, startups, and research institutes, or even do an internship, PhD, or Post-Doc.
Check the weekly list of robotics jobs on our jobs page divided by continents.
Salaries. Salaries for robot programming are quite good. You can check the average salaries for different countries on this website. See below an example for the United States:
Salaries for robotics software developer jobs (as of July 2023)
You may be able to work remotely. If you work in software for robots, chances are that you can program the robots using simulations and DevOps tools without having to be on site. See more information below.
It is super cool! Let’s face it. A software developer for robots is a lot cooler than a software developer for accounting or for pizza apps that deliver to you at the beach. Society considers robots to be one of the coolest things in technology.
You can make a difference in robotics. What is especially interesting about service robots is that the key to having a useful robot is not the hardware, but the software. The hardware that allows a robot to clean your house already exists. The most significant limiting factor for service robots is software. So, your job can contribute to finally bringing useful robots to real life.
You can make a difference in society. All Hollywood movies depict robots as bad things that will hate and enslave us. If you are a robotics developer, you can change this narrative by creating robots that are useful, good, and willing to help us. The robots of the future depend on you.
Screenshot from I Robot movie, showing a case of how robots can be useful
The point is that robots will take over many jobs currently performed by humans. It is inevitable. And those robots must be programmed by someone. Would you like it to be you?
But what about ChatGPT taking those jobs!?!?
There is a lot of hype about ChatGPT and the like taking programmer’s jobs. Let me tell you clearly: this is not going to happen soon. Maybe 20 years from now, AIs will be able to program anything, but for now, they are far from it.
AIs can improve developers’ throughput. They can make developers more efficient. They can help a single developer create a lot more than before.
But AIs cannot create the programs required for robots to work and deliver tasks. That subject is highly complex, and no AI can do it (only a few humans in the world can do it, that is us, the robotics developers!). Not now, or in the next 20 years.
Yes, you will need to incorporate AI to do better work. Yes, you will have to continuously adapt over the next 20 years and increase your productivity. That is true. But your job will not be in jeopardy for the next few years.
So now is the time to jump into software development for robotics.
Why are there not enough developers for robotics?
Reason 1: lack of a learning path
One of the main reasons is the lack of a comprehensive curriculum that leads to the necessary skills to be a robot programmer.
Until very recently, there was no need for robotics developers. Robots could not do anything useful, so robots were relegated to the labs. However, as robots become more skilled, they are leaving the lab and going to homes, facilities, etc.
It is there that robotics developers become necessary, to build robust products.
The problem is that there is no clear path to becoming a robotics developer, that is, a person who uses software skills to program robots.
Usually, in the past, robot programmers were roboticists that knew a lot about mechanics and electronics and just enough about computer programming. However, just enough is not valid anymore. If we want to build robotics products, we need pro programmers.
However, software programmers cannot go directly to work as robotics developers.
A robotics developer must be good at programming, know robotics, and know the basics to understand how robots work, the main structure of its components, get data from the sensors, send commands to the actuators, and little more. It must be able to trust that the hardware will work as expected (in the same way that the computer works as expected). To master that, robotics developers may also need to study robotics theory.
Let me say it again:
Robotics developers do not need to develop a new path planning algorithm. They don’t need to develop a new control paradigm. They don’t need to invent a new object recognition system. That is the job of the roboticists and AI at the labs. Robotics developers need to know which algorithms exist, how to use them, when to use them and how to integrate them into the complete robotics application. Hence, the global robot always does what it is expected to do.
The second main reason is that, in general, software developers do not like to deal with hardware. You are likely a developer and have never thought about entering into the robotics realm. You probably think that by programming for robots, you need to know about electronics and maybe even mechanics. You probably think that hardware and software are so coupled in robots that you cannot touch one without touching the other. That interaction with the hardware is something that many software developers don’t like. After all, they decided to become developers of software, not hardware!!
Fortunately, that hardware interaction is not required at present.
Roboticists programming robots
Due to that lack of software developers, robot programming is done by roboticists, which are the people that build the robots. Perhaps some programmers are not directly involved in creating the robot, but they have no problem getting into the hardware and trying to fix it when something goes wrong.But let’s face it. Most roboticists are not as good programmers as developers are. That is why robotics could benefit from having many expert programmers enter the field.The good news is that attracting developers to the field is easier than ever. Thanks to the Robot Operating System (ROS), you can completely abstract the hardware from the software, so you can program a robot by knowing the robot’s ROS API and testing it on a simulation. Using the ROS API, you can forget about the hardware and concentrate on the software that makes the robot do what you want.
What is the robot’s ROS API?
The ROS API lists of ROS topics, services, action servers, and messages a given robot provides to access its hardware, sensors, and actuators. If you are unfamiliar with ROS, you may not understand these terms. But in the developer’s language, topics/services/messages are like the software functions you can call on a robot to get the data from the sensors and make the robot act. It also includes the parameters you can pass to those functions.
Most modern robot builders provide off-the-shelf ROS APIs. For example, the ROS-Components shop, provides all its hardware with a ROS API.
If the robot you want to work with does not run ROS, you can make it work with ROS by ROSifying it. ROSifying means adapting your robot to work with ROS. To ROSify a robot usually requires knowledge to access the hardware. You need to learn to communicate with the electronics that provide the sensor data or access the robot’s motors. In this post, I’m not dealing with that subject because it’s out of the scope of developers. We assume you will be working with a robot already ROSified.
What is ROS anyway?
ROS stands for Robot Operating System. Even if it says so, ROS is not a real operating system since it goes on top of Linux Ubuntu. ROS is a framework on top of the O.S. that allows you to abstract the hardware from the software. And that is good news for you because this implies that you can create programs for robots without dealing with the hardware. Yay!
ROS is becoming the standard in robotics programming, at least in the service robots sector. Initially, ROS started at the university level but quickly spread into the corporate world. Every day, more companies and startups are basing their business on ROS.
Before ROS, every robot was programmed with the manufacturer’s own API. If you changed robots, you had to start the software again, apart from learning the new programming environment. Furthermore, you had to know a lot about interacting with the robot’s electronics to understand how your program was doing. The situation was similar to the computers of the 1980s when every computer had its own operating system, and you had to create the same program for every type of computer.
ROS is for robots like Windows is for PCs or Android for phones.With a ROSified robot, you can create programs to be shared among different robots. You can build a navigation program, a program that makes the robot move around autonomously without colliding, for a four-wheel robot built by company A and then use the same code to move a two-wheel robot built by company B…or even use it on a drone from company C.
ROS for industrial robots
ROS is being used in many of the current service robots. Conversely, industrial robotics companies are still not entirely convinced about using it, mainly because they will not have a proprietary system. However, several years ago, an international group called ROS-Industrial was created. They aim to make industrial manufacturers of robots understand that ROS is for them since they can use all the software off-the-shelf that other people have created for other ROS robots.
ROS for agricultural robots
In the same line as ROS-Industrial, ROS-Agriculture is another international group that aims to introduce ROS for agriculture. I highly recommend that you follow this group if you are interested in robots for agriculture because they are a very motivated team that can do crazy things with several tons of machines by using ROS. Check out, for instance, this video about an autonomous tractor running ROS, built by Kyler Laird of the ROS-Agriculture group.
How to develop for robots with ROS
Now, if you are convinced about becoming a robotics developer, here are the steps that you can take to become a robotics developer:
Setting Up
Learning
Coding
Testing
Setting Up
First, set up your system to install ROS installed on your development system. Let’s see how to set up your machine for programming robots with ROS. You have two options:
Install everything on your computer. See below.
Use our online ROS Development Studio which already provides everything set up and only requires a web browser.
Which operating system should you use with ROS?
ROS works on Linux Ubuntu or Linux Debian. ROS also can work on Windows and Mac, but the development in those systems is not as mature. I recommend installing one of Ubuntu’s latest versions on your computer. As of, May 2022, I recommend installing Ubuntu 20.04. You can skip this step if you use the ROS Development Studio.
But I’m using Windows/Mac, not Linux. Can I still develop in ROS?
If you still want to use Windows or Mac for ROS development, you have two options:
Full support exists for OSX, Gentoo, and Windows, but I don’t recommend using them yet. I’ve never met anyone who installed ROS on Windows or Mac without trouble. If you still want to go that way, check this page for more information about ROS on those systems.
Use the ROS Development Studio, where you can fully develop for ROS using a web browser. This is our online platform that provides a complete ROS environment working with your web browser and requires no installation. I especially recommend this if you are teaching and need your students to be quickly on track.
Which ROS version should I install?
Currently, there are two different versions of ROS: ROS1, also called Classic, the original. And ROS2, ROS2 is more modern and oriented toward building robotics products. If you are new to ROS, I advise you to go straight to ROS2, but depending on your goals, you may prefer to go for ROS1. I created a video to help you decide which one to go for.
Install ROS
Once you have an Ubuntu system working, install ROS. Both versions of ROS are published in distributions. A distribution is a new release of ROS, usually once every two years. Every distribution has a specific name.
First, you need to be comfortable using the Linux shell. Also, you need to know how to program in C++ or Python. If you still don’t know any of these, I recommend you start with the following free online courses:
As I already mentioned, ROS can be programmed in C++ or Python. However, if you don’t know C++, do not try to get into ROS with C++. If that is your situation, please learn ROS with Python. Of course, you can start learning C++ now because C++ is the language used in the industry, and you will need to transition from ROS Python to ROS C++ later, but your initial ROS learning should be done by programming in Python.
You will think you can handle it and take both (learn ROS and C++) simultaneously, but…bad decision. And good luck.
Finally, bindings exist for other languages, like Prolog, Lisp, Nodejs, or R. Find a complete list of currently supported languages here. I do not recommend that you learn ROS with any of those languages. ROS is complicated enough without you complicating it more with experimental languages.
Of all the methods, I recommend our online Robot Ignite Academy because it is the fastest and most comprehensive learning route. This is not something that I say, but what our customers say. Our online academy has a cost, but it will considerably speed up your learning of ROS.
On the other side, Open Robotics, which builds and maintains ROS, provides a long list of tutorials here for ROS1 and another for ROS2. The list is so extensive that it can be overwhelming. If you decide to use this method to learn ROS, then I recommend that you proceed in the following order for the tutorial for optimal learning:
Navigating the ROS file system
Creating a ROS package
Understanding topics
Writing publishers and subscribers
Examining publishers and subscribers
Writing service client
Examining service client
Creating Msg and Srv
Understanding service params
Defining custom messages
Recording and playing back data
ROS TF
ROS URDF
ROS Control
ROS Navigation
Here you can find a complete summary of the main topics you need to learn about ROS basics with clickable links.
As a final recommendation, apart from being a ROS teacher at the Master of Industry 4.0 of UPC University, I am delivering a live online class about ROS every Tuesday at 18:00 CEST/CET. I recommend you attend and practice a new ROS subject with me every Tuesday at the ROS Developers Open Class. It is free and an excellent opportunity to master a single subject of ROS with a simulated robot.
Coding
You will need a C++ or Python code editor. Follow this page to get an overview of the main IDEs you can use to write ROS code. However, I recommend only two of the available options:
If you want to install the editor on your computer, use the VSCode editor with its ROS extension. It is by far the best editor for ROS coding.
If you want to skip this step, use the ROS Development Studio instead since it comes with an integrated IDE in your browser, requires no installation, and allows the easy sharing of projects.
Testing
How can you test the programs you are developing for a robot? I am assuming here that you do not have access to a robot; actually, you do not want to have access!! Remember: developers!!Well, you have several options to test your ROS programs without using a robot.
You use mocks
If you are a developer, you already know what mocks are. You can create your own mocks that emulate the connection to the different parts of the ROS API of your robot.
On this page, you will find information about using mocks in ROS.
Working with mocks in ROS is not an easy option since it requires a lot of preparation work. Also, its usefulness is limited since it can only produce what you have put on it previously. Using mocks is an option that I don’t recommend for developing. Use it only if you cannot use any of the other options below or if you are creating unit tests. In my experience, I have never used them. I have always used one of the next two options.
You use ROS bags
ROS provides a way to record in a log file the full ROS API of a robot while running in a real-life situation and then run it back later on another computer. This log file is called a ROS bag. When you run a ROS bag on another computer, the computer will show the same ROS API to your recorded programs.You can learn how to use ROS bags here: record and replay of ROS bags.
ROS bags are a limited system because you can only use them to create an algorithm that does something from sensor data. This means that you cannot generate commands for the robot because it is only a reproduction of data, so you can get the same data as the robot got when recorded, but you cannot decide new actions for the robot.
You use simulations
So if you want to go pro without using a real robot, you should use robot simulations. Simulations are the next step in software development.
Simulations are like having the actual robot by your side but without having to care for the electronics, hardware, and physical space. Roboticists consider simulations the ugly brother of robotics. Roboticists usually hate to use simulations because it is not the real robot. Roboticists like contact with the real thing. But fortunately, we are here talking with the opposite kind of people, those who want to keep their hands off the hardware. For those, simulations are essential.
Let me tell you one thing: robot simulations are the key to intelligent robots, even if roboticists do not
admit it. More on that in future posts, but remember you read it here first!
In the case of simulations, you have a simulation of the robot running in your computer that can run and act like the real robot. Once the simulation runs, your computer will present the same ROS API programs you would have had if you were on the real robot’s comupter. That is the same as in the case of the ROS bags, with the advantage that you can now actually send commands to the robot and the simulated robot will reply accordingly. That is awesome!
To use robot simulations, ROS provides the Gazebo simulator already installed. You only need the robot simulation you want to program for running. Usually, the companies’ creators of the robot provide a simulation of their robot that you can download and run on the Gazebo simulator.
Installing simulations and making them run could be a little more of hassle. If you want to avoid that work, I recommend using our ROS Development Studio which contains the simulations ready to be launched with a single click and has ROS, IDE, and other valuable tools. Everything executed on a web browser requires no installation.
An example of the ROS Development Studio working with a simulation, ROS navigation stack, Rviz, and a terminal recording a ROS bag:
Additional robotics concepts you need to master
The final step to becoming a robotics developer is to master some robotics subjects.I’m sure that as a software engineer, you already understand many of the topics above because they closely relate to the typical job of a software developer. However, some concepts are closely tied to robotics that a software engineer needs to master (in the same sense that if you need to create invoices, you need to understand some accounting concepts). Robots are living things, I mean, things that move around. That entails a series of new concepts you need to learn because program will not stay in a fixed place anymore).What follows is a list of concepts that you need to understand to create robot programs. I have included a link to online courses that teach these subjects:
In this post, I have shown the future of becoming a robotics developer and how you can become one. Let me know in the comments what doubts you may have and which problems you encounter when following this path.
Related links about ROS that may clarify things for you
Here are some additional links that can provide more information about the future of ROS and how to become a ROS developer.
Robotics Developer Masterclass: A program that equips you with the skills to become a job-ready robotics developer. Enroll in the Robotics Developer Masterclass Batch 5 – September 2024 before August 1st and get €100 off per month!
ROS Developers Days: an online conference where you can learn by practicing with robotics developers from universities and companies worldwide.
In this video we are going to insert inertia property to each link of our robotic manipulator. At the end of the video we’ll be able to simulate this robot using Gazebo, due to having complete physic properties defined to the robot.
Step1. Create Project
Let’s start with creating a new project in ROS development studio.
Notice: If you haven’t had an account yet. You can register one here for free.
Open a shell and clone the package from bitbucket repo(https://goo.gl/1Kbt5v). You’ll have to register a bitbucket account to do that.
Since the repo is a simulation, we’ll clone it into the ~/simulation_ws/src folder.
Then the source tree should look similar like this.
Then we type the following command to compile the package.
cd ~/simulation_ws
catkin_make
source devel/setup.bash
Step2. Add physical properties in the URDF
Open the links_joints.xacro file under the urdf folder of the package and add the following tag in the link tag under the m_link_cylinder tag.
By adding this part, the link can have a more realistic physical behavior in the simulation, you also have to add the parameters for the inertia(ixx ixy ixz iyy iyz izz) in the pramas part.
We also need a collision tag to define the collision behavior when the robot collides with something in the simulation. We define the collision shape same as the visual shape here.
Now we have to define all the values for the parameters in the mrm.xacro file under the urdf folder. You can copy and replace the following code into it.
Locomotion is one of the most challenging topics in robotics. Creating the hard-coded algorithms and the kinematics models is not an easy task. Therefore, its no surprise that AI has been used to try and make robots learn how to move by themselves. In this project you will learn step by step, how to create a Monopod robot simulation working in Gazebo and then set up everything to use OpenAI-Gym infrastructure. OpenAI-Gym allows you to separate learning algorithms from the physical/simulated robot, so that you can test different learning algorithms easily. It also allows you to compare your results with other people in the same conditions.
Part 1
In this first video of a new ROS Development Studio video series, you are going to learn step by step
how to create your own hopper simulation, and may be a real version if there is high support to this videos.
You will learn in this video how to create your ROS packages, modify a URDF given by Alexander W. Winkler, https://github.com/leggedrobotics/xpp to give it control and physics and add all the needed sensors like IMU, odometry and contact sensor.
Here are the steps to create the hopper robot as shown in the video:
Step 1
Head to Robot Development Studio and create a new project.
Provide a suitable project name and some useful description.
Open the project (this will take few seconds)
Once the project is loaded run the IDE from the tools menu. Also verify that the initial directory structure should look like following:
Note that we use simulation_ws to contain all the files related to simulations. Those files not related to simulations will go to catkin_ws (like python scripts, launch files etc)
Step 2
Now we create two catkin packages with names my_legged_robots_description and my_legged_robots_sims. We will add rospy as dependency for both of them.
Start a SHELL from tools menu and navigate to ~simulation_ws/src directory as follows
$ cd simulation_ws/src
Now we create the first catkin package with following command
Once the cloning is complete, we should have a new directory with name xpp inside the ~simulation_ws/src directory
This new directory contains the mesh models for various robots such as biped, quadruped etc. We only need the mesh for monoped so we will do following
copy the meshes folder from ~simulation_ws/src/xpp/robots/xpp_hyq/ to ~simulation_ws/src/my_legged_robots_description/ directory
copy the urdf folder from ~simulation_ws/src/xpp/robots/xpp_hyq/ to ~simulation_ws/src/my_legged_robots_sims/ directory
delete the files in ~simulation_ws/src/my_legged_robots_sims/urdf/ directory except the file monoped.urdf
Step 4
Lets analyze the urdf file for the monoped.
Open the monoped.urdf file in the IDE. The file contains 4links and 3joints. Moreover the links have only visual properties which means we can’t yet simulate it. However we can load it in rviz for display.
For simulation ability we need to define inertia and collision properties into the monoped.urdf file.
We will add these properties to the monoped.urdf file. Now before we make any changes, its a good idea that we create a copy of monoped.urdf with name monoped_controlled.urdf
Here is the monoped_controlled.urdf file content after editing
In addition to the inertia and collision tags we have added a few tags.
First is the gazebo tag, this tag is required to simulate the block in gazebo, it contains information about the material hardness (whether the material is easily deformable or not) and friction values (static and dynamic).
The Second tag that was added is a link tag with name lowerleg_contactsensor_link. This link will help us detect the contact with ground in later part of this project.
Another added tag is a joint tag by name lowerleg_contactsensor_link_joint, this combined with previous link tag completes the contact sensor positioning on the robot.
Note : In the above code the values used for various inertia is calculated using inertial_calculator tool ( it is a part of ROS). Moreover, to simplify calculation of inertia of different blocks with different shapes, a bounding box approximation is applied i.e. we compute the inertia with respect to the dimensions of the bounding box. Though such value might not be perfect, it is reasonably good to work with.
Step 5
Now we will simulate this robot.
To do so first we need to create a world file. Create a directory named worlds inside ~simulation_ws/src/my_legged_robots_sims/ directory.
Using the IDE create a file low_gravity.world inside worlds directory. This is our world file. Write the following content to it:
In this file the tag gravity helps us to manipulate the gravity inside the gazebo world.
Next, to launch the world we need to create a launch file main.launch. Before creating this file we create a launch directory inside ~simulation_ws/src/my_legged_robots_sims/ directory.
Using the IDE we add the following code to main.launch file
Notice in line 9 we have provided the name of our world file low_gravity.world.
This launch file will only launch an empty world in gazebo with zero gravity. To launch it click on the Simulation menu and select Select launch file option and choose the main.launch item.
To spawn the monoped we need to create another launch file inside the ~simulation_ws/src/my_legged_robots_sims/launch/ directory.
Use IDE to create a new file named spawn_monoped.launch and add following content:
You should see the monoped load in the gazebo world.
We can change the gravity settings in low_gravity.world file and relaunch the robot to see the effect of gravity. Since we have not actuated the robot the robot will fall under the influence of gravity, which is totally fine. This finishes the steps to create the hopper robot using Robot Development Studio as shown in the video.
Checkout the URDF robot creation course in RobotIgniteAcademy: https://goo.gl/NJHwq3
[irp posts=”8194″ name=”All about Gazebo 9 with ROS”]
Part 2
In this second video of a new ROS Development Studio video series, you are going to continue to learn step by step how to create your own hopper simulation, and may be a real version if there is high support to this videos.
![[ROS Projects] OpenAI with Hopper Robot in Gazebo Step-by-Step #Part2](https://www.theconstruct.ai/wp-content/uploads/2018/06/OpenAI-with-Hopper-Robot-in-Gazebo-Step-by-Step-Part2.png)
![[ROS Projects] – My Robotic Manipulator – #Part 4 – ROS + URDF/Transmission + Gazebo Controllers](https://www.theconstruct.ai/wp-content/uploads/2018/06/My-Robotic-Manipulator-ROS-URDF-Transmission-Gazebo-Controllers.png)
![[ROS Projects] OpenAI with Hopper Robot in Gazebo Step-by-Step](https://www.theconstruct.ai/wp-content/uploads/2018/06/OpenAI-with-Hopper-Robot-in-Gazebo-Step-by-Step.png)
Salaries for robotics software developer jobs (as of July 2023)
![[ROS Projects] – My Robotic Manipulator – #Part 3 – URDF Inertia](https://www.theconstruct.ai/wp-content/uploads/2018/05/My-Robotic-Manipulator-Part-3-URDF-Inertia.png)
![[ROS Projects] Create a Hopper Robot in Gazebo Step-by-Step](https://www.theconstruct.ai/wp-content/uploads/2018/05/Create-a-Hopper-Robot-in-Gazebo-Step-by-Step-Part1.png)
