This repository contains Thomas, Vance, and Sam's semester project for Yale University's CPSC 473b Intelligent Robotics Lab course taught by Prof. Brian Scassellati. Spring 2014.
In summary, our project explored how to combine vision and actuation on a humanoid robot like Rethink Robotics' Baxter Research Robot to determine object compressibility, a property that cannot be reliably determined using either component alone.
The motivation for our project was originally guided by the question of how we could get a humanoid robot to determine whether an object could fit in a box. This motivation was a useful guiding principle, but we did not implement it in practice over the semester because of time constraints and complications involved with adding a third dimension and second camera. Nevertheless, it is not a massive leap to implement such a system; see the 'Potential Future Work' section below.
Workstation Requirements:
- Ubuntu 12.04 LTS
- ROS (groovy distribution)
- Baxter Research SDK
- Rethink Robotics baxter_simulator repo (not_required, private access only)
Materials:
- Baxter Research Robot from Rethink Robotics
- Webcam (Logitech QuickCam Pro 9000)
- Lamp (for lighting tests)
- Table
- Wooden boundary (for Baxter to push against, secured by vices)
- Reference object (to calibrate camera)
- 3D printed compressor piece
- Cloth colored similarly as the 3D printed piece (for color-based subtraction)
- Test objects (Springs with known Hooke's constants, foam, etc.)
- Tape
Note that Gazebo simulation software is not strictly required to run this project.
-
Follow the instructions in the link to setup a ROS workspace and install the RSDK https://github.com/RethinkRobotics/sdk-docs/wiki/Development-Workstation-Setup-Instructions
-
Follow the instructions in the README.md to install the Gazebo simulator https://github.com/RethinkRobotics/baxter_simulator
-
Navigate to the 'src' directory of your ROS workspace
cd ~/ros_ws/src
-
Clone this repository
git clone https://github.com/thomasweng15/cs473-baxter-project.git
-
Build and install
catkin_make
catkin_make install
We placed the table in front of Baxter and secured the wooden boundary on top of the table using vices. We placed the camera overhead the area where Baxter would be compressing objects against the wooden boundary.
We ran the glove.py
script in this repository to get Baxter to grip the 3D-printed compressor piece. We covered Baxter's arm using cloth similarly colored to the compressor piece and taped it down.
-
Navigate to your ROS workspace
cd ~/ros_ws
-
Make sure Baxter is on, then run the Baxter shell file and enable Baxter
./baxter.sh
rosrun baxter_tools enable_robot.py -e
-
Run the 'glove' script to attach Baxter's compressor piece
rosrun cs473_baxter glove.py -g grip
-
Run the main 'start' script run a trial.
rosrun cs473_baxter start.py
The program will present instructions regarding what objects to put in front of Baxter in order to run a full compression trial
Here is a rundown of the file directory structure and what is stored in the most important sections:
cs473-baxter-project/
cs473_baxter/
config/ YAML configuration file(s).
data/ Timestamped images and data for each trial.
scripts/ Python scripts for running the program.
stl/ STL files for 3D printing the compressor piece for Baxter's gripper.
CMakeLists.txt
package.xml
cs473_gazebo/
launch/ Gazebo launch file
worlds/ Gazebo world model
CMakeLists.txt
package.xml
.gitignore
.gitmodules Links the cs473vision git repo as a submodule to this one.
LICENSE
README.md
The scripts/
directory deserves special scrutiny:
scripts/
cs473vision/ submodule containing Python OpenCV code
__init__.py
glove.py Script to get Baxter to grip compressor
position_control.py Class controlling movement of Baxter's limb.
rostopic_test.py Script to test delay of accessing rostopic data.
start.py Main entry point to run compression trial.
webcam.py Class controlling webcam.
plotting.py Script to turn merge data into a CSV file.
This software is licensed under the MIT License. See LICENSE
for details.
Features:
- Move to being able to calculate 3 dimensional pixel dimensions of objects by adding another camera angle
Refactoring / Enhancements:
- Use Machine Learning to teach Baxter what its arm looks like instead of using basic color subtraction to remove its arm from the image
- Remove the webcam and use Baxter's arm and head cameras to calculate pixel dimensions
- Reduce error rates on spring constant calculations by addressing sources of error
Thank you to Professor Scassellati for selecting us for the course and for advising us through the semester on our project. Thank you to Corina for helping us set up Baxter and for fielding all of our questions about the course. Thank you to Rethink Robotics and the community for answering our questions about programming with Baxter. Thank you to the Social Robotics Lab at Yale for offering space and supplies for working on our project. Finally, a big thank you to Baxter, our big friendly robot, for a great semester!
Thomas, Vance, and Sam