In this article, we will continue to develop Arduino Bluetooth gloves. We’ll build a mechanical system for transformation of the finger’s bending. We’ll learn how to combine IMU data, pack and send data wirelessly, use Kinect, and utilize Arduino Bluetooth gloves in real-life application. If you’re new to the Arduino Glove project, we highly recommend you read Arduino Bluetooth Gloves Part 1 – Basics of this project first.
First, to measure flexion of the fingers (finger bending), we must transform the finger bending to a rotation of the potentiometer. We could make it with simple mechanical system as shown below:
Figure 1: Diagram of a mechanical system for measurement of the finger bending
To make it in real life, we have to model it in a solid state modeling editor, such as Autodesk Inventor.
Figure 2: Modeled mechanical system for measurement of the finger bending
Next, we have to 3D print the model. An STL file is a common type of 3D model among all 3D printers. Just export all your models to STL separately and send it to a 3D printer. To see the compiled 3D designs, please click here.
Figure 3: 3D printed parts for finger mechanical system
Take your potentiometer (without black knob/grip), screws, nuts, and the 3D printed parts and then assemble them as shown in Figure 4.
Figure 4: Finger base unit assembly
Screw in the joints as shown in Figure 5. Then, we’ll glue the black grip with screwed finger joint.
Figure 5: Finger joint assembly details
Figure 6: Base finger unit with fastener
Once the joints are screwed in, we’ll glue the assembled fingers to the glove. Use extra strong multi-purpose glue. Surfaces must be clean and dry before gluing. Then apply a thin layer of glue on both surfaces and wait for 10 to 15 minutes. Press the surfaces firmly for a few seconds. You can spread some additional glue around the unit if necessary. It’s recommended that you leave it to dry for about 24 hours.
*DO NOT stick your fingers into the glove while gluing. Use cardboard pieces or other material.
Figure 7: Gluing base unit onto the proximal phalanges (at the bottom of each finger)
Now we’re going to glue the finger joint. Take the joint part and another sticky fastener and stick them just above the finger’s nail (Figure 8 & 9).
Figure 8: Gluing finger joint
Figure 9: Finger joint assembly (glued)
Connect both assembled parts by inserting the black grip into the potentiometer hole.
Please be patient and repeat the same operation 9 more times with each finger.
A wire diagram of the entire module is shown below.
Figure 10: Arduino glove circuit diagram
Now, let’s wire up the fingers!
Prepare red and black wires and also three wires with white, yellow, and black markings. Color coding is strongly recommended to avoid confusion.
Solder the black-marked and yellow-marked wires to the potentiometer as it shown in Figure 11.
Figure 11: Soldering the potentiometer
Tie the wires with a plastic cable tie. This is necessary to protect it from any mechanical stress.
Figure 12: Fix wire with a cable tie
Next, you should solder the black-marked wire with the black wire & yellow and white-marked cables to the resistor. The red wire must be soldered to another side of the resistor. The white-marked wire should then be connected to the Arduino analog input.
Figure 13: Soldering one finger unit
Figure 14: Grouping black- and red-marked wires
After soldering all the units, we’re going to shrink the wires with heat-shrink tubes and group the wires with cable ties.
Figure 15: Fingers connected to the body
Figure 16: Assembled fingers connected to Arduino
Figure 17: The processor body (Arduino)
Figure 18: IMU sensor connection
Figure 19: Assembled Arduino glove
The assembly is done! Let’s move on to programming.
Install a new version of the IMU Library. Please refer back to Part 1 to learn how to install the library.
First, let’s go back and check out our previous measurements.
Figure 21: IMU sensor readings
If you investigate the L3G4200D datasheet and the IMU Library carefully, you can see that MEMS Gyroscope measures angle rate in raw digit units and IMU Library’s functions convert digit units into degree per second (dsp).
For example, the Gyroscope returns 12492 digit value:
If the Gyroscope is configured to Measurement range ±250 dps, then this value multiplies at 0.00875 dps/digit value to get 109.30 dps angle rate. (109.30 dps means that a body with this angle rate makes full rotation about the measured axis for about 4 seconds.)
If the Gyroscope is configured to Measurement range ±2000 dps, then this value multiplies at 0.07 dps/digit value to get 874.44 dps angle rate. (874.44 dps means that a body with this angle rate makes about 2.5 full rotation about the measured axis for a second.)
A sensor’s sensitivity means the smallest value change which could be detected with this sensor. Note that the wider the range the less sensivity you may get from the Gyroscope.
An angular rate is a value measured by gyroscope. Thus, a gyroscope is able to estimate how fast a body is rotated about a certain axis. In our application, it’s necessary to determine in which position our glove is. We could summarize all angular rate samples divided by change of time to estimate current angular position.
Because the measurement of angular rate is not continuous but discrete, we get an error called drift. It’s called “drift” because the sum of these changes of angular position will float in time even though a body is at rest.
Let’s look at the gyroscope drift example below:
#include // Library for I²C
#include // Library for working with IMU modules
Gyroscope gyro;// Create an object to work with Gyroscope
// Time Variables
boolfirstcall=true;// Indicator of a first iteration
// Angle Variables
Serial.begin(9600);// Open the serial port
Serial.println("Begin init ...");// Display a message on the beginning of the initialization
gyro.begin();// Initialize the gyroscope at 100Hz Output Data Rate
gyro.setRange(RANGE_2000);// set the gyroscope at 2000 dps range sensitivity
Serial.println("Init completed");// Display a message about the successful initialization
endtime=micros();// End time for the current Measurement
dt=(float)(endtime-starttime)/1000000;// Get time beetween measurements
gyro_rateX=gyro.readX_DegPerSec();// Current Angle rate around the X-axis
gyro_rateY=gyro.readY_DegPerSec();// Current Angle rate around the Y-axis
gyro_rateZ=gyro.readZ_DegPerSec();// Current Angle rate around the Z-axis
Pitch+=1*gyro_rateX *dt;// Current Angle around the X-axis
Roll+=1*gyro_rateY *dt;// Current Angle around the Y-axis
Yaw+=1*gyro_rateZ *dt;// Current Angle around the Z-axis
starttime=micros();// Start time for the next Measurement
if(ticker>30)// Print Message every 300 ms
Serial.print(Roll);// Output angular velocity around the axis X
Serial.print(Pitch);// Output of the angular velocity around the Y axis
Serial.print(Yaw);// Output of the angular velocity about the Z axis
If you leave your IMU sensor without any motion, you will see drift in action (Figure 22):
Figure 22: Gyroscope drift demonstration
As mentioned above, due to the tendency of drift, the sum will produce an error that increases in time. But we can mitigate the drift effect by calculating and utilizing zero-rate level.
The definition of the Zero-rate level is described well in the Gyroscope datasheet, at 2.6.2 Zero-rate level section. The main reason of this is a mechanical stress applied to IMU, for example, after soldering on printed curcuit board. After it’s mounted, the Zero-rate level varies slightly with temperature so this change may be neglected.
Figure 23: Zero-rate level based Gyroscope measurements
You can look at implementation details in the new version of IMU Library here.
After these enhancements, we could use the gyroscope data, but only for the short term. We still need something which is stable in the long term.
An accelerometer is a device that measures acceleration. Using accelerometers we’re able to determine angle of a sensor’s work plane based on calculated gravity vector. However, with smart gloves, a sensor will measure all other accelerations that include motions of the arm, which could corrupt the measured gravity vector.
A Low pass filter might be used to filter the gravity vector from random accelerations. The accelerometer’s measurements could be used only in the long term.
One of the most common ways to fuse sensor data is to use a powerful Kalman filter (which is difficult to understand and to implement on Arduino).
As a solution for combining IMU data, we could use a complementary filter, which is much easier to understand and implement on Arduino. The essential part of the complementary filter is as follows:
where Roll_i – is a calculated Roll angle from the previous iteration
gyroRoll_i+1 – is a change of angle from the last measurement based on the last gyroscope data
accelRoll_i+1 – is a calculated Roll angle based on the last accelerometer data
Roll_i+1 – is a new calculated Roll angle with complementary filter
This filter utilizes the gyroscope’s dynamics in the short term and gets rid of the error in the long term based on the accelerometer data. You can test it yourself, just open imu_2.ino in Arduino IDE and upload it to your Arduino UNO.
Update Message Sender
As we get measured data, we have to send this data to a PC using a wireless Bluetooth connection.
First, check that Bluetooth Bee is configured correctly. Install the Bluetooth Bee on the Xbee USB Adapter. Change the Bluetooth Bee to the AT Mode and then check UART configuration with the “AT+UART?” command. The Bluetooth Bee has to return “115200,0,0” (baud rate 115200 bits/s, 1 stop bit and none parity bit). If not, send “AT+UART=115200,0,0” command to configure the Bluetooth Bee and check UART configuration again. Don’t forget to switch off AT mode after checking!
One more thing that requires your attention is a “Serial Select” switch which is placed on the Wireless SD shield. This switch defines where to send data that was read from the wireless connection.
Switch it to the MICRO position. When in the Micro position, the Bluetooth module is connected to the Arduino. Note that the Arduino is still connected to the USB-to-serial converter. If you write data to the serial port from the Arduino, data will be sent in both ways: through USB and wirelessly.
Upload messaging.ino to the Arduino. Notice that every time the Arduino is powered on or resetting it will calibrate in 3 steps:
Gyroscope Zero-Rate Level calculation;
Measurement of minimum finger position;
Measurement of maximum finger position.
Before the Gyroscope Zero-Rate Level calculation, leave the IMU without any movement. Before measuring the minimum finger position, clench your fist with Arduino gloves on. Before measuring the maximum finger position, unclench your fist with Arduino gloves on.
Open messaging.ino in Arduino IDE and upload it to your Arduino UNO.
Power off the Arduino and place the Bluetooth Bee on the Wireless shield. Make sure that the Bluetooth Bee is not in AT Mode and that the Wireless Shield Serial select switch is switched to MICRO position.
Power your Arduino Uno with a 9V battery and pair with your Bluetooth Bee using “1234” as the password. Make sure that after pairing you can find your wireless Bluetooth connection as a Virtual COM port in the Device Manager.
Figure 25: Pairing with Bluetooth Bee
Figure 26: Virtual COM port in the Device Manager
Open Messaging.lvproj LabVIEW project and Messaging Test.vi from a Project explorer window. Choose corresponding COM port and run it.
Figure 27: Messages from Arduino Glove shown in a LabVIEW PC application
You will see that the data which Arduino Uno is writing in the Serial port is transferred via Bluetooth and read here from a Virtual COM port.
Adding Kinect Measurements
Kinect is an intuitive Xbox 360 game console device controller which is used as alternative to common joysticks. Actually, it is a powerful motion-capture controller that acquires your poses and allows you to control game processes with your hands or body. Even though it was developed for games, it can be used in real-world applications.
To use Kinect for Xbox 360 in a LabVIEW environment you need to install the following:
Open Kinect.lvproj LabVIEW Project and Kinect Test.vi from Project Explorer window, then Run it. If all requirements were installed correctly, you will see video and Kinect processing results similar to the one below:
Now we need to put everything together in one application.
Open Arduino Gloves.lvprojLabVIEW Project. Then open Host Main.vi from the Project explorer window. Choose the corresponding COM port for the Left and Right Gloves Bluetooth connection. Plug in Kinect and make sure that it’s ready to use.
Then put on Arduino Bluetooth gloves and power them on. Provide the required conditions for a calibration.
After this, run Host Main.vi and you will be able to see the following:
Figure 29: Getting all measurements in one application
Make sure that every finger flexion is measured correctly and that Roll, Pitch, and Yaw correspond to the actual attitude of your hand.
Arduino Bluetooth gloves could be useful for a real-life robot control application. All processed data could be transmitted to any third-party application with UDP messages.
Figure 30: Sending data to third-party application diagram
As an example of a third-party application, a Simulator of an industrial robot arm – Puma560 was picked and added UDP Receiver loop. Open Puma560 Simulator.lvproj, Host Main.vi and Puma560 Simulator.vi from Project Explorer Window.
Choose the corresponding COM port for the Left and Right Gloves Bluetooth connection. Plug in Kinect and make sure that its ready to use.
Then put on Arduino Bluetooth gloves and power them on, and provide the required conditions for a calibration.
After this Run both VIs and you will get the following results:
Figure 31: Exports processed data to Puma560 Simulator
Please note that you have to provide the right interpretation of the data that you get from the Smart Glove application. You shouldn’t use acquired data directly, without any additional processing. For example, the operator’s right hand coordinates are unlikely to be equal to robot’s arm gripper. You have to convert measured data from the operator’s coordinate system to your robot’s coordinate system.