Click here to read Part 1 of this article >
In this article, we will continue to develop Arduino Bluetooth gloves. We’ll build a mechanical system for transformation of the finger’s bending. We’ll learn how to combine IMU data, pack and send data wirelessly, use Kinect, and utilize Arduino Bluetooth gloves in real-life application. If you’re new to the Arduino Glove project, we highly recommend you read Arduino Bluetooth Gloves Part 1 – Basics of this project first.
Hardware
Software
Tools
First, to measure flexion of the fingers (finger bending), we must transform the finger bending to a rotation of the potentiometer. We could make it with simple mechanical system as shown below:
Figure 1: Diagram of a mechanical system for measurement of the finger bending
To make it in real life, we have to model it in a solid state modeling editor, such as Autodesk Inventor.
Figure 2: Modeled mechanical system for measurement of the finger bending
Next, we have to 3D print the model. An STL file is a common type of 3D model among all 3D printers. Just export all your models to STL separately and send it to a 3D printer.
Figure 3: 3D printed parts for finger mechanical system
Take your potentiometer (without black knob/grip), screws, nuts, and the 3D printed parts and then assemble them as shown in Figure 4.
Figure 4: Finger base unit assembly
Screw in the joints as shown in Figure 5. Then, we’ll glue the black grip with screwed finger joint.
Figure 5: Finger joint assembly details
Figure 6: Base finger unit with fastener
Once the joints are screwed in, we’ll glue the assembled fingers to the glove. Use extra strong multi-purpose glue. Surfaces must be clean and dry before gluing. Then apply a thin layer of glue on both surfaces and wait for 10 to 15 minutes. Press the surfaces firmly for a few seconds. You can spread some additional glue around the unit if necessary. It’s recommended that you leave it to dry for about 24 hours.
*DO NOT stick your fingers into the glove while gluing. Use cardboard pieces or other material.
Figure 7: Gluing base unit onto the proximal phalanges (at the bottom of each finger)
Now we’re going to glue the finger joint. Take the joint part and another sticky fastener and stick them just above the finger’s nail (Figure 8 & 9).
Figure 8: Gluing finger joint
Figure 9: Finger joint assembly (glued)
Connect both assembled parts by inserting the black grip into the potentiometer hole.
Please be patient and repeat the same operation 9 more times with each finger.
A wire diagram of the entire module is shown below.
Figure 10: Arduino glove circuit diagram
Now, let’s wire up the fingers!
Prepare red and black wires and also three wires with white, yellow, and black markings. Color coding is strongly recommended to avoid confusion.
Solder the black-marked and yellow-marked wires to the potentiometer as it shown in Figure 11.
Figure 11: Soldering the potentiometer
Tie the wires with a plastic cable tie. This is necessary to protect it from any mechanical stress.
Figure 12: Fix wire with a cable tie
Next, you should solder the black-marked wire with the black wire & yellow and white-marked cables to the resistor. The red wire must be soldered to another side of the resistor. The white-marked wire should then be connected to the Arduino analog input.
Figure 13: Soldering one finger unit
Figure 14: Grouping black- and red-marked wires
After soldering all the units, we’re going to shrink the wires with heat-shrink tubes and group the wires with cable ties.
Figure 15: Fingers connected to the body
Figure 16: Assembled fingers connected to Arduino
Figure 17: The processor body (Arduino)
Figure 18: IMU sensor connection
Figure 19: Assembled Arduino glove
The assembly is done! Let’s move on to programming.
Install a new version of the IMU Library. Please refer back to Part 1 to learn how to install the library.
Open bending.ino and upload it to the Arduino Uno.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 |
#include "glove.h" TFinger Finger1 = TFinger(A0); TFinger Finger2 = TFinger(A1); TFinger Finger3 = TFinger(A2); TFinger Finger4 = TFinger(A3); TFinger Finger5 = TFinger(A4); void setup() { Serial.begin (9600); // Open the serial port Serial.println ("Be ready to set min, in 3..."); delay(1000); Serial.println ("2..."); delay(1000); Serial.println ("1..."); delay(1000); Finger1.setmin(); Finger2.setmin(); Finger3.setmin(); Finger4.setmin(); Finger5.setmin(); Serial.println ("Minimum is set"); Serial.println ("Be ready to set max, in 3..."); delay(1000); Serial.println ("2..."); delay(1000); Serial.println ("1..."); delay(1000); Finger1.setmax(); Finger2.setmax(); Finger3.setmax(); Finger4.setmax(); Finger5.setmax(); Serial.println ("Maximum is set"); Finger1.calibrate(); Finger2.calibrate(); Finger3.calibrate(); Finger4.calibrate(); Finger5.calibrate(); Serial.println ("Calibrated"); } void loop() { Serial.print ( Finger1.read() ); Serial.print ("\t"); Serial.print (Finger2.read()); Serial.print ("\t"); Serial.print (Finger3.read()); Serial.print ("\t"); Serial.print (Finger4.read()); Serial.print ("\t"); Serial.print (Finger5.read()); Serial.println ("\t"); delay(300); // } |
You will then see a result shown in Figure 20:
Figure 20: Finger bendings reading result
Note that you have to follow calibration procedure to be able to measure data accurately.
In Arduino Bluetooth Gloves Part 1 – Basics, we covered the basics of IMU sensor. Now we will learn how to combine gyroscope and accelerometer data.
First, let’s go back and check out our previous measurements.
Figure 21: IMU sensor readings
For example, the Gyroscope returns 12492 digit value:
A sensor’s sensitivity means the smallest value change which could be detected with this sensor. Note that the wider the range the less sensivity you may get from the Gyroscope.
An angular rate is a value measured by gyroscope. Thus, a gyroscope is able to estimate how fast a body is rotated about a certain axis. In our application, it’s necessary to determine in which position our glove is. We could summarize all angular rate samples divided by change of time to estimate current angular position.
Because the measurement of angular rate is not continuous but discrete, we get an error called drift. It’s called “drift” because the sum of these changes of angular position will float in time even though a body is at rest.
Let’s look at the gyroscope drift example below:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 |
#include // Library for I²C #include // Library for working with IMU modules Gyroscope gyro; // Create an object to work with Gyroscope // Time Variables unsigned long starttime; unsigned long endtime; float dt; int ticker = 0; bool firstcall=true; // Indicator of a first iteration // Angle Variables float gyro_rateX, gyro_rateY, gyro_rateZ; float Pitch=0; float Roll=0; float Yaw=0; void setup () { Serial.begin (9600); // Open the serial port Serial.println ("Begin init ..."); // Display a message on the beginning of the initialization gyro.begin (); // Initialize the gyroscope at 100Hz Output Data Rate gyro.setRange(RANGE_2000); // set the gyroscope at 2000 dps range sensitivity Serial.println ("Init completed"); // Display a message about the successful initialization Serial.println ("Gyroscope"); } void loop () { endtime = micros(); // End time for the current Measurement if (firstcall) { firstcall=false; } else { dt = (float)(endtime - starttime)/1000000; // Get time beetween measurements gyro_rateX=gyro.readX_DegPerSec(); // Current Angle rate around the X-axis gyro_rateY=gyro.readY_DegPerSec(); // Current Angle rate around the Y-axis gyro_rateZ=gyro.readZ_DegPerSec(); // Current Angle rate around the Z-axis Pitch += 1 * gyro_rateX * dt; // Current Angle around the X-axis Roll += 1 * gyro_rateY * dt; // Current Angle around the Y-axis Yaw += 1 * gyro_rateZ * dt; // Current Angle around the Z-axis delay (10); } starttime = micros(); // Start time for the next Measurement if (ticker > 30) // Print Message every 300 ms { ticker=0; Serial.print (Roll); // Output angular velocity around the axis X Serial.print ("\t"); Serial.print (Pitch); // Output of the angular velocity around the Y axis Serial.print ("\t"); Serial.print (Yaw); // Output of the angular velocity about the Z axis Serial.print ("\t"); Serial.println (""); } else {ticker++;} } |
If you leave your IMU sensor without any motion, you will see drift in action (Figure 22):
Figure 22: Gyroscope drift demonstration
As mentioned above, due to the tendency of drift, the sum will produce an error that increases in time. But we can mitigate the drift effect by calculating and utilizing zero-rate level.
The definition of the Zero-rate level is described well in the Gyroscope datasheet, at 2.6.2 Zero-rate level section. The main reason of this is a mechanical stress applied to IMU, for example, after soldering on printed curcuit board. After it’s mounted, the Zero-rate level varies slightly with temperature so this change may be neglected.
Figure 23: Zero-rate level based Gyroscope measurements
You can look at implementation details in the new version of IMU Library here.
After these enhancements, we could use the gyroscope data, but only for the short term. We still need something which is stable in the long term.
An accelerometer is a device that measures acceleration. Using accelerometers we’re able to determine angle of a sensor’s work plane based on calculated gravity vector. However, with smart gloves, a sensor will measure all other accelerations that include motions of the arm, which could corrupt the measured gravity vector.
A Low pass filter might be used to filter the gravity vector from random accelerations. The accelerometer’s measurements could be used only in the long term.
One of the most common ways to fuse sensor data is to use a powerful Kalman filter (which is difficult to understand and to implement on Arduino).
As a solution for combining IMU data, we could use a complementary filter, which is much easier to understand and implement on Arduino. The essential part of the complementary filter is as follows:
where Roll_i – is a calculated Roll angle from the previous iteration
gyroRoll_i+1 – is a change of angle from the last measurement based on the last gyroscope data
accelRoll_i+1 – is a calculated Roll angle based on the last accelerometer data
Roll_i+1 – is a new calculated Roll angle with complementary filter
This filter utilizes the gyroscope’s dynamics in the short term and gets rid of the error in the long term based on the accelerometer data. You can test it yourself, just open imu_2.ino in Arduino IDE and upload it to your Arduino UNO.
As we get measured data, we have to send this data to a PC using a wireless Bluetooth connection.
First, check that Bluetooth Bee is configured correctly. Install the Bluetooth Bee on the Xbee USB Adapter. Change the Bluetooth Bee to the AT Mode and then check UART configuration with the “AT+UART?” command. The Bluetooth Bee has to return “115200,0,0” (baud rate 115200 bits/s, 1 stop bit and none parity bit). If not, send “AT+UART=115200,0,0” command to configure the Bluetooth Bee and check UART configuration again. Don’t forget to switch off AT mode after checking!
One more thing that requires your attention is a “Serial Select” switch which is placed on the Wireless SD shield. This switch defines where to send data that was read from the wireless connection.
Figure 24: Serial select switch on a Wireless Shield / ©Arduino LLC
Switch it to the MICRO position. When in the Micro position, the Bluetooth module is connected to the Arduino. Note that the Arduino is still connected to the USB-to-serial converter. If you write data to the serial port from the Arduino, data will be sent in both ways: through USB and wirelessly.
Upload messaging.ino to the Arduino. Notice that every time the Arduino is powered on or resetting it will calibrate in 3 steps:
Gyroscope Zero-Rate Level calculation;
Measurement of minimum finger position;
Measurement of maximum finger position.
Before the Gyroscope Zero-Rate Level calculation, leave the IMU without any movement. Before measuring the minimum finger position, clench your fist with Arduino gloves on. Before measuring the maximum finger position, unclench your fist with Arduino gloves on.
Open messaging.ino in Arduino IDE and upload it to your Arduino UNO.
Power off the Arduino and place the Bluetooth Bee on the Wireless shield. Make sure that the Bluetooth Bee is not in AT Mode and that the Wireless Shield Serial select switch is switched to MICRO position.
Power your Arduino Uno with a 9V battery and pair with your Bluetooth Bee using “1234” as the password. Make sure that after pairing you can find your wireless Bluetooth connection as a Virtual COM port in the Device Manager.
Figure 25: Pairing with Bluetooth Bee
Figure 26: Virtual COM port in the Device Manager
Open Messaging.lvproj LabVIEW project and Messaging Test.vi from a Project explorer window. Choose corresponding COM port and run it.
Figure 27: Messages from Arduino Glove shown in a LabVIEW PC application
You will see that the data which Arduino Uno is writing in the Serial port is transferred via Bluetooth and read here from a Virtual COM port.
Kinect is an intuitive Xbox 360 game console device controller which is used as alternative to common joysticks. Actually, it is a powerful motion-capture controller that acquires your poses and allows you to control game processes with your hands or body. Even though it was developed for games, it can be used in real-world applications.
To use Kinect for Xbox 360 in a LabVIEW environment you need to install the following:
Open Kinect.lvproj LabVIEW Project and Kinect Test.vi from Project Explorer window, then Run it. If all requirements were installed correctly, you will see video and Kinect processing results similar to the one below:
Figure 28: Video and Kinect processing results example / ©Kinesthesia
Now we need to put everything together in one application.
Open Arduino Gloves.lvproj LabVIEW Project. Then open Host Main.vi from the Project explorer window. Choose the corresponding COM port for the Left and Right Gloves Bluetooth connection. Plug in Kinect and make sure that it’s ready to use.
Then put on Arduino Bluetooth gloves and power them on. Provide the required conditions for a calibration.
After this, run Host Main.vi and you will be able to see the following:
Figure 29: Getting all measurements in one application
Make sure that every finger flexion is measured correctly and that Roll, Pitch, and Yaw correspond to the actual attitude of your hand.
Arduino Bluetooth gloves could be useful for a real-life robot control application. All processed data could be transmitted to any third-party application with UDP messages.
Figure 30: Sending data to third-party application diagram
As an example of a third-party application, a Simulator of an industrial robot arm – Puma560 was picked and added UDP Receiver loop. Open Puma560 Simulator.lvproj, Host Main.vi and Puma560 Simulator.vi from Project Explorer Window.
Choose the corresponding COM port for the Left and Right Gloves Bluetooth connection. Plug in Kinect and make sure that its ready to use.
Then put on Arduino Bluetooth gloves and power them on, and provide the required conditions for a calibration.
After this Run both VIs and you will get the following results:
Figure 31: Exports processed data to Puma560 Simulator
Please note that you have to provide the right interpretation of the data that you get from the Smart Glove application. You shouldn’t use acquired data directly, without any additional processing. For example, the operator’s right hand coordinates are unlikely to be equal to robot’s arm gripper. You have to convert measured data from the operator’s coordinate system to your robot’s coordinate system.
DEMO:
Innovation Success Factors from the Creator of Rapiro
Arduino Bluetooth Gloves Part 1 – Basics
Japanese Aerospace Engineer Shares Secret to Mastering Soldering
Futuristic Trends in Flexible Circuit Technologies (Cont.)
Futuristic Trends in Flexible Circuit Technologies
How To MacGyver A Laser Tripwire Using Arduino
