Bounden is a game in which two people rotate a phone for which we make use of the gyroscope to track the phone’s orientation. If you read our previous article, you know that we ran into some gyroscope issues on Android devices and were forced to delay the launch of Bounden on Android. Now, a month later, and thanks to genuine support from fans and a company called Xsens, we are proud to announce that Bounden is available on the Google Play Store now.
In short, we were lucky to solve most of the gyroscope issues. Xsens is specialized in motion tracking and offered to help us by providing us their robust Android sensor fusion library for gyroscopes. Check this before & after video:
The full technical story
The devices we carry around are filled with sensors. Many have an accelerometer, GPS, compass, gyroscope, and even humidity and pressure sensors can be found on some phones. Google has specific requirements to the output of the sensors if a hardware manufacturer wants to create an Android device, so that every developer has a consistent interface (the Android API) to receive sensor data. This sensor data is processed by algorithms created by the hardware manufacturer, and during this process, not only does it process the raw data from each sensor, it also mixes and combines different sensors to improve their results.
This sensor mixing is called ‘sensor fusion’. Sensor fusion for a gyroscope is required because of the way gyroscopes work. A gyroscope detects rotations around any axis. Those rotations are integrated (accumulated and processed) over time, giving us an orientation if it’s combined with gravity data from the accelerometer. Miniscule rotational errors accumulate in this process, causing the orientation to ‘drift’ away from a perfect gravity orientation. If drift is detected, data from the accelerometer can help to reconstruct the gyroscope orientation on two axes; the compass can help on the final axis by indicating where the magnetic north is.
Detecting this drift is very difficult, especially if the phone is constantly rotating and moving. It can take years of research and iterations to build an algorithm that does this well and precisely. The problem we faced with Bounden lies exactly in here. Google has no requirements for the quality of the algorithms to process the sensory data, so hardware manufacturers implement algorithms differently or use completely different algorithms altogether. This means that the precision of the Android API gyroscope data ranges from device to device as the responsibility of the algorithm lies with each individual manufacturer.
The sensors and sensor fusion on most of the devices we used for development worked really great. However, upon testing, the output from the gyroscope data from other Android devices was ranging from less than stellar to completely broken. The vine in our last post demonstrated that problem well.
After our article on Android fragmentation, Xsens gave us a call and invited Bojan to spend a day at their facility to fix our gyroscope accuracy issue. Xsens has been making industrial motion tracking sensors for more than 15 years, and their technology can be found in avionics, motion capture, unmanned submarines, and wearable motion sensors. They have developed a library to process gyroscope data which normally only comes with their own hardware; for us, they were willing to make an exception to help solve the Android fragmentation when it comes to motion sensors.
Their lightweight library substitutes the sensor fusion algorithms from hardware manufacturers by once again taking all sensor data into account and applying their own algorithms. We had the library implemented within a few hours and let it do all the number crunching magic to give us the proper orientation of the device. Because of that, Bounden now works on almost all Android devices that have a gyroscope.
So what do we even do with all this gyroscope data? Well, guiding you to dance, obviously! But there is more to it.
When you begin a dance, Bounden asks you to balance the screen for a certain amount of time. Not only does this make the players focused on the game, it also allows Bounden to check whether sensor data is calibrated. When the calibration is complete, the current orientation is taken as a reference for the duration of the dance, which means that all orientations from that point on are multiplied with the inverse of this reference to cancel out the difference. This technique doesn’t care about the compass axis, which we don’t specifically need for Bounden: both players are very likely to stand opposite of each other when balancing the screen, which gives us an indication on how to orientate the choreography.
Although the calibration screen worked very well, it was also becoming an increasingly frustrating interruption. Especially in practice mode, the interruption would take long enough to kill your enthusiasm of almost finishing a movement on time. We found that during practice mode, the orientation of the gyroscope had barely any drift most of the times. Because of this, we now skip the calibration screen if there isn’t too much drift, and immediately realign the virtual direction of gravity used in Bounden with the actual direction of gravity. However, we only allow for skipping the calibration screen after at least one proper calibration to make sure we’re not making big mistakes with wrong data.
There’s one more thing that happens during the calibration screen. The dances in the game will guide you to move around each other and end up in a completely different orientation. This is entirely desired behaviour and is important to the flow of the choreographies. But when you’re practising, and you start halfway through a dance, you skip the first half of the dance which, for instance, made you and the other player rotate around each other for 180 degrees. This would mean that you start the game in one direction, but the moment the calibration screen disappears and the music starts, both players would quickly have to run 180 degrees around each other to start with the actual movement. In other words; you miss the parts that lead to the right orientation that allows you to start at that point. For that purpose, every move in every dance has a starting orientation that we also multiply with the orientation reference if you start playing the game from that move and on.
After the calibration screen, you go into the game and the camera position is set at the inverted direction of the gyroscope, looking at the center of the world. And that’s how the gyroscope data is used in Bounden.
Whitelisting Android devices with gyroscope
When we published our article on Android fragmentation, we set out to whitelist all devices on which Bounden worked out of the box. It turned out that a big amount of devices were not able to run Bounden without gyroscope problems; stuttering, non-working axes, high latency, etc.
Fortunately, most gyroscope problems have been solved by using the library that Xsens gave us. The Google Play store allows us to only target devices with a gyroscope, accelerometer, and compass. These two solutions made us ready for today, as Bounden is now available on the Google Play Store for $2,99. (25% off as we celebrate it’s launch!) If your device has a gyroscope: get Bounden, see how the gyroscope works, and let us know what you think! (Oh, and if you have an iPhone, that’s over here.)