« Do you want the keys to the virtual car?

Kids these days have cast nearly all of their toys aside in favor of a digital device. The Transformers, the Teenage Mutant Ninja Turtles, and G.I. Joe lay abandoned in the bottom of the toy box as their digital counterparts get all the attention. What if there were a way to bridge that gap? A physical toy interacting in a digital space, and vice versa. niolabs connects legacy equipment with new technology every day for enterprise customers. Who is to say this can’t be done with old toys to create new experiences for today’s tech-savvy kids?

So, are toy manufactures ready for a digital transformation?

nio is here to help.


We have been exploring ways to demonstrate nio’s ability to enable truly distributed computing. So what is a simple and fun way to showcase a distributed system with real-time streaming data between the virtual and real worlds? A remote-controlled car of course! By connecting a mobile device, the cloud, and a Raspberry Pi using the nio platform, we created a car that can be controlled by anyone. There is no application or software download required, all you have to do is visit a website on your mobile device and it is transformed into the controller. The icing on the cake? All movements of the RC car are displayed simultaneously in a virtual space, bridging the gap between the real and virtual worlds.

How it works

The car takes streaming gyroscope data from a mobile device and sends it to the cloud for some calculations. The values created by the gyroscope inputs, turning your phone left or right and tilting it forward and backward, are translated into a speed and directional indicators for the car by nio services operating in the cloud. The Raspberry Pi on the car takes this data from the cloud and makes decisions based on what it’s sensing at the edge (Where am I going? Am I about to hit something?) and moves the car. But it doesn’t stop there, the car streams the data it’s receiving to a desktop UI where the true values are displayed alongside a live video stream of the cars on the track.

Once these speed demons were created, it was apparent that driving them can be tricky. Rubbin’ may be racing, but rolling your car over is counterproductive to setting a fast lap time. Instead of spending hours training the next Lewis Hamilton or Danica Patrick, we made the car aware of its surroundings. By reading an RGB color sensor and doing some basic PID controller calculations (#math), the car is able to determine if it’s on the track, and how it should turn. We call it Kid Mode. This drastically improves lap times by keeping the car on the preferred line around the track, while the driver can focus on controlling the speed. If you want to take the training wheels off just deselect Kid Mode and take on the full challenge of navigating the track. The real-time diagnostics (speed, heading, Kid Mode status) from the car are displayed on your mobile device.

Highlighted features

Real-time data streaming
Any smart phone will work
from anywhere in the world
No app download required

Biggest Challenge

How do you find out where a car is on the track? GPS isn’t accurate indoors and OpenCV requires heavy compute, which introduces latency. The answer is magnets! We added a compass to both cars to track their headings. With this data, we can determine which direction each car is headed and display it in the virtual world.

Future features

Lap Times? Competitive statistics? With nio, adding these future features doesn’t require a software architecture redesign. Create a new service and then drop in some blocks for increased interaction, and quick iteration.

Do you want to make this car 12 times bigger? Take the existing nio services and install them to your new hardware for instant scalability.

So, do you want the keys to the virtual car?

Care to take her for a spin? Just follow the steps below:

Step 1. Go to this link on your desktop https://car.demo.niolabs.com/

Step 2. Click the "operate virtual car" button on the landing page

Step 3. Pick up your mobile device and enter https://mobile.car.demo.niolabs.com into the browser

Step 4. Find the code at the bottom of the screen on your desktop

Step 5. Enter the code on your mobile device and...

3... 2... 1... GO!
*Remember this is just a demo so if it doesn't work the first time, try refreshing or making sure your phone is fully rotated and neither of your browsers are in private (incognito) mode.