Deep Reinforcement Learning for Self-Driving Cars — An intro
One of the most exciting areas in Artificial Intelligence is called Reinforcement Learning. Why? Because it’s what’s closest to human learning. In self-driving cars, a LOT of the Machine Learning involved is based on either supervised or unsupervised learning.
In Reinforcement Learning, we’re taking a totally different approach and learning to drive from experience. This is something completely crazy, but that has been implemented in existing companies, such as Wayve!
In this article, I’ll tell you more about how Deep Reinforcement Learning is used in self-driving cars!
Lately, I have noticed a lot of development platforms for reinforcement learning in self-driving cars.
Voyage Deep Drive
Voyage Deep Drive is a simulation platform released last month where you can build reinforcement learning algorithms in a realistic simulation.
It looks similar to CARLA.
A simulator is a synthetic environment created to imitate the world. Like a video game. We simulate sensors and can directly work with them, like in a real car.
AWS DEEP RACER
AWS Deep Racer is a self-driving robot where you can learn and develop reinforcement learning algorithms.
There is a simulation platform but it is primarily a physical robot equipped with cameras.
The purpose of the robot is racing cars and competitions are made by Amazon.
Deep Traffic
Deep Traffic is a course project launched by MIT where you can try and beat traffic using Deep Reinforcement Learning algorithms and a simple simulator.
If you have never done reinforcement learning before, you can simply watch the course and immediately try the project. My average speed is 70.74 km/h, try to beat me here !
If I were to show you a very practical use of Deep Reinforcement Learning in self-driving cars, I wouldn’t show you simulations, or the MIT challenge, or even a small robot… but a real self-driving car driving with Reinforcement Learning!
Finally, Wayve video demo is the one I like the most.
You can clearly identify the first and last iterations and visualize how the car learns to steer the wheel to stay in its lane and reach the objective.
All those projects look really fascinating, but what is Reinforcement Learning?
Reinforcement Learning is a learning technique like supervised and unsupervised learning.
Instead of learning from a labeled dataset (or unlabelled), you learn from mistakes generated by a reward system. We can say the agent learns from experience.
In this technique, an agent (the car) interacts with an environment (the world) to reach an optimal state or objective. To interact, the agent performs an action.
- If the action is positive, the reward will be big.
- If the action is negative (going on the sidewalk, hurting other cars, …), the reward is negative.
The goal of the agent is to choose the right action that gets the biggest reward.
We have seen a lot of reinforcement learning applied to chess or the game of Go.
In this case, the agent is able to foresee the future actions and states and anticipate which action to take now that maximizes future reward.
In short, we are able to calculate the total reward based on all rewards.
While this short article doesn’t aim to teach you reinforcement learning, it’s good to have an overview and understand why a lot of research goes there.
When working with reinforcement learning, you can design an environment and use a reinforcement learning algorithm to optimize the driving policy.
With a lot of learning, you can even work with multiple agents that explore multiple paths at the same time and returns you the optimal one.
In the movie Next, Nicolas Cage is a Multi-Agent Reinforcement Learning agent.
How to learn Deep Reinforcement Learning?
If you’re interested in Reinforcement Learning, l have compiled a few resources to learn the basics:
If you are not aware of reinforcement learning yet but know Deep Learning, start with the MIT Course I gave earlier .
📜 Also, this Medium series explains things amazingly .
I would say it’s safe to spend a lot of time in these articles and get out with a good understanding.
If you’d like to understand more use cases, Neptune.ai has a great article on 10 Real-Life Applications of Reinforcement Learning which includes Healthcare, NLP, and many more…
These are really good to get started, then you have to complete projects. You can try Deep Reinforcement Learning Nanodegree by Udacity but I don’t know any review. The good point is you get to do projects and that is essential to every learning process.
Deep Learning in Self-Driving Cars?
The best way to understand how to use Reinforcement Learning in self-driving cars is to learn “how” it belongs to “Deep Learning”.
I have compiled an entire article and MindMap on the use of Deep Learning in Self-Driving Cars!
I now invite you to join the Daily Emails on Deep Learning and Self-Driving Cars, and we’ll learn MUCH MORE together.
Including:
- How Deep Learning is Applied to different steps of Self-Driving Cars
- The most advanced algorithms used in the industry
- How to become a Deep Learning Engineer working in Self-Driving Cars
