According to the Boston, self-driving cars are in the spotlight currently. Companies including Toyota Motor Corp and self-driving startups aim to capture a share of revenue coming from autonomous vehicles. The revenue is expected to reach $42 billion by 2025.

As technology advances, it will bring more competition to an industry. A few key players have long dominated, said Bill Russo, director of advanced engineering at Vecna Robotics Inc., which makes robots programmed for any business task. “The more you add things onto these cars,” he said on Bloomberg Television’s ‘Surveillance’ with Courtney Donohoe on Friday.”It brings down the cost and expands all the possible possibilities.”

Self-Driving Cars Are Making Their Way Onto Public Roads

Bit by bit, self-driving cars are making their way onto public roads as technology companies, traditional automakers, and startups race to develop vehicles that drive themselves. In the U.S. alone, at least 27 states have laws governing autonomous cars on public roads, USA Today reported in May.

Google has been testing self-driving cars for ages. And its fleet has logged more than 1.8 million autonomous miles. Tesla Motors Inc., BkW Group, Daimler AG, Ford Motor Co., General Motors Co., Nissan Motor Co., Honda Motor Co., and Toyota are each working on their versions of autonomous cars.

Self-driving cars’ benefits are enormous.

According to a study, a shift to autonomous vehicles that would cut traffic deaths by 90% and reduce congestion could lead to $507 billion of economic savings in the U.S. alone.

“Autonomous driving is all about anticipating risk,” said Ross Anderson, chief executive officer of Strategic News Service LLC. It provides financial-market analysis and commentary for investment professionals.

He says, “The Nissan Robocar has sensors that can read posted speed limits in one country, then automatically adjust them when it crosses national borders.”

“And Google’s self-driving car learns from its mistakes as well as from other drivers’ piloting habits.”

This week, these high-tech cars are on display at the Angeles Crest Highway just outside of Palmdale, California.

This is the highest stretch of highway in Southern California, reaching altitudes up to 5,500 feet above sea level. The self-driving car Nissan unveiled, Pivo 2, navigates alongside a human-driven car on this newly constructed road.

Nissan says, “It’s only a matter of time before self-driving cars are commercially available.” And they may revolutionize the way we live and work by making driving more convenient while allowing people to spend more valuable time with family and friends or working – instead of stuck in traffic jams.

Don’t surprise if you see computer-generated images (CGI) about these new self-driving vehicles that will probably appear when the makers test this technology for software updates.

For now, we need to resolve a few issues before self-driving cars are ready for public use.

But as technology advances, it brings more competition to an industry that has long been dominated by a few big players like Baidu Inc., Alphabet Inc.’s Google, and Tesla Motors Inc. In 2015, General Motor’s Cruise Automation unit bought Strobe, a maker of 3-D lidar laser sensors, in addition to its acquisition of driverless startup Sidecar Technologies earlier that year.”

The more you add onto these vehicles,” said Bill Russo, director of advanced engineering at Boston-based Vecna Robotics. “It brings down the cost and expands the possibilities.”

Why Should We Buy Self-Driving Cars?

According to experts, they will help us avoid accidents, get jobs done and save time.

1. Self-driving cars avoiding accidents:

The human fault is accountable for more than 90% of vehicle crashes.

Self-driving cars will eliminate human errors by eliminating the human input that causes most car accidents and fatalities. New technology now lets vehicles talk to each other so they can avoid collisions, experts say.

2. Self-driving cars can be an option for the disabled and sick people:

People with disabilities, seniors, and other physically challenged individuals will have personal vehicles to transport them where they want to go. Some tech companies are already working on developing self-driving wheelchairs, according to this article in The Street in 2015.

3. Self-driving cars can help you get a job done:

Cars equipped with artificial intelligence (A.I.) will be able to follow your voice commands. So, you won’t need any special training to tell them what to do. You will speak naturally as you would to another person or a search engine like Google’s Siri. It’ll recognize when you’re playing music from Spotify in the car. And, you’ll be able to tell it to set up a playlist or to turn on your favorite radio station.

4. Self-driving cars can save time:

The most significant benefit of self-driving cars will be that we won’t have to drive them ourselves. So they should free up a ton of time — which we can spend at home with family or friends, in bed sleeping, going for a jog, or practicing our golf swing. Commuting hours each day could shrink from 50 to 10, freeing us for more productive work time.

5. Self-driving cars help save money:

Self-driving vehicles are safer than human-driven ones and cheaper to operate. It is because there is no need for maintenance staff or drivers to pay.

What are the Barriers?

Barriers include economic, technological, and legal issues ranging from liability for accidents to security concerns. For example, manufacturer defects have contributed significantly to injuries in both people and cars. Therefore manufacturers must provide a product that can reduce the risk of errors or mistakes on their part.

Taking into account that some car makers think testing self-driving cars in an urban environment may be premature; away from city centers where traffic lights and lane markings are often more evident than in dense urban areas, we need more stringent rules before they go mainstream. With all the data collected by Google’s self-driving vehicles telling us it’s time to get it right as humans will always try to find ways around laws.

As the article on Wall Street Journal shows by quoting data from Transportation Research Institute at the University of Michigan, 94% of all serious car crashes are caused by human error. The probability that self-driving cars can eliminate accidents is very low until the technology achieves perfection.

Companies such as Google, Tesla Inc, Daimler AG, and Uber Technologies work on self-driving cars.

In the words of David Strickland, former head of the National Highway Traffic Safety Administration under President Barack Obama :

“There is a tendency to think that [self-driving cars] are just around the corner … It turns out that we’re still in the science project phase of this.”

Self-driving Cars Companies:

Google has been testing its driverless technology for years. In 2016 it logged 2 million miles with its fleet of vehicles, but how many accidents have happened during those tests is unclear. Tesla CEO Elon Musk predicts full autonomy by 2020 for his electric car company.

Musk says Tesla’s lack of a steering wheel and the gas pedal will help to ease fears about self-driving cars. Uber is testing autonomous vehicles in Arizona.

Other companies include Volvo, Audi AG, Daimler AG, Ford Motor Co., General Motors Co . and startups like TuSimple Inc . and Zoox Inc . have invested heavily in fully or partially autonomous vehicle technology.

How Would Self-Driving Cars Work?

Like smartphones but without a nervous system, autonomous vehicles rely on sensors and software programs to interpret their surroundings and decide when to accelerate or a brake. They can also adjust themselves if another driver cuts them off or a police car pulls alongside.

What are the risks?

People are just beginning to consider the issues that will come with self-driving cars. These may include ethical questions about when to sacrifice one person to save another and legally liable for an accident – manufacturers, software developers, or consumers. There’s also uncertainty over whether drivers would be ready to give up control of their vehicles.

“The biggest hurdle is trust,” said Clayton Christenson, founder of Boston-based venture capital firm Andreessen Horowitz. “If you had a choice between (a driverless) car and a human driver at the controls, which would you choose?” In addition, many people live by driving and don’t want anything interfering with it:

“How do you make sure the car doesn’t get hacked?” said Jan Dawson, an independent technology analyst at Jackdaw Research. “It’s a hard question.” And there are other factors to weigh:

Driverless cars aren’t perfect, but they’re safer than humans behind the wheel. In 2016, over 40,000 traffic fatalities in the U.S., and 94% of those accidents were caused by human error. Autonomous vehicles could significantly reduce these numbers and potentially save millions of lives around the world each year.

This is our automated future … we need to bring it into reality!

Future of Self-Driving Cars

What if your car could drive you to work? Or pick up groceries on its own? Or drop you off at a restaurant and wait for you to order food, then return home while you’re at dinner with friends, all without requiring your attention or even your presence in the vehicle. The technology exists now. But would most people want it?

The next significant innovation is here — autonomous driving (A.D.). Unlike conventional A.D., vehicles that take advantage of these technologies have no proactive real-time human input. This requires new ways of thinking about safety and liability.

As a relative newcomer in this field, I recently got inspired by those excellent implementations (e.g., Autonomous). It shows excellent results from applying machine learning algorithms and applying real-world data sets.

But, how can we get such datasets? Freely available roadways are rare in modern times as many countries have invested heavily in safety on their roads to eliminate human errors and thus avoid accidents. Yes, there is always something you can do wrong, but the error margin is currently pretty low when humans share the road with self-driving cars.

Take a look at the KITTI dataset, which is open source: http://www.cvlibs.net/datasets/kitti/. I was surprised that only around 500 kilometers are publicly available for research purposes… Why so few? Various reasons could make it challenging to release vast amounts of driving data – security reasons (disclosure of company secrets), legal reasons (liability issues), and the costs to get such data.

These are the main reasons I’d like to collect my datasets that can be used for later research on self-driving cars.

Still, I know it’s not an easy task. “It takes 10 million miles to train a self-driving car” – that’s what people say about this new technology.

 

But how much driving data is needed actually? What kind of data sets can be collected?

Some companies have already started ordering their streaming datasets: Uber and Tesla, for example. Usually, these repositories contain vast images that represent different situations while driving, such as pedestrians, traffic signs, etc. Still, these sources’ data are often not recorded with GPS positions, making it hard to determine how long a specific situation has occurred. From this point, I would like to create a new dataset with the following additional information:

– GPS coordinates of the car and all objects on the road (traffic signs, pedestrians, etc.)

– Traffic camera images for each frame from existing cameras in vehicles

– Position of all cars around – that is why we need GPS coordinates and trajectories as well. This will help us understand where other cars were when some picture was taken (it could be helpful if there is an accident later).

“How can I collect such data? Data sources are rare because most traffic companies do not want to share their precious data for free. But to acquire such datasets, you have to be creative and resourceful.” – one of my students said after I shared the story with her. Let’s think about how it could be done…

First

We need a car equipped with all the necessary sensors and cameras – which is not a big deal these days. Then we can use part of the existing traffic camera infrastructure (probably on our way home) and collect images with time-stamps for every frame (this will help us understand how long some situations took place).

Second

Our car should know where other cars are located on the road. For this purpose, there is a much simpler solution for this purpose than just scanning traditional radar — why don’t we install additional GPS inside the car and use it to know where other cars will be? For example, this would work well in cities when our self-driving car stops at red lights.

Third

We can collect GPS traces that show us the trajectory of traffic signs and pedestrians. This information (as I understand) is already available somewhere but not in an accessible format. So that should also be considered.

We need some powerful future computer which should process all this data and learn from them automatically. Many companies are developing artificial neural networks (ANNs). But these products are costly – you have to order hundreds of thousands of dollars just for one server/workstation with such components in the machine. And they typically do not offer any free support or free software licenses, unlike Google – the only big company that offers free hosted Tensorflow and other related services. This is another reason to collect your datasets and train ANNs yourself 🙂

What Kind Of Tasks Can We Solve With Our Self-Driving Car?

I have been studying neural networks and trying to implement them in my research lab. There are different techniques how to use such models. Some people prefer to start working from easy piecewise linear regression, then getting more complex (and less accurate) by adding nonlinear features into a model one by one or simply starting from building a multilayer perceptron net. But there is no golden rule for a beginner or any person with no theoretical background. Why not try to start from something more complex?

It seems that our self-driving car should recognize pedestrians, traffic signs, and other cars around. This is a challenging task for it because traditional computer vision approaches usually work only with images that have fixed sizes of pixels (for example, 320×240) and leave nothing else to do but cropping this image in different ways (from left, from right, etc.) to make some classification. But what if we want our car to understand whether most of the pixels are blue or yellow in a traffic sign? How can it distinguish black color from black shadow? What will it do when it gets an image captured by many cameras, and none of them is pointing exactly at the same object?

This brings us to discussing different neural network architectures.

The scheme of self-driving algorithms consists of computer vision, robotics, and pattern recognition algorithms :

* Computer vision allows us to analyze images from a camera to find patterns, estimate the size of objects in them, detect traffic signs, etc.

* In robotics, we try to figure out what is happening around our self-driving car (how fast does it move? where are other cars positioned?) using data gathered by GPS modules and sensors inside the car itself. This data also includes camera frames captured by cameras the car is equipped with.

* Pattern recognition allows us to evaluate different models that we can learn using computer vision and robotics data (these models show the probability of object presence in a particular camera frame).   We can compare these probabilities with real values obtained from an image, so we can decide whether we have just captured such a traffic sign or not (+ this approach allows us to identify such concepts as “black” or “blue” color, etc.). 

This approach seems quite flexible because it allows you to train your ANNs to segment objects and other challenging tasks like identifying pedestrians based on their gait and recognizing their gender, age, or skin complexion. A chance to integrate different neural networks into one because our project could be inefficient if we have several (and even more) of them. But I believe it’s possible with modern software tools like Caffe and Tensorflow.

But, how do you train such an ANN? How do you get tons of labeled training data in the first place? By collecting thousands of images from online sources, or is there any other way?

…At this point, I ran out of time and searched for a video that would answer my questions but unfortunately couldn’t find something better than a 2:27h talk by Andrej Karpathy at Jeff Hammerbacher’s OpenAI class: “Problems in A.I. — Unconventional Wisdom.”

But still, I would recommend that you watch this one to understand different approaches and use cases for deep learning.

I hope I’ll have time to write about classification, regression, recurrent neural networks, convolutional neural networks, etc., but if not, don’t hesitate to ask your questions in comments or mention them on Reddit.

I can’t say that the future of self-driving cars will be a bright one, but we need to get it right as these cars will become mainstream for us sooner rather than later. The potential benefits are enormous! Imagine not having to stress out during rush hour anymore ;D Maybe I’ll enjoy sleeping more – who knows? Let me know what you think about self-driving cars. Do you think they are a good thing? Will you buy one when it arrives on the market?

Good luck with building your self-driving car!

 

References:

Self-driving car architecture could be different depending on the task it is trying to solve. Source: autonomouscars.com

Intro to ConvNets (with Python code) http://colah.github.io/posts/2015-07-ConvNets-in-200-Words/

Deep Learning for Self Driving Cars: Part 2 – CNN and RNN by Jovon Gombash – Medium.com https://medium.com/@jovongombas/deep-learning-for-self-driving-cars-part-2-cnn-and-rnn–7e4a3487bc3d

Self Driving Cars: How They Work and What You Need to Know by Kavin Senapathy – Forbes.com https://www.forbes.com/sites/kavinsenapathy/2015/03/12/self-driving-cars-how-they-work/#5fa6b3a03961

The Promise and Peril of Self Driving Cars – The New Yorker https://www.newyorker.com/magazine/2016/03/14/the-promise-and-peril-of-self-driving-cars

Self Driving Cars: What’s the Future? https://www.youtube.com/watch?v=2H8LISqFnJ4