Deep Diesel: Machine & Deep Learning for diesel car detection

In this article series, we show different machine and deep learning approaches on the use case of detecting diesel cars as well as environmental zone badges and type labels on the cars. This article gives an introduction and an overview of the article series.

Automated detection of a environmental zone badge in flowing traffic.

Important links related to this series (released week by week)
>codecentric.ai Homepage
>Deep Diesel – Part 1: Machine & Deep Learning for diesel car detection
>Deep Diesel – Part 2: Machine-Learning-Diesel Car Detection using a HOG detector
>Deep Diesel – Part 3: Deep Learning Diesel Car Detection using the AWS DeepLens
>codecentric.ai youtube channel

The diesel scandal, or a possible ban on diesel driving, is currently affecting the entire country: diesel drivers, city councils, OEMs, NGOs and the general public. As a diesel driver (still), I am concerned by this, as I commute through an environmental zone every day. Since I hardly have any influence on the further development anyway, I try to make the best of it for now. This being said, the task doesn’t really suit me personally, as I drive an approx. three-year-old EURO 5 diesel vehicle and would also like to continue driving it without restrictions. On the other hand, I understand that something needs to change about the overall situation and mobility concepts in cities and for commuters. So I’ll use the topic as a playing field for evaluating machine learning and deep learning algorithms.

I am not welcome anymore…

This series of articles is intended to demonstrate various machine and deep learning methods, using a practical example, to illustrate their advantages and disadvantages. The different parts will deal with different algorithms, libraries, and devices. The focus here is more on conveying practical knowledge and experience using a relevant example than on a methodologically correct, scientific approach.

The following parts are planned:

Part 1 – Deep Diesel: Intro (this part)

Part 2 – Machine Learning Diesel Filter

Part 3 – Deep-Learning Diesel Filter based on AWS Deep Lens

Driving bans based on environmental zone badges

For some time now, the environmental impact of diesel vehicles in urban areas has been the subject of debate. In this context, driving bans are controversially discussed. Even if these have generally been legally legitimized by the Federal Administrative Court of Leipzig, the question arises as to the enforceability of this measure. Even with the blue badge under discussion, the inspection of vehicles entering urban areas is difficult to carry out without massive obstructions, as the vehicles concerned can only be inspected by the police in individual proceedings. Therefore, it is reasonable to assume that the local authorities will probably carry out the controls primarily in stationary traffic, and, what’s more, only in public parking areas. Currently, however, the vehicles in question are already on the roads of urban areas and contributing to air pollution there.

Our scenario

The use case on the basis of which we want to evaluate the different systems is a supporting system for mobile traffic control based on current computer vision algorithms, which supports the identification of unauthorized vehicles. To simplify the procedure, we start with the recognition of the green badge. This is sufficient for the time being, as we assume in our scenario that the blue badge will be introduced sooner or later and that green badges, in particular, will be identified when entering the cities. In addition, we can generalize to the recognition of further badges by means of extended training sets.

When you’ve spent some time on this subject, it’s hard to walk by vehicles without looking at their windscreens. It is striking that a significant proportion of cars does not have emissions stickers (estimated to be in the single-digit percentage range). The detection of missing stickers is excluded in the first step, but would also have to be handled. Theoretically, vehicles could be classified by make, type marking and exhaust pipe (often bent downwards for diesel engines) on the mounting of the rear of the vehicle – but this is a rather high-dimensional problem, which requires an immense amount of training data and seems rather impractical. The reading of license plates and a live comparison with the registration database would certainly be possible – but completely disproportionate and undesirable since it is highly invasive in terms of privacy – and could be used to create motion profiles. Ideally, such a system should function completely offline and not store any data.

The approach is to be implemented in such a way that a free-floating traffic boundary condition can be met – the current inner-city traffic (up to 70km/h) is not to be influenced by the ongoing control, i.e. it can flow into the city without obstructions.

Machine learning and deep learning

If we present machine learning and deep learning as separate methods in the articles, this is only partially true. Deep learning is actually a special area of machine learning. Machine learning uses procedures and algorithms to make decisions. An example would be an algorithm that leads to a rule based on RGB color channels in a decision tree (R<50,G>200,B<50) → Green badge ; (R<50,G<50,B<200) → Blue badge ;… or in our case characteristic structural elements for the regions of interest.

When we talk about deep learning, we refer to neural networks and their training. The decisions of a neural network are sometimes hard to understand. On the topic “What is Deep Learning?” some videos will follow soon on our YouTube channel codecentric.ai.

Also an  example for object detection based on neuronal networks can be found on our youtube channel:

By loading the video, you agree to YouTube’s privacy policy.
Learn more

Load video

Always unblock YouTube

Procedure and expected results

For both topics, similar steps are necessary to implement a functioning Object Detector.

1. Collect training data
2. Labeling of the data
3. Training and parameterization of the algorithm
4. Testing of the classifier
5. Evaluation of the live data

The collection of training data for the PoC happens at our company’s parking lot and primarily with the company cars. We specifically pointed out in advance when and where we took photos of the vehicles.

Data labeling is done manually and provides the algorithms with examples of objects that should be identified in the images. The result is a list of the objects to be identified in an image with the name and coordinates of the so-called bounding box.

Example for a labeled region in our trainings set.

As soon as the training data is available, the classifier is parameterized and trained. A model is derived which reflects the characteristics of the image objects to be recognized. The specific models used in the applied classifiers will be the subject of subsequent articles.

Performance is quantified by testing the classifier with test data that differs from the training data. The test data must also be labeled. How many objects were detected correctly? How many image sections were incorrectly recognized as objects (false positive)? How many objects were not detected (false negative)?

If the model has been trained with sufficient performance on the test data, the classifier can be used. In our case, this detection  should  be achieved in real time or near time. This means that we want to classify a video stream. The objects are detected on a single-frame level. We need to get the data and have it evaluated by our classifier.

Our implementation plan

In order to be able to test a first functional PoC as quickly as possible, we want to start with a machine learning algorithm. We decided to start with a HOG Detector (Histogram of Oriented Gradients ), which primarily refers to the geometric properties of the object. The big advantage of this detector is that very little training data is required. We will discuss the major disadvantages of the HOG Detector expected by us in the article on implementation. We will base our implementation on dlib, which comes with an extensive toolbox of different classifiers. dlib is especially recommended for starters in the Computer Vision environment, and we will give some implementation hints in the next article.

Afterwards, we will give an overview of detection methods with AWS Deep Lens. Deep Lens is particularly interesting for us because it is a simple testbed for our purposes that can even bridge the distance to our own dummy control point via WLAN. It is not yet clear what method is to be used for object recognition. In particular, we want to gain experience with the device we brought back from AWS re:Invent 2017, which is not yet available in Germany.

Basically you are able to use different acquired data for detecting diesel cars in order to increase confidence in your finding. Typical vectors could be visual data, sound data, heat signatures (probably not working so well) or leaking electromagnetic emissions. So we did experiment with quite a wide spectrum of those inputs with:

Experimenting with software defined radio to acquire data to discriminate between diesel and benzin engines

Expected challenges

The main challenge for us will certainly be to collect relevant training data. For the HOG Detector, usually only little data is needed. The training of deep learning algorithms, however, requires a lot of training data. Possibly the implementation of the deep learning approach will also fail. A priori we expect that the consideration of changing environmental conditions, such as reflections, poor lighting conditions, wetness, and reflections, will be rather difficult to achieve within the PoCs.

The area of performance optimization of detection is generally a complex and lengthy process and will not be pursued further by us for the PoC. However, it should be clear that this will always follow implementations of this kind.

Important links related to this series
>codecentric.ai Homepage
>Deep Diesel – Part 1: Machine & Deep Learning for diesel car detection
>Deep Diesel – Part 2: Machine-Learning-Diesel Car Detection using a HOG detector (next week)
>Deep Diesel – Part 3: Deep Learning Diesel Car Detection using the AWS DeepLens (in two weeks)
>codecentric.ai youtube channel