A model to predict steering torque fully end-to-end

Overview

torque_model

The torque model is a spiritual successor to op-smart-torque, which was a project to train a neural network to control a car's steering fully end to end.

The input is the current wheel angle and future wheel angle (among other things), and the net's output is what torque the human was applying at the time to reach that future state smoothly and confidently. This bypasses the need to manually tune a PID, LQR, or INDI controller, while gaining human-like control over the steering wheel.

Needs to be cloned into an openpilot repo to take advantage of its tools.

The problem

As talked about in great detail and with a simple thought experiment in comma.ai's blog post here about end to end lateral planning, the same concept of behavioral cloning not being able to recover from disturbances applies here.

Behavior cloning and lack of perturbations

The way we generate automatically-labeled training data for a model that predicts how to control a steering wheel is rather simple; any time a human is driving we just take the current (t0s) and future (t0.3s) steering wheel angles and then just have the model predict whatever torque the human was applying at t0s to get us there.

This seems to work great, and the validation loss also seems to be really low! However, when you actually try to drive on this model or put it in a simulator, you can quickly see that any small disturbances (like wind, road camber, etc) quickly lead to a feedback loop or just plain inability to correct back to our desired steering angle.

This is due to the automatically-generated training and validation data containing only samples where the current and future (desired during runtime) steering wheel angles are very close together (just a couple degrees), as a symptom of only using data where the future angle is just fractions of a second away.

To fully realize the problem, think about what would happen if you wanted this model to predict what a human would actuate if the steering wheel is centered, but our desired angle is something like 90 degrees. As the model has never seen a difference of angles higher than just a couple of degrees, it either outputs a very small torque value, or just nonsense, as this input is vastly outside of its training distribution.

The solution

The solution talked about in the blog post above is to use a very simple simulator to warp the input video to be offset left or right, and then tell the model what path the human actually drove. A similar approach can also be taken here, where we generate random samples with an arbitrary steering wheel angle error, and then use a simple model of steering wheel torque, like a PF (proportional-feedforward) controller as the output to predict.

For the example above where we start at 0 degrees and want to reach 90 degrees, we can inject samples into the training data where we have that exact situation and then have the output be what a simple PF controller would output. Then during runtime in the car, when the model corrects for this arbitrary high angle error situation, the current and desired steering wheel angles become much closer together, and the model can then use its knowledge of how humans control under these circumstances.

The future

The current model described and implememted here is non-temporal, meaning the model has no knowledge of the past, where the steering wheel was, and inferring where it's heading. While the input data includes the steering angle rate, there's a lot of information missing it could use to improve its predictions, as well as a model bug where including the angle rate during runtime causes very smoothed and laggy predictions (probably due to the generated synthetic samples not taking any angle rate into account).

Ideally the model has some knowledge of the past, however this means we need an accurate simulator to train the model with perturbations added, so it can correct for disturbances in the real world.

Owner
Shane Smiskol
I mess around with self driving cars, neural networks, and real world data!
Shane Smiskol
ml4h is a toolkit for machine learning on clinical data of all kinds including genetics, labs, imaging, clinical notes, and more

ml4h is a toolkit for machine learning on clinical data of all kinds including genetics, labs, imaging, clinical notes, and more

Broad Institute 65 Dec 20, 2022
Python module for data science and machine learning users.

dsnk-distributions package dsnk distribution is a Python module for data science and machine learning that was created with the goal of reducing calcu

Emmanuel ASIFIWE 1 Nov 23, 2021
Deepchecks is a Python package for comprehensively validating your machine learning models and data with minimal effort

Deepchecks is a Python package for comprehensively validating your machine learning models and data with minimal effort

2.3k Jan 04, 2023
List of Data Science Cheatsheets to rule the world

Data Science Cheatsheets List of Data Science Cheatsheets to rule the world. Table of Contents Business Science Business Science Problem Framework Dat

Favio André Vázquez 11.7k Dec 30, 2022
Scikit-Garden or skgarden is a garden for Scikit-Learn compatible decision trees and forests.

Scikit-Garden or skgarden (pronounced as skarden) is a garden for Scikit-Learn compatible decision trees and forests.

260 Dec 21, 2022
K-Means clusternig example with Python and Scikit-learn

Unsupervised-Machine-Learning Flat Clustering K-Means clusternig example with Python and Scikit-learn Flat clustering Clustering algorithms group a se

Emin 1 Dec 13, 2021
A repository of PyBullet utility functions for robotic motion planning, manipulation planning, and task and motion planning

pybullet-planning (previously ss-pybullet) A repository of PyBullet utility functions for robotic motion planning, manipulation planning, and task and

Caelan Garrett 260 Dec 27, 2022
Exemplary lightweight and ready-to-deploy machine learning project

Exemplary lightweight and ready-to-deploy machine learning project

snapADDY GmbH 6 Dec 20, 2022
Neighbourhood Retrieval (Nearest Neighbours) with Distance Correlation.

Neighbourhood Retrieval with Distance Correlation Assign Pseudo class labels to datapoints in the latent space. NNDC is a slim wrapper around FAISS. N

The Learning Machines 1 Jan 16, 2022
TorchDrug is a PyTorch-based machine learning toolbox designed for drug discovery

A powerful and flexible machine learning platform for drug discovery

MilaGraph 1.1k Jan 08, 2023
Uplift modeling and causal inference with machine learning algorithms

Disclaimer This project is stable and being incubated for long-term support. It may contain new experimental code, for which APIs are subject to chang

Uber Open Source 3.7k Jan 07, 2023
A simple example of ML classification, cross validation, and visualization of feature importances

Simple-Classifier This is a basic example of how to use several different libraries for classification and ensembling, mostly with sklearn. Example as

Rob 2 Aug 25, 2022
Evaluate on three different ML model for feature selection using Breast cancer data.

Anomaly-detection-Feature-Selection Evaluate on three different ML model for feature selection using Breast cancer data. ML models: SVM, KNN and MLP.

Tarek idrees 1 Mar 17, 2022
ML Kaggle Titanic Problem using LogisticRegrission

-ML-Kaggle-Titanic-Problem-using-LogisticRegrission here you will find the solution for the titanic problem on kaggle with comments and step by step c

Mahmoud Nasser Abdulhamed 3 Oct 23, 2022
A Collection of Conference & School Notes in Machine Learning 🦄📝🎉

Machine Learning Conference & Summer School Notes. 🦄📝🎉

558 Dec 28, 2022
This is an implementation of the proximal policy optimization algorithm for the C++ API of Pytorch

This is an implementation of the proximal policy optimization algorithm for the C++ API of Pytorch. It uses a simple TestEnvironment to test the algorithm

Martin Huber 59 Dec 09, 2022
Merlion: A Machine Learning Framework for Time Series Intelligence

Merlion is a Python library for time series intelligence. It provides an end-to-end machine learning framework that includes loading and transforming data, building and training models, post-processi

Salesforce 2.8k Jan 05, 2023
A high-performance topological machine learning toolbox in Python

giotto-tda is a high-performance topological machine learning toolbox in Python built on top of scikit-learn and is distributed under the G

giotto.ai 632 Dec 29, 2022
Machine learning template for projects based on sklearn library.

Machine learning template for projects based on sklearn library.

Janez Lapajne 17 Oct 28, 2022
LinearRegression2 Tvads and CarSales

LinearRegression2_Tvads_and_CarSales This project infers the insight that how the TV ads for cars and car Sales are being linked with each other. It i

Ashish Kumar Yadav 1 Dec 29, 2021