A Collection of LiDAR-Camera-Calibration Papers, Toolboxes and Notes

Overview

Awesome-LiDAR-Camera-Calibration

Awesome

A Collection of LiDAR-Camera-Calibration Papers, Toolboxes and Notes.

Outline

0. Introduction

For applications such as autonomous driving, robotics, navigation systems, and 3-D scene reconstruction, data of the same scene is often captured using both lidar and camera sensors. To accurately interpret the objects in a scene, it is necessary to fuse the lidar and the camera outputs together. Lidar camera calibration estimates a rigid transformation matrix (extrinsics, rotation+translation, 6 DoF) that establishes the correspondences between the points in the 3-D lidar plane and the pixels in the image plane.

Example

1. Target-based methods

Paper Target Feature Optimization Toolbox Note
Extrinsic Calibration of a Camera and Laser Range Finder (improves camera calibration), 2004 checkerboard C:Plane (a), L: pts in plane (m) point-to-plane CamLaserCalibraTool CN
Fast Extrinsic Calibration of a Laser Rangefinder to a Camera, 2005 checkerboard C: Plane (a), L: Plane (m) plane(n/d) correspondence, point-to-plane LCCT *
Extrinsic calibration of a 3D laser scanner and an omnidirectional camera, 2010 checkerboard C: plane (a), L: pts in plane (m) point-to-plane cam_lidar_calib *
LiDAR-Camera Calibration using 3D-3D Point correspondences, 2017 cardboard + ArUco C: 3D corners (a), L: 3D corners (m) ICP lidar_camera_calibration *
Reflectance Intensity Assisted Automatic and Accurate Extrinsic Calibration of 3D LiDAR and Panoramic Camera Using a Printed Chessboard, 2017 checkerboard C: 2D corners (a), L: 3D corners (a) PnP, angle difference ILCC *
Extrinsic Calibration of Lidar and Camera with Polygon, 2018 regular cardboard C: 2D edge, corners (a), L: 3D edge, pts in plane (a) point-to-line, point-inside-polygon ram-lab/plycal *
Automatic Extrinsic Calibration of a Camera and a 3D LiDAR using Line and Plane Correspondences, 2018 checkerboard C: 3D edge, plane(a), L: 3D edge, pts in plane (a) direcion/normal, point-to-line, point-to-plane Matlab LiDAR Toolbox *
Improvements to Target-Based 3D LiDAR to Camera Calibration, 2020 cardboard with ArUco C: 2d corners (a), L: 3D corners (a) PnP, IOU github *
ACSC: Automatic Calibration for Non-repetitive Scanning Solid-State LiDAR and Camera Systems, 2020 checkerboard C: 2D corners (a), L: 3D corners (a) PnP ACSC *
Automatic Extrinsic Calibration Method for LiDAR and Camera Sensor Setups, 2021 cardboard with circle & Aruco C: 3D points (a), L: 3D points (a) ICP velo2cam_ calibration *

C: camera, L: LiDAR, a: automaic, m: manual

2. Targetless methods

2.1. Motion-based methods

Paper Feature Optimization Toolbox Note
LiDAR and Camera Calibration Using Motions Estimated by Sensor Fusion Odometry, 2018 C: motion (ICP), L: motion (VO) hand-eye calibration * *

2.2. Scene-based methods

2.2.1. Traditional methods

Paper Feature Optimization Toolbox Note
Automatic Targetless Extrinsic Calibration of a 3D Lidar and Camera by Maximizing Mutual Information, 2012 C:grayscale, L: reflectivity mutual information, BB steepest gradient ascent Extrinsic Calib *
Automatic Calibration of Lidar and Camera Images using Normalized Mutual Information, 2013 C:grayscale, L: reflectivity, noraml normalized MI, particle swarm * *
Automatic Online Calibration of Cameras and Lasers, 2013 C: Canny edge, L: depth-discontinuous edge correlation, grid search * *
SOIC: Semantic Online Initialization and Calibration for LiDAR and Camera, 2020 semantic centroid PnP * *
A Low-cost and Accurate Lidar-assisted Visual SLAM System, 2021 C: edge(grayscale), L: edge (reflectivity, depth projection) ICP, coordinate descent algorithms CamVox *
Pixel-level Extrinsic Self Calibration of High Resolution LiDAR and Camera in Targetless Environments,2021 C:Canny edge(grayscale), L: depth-continuous edge point-to-line, Gaussian-Newton livox_camera_calib *
CRLF: Automatic Calibration and Refinement based on Line Feature for LiDAR and Camera in Road Scenes, 2021 C:straight line, L: straight line perspective3-lines (P3L) * CN

2.2.2. Deep-learning methods

Pape Toolbox Note
RegNet: Multimodal sensor registration using deep neural networks, 2017,IV regnet *
CalibNet: Geometrically supervised extrinsic calibration using 3d spatial transformer networks,2018,IROS CalibNet *

3. Other toolboxes

Toolbox Introduction Note
Apollo sensor calibration tools targetless method, no source code CN
Autoware camera lidar calibrator pick points mannually, PnP *
Autoware calibration camera lidar checkerboard, similar to LCCT CN
livox_camera_lidar_calibration pick points mannually, PnP *
An open-source, low-cost, image-based weed detection device for fallow scenarios.

Welcome to the OpenWeedLocator (OWL) project, an opensource hardware and software green-on-brown weed detector that uses entirely off-the-shelf compon

Guy Coleman 145 Jan 05, 2023
Mouse Brain in the Model Zoo

Deep Neural Mouse Brain Modeling This is the repository for the ongoing deep neural mouse modeling project, an attempt to characterize the representat

Colin Conwell 15 Aug 22, 2022
Geometry-Aware Learning of Maps for Camera Localization (CVPR2018)

Geometry-Aware Learning of Maps for Camera Localization This is the PyTorch implementation of our CVPR 2018 paper "Geometry-Aware Learning of Maps for

NVIDIA Research Projects 321 Nov 26, 2022
This repo is about to create the Streamlit application for given ML model.

HR-Attritiion-using-Streamlit This repo is about to create the Streamlit application for given ML model. Problem Statement: Managing peoples at workpl

Pavan Giri 0 Dec 10, 2021
Code for our paper "SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization", ACL 2021

SimCLS Code for our paper: "SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization", ACL 2021 1. How to Install Requirements

Yixin Liu 150 Dec 12, 2022
Async API for controlling Hue Lights

Hue API Async API for controlling Hue Lights Documentation: hue-api.nirantak.com Source: github.com/nirantak/hue-api Installation This is an async cli

Nirantak Raghav 4 Nov 16, 2022
Sequence-to-Sequence learning using PyTorch

Seq2Seq in PyTorch This is a complete suite for training sequence-to-sequence models in PyTorch. It consists of several models and code to both train

Elad Hoffer 514 Nov 17, 2022
Self-Supervised Learning with Kernel Dependence Maximization

Self-Supervised Learning with Kernel Dependence Maximization This is the code for SSL-HSIC, a self-supervised learning loss proposed in the paper Self

DeepMind 29 Dec 29, 2022
Code to generate datasets used in "How Useful is Self-Supervised Pretraining for Visual Tasks?"

Synthetic dataset rendering Framework for producing the synthetic datasets used in: How Useful is Self-Supervised Pretraining for Visual Tasks? Alejan

Princeton Vision & Learning Lab 21 Apr 29, 2022
Official Pytorch implementation of ICLR 2018 paper Deep Learning for Physical Processes: Integrating Prior Scientific Knowledge.

Deep Learning for Physical Processes: Integrating Prior Scientific Knowledge: Official Pytorch implementation of ICLR 2018 paper Deep Learning for Phy

emmanuel 47 Nov 06, 2022
The official implementation of ICCV paper "Box-Aware Feature Enhancement for Single Object Tracking on Point Clouds".

Box-Aware Tracker (BAT) Pytorch-Lightning implementation of the Box-Aware Tracker. Box-Aware Feature Enhancement for Single Object Tracking on Point C

Kangel Zenn 5 Mar 26, 2022
This implements one of result networks from Large-scale evolution of image classifiers

Exotic structured image classifier This implements one of result networks from Large-scale evolution of image classifiers by Esteban Real, et. al. Req

54 Nov 25, 2022
Pseudo lidar - (CVPR 2019) Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving

Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving This paper has been accpeted by Conference o

Yan Wang 881 Dec 27, 2022
This is a Keras implementation of a CNN for estimating age, gender and mask from a camera.

face-detector-age-gender This is a Keras implementation of a CNN for estimating age, gender and mask from a camera. Before run face detector app, expr

Devdreamsolution 2 Dec 04, 2021
Vpw analyzer - A visual J1850 VPW analyzer written in Python

VPW Analyzer A visual J1850 VPW analyzer written in Python Requires Tkinter, Pan

7 May 01, 2022
Joint Gaussian Graphical Model Estimation: A Survey

Joint Gaussian Graphical Model Estimation: A Survey Test Models Fused graphical lasso [1] Group graphical lasso [1] Graphical lasso [1] Doubly joint s

Koyejo Lab 1 Aug 10, 2022
This is an example of object detection on Micro bacterium tuberculosis using Mask-RCNN

Mask-RCNN on Mycobacterium tuberculosis This is an example of object detection on Mycobacterium Tuberculosis using Mask RCNN. Implement of Mask R-CNN

Jun-En Ding 1 Sep 16, 2021
This repository contains the PyTorch implementation of the paper STaCK: Sentence Ordering with Temporal Commonsense Knowledge appearing at EMNLP 2021.

STaCK: Sentence Ordering with Temporal Commonsense Knowledge This repository contains the pytorch implementation of the paper STaCK: Sentence Ordering

Deep Cognition and Language Research (DeCLaRe) Lab 23 Dec 16, 2022
Deep Learning (with PyTorch)

Deep Learning (with PyTorch) This notebook repository now has a companion website, where all the course material can be found in video and textual for

Alfredo Canziani 6.2k Jan 07, 2023
CompilerGym is a library of easy to use and performant reinforcement learning environments for compiler tasks

CompilerGym is a library of easy to use and performant reinforcement learning environments for compiler tasks

Facebook Research 721 Jan 03, 2023