A Collection of LiDAR-Camera-Calibration Papers, Toolboxes and Notes

Overview

Awesome-LiDAR-Camera-Calibration

Awesome

A Collection of LiDAR-Camera-Calibration Papers, Toolboxes and Notes.

Outline

0. Introduction

For applications such as autonomous driving, robotics, navigation systems, and 3-D scene reconstruction, data of the same scene is often captured using both lidar and camera sensors. To accurately interpret the objects in a scene, it is necessary to fuse the lidar and the camera outputs together. Lidar camera calibration estimates a rigid transformation matrix (extrinsics, rotation+translation, 6 DoF) that establishes the correspondences between the points in the 3-D lidar plane and the pixels in the image plane.

Example

1. Target-based methods

Paper Target Feature Optimization Toolbox Note
Extrinsic Calibration of a Camera and Laser Range Finder (improves camera calibration), 2004 checkerboard C:Plane (a), L: pts in plane (m) point-to-plane CamLaserCalibraTool CN
Fast Extrinsic Calibration of a Laser Rangefinder to a Camera, 2005 checkerboard C: Plane (a), L: Plane (m) plane(n/d) correspondence, point-to-plane LCCT *
Extrinsic calibration of a 3D laser scanner and an omnidirectional camera, 2010 checkerboard C: plane (a), L: pts in plane (m) point-to-plane cam_lidar_calib *
LiDAR-Camera Calibration using 3D-3D Point correspondences, 2017 cardboard + ArUco C: 3D corners (a), L: 3D corners (m) ICP lidar_camera_calibration *
Reflectance Intensity Assisted Automatic and Accurate Extrinsic Calibration of 3D LiDAR and Panoramic Camera Using a Printed Chessboard, 2017 checkerboard C: 2D corners (a), L: 3D corners (a) PnP, angle difference ILCC *
Extrinsic Calibration of Lidar and Camera with Polygon, 2018 regular cardboard C: 2D edge, corners (a), L: 3D edge, pts in plane (a) point-to-line, point-inside-polygon ram-lab/plycal *
Automatic Extrinsic Calibration of a Camera and a 3D LiDAR using Line and Plane Correspondences, 2018 checkerboard C: 3D edge, plane(a), L: 3D edge, pts in plane (a) direcion/normal, point-to-line, point-to-plane Matlab LiDAR Toolbox *
Improvements to Target-Based 3D LiDAR to Camera Calibration, 2020 cardboard with ArUco C: 2d corners (a), L: 3D corners (a) PnP, IOU github *
ACSC: Automatic Calibration for Non-repetitive Scanning Solid-State LiDAR and Camera Systems, 2020 checkerboard C: 2D corners (a), L: 3D corners (a) PnP ACSC *
Automatic Extrinsic Calibration Method for LiDAR and Camera Sensor Setups, 2021 cardboard with circle & Aruco C: 3D points (a), L: 3D points (a) ICP velo2cam_ calibration *

C: camera, L: LiDAR, a: automaic, m: manual

2. Targetless methods

2.1. Motion-based methods

Paper Feature Optimization Toolbox Note
LiDAR and Camera Calibration Using Motions Estimated by Sensor Fusion Odometry, 2018 C: motion (ICP), L: motion (VO) hand-eye calibration * *

2.2. Scene-based methods

2.2.1. Traditional methods

Paper Feature Optimization Toolbox Note
Automatic Targetless Extrinsic Calibration of a 3D Lidar and Camera by Maximizing Mutual Information, 2012 C:grayscale, L: reflectivity mutual information, BB steepest gradient ascent Extrinsic Calib *
Automatic Calibration of Lidar and Camera Images using Normalized Mutual Information, 2013 C:grayscale, L: reflectivity, noraml normalized MI, particle swarm * *
Automatic Online Calibration of Cameras and Lasers, 2013 C: Canny edge, L: depth-discontinuous edge correlation, grid search * *
SOIC: Semantic Online Initialization and Calibration for LiDAR and Camera, 2020 semantic centroid PnP * *
A Low-cost and Accurate Lidar-assisted Visual SLAM System, 2021 C: edge(grayscale), L: edge (reflectivity, depth projection) ICP, coordinate descent algorithms CamVox *
Pixel-level Extrinsic Self Calibration of High Resolution LiDAR and Camera in Targetless Environments,2021 C:Canny edge(grayscale), L: depth-continuous edge point-to-line, Gaussian-Newton livox_camera_calib *
CRLF: Automatic Calibration and Refinement based on Line Feature for LiDAR and Camera in Road Scenes, 2021 C:straight line, L: straight line perspective3-lines (P3L) * CN

2.2.2. Deep-learning methods

Pape Toolbox Note
RegNet: Multimodal sensor registration using deep neural networks, 2017,IV regnet *
CalibNet: Geometrically supervised extrinsic calibration using 3d spatial transformer networks,2018,IROS CalibNet *

3. Other toolboxes

Toolbox Introduction Note
Apollo sensor calibration tools targetless method, no source code CN
Autoware camera lidar calibrator pick points mannually, PnP *
Autoware calibration camera lidar checkerboard, similar to LCCT CN
livox_camera_lidar_calibration pick points mannually, PnP *
Official repository of Semantic Image Matting

Semantic Image Matting This is the official repository of Semantic Image Matting (CVPR2021). Overview Natural image matting separates the foreground f

192 Dec 29, 2022
ViSER: Video-Specific Surface Embeddings for Articulated 3D Shape Reconstruction

ViSER: Video-Specific Surface Embeddings for Articulated 3D Shape Reconstruction. NeurIPS 2021.

Gengshan Yang 59 Nov 25, 2022
METER: Multimodal End-to-end TransformER

METER Code and pre-trained models will be publicized soon. Citation @article{dou2021meter, title={An Empirical Study of Training End-to-End Vision-a

Zi-Yi Dou 257 Jan 06, 2023
Segmentation models with pretrained backbones. Keras and TensorFlow Keras.

Python library with Neural Networks for Image Segmentation based on Keras and TensorFlow. The main features of this library are: High level API (just

Pavel Yakubovskiy 4.2k Jan 09, 2023
N-RPG - Novel role playing game da turfu

N-RPG Ce README sera la page de garde du projet. Contenu Il contiendra la présen

4 Mar 15, 2022
Official implementation for the paper "Attentive Prototypes for Source-free Unsupervised Domain Adaptive 3D Object Detection"

Attentive Prototypes for Source-free Unsupervised Domain Adaptive 3D Object Detection PyTorch code release of the paper "Attentive Prototypes for Sour

Deepti Hegde 23 Oct 17, 2022
Official implementation for "Low-light Image Enhancement via Breaking Down the Darkness"

Low-light Image Enhancement via Breaking Down the Darkness by Qiming Hu, Xiaojie Guo. 1. Dependencies Python3 PyTorch=1.0 OpenCV-Python, TensorboardX

Qiming Hu 30 Jan 01, 2023
Evolving neural network parameters in JAX.

Evolving Neural Networks in JAX This repository holds code displaying techniques for applying evolutionary network training strategies in JAX. Each sc

Trevor Thackston 6 Feb 12, 2022
ESTDepth: Multi-view Depth Estimation using Epipolar Spatio-Temporal Networks (CVPR 2021)

ESTDepth: Multi-view Depth Estimation using Epipolar Spatio-Temporal Networks (CVPR 2021) Project Page | Video | Paper | Data We present a novel metho

65 Nov 28, 2022
This is the face keypoint train code of project face-detection-project

face-key-point-pytorch 1. Data structure The structure of landmarks_jpg is like below: |--landmarks_jpg |----AFW |------AFW_134212_1_0.jpg |------AFW_

I‘m X 3 Nov 27, 2022
This is a code repository for paper OODformer: Out-Of-Distribution Detection Transformer

OODformer: Out-Of-Distribution Detection Transformer This repo is the official the implementation of the OODformer: Out-Of-Distribution Detection Tran

34 Dec 02, 2022
Object detection and instance segmentation toolkit based on PaddlePaddle.

Object detection and instance segmentation toolkit based on PaddlePaddle.

9.3k Jan 02, 2023
FrankMocap: A Strong and Easy-to-use Single View 3D Hand+Body Pose Estimator

FrankMocap pursues an easy-to-use single view 3D motion capture system developed by Facebook AI Research (FAIR). FrankMocap provides state-of-the-art 3D pose estimation outputs for body, hand, and bo

Facebook Research 1.9k Jan 07, 2023
Code for "The Box Size Confidence Bias Harms Your Object Detector"

The Box Size Confidence Bias Harms Your Object Detector - Code Disclaimer: This repository is for research purposes only. It is designed to maintain r

Johannes G. 24 Dec 07, 2022
Answer a series of contextually-dependent questions like they may occur in natural human-to-human conversations.

SCAI-QReCC-21 [leaderboards] [registration] [forum] [contact] [SCAI] Answer a series of contextually-dependent questions like they may occur in natura

19 Sep 28, 2022
Breaking the Curse of Space Explosion: Towards Efficient NAS with Curriculum Search

Breaking the Curse of Space Explosion: Towards Effcient NAS with Curriculum Search Pytorch implementation for "Breaking the Curse of Space Explosion:

guoyong 17 Jan 03, 2023
TrackFormer: Multi-Object Tracking with Transformers

TrackFormer: Multi-Object Tracking with Transformers This repository provides the official implementation of the TrackFormer: Multi-Object Tracking wi

Tim Meinhardt 321 Dec 29, 2022
Code for Low-Cost Algorithmic Recourse for Users With Uncertain Cost Functions

EMS-COLS-recourse Initial Code for Low-Cost Algorithmic Recourse for Users With Uncertain Cost Functions Folder structure: data folder contains raw an

Prateek Yadav 1 Nov 25, 2022
Official repository of the paper "GPR1200: A Benchmark for General-PurposeContent-Based Image Retrieval"

GPR1200 Dataset GPR1200: A Benchmark for General-Purpose Content-Based Image Retrieval (ArXiv) Konstantin Schall, Kai Uwe Barthel, Nico Hezel, Klaus J

Visual Computing Group 16 Nov 21, 2022
Awesome Artificial Intelligence, Machine Learning and Deep Learning as we learn it

Awesome Artificial Intelligence, Machine Learning and Deep Learning as we learn it. Study notes and a curated list of awesome resources of such topics.

mani 1.2k Jan 07, 2023