Scan the MRZ code of a passport and extract the firstname, lastname, passport number, nationality, date of birth, expiration date and personal numer.

Overview

PassportScanner

Works with 2 and 3 line identity documents.

Build Status Issues Stars Version License Platform Documentation

Git Twitter LinkedIn eMail

What is this

With PassportScanner you can use your camera to scan the MRZ code of a passport. It will extract all data like firstname, lastname, passport number, nationality, date of birth, expiration date and personal numer. Theres is Support for the TD1 and TD3 format (2 or 3 lines)

IMPORTANT NOTICE: SCANNING IDENTITY DOCUMENTS IS IN MOST CASES RESTRICTED BY LAW. OBSERVE THE APPLICABLE LAWS USING THIS TOOL. THE COPYRIGHT HOLDER IS NOT IN ANY WAY LIABLE FOR UNLAWFUL USAGE OF THIS TOOL.

PassportScanner is trying to optimize OCR results by first performing some graphic filters. The exposure filter is dynamic. This means that if the image is dark it tries to light it up and visa versa. As you can see in the demo animation below you will be able to scan a passport about once every 3 seconds.

wait a moment until the .gif below is downloaded...

animated

Building the PassportScanner demo

The current version is tested with Xcode 11.3, Swift 5

  1. Clone the repo to a working directory

  2. CocoaPods is used to manage dependencies. Pods are setup easily and are distributed via a ruby gem. Follow the simple instructions on the website to setup. After setup, run the following command from the toplevel directory of PassportScanner to download the dependencies:

pod install
  1. Open the PassportScanner.xcworkspace in Xcode.

  2. Build and Run the app.

External components for the demo

PassportScanner is using the following components which can be installed using CocoaPods.

  • TesseractOCRiOS Tesseract OCR iOS is a Framework for iOS7+.
  • GPUImage An open source iOS framework for GPU-based image and video processing
  • UIImage-Resize Category to add some resizing methods to the UIImage class, to resize it to a given CGSize — or fit in a CGSize keeping aspect ratio

Using PassportScanner in your own App

'PassportScanner' is now available through the dependency manager CocoaPods. At this moment this can be installed by executing:

[sudo] gem install cocoapods

If you have installed cocoapods you can just add PassportScanner to your workspace by making sure the following lines are in your Podfile:

source 'https://github.com/CocoaPods/Specs.git'
platform :ios, '8.0'
use_frameworks!
pod "PassportScanner"

Add an import at the top of your swift file like this:

import PassportScanner

If you want support for older versions than iOS 8.0, then you can also just copy the Pod folder containing the 2 classes MRZ and PassportScannerController to your app. Also add the above mentioned external components to your podfile.

When PassportScanner is added to your project, then copy the PassportScanner view from the Main.storyboard of the demo app to your own storyboard. After that you can customize that view. Also copy the MyScanViewController.sift to your own project. See the ViewConroller.swift to see how you can initiate the view and get the MRZ data

also copy over the tessdata folder to our app. That folder contains the trained data for tesseract (The OCR engine)

Here is sample code for a PassportScannerController:

protocol ProcessMRZ {
    func processMRZ(mrz:MRZ)
}

class MyScanViewController: PassportScannerController {

    var delegate: ProcessMRZ?

    // the .StartScan and .EndScan are IBOutlets and can be linked to your own buttons

    // For now just start scanning the moment this view is loaded
    override func viewDidLoad() {
        super.viewDidLoad();
        self.debug = true // So that we can see what's going on (scan text and quality indicator)
        self.accuracy = 1  // 1 = all checksums should pass (is the default so we could skip this line)
        self.mrzType = .auto // Performs a little better when set to td1 or td3
        self.showPostProcessingFilters = true // Set this to true to to give you a good indication of the scan quality
        self.StartScan(self)
    }
    
    override func succesfullScan(mrz: MRZ) {
        print("mrz: {\(mrz.description)\n}")
        delegate?.processMRZ(mrz)
        self.dismissViewControllerAnimated(true, completion: nil)
    }

    override func abbortScan() {
        self.dismissViewControllerAnimated(true, completion: nil)
    }
}

Then in your own code you can access this PassportScannerController and process the MRZ data with something like this:

class ViewController: UIViewController, ProcessMRZ {

    @IBOutlet weak var mrzLabel: UILabel!

    override func supportedInterfaceOrientations() -> UIInterfaceOrientationMask {
        return UIInterfaceOrientationMask.Portrait
    }

    override func prefersStatusBarHidden() -> Bool {
        return true
    }

    @IBAction func StartScan(sender: AnyObject) {
        let storyboard: UIStoryboard = UIStoryboard(name: "Main", bundle: nil)
        let scanVC: MyScanViewController = storyboard.instantiateViewControllerWithIdentifier("PassportScanner") as! MyScanViewController
        scanVC.delegate = self
        self.presentViewController(scanVC, animated: true, completion: nil)
    }

    func processMRZ(mrz:MRZ) {
        self.mrzLabel.text = mrz.description
        mrzLabel.sizeToFit()
    }
}

License

PassportScanner is available under the MIT 3 license. See the LICENSE file for more info.

My other libraries:

Also see my other open source iOS libraries:

  • EVReflection - Reflection based (Dictionary, CKRecord, JSON and XML) object mapping with extensions for Alamofire and Moya with RxSwift or ReactiveSwift
  • EVCloudKitDao - Simplified access to Apple's CloudKit
  • EVFaceTracker - Calculate the distance and angle of your device with regards to your face in order to simulate a 3D effect
  • EVURLCache - a NSURLCache subclass for handling all web requests that use NSURLReques
  • AlamofireOauth2 - A swift implementation of OAuth2 using Alamofire
  • EVWordPressAPI - Swift Implementation of the WordPress (Jetpack) API using AlamofireOauth2, AlomofireJsonToObjects and EVReflection (work in progress)
  • PassportScanner - Scan the MRZ code of a passport and extract the firstname, lastname, passport number, nationality, date of birth, expiration date and personal numer.
  • AttributedTextView - Easiest way to create an attributed UITextView with support for multiple links (url, hashtags, mentions).
Comments
  • Add support to scan the MRZ of an ID card

    Add support to scan the MRZ of an ID card

    As reported by @jrodriguezq there is no support for the MRZ for an ID card. I will add this as an enhanecment later.

    Original question:

    The last commit fixes the issue that I had with the camera, now it looks like this: img_3139

    The OCR it's working, but I'm not able to correctly read the MRZ. As you can see, the image appears to be clear enough. The usual case of the mrz captured it's something like this:

    INCHL1D41230409D26<<<<<<<<<<<< 9002063M2002064CHL17409866<2<6 CORTES<CONTRERAS<<SERGIO<JAV ll 12

    This appears to be almost correct, but some numbers and blank characters usually stand in the way and I'm not able to scan the MRZ. Is there any configuration that I'm missing?

    Again, many thanks for the library.

    enhancement 
    opened by evermeer 17
  • Camera Scanning is not clear

    Camera Scanning is not clear

    I am developing Passport Scanner app by referring PassportScanner program (https://github.com/evermeer/PassportScanner). I am testing above code on iPod 5th Gen. But I am facing problem in that, the problem is “camera to scan the MRZ code of a passport”, once it is launched is not coming clear and Pink dots are coming. How to get clear camera scanning, Do I have to change any camera parameter? Please help me to improve camera scanning.

    passportscanner

    opened by krmadhan 10
  • After scanning for some seconds with a Scan quality insufficient : 0.0, I get a

    After scanning for some seconds with a Scan quality insufficient : 0.0, I get a "GPUImageFramebuffer unlock" Exception.

    2016-03-21 17:50:02.556 PassportScanner[9842:4849034] *** Assertion failure in -[GPUImageFramebuffer unlock], /Users/frlobo/Downloads/PassportScanner-master-2/Pods/GPUImage/framework/Source/GPUImageFramebuffer.m:269 2016-03-21 17:50:02.558 PassportScanner[9842:4849034] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Tried to overrelease a framebuffer, did you forget to call -useNextFrameForImageCapture before using -imageFromCurrentFramebuffer?' *** First throw call stack: (0x18382d900 0x182e9bf80 0x18382d7d0 0x1841a099c 0x100197c80 0x100197e2c 0x184ac447c 0x184ac3510 0x18370d664 0x184ac3210 0x18370d664 0x18851cc24 0x182eb5ae8 0x18371142c 0x18411741c 0x1841d6728 0x101b65bb0 0x101b6b658 0x1837e4bb0 0x1837e2a18 0x183711680 0x184c20088 0x188588d90 0x1001005e4 0x1832b28b8) libc++abi.dylib: terminating with uncaught exception of type NSException

    bug Fixed? 
    opened by kikolobo 9
  • Crash while scanning

    Crash while scanning

    Hi, I encountered the following error:

    2015-12-31 13:47:49.125 PassportScanner[3038:1427113] *** Assertion failure in -[GPUImageFramebuffer unlock], /Users/francis/Downloads/PassportScanner-master/Pods/GPUImage/framework/Source/GPUImageFramebuffer.m:269 2015-12-31 13:47:49.130 PassportScanner[3038:1427113] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Tried to overrelease a framebuffer, did you forget to call -useNextFrameForImageCapture before using -imageFromCurrentFramebuffer?'

    Apart from that this is fantastic! Thanks for your work.

    Fixed? 
    opened by francisjervis 7
  • Reduce Tesseract delay?

    Reduce Tesseract delay?

    Hi Edwin!

    I've been making tests with this scanner and I noticed every time Tesseract is called to analyze the processed capture (according to the code is every 0.2 secs) cause a delay in the camera view and the view get freeze constantly. I guess Tesseract take time to read the eng.traineddata and that is the cause of the delay.

    Is there some way to reduce that delay and get a camera view with no freezing?

    opened by alejandroruizponce 5
  • Feature request: Disable camera filters in the preview

    Feature request: Disable camera filters in the preview

    Hi everybody, Currently the library show a manipulated video in the camera preview view. The image is black and white, several filters have been applied in order to facilitate the acquisition with the ocr. For some uses this is not acceptable, I'm trying to figure out if it is possible to restore the "standard camera preview" disabling all filters, re applying them in post processing when the frame is passed to the recognition engine.

    Any suggestion in where to start? Thanks!

    opened by punto2018 5
  • Camera freeze in white screen

    Camera freeze in white screen

    The issue can be reproduced as follow:

    • start the ocr by pressing "scan" in the demo
    • the camera view show "some frame" for a while
    • than it "fade" in white nothing can be done to acquire the MRZ

    Tested on iPhone 6S and iOS 12.

    img_0061trim

    opened by punto2018 4
  • How to validate 3-lines MRZ?

    How to validate 3-lines MRZ?

    Hello @evermeer, awesome work!

    I met the challenge - how to validate second line of triple-lined MRZ. I saw the wikipedia but there is no exhaustive info about how to do that. And other question... in your MRZ.swift i saw that you extract passportNumber like two leading letters and other is numbers.. where you found this format? I want the same explanation for document number for triple-lined MRZ's.

    Thanks in advance!

    question Fixed? 
    opened by andymedvedev 4
  • Possible memory leak or normal behavior

    Possible memory leak or normal behavior

    I've just noticed that after each scan the memory allocated is getting 8-10Mb bigger (after about 10 scans memory usage exceeds 100 Mb ). Is it OK, or maybe an issue of Debug Navigator? It seems that when the PassportScannerController dismisses, it has not been fully deallocated. Tesseract cash clean up doesn't help.

    opened by babucha 3
  • added ocr parsing rect setter, added tesseract training data path set…

    added ocr parsing rect setter, added tesseract training data path set…

    Hi, I've added some features:

    • ocr parsing rect setter Now it is possible to set the rect to be used by tesseract for the ocr. It is useful in case you need to overlap a mask to the render view where the mrz is not in the middle of the screen. The ocr area is now visible in red when debug == true Note: the default mask (overlay.png) in the example project is not correct, maybe it now can be removed (or fixed)

    • added tesseract training data path setter It is useful if the training data are not in the main bundle

    added controls in order to allow the use the view without mounting the vc

    • sometimes the developer can't mount a vc but need only the view in its vc hierarchy. In this case the viewDidLoad and viewDidAppear was not well managed.

    Thanks and Enjoy

    opened by punto2018 3
  • objc support, viewcontroller as delegate

    objc support, viewcontroller as delegate

    Changelog: -added support for objective c projects

    • PassportScannerController can be used as delegate, without need to subclass it
    • Added the processed image getter
    • it is already merged with the version 4.0.1
    opened by punto2018 3
  • When scanning a real passport, it has a hard time to scan the MRZ

    When scanning a real passport, it has a hard time to scan the MRZ

    Not sure why. I have both a Canadian and US passport. The Canadian did scan on the first try. I got one shot at the US but subsequent attempts failed. I just get this warning in the console:

    Scan quality insufficient : 0.0
    2020-12-21 12:03:57.630215-0500 PassportScanner[1519:784903] [Unknown process name] CGImageCreate: invalid image alphaInfo: kCGImageAlphaNone. It should be kCGImageAlphaNoneSkipLast
    - Start recognize
    Scan result : bylaw 1w xf I  L
    
    

    I have ideal light, the code appears clearly on the iPhone 11 that I'm using.

    opened by LaurentDaudelin 1
Releases(4.7.0)
Owner
Edwin Vermeer
Edwin Vermeer
Detect handwritten words in a text-line (classic image processing method).

Word segmentation Implementation of scale space technique for word segmentation as proposed by R. Manmatha and N. Srimal. Even though the paper is fro

Harald Scheidl 190 Jan 03, 2023
Ackermann Line Follower Robot Simulation.

Ackermann Line Follower Robot This is a simulation of a line follower robot that works with steering control based on Stanley: The Robot That Won the

Lucas Mazzetto 2 Apr 16, 2022
Total Text Dataset. It consists of 1555 images with more than 3 different text orientations: Horizontal, Multi-Oriented, and Curved, one of a kind.

Total-Text-Dataset (Official site) Updated on April 29, 2020 (Detection leaderboard is updated - highlighted E2E methods. Thank you shine-lcy.) Update

Chee Seng Chan 671 Dec 27, 2022
Code for the head detector (HeadHunter) proposed in our CVPR 2021 paper Tracking Pedestrian Heads in Dense Crowd.

Head Detector Code for the head detector (HeadHunter) proposed in our CVPR 2021 paper Tracking Pedestrian Heads in Dense Crowd. The head_detection mod

Ramana Subramanyam 76 Dec 06, 2022
Um simples projeto para fazer o reconhecimento do captcha usado pelo jogo bombcrypto

CaptchaSolver - LEIA ISSO 😓 Para iniciar o codigo: pip install -r requirements.txt python captcha_solver.py Se você deseja pegar ver o resultado das

Kawanderson 50 Mar 21, 2022
This is a Computer vision package that makes its easy to run Image processing and AI functions. At the core it uses OpenCV and Mediapipe libraries.

CVZone This is a Computer vision package that makes its easy to run Image processing and AI functions. At the core it uses OpenCV and Mediapipe librar

CVZone 648 Dec 30, 2022
Aloception is a set of package for computer vision: aloscene, alodataset, alonet.

Aloception is a set of package for computer vision: aloscene, alodataset, alonet.

Visual Behavior 86 Dec 28, 2022
A buffered and threaded wrapper for the OpenCV VideoCapture object. Can speed up video decoding significantly. Supports

A buffered and threaded wrapper for the OpenCV VideoCapture object. Can speed up video decoding significantly. Supports "with"-syntax.

Patrice Matz 0 Oct 30, 2021
Code for AAAI 2021 paper: Sequential End-to-end Network for Efficient Person Search

This repository hosts the source code of our paper: [AAAI 2021]Sequential End-to-end Network for Efficient Person Search. SeqNet achieves the state-of

Zj Li 218 Dec 31, 2022
A python programusing Tkinter graphics library to randomize questions and answers contained in text files

RaffleOfQuestions Um programa simples em python, utilizando a biblioteca gráfica Tkinter para randomizar perguntas e respostas contidas em arquivos de

Gabriel Ferreira Rodrigues 1 Dec 16, 2021
An Optical Character Recognition system using Pytesseract/Extracting data from Blood Pressure Reports.

Optical_Character_Recognition An Optical Character Recognition system using Pytesseract/Extracting data from Blood Pressure Reports. As an IOT/Compute

Ramsis Hammadi 1 Feb 12, 2022
Pre-Recognize Library - library with algorithms for improving OCR quality.

PRLib - Pre-Recognition Library. The main aim of the library - prepare image for recogntion. Image processing can really help to improve recognition q

Alex 80 Dec 30, 2022
Links to awesome OCR projects

Awesome OCR This list contains links to great software tools and libraries and literature related to Optical Character Recognition (OCR). Contribution

Konstantin Baierer 2.2k Jan 02, 2023
Thresholding-and-masking-using-OpenCV - Image Thresholding is used for image segmentation

Image Thresholding is used for image segmentation. From a grayscale image, thresholding can be used to create binary images. In thresholding we pick a threshold T.

Grace Ugochi Nneji 3 Feb 15, 2022
第一届西安交通大学人工智能实践大赛(2018AI实践大赛--图片文字识别)第一名;仅采用densenet识别图中文字

OCR 第一届西安交通大学人工智能实践大赛(2018AI实践大赛--图片文字识别)冠军 模型结果 该比赛计算每一个条目的f1score,取所有条目的平均,具体计算方式在这里。这里的计算方式不对一句话里的相同文字重复计算,故f1score比提交的最终结果低: - train val f1score 0

尹畅 441 Dec 22, 2022
Handwritten Number Recognition using CNN and Character Segmentation

Handwritten-Number-Recognition-With-Image-Segmentation Info About this repository This Repository is aimed at reading handwritten images of numbers an

Sparsha Saha 17 Aug 25, 2022
A program that takes in the hand gesture displayed by the user and translates ASL.

Interactive-ASL-Recognition Using the framework mediapipe made by google, OpenCV library and through self teaching, I was able to create a program tha

Riddhi Bajaj 3 Nov 22, 2021
Regions sanitàries (RS), Sectors Sanitàris (SS) i Àrees Bàsiques de Salut (ABS) de Catalunya

Regions sanitàries (RS), Sectors Sanitaris (SS), Àrees de Gestió Assistencial (AGA) i Àrees Bàsiques de Salut (ABS) de Catalunya Fitxers GeoJSON de le

Glòria Macià Muñoz 2 Jan 23, 2022
Source code of RRPN ---- Arbitrary-Oriented Scene Text Detection via Rotation Proposals

Paper source Arbitrary-Oriented Scene Text Detection via Rotation Proposals https://arxiv.org/abs/1703.01086 News We update RRPN in pytorch 1.0! View

428 Nov 22, 2022
This is the open source implementation of the ICLR2022 paper "StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis"

StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image

Meta Research 840 Dec 26, 2022