OverFeat

OverFeat is a Convolutional Network-based image classifier and feature extractor.

OverFeat was trained on the ImageNet dataset and participated in the ImageNet 2013 competition.

This package allows researchers to use OverFeat to recognize images and extract features.

A library with C++ source code is provided for running the OverFeat convolutional network, together with wrappers in various scripting languages (Python, Lua, Matlab coming soon).

OverFeat was trained with the Torch7 package ( http://www.torch.ch ). The OverFeat package provides tools to run the network in a standalone fashion. The training code is not distributed at this time.

CREDITS, LICENSE, CITATION

OverFeat is Copyright NYU 2013. Authors of the present package are Michael Mathieu, Pierre Sermanet, and Yann LeCun.

The OverFeat system is by Pierre Sermanet, David Eigen, Xiang Zhang, Michael Mathieu, Rob Fergus, and Yann LeCun.

Please refer to the LICENSE file in the same directory as the present file for licensing information.

If you use OverFeat in your research, please cite the following paper:

"OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks", Pierre Sermanet, David Eigen, Xiang Zhang, Michael Mathieu, Rob Fergus, Yann LeCun http://arxiv.org/abs/1312.6229

INSTALLATION:

Download the archive from http://cilvr.cs.nyu.edu/doku.php?id=software:overfeat:start

Extract the files:

tar xvf overfeat-vXX.tgz
cd overfeat

Overfeat uses external weight files. Since these files are large and do not change often, they are not included in the archive. We provide a script to automatically download the weights :

./download_weights.py

The weight files should be in the folder data/default in the overfeat directory.

Overfeat can run without BLAS, however it would be very slow. We strongly advice you to install openblas on linux (on MacOS, Accelerate should be available without any installation). On Ubuntu/Debian you should compile it (it might take a while, but it is worth it) :

sudo apt-get install build-essential gcc g++ gfortran git libgfortran3
cd /tmp
git clone https://github.com/xianyi/OpenBLAS.git
cd OpenBLAS
make NO_AFFINITY=1 USE_OPENMP=1
sudo make install

For some reason, on 32 bits Ubuntu, libgfortran doesn't create the correct symlink. If you have issues linking with libgfortran, locate where libgfortran is installed (for instance /usr/lib/i386-linux-gnu) and create the correct symlink :

cd 
   
    
sudo ln -sf libgfortran.so.3 libgfortran.so

   

The precompiled binaries use BLAS. If you don't want to (or can't, for some reason) use BLAS, you must recompile overfeat.

RUNNING THE PRE-COMPILED BINARIES

Pre-compiled binaries are provided for Ubuntu Linux (32 bits and 64 bits) and Mac OS. The pre-requisites are python and imagemagick, which are installed by default on most popular Linux distros.

Important note: OverFeat compiled from source on your computer will run faster than the pre-compiled binaries.

Example of image classification, printing the 6 highest-scoring categories:

bin/YOUR_OS/overfeat -n 6 samples/bee.jpg

where YOUR_OS can be either linux_64, linux_32, or macos.

Running the webcam demo:

bin/YOUR_OS/webcam

GPU PRE-COMPILED BINARIES (EXPERIMENTAL)

We are providing precompiled binaries to run overfeat on GPU. Because the code is not released yet, we do not provide the source for now. The GPU release is experimental and for now only runs on linux 64bits. It requires a Nvidia GPU with CUDA architecture >= 2.0 (that covers all recent GPUs from Nvidia).

You will need openblas to run the GPU binaries.

The binaries are located in

bin/linux_64/cuda

And work the same way as the CPU versions. You can include the static library the same way as the CPU version.

COMPILING FROM SOURCE

Install dependencies : python, imagemagick, git, gcc, cmake (pkg-config and opencv required for the webcam demo). On Ubuntu/Debian :

apt-get install g++ git python imagemagick cmake

For the webcam demo :

apt-get install pkg-config libopencv-dev libopencv-highgui-dev

Here are the instructions to build the OverFeat library and tools:

Go to the src folder :

cd src

Build the tensor library (TH), OverFeat and the command-line tools:

make all

Build the webcam demo (OpenCV required) :

make cam

On Mac OS, the default gcc doesn't support OpenMP. We strongly recommend to install a gcc version with OpenMP support. With MacPort :

sudo port install gcc48

Which will provide g++-mp-48 . If you don't install this version, you will have to change the two corresponding lines in the Makefile.

UPDATING

A git repository is provided with the archive. You can update by typing

git pull

from the overfeat directory.

HIGH LEVEL INTERFACE:

The feature extractor requires a weight file, containing the weights of the network. We provide a weight file located in data/default/net_weight . The software we provide should be able to locate it automatically. In case it doesn't, the option -d can be used to manually provide a path.

Overfeat can use two sizes of network. By default, it uses the smaller one. For more accuracy, the option -l can be used to use a larger, but slower, network.

CLASSIFICATION:

In order to get the top (by default, =5) classes from a number of images :

bin/linux_64/overfeat [-n 
   
    ] [-d 
    
     ] [-l] path_to_image1 [path_to_image2 [path_to_image3 [... ] ] ]

    
   

To use overfeat online (feeding an image stream), feed its stdin stream with a sequence of ppm images (ended by end of file ('\0') character). In this case, please use option -p. For instance :

convert image1.jpg image2.jpg -resize 231x231 ppm:- | ./overfeat [-n 
   
    ] [-d 
    
     ] [-l] -p

    
   

Please note that to get the classes from an image, the image size should be 231x231. The image will be cropped if one dimension is larger than 231, and the network won't be able to work if both dimension are larger. For feature extraction without classification, it can be any size greater or equal to 231x231 for the small network, and 221x221 for the large network .

FEATURE EXTRACTION:

In order to extract the features instead of classifying, use -f option. For instance :

bin/linux_64/overfeat [-d 
   
    ] [-l] -f image1.png image2.jpg

   

It is compatible with option -p. The option -L (overrides -f) can be used to return the output of any layer. For instance

bin/linux_64/overfeat [-d 
   
    ] [-l] -L 12 image1.png

   

returns the output of layer 12. The option -f corresponds to layer 19 for the small layer and 22 for the large one.

It writes the features on stdout as a sequence. Each feature starts with three integers separated by spaces, the first is the number of features (n), the second is the number of rows (h) and the last is the number of columns (w). It is followed by a end of line ('\n') character. Then follows nhw floating point numbers (written in ascii) separated by spaces. The feature is the first dimension (so that to obtain the next feature, you must add wh to your index), followed by the row (to obtain the next row, add w to your index). That means that if you want the features corresponding to the top-left window, you need to read pixels ih*w for i=0..4095 .

The output is going to be a 3D tensor. The first dimension correspond to the features, while dimensions 2 and 3 are spatial (y and x respectively). The spatial dimension is reduced at each layer, and with the default network, using option -f, the output has size nFeatures * h * w where

• for the small network,
  • nFeatures = 4096
  • h = ((H-11)/4 + 1)/8-6
  • w = ((W-11)/4 + 1)/8-6
• for the large network,
  • nFeatures = 4096
  • h = ((H-7)/2 + 1)/18-5
  • w = ((W-7)/2 + 1)/18-5 if the input has size 3HW . Each pixel in the feature map corresponds to a localized window in the input. With the small network, the windows are 231x231 pixels, overlapping so that the i-th window begins at pixel 32i, while for the large network, the windows are 221x221, and the i-th window begins at pixel 36i.

WEBCAM:

We provide a live classifier based on the webcam. It reads images from the webcam, and displays the most likely classes along with the probabilities. It can be run with

bin/linux_64/webcam [-d 
   
    ] [-l] [-w 
    
     ]

    
   

BATCH:

We also provide an easy way to process a whole folder :

./bin/linux_64/overfeat_batch [-d 
   
    ] [-l] -i 
    
      -o 
     

     
    
   

It process each image in the input folder and produces a corresponding file in the output directory, containing the features,in the same format as before.

EXAMPLES:

Classify image samples/bee.jpg, getting the 3 most likely classes :

bin/linux_64/overfeat -n 3 samples/bee.jpg

Extract features from samples/pliers.jpg with the large network :

bin/linux_64/overfeat -f -l samples/pliers.jpg

Extract the features from all files in samples :

./bin/linux_64/overfeat_batch -i samples -o samples_features

Run the webcam demo with the large network :

bin/linux_64/webcam -l

ADVANCED:

The true program is actually overfeatcmd, where overfeat is only a python script calling overfeatcmd. overfeatcmd is not designed to be used by itself, but can be if necessary. It taked three arguments :

bin/linux_64/overfeatcmd