Aloception
Aloception is a set of packages for computer vision built on top of popular deep learning libraries: pytorch and pytorch lightning.
Aloscene
Aloscene extend the use of tensors with Augmented Tensors designed to facilitate the use of computer vision data (such as frames, 2d boxes, 3d boxes, optical flow, disparity, camera parameters...).
frame = aloscene.Frame("/path/to/image.jpg")
frame = frame.to("cpu")
frame.get_view().render()
Alodataset
Alodataset implement ready-to-use datasets for computer vision with the help of aloscene and augmented tensors to make it easier to transform and display your vision data.
coco_dataset = alodataset.CocoDetectionDataset(sample=True)
for frame in coco_dataset.stream_loader():
frame.get_view().render()
Alonet
Alonet integrates several promising computer vision architectures. You can use it for research purposes or to finetune and deploy your model using TensorRT. Alonet is mainly built on top of lightning with the help of aloscene and alodataset.
Training
# Init the training pipeline
detr = alonet.detr.LitDetr()
# Init the data module
coco_loader = alonet.detr.CocoDetection2Detr()
# Run the training using the two components
detr.run_train(data_loader=coco_loader, project="detr", expe_name="test_experiment")
Inference
# Load model
model = alonet.detr.DetrR50(num_classes=91, weights="detr-r50").eval()
# Open and normalized frame
frame = aloscene.Frame("/path/to/image.jpg").norm_resnet()
# Run inference
pred_boxes = model.inference(model([frame]))
# Add and display the predicted boxes
frame.append_boxes2d(pred_boxes[0], "pred_boxes")
frame.get_view().render()
Note
One can use aloscene independently than the two other packages to handle computer vision data, or to improve its training pipelines with augmented tensors.
Install
Aloception's packages are built on top of multiple libraries. Most of them are listed in the requirements.txt
pip install -r requirements.txt
Once the others packages are installed, you still need to install pytorch based on your hardware and environment configuration. Please, ref to the pytorch website
_ for this install.
Getting started
- Getting started
- Aloscene: Computer vision with ease
- Alodataset: Loading your vision datasets
- Alonet: Loading & training your models
- About augmented tensors
Tutorials
- How to setup your data?
- Training Detr
- Finetuning DETR
- Training Deformable DETR
- Finetuning Deformanble DETR
- Exporting DETR / Deformable-DETR to TensorRT
Alonet
Models
| Model name | Link | alonet location | Learn more |
|---|---|---|---|
| detr-r50 | https://arxiv.org/abs/2005.12872 | alonet.detr.DetrR50 | Detr |
| deformable-detr | https://arxiv.org/abs/2010.04159 | alonet.deformable_detr.DeformableDETR | Deformable detr |
| RAFT | https://arxiv.org/abs/2003.12039 | alonet.raft.RAFT | RAFT |
Detr
Here is a simple example to get started with Detr and aloception. To learn more about Detr, you can checkout the Tutorials or the detr README.
# Load model
model = alonet.detr.DetrR50(num_classes=91, weights="detr-r50").eval()
# Open and normalized frame
frame = aloscene.Frame("/path/to/image.jpg").norm_resnet()
# Run inference
pred_boxes = model.inference(model([frame]))
# Add and display the predicted boxes
frame.append_boxes2d(pred_boxes[0], "pred_boxes")
frame.get_view().render()
Deformable Detr
Here is a simple example to get started with Deformable Detr and aloception. To learn more about Deformable, you can checkout the Tutorials or the deformable detr README.
# Loading Deformable model
model = alonet.deformable_detr.DeformableDetrR50(num_classes=91, weights="deformable-detr-r50").eval()
# Open, normalize frame and send frame on the device
frame = aloscene.Frame("/home/thibault/Desktop/yoga.jpg").norm_resnet().to(torch.device("cuda"))
# Run inference
pred_boxes = model.inference(model([frame]))
# Add and display the predicted boxes
frame.append_boxes2d(pred_boxes[0], "pred_boxes")
frame.get_view().render()
RAFT
Here is a simple example to get started with RAFT and aloception. To learn more about RAFT, you can checkout the raft README.
# Use the left frame from the Sintel Flow dataset and normalize the frame for the RAFT Model
frame = alodataset.SintelFlowDataset(sample=True).getitem(0)["left"].norm_minmax_sym()
# Load the model using the sintel weights
raft = alonet.raft.RAFT(weights="raft-sintel")
# Compute optical flow
padder = alonet.raft.utils.Padder()
flow = raft.inference(raft(padder.pad(frame[0:1]), padder.pad(frame[1:2])))
# Render the flow along with the first frame
flow[0].get_view().render()
Alodataset
Here is a list of all the datasets you can use on Aloception. If you're dataset is not in the list but is important for computer vision. Please let us know using the issues or feel free to contribute.
Datasets
| Dataset name | alodataset location | To try |
|---|---|---|
| CocoDetection | alodataset.CocoDetectionDataset | python alodataset/coco_detection_dataset.py |
| CrowdHuman | alodataset.CrowdHumanDataset | python alodataset/crowd_human_dataset.py |
| Waymo | alodataset.WaymoDataset | python alodataset/waymo_dataset.py |
| ChairsSDHom | alodataset.ChairsSDHomDataset | python alodataset/chairssdhom_dataset.py |
| FlyingThings3DSubset | alodataset.FlyingThings3DSubsetDataset | python alodataset/flyingthings3D_subset_dataset.py |
| FlyingChairs2 | alodataset.FlyingChairs2Dataset | python alodataset/flying_chairs2_dataset.py |
| SintelDisparityDataset | alodataset.SintelDisparityDataset | python alodataset/sintel_disparity_dataset.py |
| SintelFlowDataset | alodataset.SintelFlowDataset | python alodataset/sintel_flow_dataset.py |
| MOT17 | alodataset.Mot17 | python alodataset/mot17.py |
Unit tests
python -m pytest
Licence
Shield:
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
91 Nov 22, 2022
111 Nov 27, 2022
2.2k Jan 02, 2023
29.2k Jan 05, 2023
152 Oct 16, 2022
41 Dec 30, 2022
67 Nov 30, 2022
36 Dec 15, 2022
541 Dec 28, 2022
46 Jun 08, 2022
42 Jun 05, 2022
170 Dec 21, 2022
3 Sep 29, 2022
38 Dec 05, 2022
487 Nov 11, 2022
21 Mar 11, 2022
5.2k Jan 04, 2023
3 Jun 07, 2021
27 Jan 08, 2023
1 Oct 27, 2022