Augmenty: The cherry on top of your NLP pipeline
Augmenty is an augmentation library based on spaCy for augmenting texts. Besides a wide array of highly flexible augmenters, Augmenty provides a series of tools for working with augmenters, including combining and moderating augmenters. Augmenty differs from other augmentation libraries in that it corrects (as far as possible) the assigned labels under the augmentation, thus making many of the augmenters valid for training more than simply sentence classification.
🔧
Installation
To get started using augmenty simply install it using pip by running the following line in your terminal:
pip install augmenty
Do note that this is a minimal installation. As some augmenters requires additional packages please write the following line to install all dependencies.
pip install augmenty[all]
For more detailed instructions on installing augmenty, including specific language support, see the installation instructions.
🍒
Simple Example
The following shows a simple example of how you can quickly augment text using Augmenty. For more on using augmenty see the usage guides.
import spacy
import augmenty
nlp = spacy.load("en_core_web_sm")
docs = nlp.pipe(["Augmenty is a great tool for text augmentation"])
entity_augmenter = augmenty.load("ents_replace.v1",
ent_dict = {{"ORG": [["spaCy"], ["spaCy", "Universe"]]})
for doc in augmenty.docs(docs, augmenter=entity_augmenter)
print(doc)
spaCy Universe is a great tool for text augmentation.
📖
Documentation
| Documentation | |
|---|---|
| |
Guides and instruction on how to use augmenty and its features. |
| |
New additions, changes and version history. |
| |
The detailed reference for augmenty's API. Including function documentation |
| |
Contains a full list of current augmenters in augmenty. |
| |
A simple streamlit demo to try out the augmenters. |
💬
Where to ask questions
| Type | |
|---|---|
| |
GitHub Issue Tracker |
| |
GitHub Issue Tracker |
| |
GitHub Discussions |
| |
GitHub Discussions |
| |
Adding an augmenter |
🤔
FAQ
How do I test the code and run the test suite?
augmenty comes with an extensive test suite. In order to run the tests, you'll usually want to clone the repository and build augmenty from the source. This will also install the required development dependencies and test utilities defined in the requirements.txt.
pip install -r requirements.txt
pip install pytest
python -m pytest
which will run all the test in the augmenty/tests folder.
Specific tests can be run using:
python -m pytest augmenty/tests/test_docs.py
Code Coverage If you want to check code coverage you can run the following:
pip install pytest-cov
python -m pytest --cov=.
Does augmenty run on X?
augmenty is intended to run on all major OS, this includes Windows (latest version), MacOS (Catalina) and the latest version of Linux (Ubuntu). Below you can see if augmenty passes its test suite for the system of interest. Please note these are only the systems augmenty is being actively tested on, if you run on a similar system (e.g. an earlier version of Linux) augmenty will likely run there as well, if not please create an issue.
| Operating System | Status |
|---|---|
| Ubuntu/Linux (Latest) | |
| MacOS (Catalina) |  |
| Windows (Latest) |  |
How is the documentation generated?
augmenty uses sphinx to generate documentation. It uses the Furo theme with a custom styling.
To make the documentation you can run:
# install sphinx, themes and extensions
pip install sphinx furo sphinx-copybutton sphinxext-opengraph
# generate html from documentations
make -C docs html
Many of these augmenters are completely useless for training?
That is true, some of the augmenters are rarely something you would augment with during training. For instance randomly adding or removing spacing. However, augmentation can just as well be used to test whether a model is robust to certain variations.
Can I use augmenty without using spacy?
Indeed augmenty contains convenience functions for applying augmentation directly to raw texts. Check out the getting started guide to learn how.
🎓
Citing this work
If you use this library in your research, please cite:
@inproceedings{augmenty2021,
title={Augmenty, the cherry on top of your NLP pipeline},
author={Enevoldsen, Kenneth and Hansen, Lasse},
year={2021}
}
850 Dec 28, 2022
20 Apr 30, 2022
11 Sep 19, 2022
94 Dec 08, 2022
1.6k Jan 05, 2023
47 Aug 14, 2022
12 May 31, 2022
53 Mar 01, 2022
1 Oct 25, 2021
1 Dec 23, 2021
29 Dec 14, 2022
1 Jan 01, 2022
47 Sep 05, 2022
81 Dec 09, 2022
85 Dec 28, 2022
1 Feb 04, 2022
9.1k Jan 02, 2023
21 Dec 17, 2022
2.5k Jan 03, 2023
1 Feb 16, 2022