Trove
Trove is a research framework for building weakly supervised (bio)medical named entity recognition (NER) and other entity attribute classifiers without hand-labeled training data.
The COVID-19 pandemic has underlined the need for faster, more flexible ways of building and sharing state-of-the-art NLP/NLU tools to analyze electronic health records, scientific literature, and social media. Likewise, recent research into language modeling and the dangers of uncurated, "unfathomably" large-scale training data underlines the broader need to approach training set creation itself with more transparency and rigour.
Trove provides tools for combining freely available supervision sources such as medical ontologies from the Unified Medical Language System (UMLS), common text heuristics, and other noisy labeling sources for use as entity labelers in weak supervision frameworks such as Snorkel, FlyingSquid and others. Technical details are available in our manuscript.
Trove has been used as part of several COVID-19 reseach efforts at Stanford.
- Continuous symptom profiling of patients screened for SARS-CoV-2. We used a daily feed of patient notes from Stanford Health Care emergency departments to generate up-to-date COVID-19 symptom frequency data. Funded by the Bill & Melinda Gates Foundation.
- Estimating the efficacy of symptom-based screening for COVID-19 published in npj Digitial Medicine.
- Our COVID-19 symptom data was used by CMU's DELPHI group to prioritize selection of informative features from Google's Symptom Search Trends dataset.
Getting Started
Tutorials
See tutorials/ for Jupyter notebooks walking through an example NER application.
Installation
Requirements: Python 3.6 or later. We recomend using pip to install
pip install -r requirements.txt
Contributions
We welcome all contributions to the code base! Please submit a pull request and/or start a discussion on GitHub Issues.
Weakly supervised methods for programatically building and maintaining training sets provides new opportunities for the larger community to participate in the creation of important datasets. This is especially exciting in domains such as medicine, where sharing labeled data is often challening due to patient privacy concerns.
Inspired by recent efforts such as HuggingFace's Datasets library, we would love to start a conversation around how to support sharing labelers in service of mantaining an open task library, so that it is easier to create, deploy, and version control weakly supervised models.
Citation
If use Trove in your research, please cite us!
Fries, J.A., Steinberg, E., Khattar, S. et al. Ontology-driven weak supervision for clinical entity classification in electronic health records. Nat Commun 12, 2017 (2021). https://doi-org.stanford.idm.oclc.org/10.1038/s41467-021-22328-4
@article{fries2021trove,
title={Ontology-driven weak supervision for clinical entity classification in electronic health records},
author={Fries, Jason A and Steinberg, Ethan and Khattar, Saelig and Fleming, Scott L and Posada, Jose and Callahan, Alison and Shah, Nigam H},
journal={Nature Communications},
volume={12},
number={1},
year={2021},
publisher={Nature Publishing Group}
}
174 Dec 19, 2022
18 Sep 10, 2022
12 Dec 14, 2022
1 Dec 17, 2021
388 Dec 18, 2022
24 May 30, 2022
615 Dec 25, 2022
1.2k Jan 4, 2023
6 Dec 25, 2022
117 Dec 24, 2022
2.1k Jan 05, 2023
8.9k Jan 04, 2023
143 Dec 26, 2022
2 May 13, 2022
21 Oct 19, 2022
12 Mar 14, 2022
150 Nov 14, 2022
8 Nov 24, 2022
357 Jan 04, 2023
10 Nov 09, 2022
130 Dec 29, 2022
75 Dec 12, 2022
33 Dec 04, 2022
791 Jan 06, 2023
715 Nov 05, 2022
157 Nov 20, 2022
40 Dec 18, 2022
1 Nov 30, 2021
448 Jan 02, 2023