Skip to content

ravikanagpal/Toxic_span_detection

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Open in Visual Studio Code

SemEval 2021 Task 5: Toxic Spans Detection:

Task:

Link to SemEval-2021: Task 5 Toxic Span Detection is https://competitions.codalab.org/competitions/25623

References:

  1. https://huggingface.co/docs/transformers/training - To understand how to train model.
  2. https://huggingface.co/docs/transformers/model_doc/roberta - To understand Roberta model and corresponding tokenizer
  3. https://huggingface.co/docs/transformers/model_doc/distilbert - To understand DistilBert and corresponding rokeniser
  4. huggingface/transformers#14305 - to understand postprocessing of predicted labels to spans
  5. https://github.com/huggingface/notebooks/blob/master/examples/token_classification-tf.ipynb - Copied function tokenize_and_align_labels() from this tutorial notebook from huggingface and followed the certain steps to fine tune model on custom dataset.
  6. https://github.com/ipavlopoulos/toxic_spans/blob/master/evaluation/metrics.py - F1 score function provided by competition is modified to accomodate our model output

About

Using BERT-based models for toxic span detection

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published