Code for Massive-scale Decoding for Text Generation using Lattices

Last update: Dec 18, 2022

Related tags

Overview

Massive-scale Decoding for Text Generation using Lattices

TL;DR: a new search algorithm to construct lattices encoding many generation options; two key technical contributions: (1) best-first search, (2) path recombination.

Visualization

We provide a few examples in the vis folder and on my homepage. You need to download the html files to view and interact with the model outputs.

The complete set of outputs are available on Box.

Getting started

model contains all of the methods, including baselines like beam search, nucleus sampling, and our methods.
evaluation contains scripts for evaluation.
command are the prompts and shells we use to run the experiment.

Beam Search:

PYTHONPATH=./ python src/recom_search/command/run_pipeline.py -nexample 100  -ngram_suffix 4  -beam_size 16 -min_len 10 -max_len 35   -model bs

Best-first Search:

PYTHONPATH=./ python src/recom_search/command/run_pipeline.py -nexample 100  -ngram_suffix 4  -beam_size 16 -min_len 10 -max_len 35   -model astar_baseline

Best-first Search with Recomb:

PYTHONPATH=./ python src/recom_search/command/run_pipeline.py -nexample 100  -ngram_suffix 4 -beam_size 16 -min_len 10 -max_len 35 -model astar -merge imp  -avg_score 0.75  -adhoc

Best-first Search with Zip:

PYTHONPATH=./ python src/recom_search/command/run_pipeline.py -nexample 100  -ngram_suffix 4 -beam_size 16 -min_len 10 -max_len 35 -model astar -merge zip  -avg_score 0.75  -adhoc

More detailed instructions coming soon!

Citation

@misc{xu-durrett-2021-massive,
    title={Massive-scale Decoding for Text Generation using Lattices},
    author={Jiacheng Xu and Greg Durrett},
    year={2021},
    eprint={2112.07660},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

Contact

[email protected]

Code for Massive-scale Decoding for Text Generation using Lattices

Related tags

Overview

Massive-scale Decoding for Text Generation using Lattices

Visualization

Getting started

Citation

Contact

Owner

Jiacheng Xu

v objective diffusion inference code for JAX.

Spontaneous Facial Micro Expression Recognition using 3D Spatio-Temporal Convolutional Neural Networks

Source code for The Power of Many: A Physarum Swarm Steiner Tree Algorithm

Toward Spatially Unbiased Generative Models (ICCV 2021)

VGGFace2-HQ - A high resolution face dataset for face editing purpose

PyTorch implementation of popular datasets and models in remote sensing

Realtime_Multi-Person_Pose_Estimation

Numerical Methods with Python, Numpy and Matplotlib

KoCLIP: Korean port of OpenAI CLIP, in Flax

Optical Character Recognition + Instance Segmentation for russian and english languages

Implementation of Cross Transformer for spatially-aware few-shot transfer, in Pytorch

Projects of Andfun Yangon

ADOP: Approximate Differentiable One-Pixel Point Rendering

Collect some papers about transformer with vision. Awesome Transformer with Computer Vision (CV)

Torch Implementation of "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network"

Model Zoo of BDD100K Dataset

The pytorch implementation of the paper "text-guided neural image inpainting" at MM'2020

Yolo algorithm for detection + centroid tracker to track vehicles

FishNet: One Stage to Detect, Segmentation and Pose Estimation

Object detection using yolo-tiny model and opencv used as backend