pair2vec: Compositional Word-Pair Embeddings for Cross-Sentence Inference

Introduction

This repository contains the code for replicating results from

• pair2vec: Compositional Word-Pair Embeddings for Cross-Sentence Inference
• Mandar Joshi, Eunsol Choi, Omer Levy, Dan Weld, and Luke Zettlemoyer

Getting Started

• Install python3 requirements: pip install -r requirements.txt

Using pretrained pair2vec embeddings

• Download pretrained pair2vec: ./download_pair2vec.sh
  • If you want to reproduce results from the paper on QA/NLI, please use the following:
    • Download and extract the pretrained models tar file
    • Run evaluation:

  python -m allennlp.run evaluate [--output-file OUTPUT_FILE]
  --cuda-device 0
  --include-package endtasks
  ARCHIVE_FILE INPUT_FILE
  

  • If you want to train your own QA/NLI model:

  python -m allennlp.run train <config_file> -s <serialization_dir> --include-package endtasks
  

See the experiments directory for relevant config files.

Training your own embeddings

• Download the preprocessed corpus if you want to train pair2vec from scratch: ./download_corpus.sh
• Training: This starts the training process which typically takes 7-10 days. It takes in a config file and a directory to save checkpoints.

python -m embeddings.train --config experiments/pair2vec_train.json --save_path <directory>

Miscellaneous

• If you use the code, please cite the following paper

@inproceedings{joshi-etal-2019-pair2vec,
title = "pair2vec: Compositional Word-Pair Embeddings for Cross-Sentence Inference",
author = "Joshi, Mandar and
Choi, Eunsol and
Levy, Omer and
Weld, Daniel and
Zettlemoyer, Luke",
booktitle = "Proceedings of the 2019 Conference of the North {A}merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)",
month = jun,
year = "2019",
address = "Minneapolis, Minnesota",
publisher = "Association for Computational Linguistics",
url = "https://www.aclweb.org/anthology/N19-1362",
pages = "3597--3608"
}