This is our code for the paper Geometry-Aware Supertagging with Heterogeneous Dynamic Convolutions.

While unpublished, you can cite the arxiv preprint:

@misc{https://doi.org/10.48550/arxiv.2203.12235,
  doi = {10.48550/ARXIV.2203.12235},  
  url = {https://arxiv.org/abs/2203.12235},
  author = {Kogkalidis, Konstantinos and Moortgat, Michael},
  keywords = {Computation and Language (cs.CL), Machine Learning (cs.LG), FOS: Computer and information sciences, FOS: Computer and information sciences},
  title = {Geometry-Aware Supertagging with Heterogeneous Dynamic Convolutions},
  publisher = {arXiv},
  year = {2022},
  copyright = {Creative Commons Attribution 4.0 International}
}

The model presents a new approach to constructive supertagging, based on an explicit graph representation that accounts for both the intra-tree interactions between a single supertag and the inter-tree interactions between (partially decoded) supertag sequences. To account for the disparity between the various modalities in the graph (i.e. sentential word order, subword contextualized vectors, tree-sequence order and intra-tree edges) we adopt a heterogeneous formulation. Decoding is performed in parallel over trees, each temporal step associated with an increased tree depth. Statefulness is achieved by representing each partially decoded tree with a single state-tracking vector, which is updated twice at each step: once with feedback from its own tree's last decoded fringe, and once with feedback from surrounding trees. The result is a highly parallel yet partially auto-regressive architecture with input-scaling memory complexity and near-constant decoding time that achieves new state-of-the-art scores on four datasets while retaining the ability to predict rare supertags reliably.

Averages of 6 repetitions compared to recent results on the respective datasets (crawled 23/03/2022).

dyngraphst.neural contains the model architecture , and dyngraphst.data contains the data preprocessing code; see the READMEs of the respective directories for more details. While anticipating the next stable release of PyTorchGeometric, the code will require two distinct python environments: python3.9 with pytorch 1.10.2 and torch_geometric 2.0.3 for training/inference, and python3.10 for data processing and evaluation. The two will be coallesced at a later stage.

Detailed instructions coming soon.

If you have any questions or comments or would like a grammar/language specific pretrained model, feel free to get in touch.