Compound Word Transformer

Authors: Wen-Yi Hsiao, Jen-Yu Liu, Yin-Cheng Yeh and Yi-Hsuan Yang

Paper (arXiv) | Audio demo (Google Drive) | Blog

Official PyTorch implementation of AAAI2021 paper "Compound Word Transformer: Learning to Compose Full-Song Musicover Dynamic Directed Hypergraphs".

We presented a new variant of the Transformer that can processes multiple consecutive tokens at once at a time step. The proposed method can greatly reduce the length of the resulting sequence and therefore enhance the training and inference efficiency. We employ it to learn to compose expressive Pop piano music of full-song length (involving up to 10K individual to23 kens per song). In this repository, we open source our Ailabs.tw Pop17K dataset, and the codes for unconditional generation.

Dependencies

• python 3.6
• Required packages:
  • madmom
  • miditoolkit
  • pytorch-fast-transformers

chorder is our in-house rule-based symbolic chord recognition algorithm, which is developed by our former intern - joshuachang2311. He is also a jazz pianist 🎹.

Model

In this work, we conduct two scenario of generation:

• unconditional generation

  • To see the experimental results and discussion, please refer to here.

• conditional generation, leadsheet to full midi (ls2midi)

  • [Work in progress] We plan to open source the code associated with this part in the future.
    • melody extracyion (skyline)
    • objective metrics
    • model

Dataset

To prepare your own training data, please refer to documentaion for further understanding.
Or, you can start with our AIlabs.tw Pop17K, which is available here.

Acknowledgement

• PyTorch codes for transformer-XL is modified from kimiyoung/transformer-xl.
• Thanks Yu-Hua Chen and Hsiao-Tzu Hung for helping organize the codes.