- Distributed Word Representations
- Distributed Sentence Representations
- Entity Recognition (Sequence Tagging)
- Language Model (LM for pre-training)
- Machine Translation
- Question Answering (Machine Reading Comprehension)
- Recommendation Systems
- Relation Extraction
- Sentences Matching (Natural Language Inference/Textual Entailment)
- Text Classification (Sentiment Classification)
- Materials/Toolkits
- 2017-11
- Faruqui and Dyer - 2014 - Improving vector space word representations using multilingual correlation [pdf] [note]
- Maaten and Hinton - 2008 - Visualizing data using t-SNE [pdf] [pdf (annotated)] [note]
- Ling et al. - 2015 - Finding function in form: Compositional character models for open vocabulary word representation [pdf] [pdf (annotated)] [note]
- Bojanowski et al. - 2016 - Enriching word vectors with subword information [pdf] [pdf (annotated)] [note]
- 2017-12
- Bengio and Senécal - 2003 - Quick Training of Probabilistic Neural Nets by Importance Sampling [pdf] [pdf(annotated)] [note]
- references
- 2017-11
- Le and Mikolov - 2014 - Distributed representations of sentences and documents [pdf] [pdf (annotated)] [note]
- 2018-12
- Li and Hovy - 2014 - A Model of Coherence Based on Distributed Sentence Representation [pdf] [pdf (annotated)] [note]
- Kiros et al. - 2015 - Skip-Thought Vectors [pdf] [pdf (annotated)] [note]
- Hill et al. - 2016 - Learning Distributed Representations of Sentences from Unlabelled Data [pdf] [pdf (annotated)] [note]
- Arora et al. - 2016 - A simple but tough-to-beat baseline for sentence embeddings [pdf] [pdf (annotated)] [note]
- Pagliardini et al. - 2017 - Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features (sent2vec) [pdf] [pdf (annotated)] [note]
- Logeswaran et al. - 2018 - An efficient framework for learning sentence representations (Quick-Thought Vectors) [pdf] [pdf (annotated)] [note]
- 2019-01
- Wieting et al. - 2015 - Towards universal paraphrastic sentence embeddings [pdf] [pdf (annotated)] [note]
- Adi et al. - 2016 - Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks [pdf] [pdf (annotated)] [note]
- Conneau et al. - 2017 - Supervised Learning of Universal Sentence Representations from Natural Language Inference Data (InferSent) [pdf] [pdf (annotated)] [note]
- Cer et al. - 2018 - Universal Sentence Encoder [pdf] [pdf (annotated)] [note]
- references
- 2018-10
- Lample et al. - 2016 - Neural Architectures for Named Entity Recognition [pdf]
- Ma and Hovy - 2016 - End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF [pdf]
- Yang et al. - 2017 - Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks [pdf]
- Peters et al. - 2017 - Semi-supervised sequence tagging with bidirectional language models [pdf]
- Shang et al. - 2018 - Learning Named Entity Tagger using Domain-Specific Dictionary [pdf]
- references
- 2017-11
- 2019-02
- Peters et al. - 2018- Deep contextualized word representations(ELMo) [pdf] [note]
- Howard and Ruder - 2018 - Universal language model fine-tuning for text classification(ULMFit) [pdf]
- Radford et al. - 2018 - Improving language understanding by generative pre-training [pdf]
- Devlin et al. - 2018 - Bert: Pre-training of deep bidirectional transformers for language understanding [pdf]
- references
- Blog:The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning)
- ELMo
- Quick Start: Training an IMDb sentiment model with ULMFiT
- finetune-transformer-lm: Code and model for the paper "Improving Language Understanding by Generative Pre-Training"
- BERT
- google-research/bert: officical TensorFlow code and pre-trained models for BERT
- huggingface/transformers provides state-of-the-art general-purpose architectures (BERT, GPT-2, RoBERTa, XLM, DistilBert, XLNet, CTRL...) for NLU and NLG using TensorFlow 2.0 and PyTorch.
- awesome-bert: bert nlp papers, applications and github resources ， BERT 相关论文和 github 项目
- 2017-12
- Oda et al. - 2017 - Neural Machine Translation via Binary Code Predict [pdf] [note]
- Kalchbrenner et al. - 2016 - Neural machine translation in linear time [pdf] [pdf (annotated)] [note]
- 2018-05
- Sutskever et al. - 2014 - Sequence to Sequence Learning with Neural Networks [pdf]
- Cho et al. - 2014 - Learning Phrase Representations using RNN Encoder-Decoder for NMT [pdf]
- Bahdanau et al. - 2014 - NMT by Jointly Learning to Align and Translate [pdf]
- Luong et al. - 2015 - Effective Approaches to Attention-based NMT [pdf]
- 2018-06
- Gehring et al. - 2017 - Convolutional sequence to sequence learning [pdf]
- Vaswani et al. - 2017 - Attention is all you need [pdf] [note1:The Illustrated Transformer] [note2:The Annotated Transformer]
- references
- 2018-03
- 2018-04
- Clark and Gardner. - 2017 - Simple and Effective Multi-Paragraph Reading Comprehension [pdf]
- Wang et al. - 2017 - Gated Self-Matching Networks for Reading Comprehension and Question Answering [pdf]
- Yu et al. - 2018 - QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension [pdf]
- references
- 2019-05
- Rendle S. - 2010 - Factorization machines [pdf] [note]
- Cheng et al. - 2016 - Wide & Deep Learning for Recommender Systems [pdf]
- Guo et al. - 2017 - DeepFM: A Factorization-Machine based Neural Network for CTR Prediction [pdf]
- He and Chua. - 2017 - Neural Factorization Machines for Sparse Predictive Analytics [pdf]
- references
- 2018-08
- Mintz et al. - 2009 - Distant supervision for relation extraction without labeled data [pdf]
- Zeng et al. - 2015 - Distant Supervision for Relation Extraction via Piecewise Convolutional Neural Networks [pdf]
- Zhou et al. - 2016 - Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification [pdf]
- Lin et al. - 2016 - Neural Relation Extraction with Selective Attention over Instances [pdf]
- 2018-09
- references
- 2017-12
- Hu et al. - 2014 - Convolutional neural network architectures for Matching Natural Language Sentences [pdf] [pdf (annotated)] [note]
- 2018-07
- Nie and Bansal - 2017 - Shortcut-Stacked Sentence Encoders for Multi-Domain Inference [pdf] [note]
- Wang et al. - 2017 - Bilateral Multi-Perspective Matching for Natural Language Sentences [pdf] [note]
- Tay et al. - 2017 - A Compare-Propagate Architecture with Alignment Factorization for Natural Language Inference [pdf]
- Chen et al. - 2017 - Enhanced LSTM for Natural Language Inference [pdf] [note]
- Ghaeini et al. - 2018 - DR-BiLSTM: Dependent Reading Bidirectional LSTM for Natural Language Inference [pdf]
- references
- 2017-09
- Joulin et al. - 2016 - Bag of tricks for efficient text classification [pdf] [pdf (annotated)] [note]
- 2017-10
- Kim - 2014 - Convolutional neural networks for sentence classification [pdf] [pdf (annotated)] [note]
- Zhang and Wallace - 2015 - A Sensitivity Analysis of (and Practitioners' Guide to) Convolutional Neural Networks for Sentence Classification [pdf] [pdf (annotated)] [note]
- Zhang et al. - 2015 - Character-level convolutional networks for text classification [pdf] [pdf (annotated)] [note]
- Lai et al. - 2015 - Recurrent Convolutional Neural Networks for Text Classification [pdf] [pdf (annotated)] [note]
- Yang et al. - 2016 - Hierarchical attention networks for document classification [pdf]
- 2017-11
- Iyyer et al. - 2015 - Deep unordered composition rivals syntactic methods for Text Classification [pdf] [pdf (annotated)] [note]
- 2019-04 (Aspect level sentiment classification)
- Wang et al. - 2016 - Attention-based LSTM for aspect-level sentiment classification [pdf]
- Tang et al. - 2016 - Aspect level sentiment classification with deep memory network [pdf]
- Chen et al. - 2017 - Recurrent Attention Network on Memory for Aspect Sentiment Analysis [pdf]
- Xue and Li - 2018 - Aspect Based Sentiment Analysis with Gated Convolutional Networks [pdf]
- references
-
optimization algorithms
-
StanfordNLP: Official Stanford NLP Python Library for Many Human Languages
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent