NLP Fundamentals — Text Classifier(P3)

Word Embeddings

Subword Embeddings and fastText

  • This approach can handle words that did not appear in training data (OOV).
  • The implementation facilitates extremely fast learning on even very large corpora.

Deep Learning for Text Classification

  1. Tokenize the texts and convert them into word index vectors
  2. Pad the text sequences so that all text vectors are of the same length
  3. Map every word index to an embedding vector. We do so by multiplying word index vectors with the embedding matrix. The embedding matrix can either be populated using pre-trained embeddings or be trained for embeddings on this corpus.
  4. Use the output from Step 3 as the input to a neural network architecture.’

CNNs for Text Classification

LSTMs for Text Classification

Text Classification with large pre-trained language models

Learning with No or Less Data, and Adapting to New Domains

Less Training Data: Active Learning and Domain Adaptation

  1. Start with a large, pre-trained language model trained on a large dataset of the source domain (e.g., Wikipedia data).
  2. Fine-tune this model using the target language’s unlabeled data
  3. Train a classifier on the labeled target domain data, by extracting feature representations from the fine-tuned language model from Step 2.

Practical Advice

--

--

--

AI Researcher - NLP Practitioner

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Implementing Kronecker Product Decomposition with NumPy

Getting Start in Sentiment Analysis using NLP

OCR for Invoice Digitization Is No Longer the Only Key Ingredient of Accuracy

Google AutoML Vision for Image Classification

An Overview of ResNet and its Variants

Improve pedestrian detection by using general object detector with Cascade RCNN

Way to Go — — Things to consider after building a machine learning classification model

Unblurring defocused images using multi-branch neural networks

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Duy Anh Nguyen

Duy Anh Nguyen

AI Researcher - NLP Practitioner

More from Medium

NLP with Disaster Tweets

How has Criterion Evolved and Embraced OTT?

Making Sense of Inverse Data Flow — React

Application Framework