NLP Fundamentals — Text Classifier(P3)

Word Embeddings

Subword Embeddings and fastText

  • This approach can handle words that did not appear in training data (OOV).
  • The implementation facilitates extremely fast learning on even very large corpora.

Deep Learning for Text Classification

  1. Tokenize the texts and convert them into word index vectors
  2. Pad the text sequences so that all text vectors are of the same length
  3. Map every word index to an embedding vector. We do so by multiplying word index vectors with the embedding matrix. The embedding matrix can either be populated using pre-trained embeddings or be trained for embeddings on this corpus.
  4. Use the output from Step 3 as the input to a neural network architecture.’

CNNs for Text Classification

LSTMs for Text Classification

Text Classification with large pre-trained language models

Learning with No or Less Data, and Adapting to New Domains

Less Training Data: Active Learning and Domain Adaptation

  1. Start with a large, pre-trained language model trained on a large dataset of the source domain (e.g., Wikipedia data).
  2. Fine-tune this model using the target language’s unlabeled data
  3. Train a classifier on the labeled target domain data, by extracting feature representations from the fine-tuned language model from Step 2.

Practical Advice

--

--

--

AI Researcher - NLP Practitioner

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Building a realtime Twitter sentiment dashboard with Firebase and NLP

Convolutional Neural Networks

Improve road lane detection by using multiple images with neural networks

Few practical examples of cheating AI models including GA (Genetic Algorithm) and FGSM (Fast…

Generative Chatbot using Natural Language Processing(NLP)

k-mean clustering and its use cases in security

Training an agent to beat an Atari game with as little as 50 million steps

M2M Day 344: Would most people consider this cruel and unusual punishment?

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Duy Anh Nguyen

Duy Anh Nguyen

AI Researcher - NLP Practitioner

More from Medium

NLP, first steps using spaCy

Using Text to Understand Users: NLP Classification with Reddit

Case study: Smart City Vigilance

Challenges in Designing Emotionally Sentient Agent