This event has ended. Visit the official site or create your own event on Sched.

The Seventh Annual Tom Tom Founders Festival is a week-long celebration of innovators, visionaries, and artists who are shaping small cities. The Festival occurs at dozens of venues throughout downtown Charlottesville. 

View analytic
Thursday, April 12 • 11:30am - 12:00pm
Embed, Encode, Attend, Predict: A four-step framework for understanding neural network approaches to Natural Language Understanding problems

Log in to save this to your schedule and see who's attending!

Feedback form is now closed.
While there is a wide literature on developing neural networks for natural language understanding, the networks all have the same general architecture, determined by basic facts about the nature of linguistic input. In this talk I name and explain the four components (embed, encode, attend, predict), give a brief history of approaches to each subproblem, and explain two sophisticated networks in terms of this framework -- one for text classification, and another for textual entailment. The talk assumes a general knowledge of neural networks and machine learning. The talk should be especially suitable for people who have been working on computer vision or other problems. Just as computer vision models are designed around the fact that images are two or three-dimensional arrays of continuous values, NLP models are designed around the fact that text is a linear sequence of discrete symbols that form a hierarchical structure: letters are grouped into words, which are grouped into larger syntactic units (phrases, clauses, etc), which are grouped into larger discursive structures (utterances, paragraphs, sections, etc). Because the input symbols are discrete (letters, words, etc), the first step is "embed": map the discrete symbols into continuous vector representations. Because the input is a sequence, the second step is "encode": update the vector representation for each symbol given the surrounding context. You can't understand a sentence by looking up each word in the dictionary --- context matters. Because the input is hierarchical, sentences mean more than the sum of their parts. This motivates step three, attend: learn a further mapping from a variable-length matrix to a fixed-width vector, which we can then use to predict some specific information about the meaning of the text.

avatar for Matthew Honnibal

Matthew Honnibal

Founder, Explosion AI
Matthew Honnibal is the creator and lead developer of spaCy, one of the most popular libraries for Natural Language Processing. He has been publishing research on NLP since 2005, with a focus on syntactic parsing and other structured prediction problems. He left academia to start... Read More →

avatar for Capital One

Capital One

Capital One is a diversified bank that offers a broad array of financial products and service to consumers, small businesses and commercial clients. 
avatar for S&P Global

S&P Global

S&P Global Inc. (prior to April 2016 McGraw Hill Financial, Inc., and prior to 2013 McGraw Hill Companies) is an American publicly traded corporation headquartered in New York City. Its primary areas of business are financial information and analytics. It is the parent company of S&P Global Ratings, S&P Global Market Intelligence, and... Read More →

Twitter Feed