About

Table of contents

  1. Goals of the Course
  2. 447 vs. 517
  3. Lectures
  4. Grades
    1. Assignments
    2. Project
  5. Computing
  6. Resources

Goals of the Course

Natural language processing (NLP) seeks to endow computers with the ability to intelligently process human language. NLP components are used in conversational agents and other systems that engage in dialogue with humans, automatic translation between human languages, automatic answering of questions using large text collections and other kinds of data, the extraction of structured information from text, tools that help human authors, and many, many more. This course will teach you the fundamental ideas used in key NLP components. The course aims to integrate building blocks with discussion of cutting-edge technology.

At a high level, the instructor hopes for CSE 517 and 447 students to be able to draw the right inferences from what they see in the news (or a press release, or a job talk) about NLP.

447 vs. 517

The courses are similar in breadth and use the same lecture content. The projects are quite different; 447’s project is a predefined implementation problem that gives teams freedom in developing a solution. It is designed to encourage iterative improvement and an understanding of inherent tradeoffs in building an NLP system. 517’s project is more research-oriented; it asks teams to reproduce experiments in recently published NLP papers. Teams have great flexibility in the choice of a paper to reproduce.

Additionally, there may be differences in the assignments.

Lectures

You are encouraged to come to live lectures and participate in the discussions. The instructor may provide notes, voice recordings, and/or transcripts of the lectures. If there are lecture slides, they’ll be shared on the course website.

Videorecorded lectures from 2021 are provided as a supplementary resource and in case you can’t make it to a lecture. Note that the ordering of the lectures is different year to year; the field changes every year, so what’s presented live will become increasingly different as we move farther away from 2021 in time. Also, be warned that these recorded lectures are “compressed” in the sense that they go about 30% faster than a live lecture with student participation. Most students will prefer to watch them in small segments, pausing frequently to take notes, and perhaps watch some segments again.

  1. Classification and multinomial logistic regression; with captions; transcript; slides
  2. Language modeling, especially with neural networks; with captions; transcript; slides
  3. Vector embeddings for documents and words; with captions; transcript; slides
  4. Morphology and weighted finite-state transducers; with captions; transcript; slides
  5. Sequence labeling and conditional random fields; with captions; transcript; slides
  6. Translation and sequence-to-sequence models; with captions; transcript; slides
  7. Syntax, semantics, and linguistic structure prediction; with captions; transcript; slides

Grades

You will be evaluated based on individually completed assignments (50%) and a project completed in a team of three (50%). Course staff may grant extra credit (up to 5%) to students who actively and meaningfully engage on the course discussion board.

Assignments

There will be six graded assignments, with deadlines every one to two weeks. You are encouraged to complete all of them on time. If you submit an assignment within seven days of the due date, the TAs will calculate the grade you would have received if it had been on time, but you will receive a zero. Your total assignment grade will be calculated as a weighted sum of your six assignments’ grades. Most assignments will have a weight of one; your best assignment will be doubly weighted, and your worst assignment will get a weight of zero. (This means that your highest-graded assignments will be worth one sixth of your grade, the middle four will be worth one twelfth of your grade apiece, and the lowest-graded assignment will be worth 0% of your grade.) Because we have built this slack into the grading system, and because you will still receive feedback on work that is slightly late, there will be no exceptions to our policy of zero credit for late work. Precise assignment deadlines are shown on the course calendar.

  • Assignment 0, which is ungraded, is a tool for you to assess your preparedness for this course.
  • Assignment 1 is designed to advance your understanding of text classification, feature design and selction, the evaluation of classifiers, the mathematics of some important classification models, frequency information in text data, and some mathematical properties of language models.
  • Assignment 2 is designed to advance your understanding of neural networks for NLP.
  • Assignment 3 is designed to advance your understanding of word embeddings and NLP models that make use of them.
  • Assignment 4 is designed to advance your understanding of basic language models.
  • Assignment 5 is designed to advance your understanding of the transformer architecture.
  • Assignment 6 is designed to advance your understanding of text generation with language models.

Solving extra credit problems in an assignment increases your grade for that particular assignment.

NLP is now being deployed in exciting new settings, e.g., code generation. In the spirit of gaining a better understanding of such applications, you are allowed to use code-generation assistants for the assignments. If you do so, you must acknowledge their use the writeup, and include a paragraph describing your use and analyzing the experience. In short, how well did it work and what went wrong? Please also note that using such tools may imply donating your data to the companies that deployed them. Please take reasonable steps to avoid making our assignments easier in future iterations of the course (e.g., once the tool provides a correct answer, don’t give it positive feedback).

Project

Details on the project are given in the instructions document below. Project deliverables must be turned in on time; there will be zero credit for late submissions. Project deadlines are shown on the course calendar.

Computing

Please see information about computing resources here.

Resources

Enrichment lectures and more: