Skip to main content Link Search Menu Expand Document (external link)
Natural Language Processing

About

Table of contents

  1. Goals of the Course
  2. 447 vs. 517
  3. Lectures
  4. Grades
    1. Assignments
    2. Project
  5. Computing
  6. Resources

Goals of the Course

Natural language processing (NLP) seeks to endow computers with the ability to intelligently process human language. NLP components are used in conversational agents and other systems that engage in dialogue with humans, automatic translation between human languages, automatic answering of questions using large text collections and other kinds of data, the extraction of structured information from text, tools that help human authors, and many, many more. This course will teach you the fundamental ideas used in key NLP components. It is organized into several “greatest hits” topics, each with a more-or-less self-contained lecture and associated readings, problems, and implementation exercises.

At a high level, the instructor hopes for CSE 517 students to be able to draw the right inferences from what they see in the news (or a press release, or a job talk) about NLP.

447 vs. 517

The courses are similar in breadth and use the same lecture content. The projects are quite different; 447’s project is a predefined implementation problem that gives teams freedom in developing a solution. It is designed to encourage iterative improvement and an understanding of inherent tradeoffs in building an NLP system. 517’s project is more research-oriented; it asks teams to reproduce experiments in recently published NLP papers. Teams have great flexibility in the choice of a paper to reproduce.

Additionally, there will be differences in the assignments. In 2023, CSE 447 and CSE 517 are being taught separately.

Lectures

You are encouraged to come to live lectures and participate in the discussions. Videorecorded lectures from 2021 are provided as a supplementary resource and in case you can’t make it to a lecture (e.g., because you’re not feeling well). Note that the ordering of the lectures may be slightly different year to year, especially later in the quarter. Note that the field changes every year, so what’s presented live will become increasingly different as we move farther away from 2021 in time. Also, be warned that these lectures are “compressed” in the sense that they go about 30% faster than a live lecture with student participation. Most students will prefer to watch them in small segments, pausing frequently to take notes, and perhaps watch some segments again.

  1. Classification and multinomial logistic regression; with captions; transcript; slides
  2. Language modeling, especially with neural networks; with captions; transcript; slides
  3. Vector embeddings for documents and words; with captions; transcript; slides
  4. Morphology and weighted finite-state transducers; with captions; transcript; slides
  5. Sequence labeling and conditional random fields; with captions; transcript; slides
  6. Translation and sequence-to-sequence models; with captions; transcript; slides
  7. Syntax, semantics, and linguistic structure prediction; with captions; transcript; slides

Grades

You will be evaluated based on individually completed assignments (50%) and a project completed in a team of three (50%). Course staff may grant extra credit (up to 5%) to students who actively and meaningfully engage on the course discussion board.

Assignments

There will be five graded assignments, roughly one every two weeks. You are encouraged to complete all of them on time. If you submit an assignment within seven days of the due date, the TAs will calculate the grade you would have received if it had been on time, but you will receive a zero. Your total assignment grade will be calculated as a weighted sum of your five assignments’ grades. Most assignments will have a weight of one; your best assignment will be doubly weighted, and your worst assignment will get a weight of zero. (This means that your highest-graded assignments will be worth 20% of your grade, the middle three will be worth 10% of your grade apiece, and the lowest-graded assignment will be worth 0% of your grade.) Because we have built this slack into the grading system, and because you will still receive feedback on work that is slightly late, there will be no exceptions to our policy of zero credit for late work. Precise assignment deadlines are shown on the course calendar.

  • Assignment 0, which is ungraded, is a tool for you to assess your preparedness for this course.
  • Assignment A is designed to advance your understanding of text classification, feature design and selction, the evaluation of classifiers, the mathematics of some important classification models, frequency information in text data, and some mathematical properties of language models.
  • Assignment B is designed to advance your understanding of language models, word embeddings, and NLP models that make use of them.
  • Assignment C is designed to advance your understanding of ethical matters that arise in NLP and sequence models.
  • Assignment D is designed to give you more practical hands-on experience with sequence labeling and understanding of structured problems in NLP that deal with segmentation, trees, and logical forms.
  • Assignment E is designed to give you more practical hands-on experience with sequence-to-sequence methods.

Solving extra credit problems in an assignment increases your grade for that particular assignment.

NLP is now being deployed in exciting new settings, e.g., code generation. In the spirit of gaining a better understanding of such applications, you are allowed to use code-generation assistants for the assignments. If you do so, you must acknowledge their use the writeup, and include a paragraph describing your use and analyzing the experience. In short, how well did it work and what went wrong? Please also note that using such tools may imply donating your data to the companies that deployed them. Please take reasonable steps to avoid making our assignments easier in future iterations of the course (e.g., once the tool provides a correct answer, don’t give it positive feedback).

Project

Details on the project are given in the instructions document below. Project deliverables must be turned in on time; there will be zero credit for late submissions. Project deadlines are shown on the course calendar.

Computing

Please see information about computing resources here.

Resources

Enrichment lectures and more: