When a better human model means worse reward inference

Imagine I just lost a game of chess. You might infer that I’m disappointed or not very good at chess. Without any additional information, you probably wouldn’t infer that I wanted to lose the game. Yet, that is the inference that most inverse reinforcement learning (IRL) methods would make. Nearly...

Pearl's Causal Ladder

Pearl frequently refers to what he calls a “causal ladder”, a hierarchy of three types of problems of increasing difficulty: (1) prediction (2) intervention, and (3) counterfactuals. If we wish to ascend the so-called ladder, and increase the range of causal questions we can answer, it is crucial to understand...

Paradigm Shift

She stared at the keys of her laptop. It wasn’t that she had nothing to write about—so much had happened. Her mind had effortfully packaged the ideas into words that were ready to be transmitted. However, her hands, which rested upon her keyboard, remained unresponsive, deaf to the requests sent...

Bounded Optimality

A friend recently asked me why I find bounded optimality interesting. Here’s why:

  1. It is necessary to have a normative framework for how agents should act under computational pressure because this is what the real world is like. In the real world an agent should understand not to...

Kneser-Ney Smoothing

Language modeling is important for almost all natural language processing tasks: speech recognition, spelling correction, machine translation, etc. Today I’ll go over Kneser-Ney smoothing, a historically important technique for language model smoothing.

Language Models

A language model estimates the probability of an n-gram from a training corpus. The simplest...