When a better human model means worse reward inference

Imagine I just lost a game of chess. You might infer that I’m disappointed or not very good at chess. Without any additional information, you probably wouldn’t infer that I wanted to lose the game. Yet, that is the inference that most inverse reinforcement learning (IRL) methods would make. Nearly...

Pearl's Causal Ladder

Pearl frequently refers to what he calls a “causal ladder”, a hierarchy of three types of problems of increasing difficulty: (1) prediction (2) intervention, and (3) counterfactuals. If we wish to ascend the so-called ladder, and increase the range of causal questions we can answer, it is crucial to understand...

Bounded Optimality

A friend recently asked me why I find bounded optimality interesting. Here’s why:

  1. It is necessary to have a normative framework for how agents should act under computational pressure because this is what the real world is like. In the real world an agent should understand not to...

Kneser-Ney Smoothing

Language modeling is important for almost all natural language processing tasks: speech recognition, spelling correction, machine translation, etc. Today I’ll go over Kneser-Ney smoothing, a historically important technique for language model smoothing.

Language Models

A language model estimates the probability of an n-gram from a training corpus. The simplest...