Low-Stakes Alignment

AI Safety Fundamentals: Alignment

Right now I’m working on finding a good objective to optimize with ML, rather than trying to make sure our models are robustly optimizing that objective. (This is roughly “outer alignment.”) That’s pretty vague, and it’s not obvious whether “find a good objective” is a meaningful goal rather than being inherently confused or sweeping key distinctions under the rug. So I like to focus on a more precise special case of alignment: solve alignment when decisions are “low stakes.” I think this case effectively isolates the problem of “find a good objective” from the problem of ensuring robustness and is precise enough to focus on productively. In this post I’ll describe what I mean by the low-stakes setting, why I think it isolates this subproblem, why I want to isolate this subproblem, and why I think that it’s valuable to work on crisp subproblems.

Source:

https://www.alignmentforum.org/posts/TPan9sQFuPP6jgEJo/low-stakes-alignment

Narrated for AI Safety Fundamentals by TYPE III AUDIO.

---

A podcast by BlueDot Impact.
Learn more on the AI Safety Fundamentals website.

Hoofdstukken

1. Low-Stakes Alignment (00:00:00)

2. 1. What is the low-stakes setting? (00:01:07)

3. 2. Why do low stakes require only outer alignment? (00:01:49)

4. 3. Why focus on this subproblem first? (00:03:26)

5. 4. Is the low-stakes setting actually scary? (00:05:10)

6. 5. Why focus on "low stakes" rather than "outer alignment"? (00:06:09)

7. 6. More formal definition of low-stakes (00:07:23)

8. 7. More formal argument that outer alignment is sufficient (00:08:55)

9. 8. Why expect SGD to work online even for neural networks? (00:11:38)