Raphaël Millière on the Limits of Deep Learning and AI x-risk Skepticism

Raphaël Millière is presented as a skeptic of AI existential risk who still follows current deep learning research closely and is impressed by recent progress. The post centers on his claim that limits in understanding, compositionality, and generalization make it hard to extrapolate from current models to AGI.

Reading notes

#

• The interview is meant to steelman skepticism about AI existential risk by focusing on someone who is impressed by current progress but still dismisses x-risk from AI.
• Millière’s objections are linked to limits in deep learning around understanding, compositionality, and generalization.
• He uses François Chollet’s distinction between local generalization and broad generalization.
• Local generalization means handling unseen examples within a specific task.
• Broad generalization means adapting to unknown unknowns across many tasks, including novel situations and long-tail edge cases.
• He suggests current language models may show few-shot learning without reaching broad generalization.
• He says arithmetic tasks may still be within the training distribution of GPT-3 and PaLM.
• He argues scaling plots measure loss reduction in autoregressive models, not intelligence itself.
• He warns against extrapolating those plots to human general intelligence.
• He says recent systems such as DALL·E 2, Gato, PaLM, and Imagen still involved at least minor architectural changes.
• He notes that data format and modality handling matter, including serialization of discrete and continuous data in Gato.
• He says models cannot learn anything without inductive bias, and that the amount of prior knowledge required is the key question.
• He would change his mind if transformer-like systems reached human level on hard benchmarks such as ARC, BIG-bench tasks, and Winoground with minimal architectural changes.
• He says it is consistent to be impressed by current systems while still discussing their limits.
• He thinks progress toward more general intelligence is happening, but that humans still show an extreme level of generalization models have not reached.