Proximal Policy Optimization (PPO) was introduced in 2017 for simulated robots and Atari games, it became the optimization engine behind reinforcement learning from human feedback (RLHF) for large language models.

"The agent that learns to predict its own future is the agent that learns to control it." — Richard Sutton

the Bellman equation captures one profoundly simple insight: the value of being somewhere now equals the immediate reward you receive, plus the discounted value of wherever you end up next. That recursive observation, formalized by the American mathematician Richard Ernest Bellman in the early 1950s, transformed decision-making theory and quietly became the mathematical backbone of modern artificial intelligence.

From self-driving cars to game-playing machines to self-learning robots—how intelligent systems learn by interacting with the world