Reinforcement Learning: An Introduction
Sutton and Barto build your intuition from multi-armed bandits all the way to policy gradients with remarkable clarity. This book made temporal difference learning and Bellman equations intuitive for me — it’s the kind of book where every chapter unlocks the next.