RL theory I [3 P]

Prove Corollary 1.3 (p. 9) from the script Theory of Reinforcement Learning ³:

Every policy $\pi$ for which $V^{\pi}$ satisfies the Bellman optimality equations

$$V^{\pi}(s) = \max_{a \in A_s} Q^{\pi}(s, a) \forall s \in S$$

is optimal.