I draw visuals in order to understand concepts. Sometimes friends have found these helpful, so I thought I'd make them public.
These are raw notes from when I was learning each subject (sometimes many many years ago!), so they could be wrong! If you have a question about something, feel free to ask.
That moment when it clicks why KL divergence between and equals mutual information ! Thanks to Chris Olah for the rain/sun coat/no coat joint distribution example shown here.