Ah it's because I use the tray icon notification and not the e-mail one. I have MultiPing installed on my computer and to me the best and fastest way to get alerted is by the tray icon.

Perhaps a feature for the next release?