Why was he doubly irritated?

Because he had forgotten and because he remembered that he had reminded himself twice not to forget.

James Joyce - Ulysses