The only thing that comes to my mind is to verify if your whitelist file is correctly written.
Maybe you have some invisible characters or some incorrect charset (it depends by which editor you used to write your file)?
Or maybe the last line of your text file is not terminated with a newline? This happens pretty often...
try to delete your whitelist file, rewrite it with an editor like vim or nano and restart amavisd.
Let me know 