Invasion of privacy, am I in the wrong? (Long Q)?

Kind of. If you didn't want him to know about your appointment, you shouldn't have left the paper lying around for him to see. Of course he will be suspicious if you say you don't want to talk about it, no matter which way you put it. You probably shouldn't have read his emails. If he was on a dating website while he was with you, that's a bad sign from the start! Trust is an important issue, so is being able to talk to your partner about anything and everything.