Affect for me, is something you experience inside first - it could be the sudden change of your emotional state or a memory arriving out of knowhere, but you become aware of the fact of it before you become aware of the cause.

Effect is something that hits you from outside - you see it coming and know that it is going to change you.