The first time you "select a target" it isn't really selecting the target at all, you are just saying you want to perform an action.
When you select your spell to cast, then you get a purple spinning icon over the target, that is the real "targeting" icon. It defaults to the last thing you had...