Sounds like your shoulders are not lined up on your target line and you are reaching. Please go see a pro and get this straightened out. We cannot correct you issues online.

Hi Andy,

Make sure you pick a spot behind and in front of the ball. Just like drawing a line on a piece of paper the third reference point helps not only your alignment but gives you a mental path for the clubhead.

I do something similar. It could be that you are seeing that the club face is aligned to your target point, but your buddies see your shoulder or body alignment going someplace else. You could also try picking a spot a few feet instead of inches. The further out you can pick a spot, the more accurate the imaginary line will be to the target. In terms of shooting accurately with a rifle, they would call that "sight radius".

I was having a similar issue with my driver. Clubhead, feet etc all lined up but had my shoulders open - since working on my shoulders I'm much more accurate off the tee.
It feels really wrong to start with but becomes the norm with practice.

Andy
You didn't say how it works for you. If it works, and your buddies are criticizing, you're probably kicking their butts out there. Smile and enjoy it.
If your shot aren't hitting their targets, get back with us and we'll give you a bunch of amateur advice that will really mess you up, and your buddies will be the ones smiling. Then you'll be forced to find a good swing coach.