no, it doesn't have to come out. It is turns to allow camber adjustment(the hole is slightly off-set). the best advise I can give is"GET A BIGGER HAMMER" if that doesn't work then apply heat"torch" to the axle housing around where the ball joint stud sticks out. This should help but your still going to HAVE to use a HAMMER.

Frank