Are we to assume that your good groups are coming from new brass that you have not resized at all? Or, were they FL sized or neck sized before using?
You don't say what brand of "neck" die you have. It's likey that your neck die is sizing down much too far so when the expander button comes out the neck will get "bent", causing more bullet run-out. AND/OR, the neck isn't being expanded quite enough, the "tension " is too high so the bullets actually act as an expander as they are seated. If so, the case-to bullet junction is not supported when seating starts, so you get a lot of run-out.
Suggest you try a Lee Collet neck die, it leaves very straight necks that are big enough to allow bullets to enter properly.