In rifle ammo, crimps are only needed to hold the bullet in place during cycling or recoil, just as you stated. In a bolt action, a crimp will often impair accuracy.
Neck tension is quite a different subject. The bullet needs to be firmly seated (neck tension) so the powder will ignite properly. Loose or worse yet, inconsistent neck tension will result in non-uniform chamber pressures, thus inconsistent velocities. Of course this is will show up down range as poor groups.
A good way to insure uniform neck tension is to carefully inspect your brass after it has been sized. Examine the case mouths and look for splits, potential splits, and thin areas. When you are seating bullets, if you detect an unusually light or heavy pull on the press handle, you probably found a bad case.
Being an old bench rest shooter, I get anal about cases. If one doesn't look right I'll throw it in the cull box or the trash can.