A near to max load of H-4350 and a 120 gr Nosler ballistic tip is all you will need for yote/deer out to around 600 yards. If all my shots were going to be over 400 yards I would go with the 140 Hornady A-Max or the 130 Berger VLD Hunting. No bullet in .264 from a 260 Rem will be fur friendly. I would not go shorter than 24" for a barrel myself. My buddy got one of the first Rem 700 SS rifles to come off the line in 260 Rem. It had a 24" barrel. This was before you could even get a hold of ammo. We made cases out of Rem 243 Win cases because the necks come out long enough. 308 cases necks come out really short. There was no load data out at the time so we were WILDCATTING. The loads we settled on were a bit more H-4350 than what the manuals call for these days but according to all the pressure signs we were OK with our loads. The fellow that my buddy eventually traded the rifle to, ( he never keeps anything long) has been using the same loads all these years and is still smacking and stacking deer every year with it with no problems. I shoot a 6.5X55 Swede which does the same velocity as the 260 Rem quite a bit. I shoot groundhogs and deer with it. I have shot many different bullets and have settled on just one load with the 120 Nosler ballistic tip at 3000 fps. Most of my shots are under 400 yards. You don't get the RED MIST effect on groundhogs that the lighter bullets like the Sierra 100 gr HP gives you but they are just as dead. On deer it is about as close to perfect a bullet you can find. For deer size game the 140 Nosler Accubond may be too much of a good thing for longer ranges. I shoot the 130 accubond in my 264 Win mag at 3350 fps and it does good on deer from 80 out to 250 yards which is as far as I have shot one with it in the two years I have been using it. For game like elk either of the accubonds would be great but I would keep my shots inside 300 yards or so with the 260 Rem. As for barrel twist, if you are going to be shooting 140 gr bullets you need an 8 twist. If 130 gr and under a 9 twist will do fine. The more I shoot the .264 calibers the better I like them. By the way I used to help my buddy do crop damage control on a big farm in NC for 15 years where we had to take 100 deer each year. When I said I tried different bullets and loads, I gave them a pretty good work out. I have killed deer with about everything from an arrow through the 45-70. Good luck and good shooting.
