You may have to adjust your velocity with the 180s to get them to stabilize. You also want these bullets seated out in the case so they are near the rifling. There are several ways to get this measurement. Get a cleaning rod that will go from your muzzle all the way to the breach face of your frame. Get a piece of masking tape and place it around the cleaning rod at the muzzle end. Put cleaning rod down the barrel until the tip is against the breach face. Mark the masking tape with a pen at the muzzle end. Now have someone hold your gun for you with the action open. Drop one of your 180 grain bullets down the chamber. Have them watch the bullet as you slowly push the cleaning rod up into the the barrel from the muzzle. When the cast bullets just moves from contact with the cleaning rod, mark your masking tape again. The distance between the marks on your cleaning rod should be your max OAL length. Subtract about .015" and you should have a starting point for seating your bullets.
Some of my contender barrels like my cast bullets sized .309. Others shoot tighter groups with them sized .310. (slower loads are usually the .310s). The Lyman Cast Bullet Manual has some excellent starting loads. Mine is in storage so I cannot help you with loads.
You will want to get a Lyman "M" die or a LEE or RCBS belling die so you can put a slight bell in your brass to get the cast bullet started. Set your seating die to just eliminate that bell. You do NOT need to crimp a cast bullet in the .308. A couple of my barrels have long throats so there is actually lube grooves showing. I just use my sizer with a little harder lube and keep the ammo in a carrier that closes to keep the dirt away.
I have found most of my cast bullet loads to have TWO accuracy sweet spots. One is a relatively mild load, and the next is nearing max. In between loads were so-so and after testing a number of barrels I went right to the lower velocity book loads to look for my mild load and toward the max end of book loads to find my high velocity loads.
If you are using a powder that does not fill the case 75% or more, you want to be selective in primers also. I was using a mild load of AA1680 in a 7TCU and found quite a difference in accuracy between Remington 7 1/2, CCI BR4, Winchester SR, and Federal 205M primers. .5" groups to 3" groups with only difference being the primer. Most of my cast bullet loads are using similar powder burning rates and are usually lit with Winchester primers.
Good luck with your barrel and I hope you get those 180s to shoot.
Steve