Yea I have a little single six that I purchased from Flatlander (much to 2dogs dismay) that is just a heck of a lot of fun. It's funny and obvious of course that shooting my 32 mags makes me better with my 45 colts! It's all sight and trigger control without being knocked about at all. I have to admit that a Dillon set up in 32 mag would be nice rather than the single stage RCBS I have used for 20 years now. I have found basically 3 loads I shoot 4.0gr W231 with a 95gr RNFP for a plinking round and 11gr W296 with a 100gr XTP bullet (rarely) or most of the time a 120gr LBTGC bullet that really shoots well. These shoot very well in both my SS and my Buckeye Bisley, now I just need a holster for the little SS.
Besides Lloyd that little 32 kick should not bother your back at all.