Yes a patch ball would work better but there are problems.
First because of the greater and greater size of cannon balls you will find it harder and harder to properly seat the ball due to the greater friction of the greater area of the greater circumference of the larger ball. The more the fouling the more the difficult to seat the ball. if you can even seat the ball at all.You would problay need a sledge hammer to seat the ball. And heaven forbid if you had to pull the ball. Cannoneers discovered the problem years ago and always provided windage.
The second problem is the dynamics of internal ballistics. If you tight seal these larger bores you will raise pressures substantially. Since pressures are measured in square inches think about the number of square inches in the chamber of a .50 cal chamber versus a 2 inch chamber. Now add resistance to the release of the pressure in the chamber. The larger resistance you had seating the ball now comes into play unseating or firing the ball.
This is also why in these larger bores lead balls are not recommended. The larger balls in lead require greater pressure to overcome the inertia of their greater mass versus balls of Zinc. In our reference section we have posted a wonderful round ball weight calculator created by GGaskill. Plug in some ever increasing diameters and sees how the weights substantially run up real fast. George's Round Ball Calculator
http://lassengunsmithing.com/html/ShotWeight.htm.
Another factor is that the cannon is an area weapon. They weren't intend to hit a pinpoint target.
Should you make a trip to to the East coast I strongly recommend you time your trip so you can go to Ft. Shenandoah and see the N-SSA nationals and the cannon shoots. These guys don't use a patch. They aren't using a full 1/40 windage either, but windage they do have. But the accuracy these guys and gals get is incredible.