Yes, I've stalked vermin here with a rimfire, but only when I could do so on the properties of friends and wasn't paying a heap for it. I don't get many of those opportunities anymore and large scale culling opportunities that do exist (for vermin) aren't attractive to me.
Dave, you are correct, it's fun and often very challenging to hunt the smaller species. The smaller antelope species do not get the recognition that they deserve and are often cheaper than the larger species and the mounts take up less space in the bedroom at home! (See my other post in reply to the question on $$). But they usually need to form part of a safari to make it worthwhile for the operator and PH.
Smaller antelope (no real "deer" here that I'm aware of except introduced deer), like Duiker (many species ranging from really tiny that you'd fit in a shoebox to larger and some very exotic), Steenbuck, Klipspringer and Oribi (threatened status in some areas I think - JJ can correct me) etc. have all been hunted with rimfires (.22 LR), but that's generally not legal that I'm aware of and they really deserve a centrefire.
Grumulkin is correct - rimfires are illegal for much except birds and vermin, which depends on where you are and what the specifics are. Either way, I doubt you'll likely be able to legally hunt much more than some birds, rabbits, springhare and rock hyrax ("dassie"). The smaller antelope and Jackal can be successfully hunted with the .222 and larger and yes, the Hornet is quite popular for this in some areas where the range is not long (but not my choice) and for things like rock hyrax. The Hornet is also used for night time culling by some as Grumulkin point out, but generally the .222 and larger is preferred.
In short, the 17HMR, .22 LR and .22 Mag are for small vermin in my opinion. These are species usually incidental to a hunt and to bring a rifle and pay day rates for that is unlikley to be attractive to you, but you could certainly enquire.
Not to generalise too much, but in most areas the small species are not that numerous that they are a primary choice for "camp meat" and of course they are small.
In short a hunt of smaller antelope species is really worth looking at, particularly if you've done and can combine with some of the others. But I'd not use a rimfire and probably not a Hornet.