MGM, you are right on with most of your post. There simply isn't enough significant data to reach a scientific conclusion on the effect of man's activity on global temperature change at this point in time. We have neither a complete enough historical data base for comparison to modern data, nor a sound enough theory of the modern effects of our activities (emissions, thermal output, etc.) to say whether or not climate is indeed changing significantly due to our activities.
We do know from ice core sample studies that climate on Earth has varied greatly in the past, from centuries-long ice ages that affected up to 1/2 of Earth's landmass, to the existence of vast deserts across a great portion of the Northern Hemisphere, to tropical rainforests ranging as far north as present-day New York State. We also know that these climate changes have sometimes occurred quickly in the geologic time scale, such as mini-ice ages that have descended in a matter of decades.
I watched an old episode of "In Search Of", a quasi-science series from the late 70s and early 80s hosted by Leonard Nimoy, just a few months ago. This program was recorded in the late 70s, and the episode was focused on the theory that a new ice age was descending upon Earth. Evidence was a rash of severe cold that gripped North America for several years at the time. I chuckled to myself that 25 years later we are deep into a debate over global warming. Many climatologists have in fact theorized that the current "global warming" is a mere hiccup in a general trend toward a new ice age that will manifest in the next 300 or so years.
The fact is, we are only beginning to unravel the mystery of climate and its causes. It is far too presumptuous for anyone to predict any climate trends based on a few centuries of data, much of which is limited in scope and accuracy.