I don't know where you live, must most larger cities have either a historic society, an archives, or a public library with reference material. You need access to the census. If you know your grandfather's name and date of birth, you can find him on the census. Census data is made public after 35 years. Start with what you know, use original sources, like the census, tax records, birth and death records, etc. If you know your father's name, you can order his death certificate, which will give his father's name, etc. These sources will get you back for about three generations. Baptismal records of churchs, etc. can be accessed.
I you email me, I will send you some links to free sites.
Don't worry about the software at the beginning. Use hardcopy until you get enough data it becomes unmanageable.
I've traced several branches of mine and the wifes family back 13 generations. I now have over 7500 names in a database. But it took a while..be patient.
liftmechanic@hotmail.com
Ben