That ejector consistists of a round plunger driven by a spring. Both are contained in the bolt body. Failure to eject is is usually because the spring or plunger is rusted or oil gunked in place, as you suspect. It is fairly easy to disassemble a Rem bolt bit it's not easy to discribe without photos or drawings. There are several old NRA and Gun Digest disassembly books on the market, perhaps you can find one at a gun show or ebay.
If it is just old, hardened oil gunk, as a quick fix you may want to try soaking the bolt in mineral spirits to see if you can dissolve the sticky stuff. Pushing in on the ejector occasionally as it soaks will help break up, loosen and float the dried goo out faster. Cheap spray-on "brake cleaner" works really good too.
If it is rust (or if the spring is broken, but that's unlikely) you have a more difficult problem. You really need to take the ejector out to remove rust from the parts and their hole so maybe it would be worth the cost of having a gunsmith take a look at it.
You have a jewel of a deer cartridge in that old rifle!
