It makes sense that infant mortality rates would play a large role, but it wouldn't be logical to believe people don't live longer today after they are past the age of 1 or 2, for example. My child would have probably died at the age of 8 from a bad appendix. I would have probably been dead by now do to a medical condition acquired in my 20's. My father probably would have died earlier due to a massive heart attack in his 40's. My mother would probably have perished due to cancer around 10 years ago if not for radical surgery. Well, you get the point. All you have to do is think about all the life saving surgeries that are done daily, not to mention the medicines and vaccines acquired to fight infection, and deadly diseases, etc.
No that's not it. It was in an anthropology journal. I'll dig it up. Basically everything you are saying is not true. Things have improved since the stone age, but not as much as you would think. If you made it past your 15th year in the stone age you had a better than average chance to make it to 65. Again, a much higher percentage of the population died very young in the stone age, but those that didn't generally lived into their 60s. Consider also that throughout recorded history men generally married at 18-20 and women at 13-15 and it isn't hard to understand that longstanding marriages have never been uncommon. http://scienceblogs.com/gregladen/2011/05/01/falsehood-if-this-was-the-ston/
