I've been meaning to do a population simulation for a long time. The simulation is fairly simple. Start with a population of 1,000,000,000 people. There are two types of "genes" in this population. The most common and also dominant gene is the "barren gene" and compels its individual to produce one child on average. Given that it takes two parents to produce a child, if this were the only "gene," the population would halve every generation and mankind would indeed go extinct in only a few hundred years.
The second "gene" is the "fruitful gene" and potentially compels its individual to have three children on average. However, since this "gene" is recessive, the individual is only compelled to have three children if he or she has two of these "genes." At the start of the simulation, only five-percent of the genes are of this type. So not only is it recessive, it's also rare. If a double "fruitful" mates with a someone with at least one "barren gene", they split the difference and have two children.
The graph above shows the results of one run of the simulation (since it randomly combines each generation, the results are slightly different each time). The population drops for 15 to 20 generations before turning the corner and recovering fully somewhere around the 40th generation. (If anybody wants the matlab code, let me know, I'll be happy to send it to you).
I think that this simulation is very pessimistic (i.e. the population will never drop that far). I don't think the "fruitful gene" is fully recessive, I think it's more prevalent than five-percent, I think that "fruitful" people would seek each other out instead of mating randomly, and I think that the "fruitful gene" is probably more fruitful than just three kids. For example: if the "fruitful" folk seek each other out instead of mating randomly, the population typically drops less than a factor of ten over 4 or 5 generations before recovering fully within 10; or when starting the simulation with 10% "fruitful genes" and having two fruitful parents produce four children, the population for only around 10 generations before recovering fully after 20.
The point of this is not to convince anybody, but rather to show why I'm highly skeptical of the claim that mankind will go extinct due to low birthrates. I know too many highly educated, non-religious couples with access to birth control who have three or more children by choice.
The barren will go extinct, but not the fruitful.