This website by Steve Conover has some very good material along this line of reasoning. (updated link here)
As a thought experiment: if we ran budget deficits every year which averaged the same as the long-term growth rate of the economy then the outstanding indebtedness (accumulated debt) would remain constant relative to annual economic activity. Yes, deficits every year but no increasing burden in relative terms. As a practical matter, it's probably best to run smaller deficits (relative to the size of the economy - no hysteria about the absolute numbers please) during a boom and then tolerate somewhat larger deficits in a recession. Unlike individuals, who usually wish to payoff outstanding debts before they die, our government should operate as an "ongoing entity" and manage the debt to good advantage. Also, unlike individuals, the tax base is diversified across all activity, not just a single job.
Realize that obsession over deficits without considering the policy mix which brings them about is an extremely "government-centric" view of the economy. It is very important to consider the impact of incentives upon individuals and businesses in the much larger private sector (for-profit and not-for-profit).
This is a link to an earlier post, Let's get fiscal, on the more recent history of deficits and the economy.
Below are some excerpts(emphasis mine) from a renowned book by George Gilder titled Wealth and Poverty which are relevant to this subject. Here is a review of the book from when it was written in 1981. (updated link here)
Deficits - Wealth & Poverty
Starting on p.239 of Wealth & Poverty
…The purpose of the (tax) cuts, it must be stressed, is to expand the tax base – to make the rich pay more taxes by inducing them to consume less and to work and invest more. If deficits arise as an initial result, no one should panic. In an economy with an overweening public sector, deficit spending, even in substantial amounts, is decidedly preferable to tax increases.
...To imagine that issuing a bond is better than levying a tax is to suffer from "bond illusion"- the idea that we can get something for nothing by manipulating financial instruments. There is no such thing, we are told, as a free lunch; all lunches must come out of the real production of the economy. This is a hard-earned and valuable truth. But it is a half-truth, for it implies that "the economy" and other such concepts, like "the money supply" or "industrial capacity" or "the supply of labor" or "the reserves of natural resources" are definite and measurable things, subject to mathematical laws.
...There is no such thing as a free lunch. But there is no such thing as a measurable "economy" or an absolute "supply" of money, labor, or resources.
Labor and resources, for example, are enormously elastic. ... In an overtaxed system, the statistics of limits and capacity are mostly mush.
Any resource depends largely on ideas and technologies, which change rapidly. ...
When the government chooses to spend a billion dollars more, what chiefly matters is not the extraction of resources or use of capacity that the spending entails, but the impact on the incentives and the creativity of businesses and workers. ...
Deficit spending can be a way of protecting the private sector and its most catalytic investments from the effects of direct taxes. ... Taxes, though, are an immediate and direct burden, with effects that are magnified by the high return on investment of rapidly growing firms.
It is always best to cut government spending wherever its yield or benefit is less than private spending. But in the absence of this best policy, governments should not, as nowadays they so often do, resort to the worst policy: increasing marginal tax rates or rejecting their retrenchment in order to achieve a "balanced budget" and fight inflation. It will not work.
Where does this leave the canons of fiscal integrity that have so long been preached by most responsible economists? It turns fiscal integrity from a numerical exercise into the vital art of preserving and expanding the real and seminal wealth of the nation. This means not merely balancing income and outgo but balancing the prospective yields of government and private activity and financing profitable long-term projects in either sector by appropriate issues of debt. Under current circumstances reversion to the conventional kind of fiscal integrity of accounting balances will always result in an irresponsible and destructive resistance to tax cuts. As long as capacity is considered to be an inelastic number - as long as the economy is treated as a sum of measurable quantities - increases in government spending will inevitably require rises in tax rates and the closing of the horizons of growth.
But within that calculus of pay-as-you-go lunches there is no room for creativity and inspiration, no room for the incalculable and unprovable supply-side responses that are the mustard seeds of the capitalist miracles of growth. …it is only the physical part of our wealth that is finite. Its metaphysical sources (imagination and creativity) are infinite. There are free lunches under capitalism because there are free minds and free men – because of the limitless returns on metaphysical capital… But these benefits always elude the computations of aggregate economics.
Debt that is wantonly monetized or accompanied by chaos, as during the French revolutionary era, or debt that is prodigally incurred in order to destroy the unit of account of other debt, as in Weimar Germany, can bring ruin…. But debt that is incurred for capital projects of benefit to citizens and their productivity, or debt that is incurred to avoid inflicting destructive taxes on growing firms – such liabilities can become vital assets of growth and progress. The worst economic disaster is to blight the future by suppressing the catalytic ventures on the economic frontier.
In an expanding economy money available now for investment is many times more valuable than money paid later in interest… Only in a static, uncreative economy does it pay to pay as you go.
Budgeting in a growing capitalist economy is not zero-sum, and time is not linear. Money to be paid beyond a certain future point dissolves almost entirely; even in accounting, an annual payment of a hundred dollars forever has nearly the same value as a payment of a hundred dollars for twenty years. Time is capital in a growing system and dwarfs all debt.
The United States must overcome the materialist fallacy: the illusion that resources and capital are essentially things, which can run out, rather than products of human will and imagination, which in freedom are inexhaustible. This fallacy is one of the oldest of economic delusions, from the period of empire when men believed that wealth was land, to the period of mercantilism when they fantasized that it was gold, to the contemporary period when they suppose it is oil… Throughout history, from Venice to Hong Kong, the fastest growing countries have been the lands best endowed not with things but with free minds and private right to property.