Here's the thing, though Gene. There are a lot of folks that don't pay their full amount of taxes due. I know this for a fact.
See, I'm a working father of 4. We have only my salary to live on (and subsequently, to be taxed). With everything that's going on (educator's tax credits, educator's expenses tax writeoff's, child tax credits for 4 qualifying children, ... ), I end up not having ANY tax liability. So, here I am, a middle-class working stiff that doesn't pay taxes. Oh, I have them taken out, and I file my 1040A (and all other necessary forms, as required) every February. And then Uncle Sam (via that lovely agency, the IRS) returns everything that was withheld. Is this fair? Not in the least. Is this what happens under the current tax system? Most definitely.
To be fair, every person that works should have to put into the pot a portion of what they have earned. Make it a flat percentage so that the amount put by Party A is proportional to the amount put in by Party B. The actual numbers don't matter because the proportions of taxes paid to earnings kept will be the same. That's fair. It only seems unfair (and where most opponents to a flat tax latch on to) when you look at Party A only paying $2700 in taxes (off a base earnings of $18,000, keeping $15,300) versus Party B paying $27,000 (off a base earnings of $180,000, keeping $153,000).
Seems like Party B is getting to keep so much more money than Party A, right? In raw numbers, yes, he is. Why does Party B get to keep 10 times more money than Party A? Because Party B earned 10 times more money than Party A. The proportion of taxes paid is the same, though.
How does this help get the deficit down? More people paying taxes. No more loopholes to avoid paying taxes. Therefore, no way to avoid paying taxes = more money coming in.
Think of it like a company. On paper, I might have $2.5 million dollars in revenue. But, I can't actually spend it if my Accounts Receivable folks drop the ball and don't actually collect the money that I'm owed. Same here, if the people that are currently not paying any taxes (because of refunds or simple avoidance or other loopholes), then revenue goes up.
Also, measures would need to be put in place to curtail not filing, at all.