In April 2012, the Idaho Supreme Court upheld previous rulings on Hart and his unpaid income taxes. The owed taxes stretched back to 1996, when Hart stopped filing federal and state income tax returns in protest over their perceived unconstitutionality. He resumed payments, but according to the IRS was hundreds of thousands of dollars behind schedule. In addition, the Idaho Tax Commission found that Hart owed them $53,523 in back taxes as of 2009, which had continued to accrue interest. The issue decided by the court was not the legality of collecting income taxes in general, but whether Hart waited too long to file his initial appeal and if his most recent hearing should have been delayed, since he was participating in lawmaking in the Idaho House of Representatives. On the first point, two government bodies had already deemed that the six months Hart waited was much longer than the 91 day appeal period and that his work as a legislator would not sway that point. As to the second point, an exception was provided by First Judicial District Judge John Mitchell. Mitchell found that since that hearing only relied on legal arguments of the attorneys, Hart was not required to be there. He was however, able to call in by phone, which he declined to do. On November 5, Hart appeared before U.S. District of Idaho Judge Edward Lodge for a federal tax trial. The IRS claimed that Representative Hart owed the United States $550,000 in back taxes. In July 2015, Hart settled his tax dispute with the federal government. As part of the settlement, Hart agreed to make both monthly and annual payments to the IRS for the next nine years toward his $586,000 federal tax debt. The federal government also auctioned off his home for unpaid back taxes.

Idaho Speaker of the House Lawerence Denney (R-Midvale) removed Hart from the Revenue and Taxation Committee on November 9, 2010. Denney said Hart sent a letter requesting removal from the panel. Rep. John Rusche (D-Lewiston) urged an investigation into Hart for his alleged dealings with the Idaho State Tax Commission and the IRS.