Anthony Ray Hinton, 58, was convicted of murdering two restaurant managers in Birmingham in 1985, but was granted a new trial last year.
Tests on bullets found at the crime scene could not be connected to a gun found at Mr Hinton's home, prompting prosecutors to drop the case.
His lawyer, Bryan Stevenson, said his client was convicted because he could not afford better legal counsel.
Walking out of the Jefferson Country Jail in Birmingham, Mr Hinton hugged family members and said: "Thank you, Jesus."
"All they had to do was test the gun," Mr Hinton said, explaining why he shouldn't have had to sit on death row for nearly three decades.
Bullets from the crime scene were the only evidence that linked Mr Hinton to the murders.
Prosecutors said that modern forensic methods failed to link the bullets to a revolver found in Mr Hinton's home.
"Every day, every month, every year that the state took from him, they took something that they don't have the power to give back," said Mr Stevenson.
At his first trial, Mr Hinton's lawyer believed his client only had $1,000 (£670) to hire an expert that could defend against the prosecution's allegations about the bullets.
The lawyer hired the only expert willing to take the job at that rate, and jurors reportedly laughed as the expert struggled to answer questions during cross-examination.
The US Supreme Court ruled last year that Mr Hinton did not have adequate legal counsel at the first trial and said the case should be reconsidered at a second trial.