The Crown Prosecution Service (CPS) has admitted its recent comments about the Bible were "inappropriate".
In a letter to lawyers at the Christian Legal Centre, the CPS had claimed that parts of the Bible are "no longer appropriate in modern society" and "would be deemed offensive if stated in public".
The comments were made in a statement to the CLC in relation to the case of their client, Christian preacher, John Dunn.
Mr Dunn was accused of a hate crime while preaching from the book of Corinthians on Swindon High Street but the case against him was later dropped.
In the House of Lords this week, Baroness Hoey asked the government what assessment they had made of the CPS's statement.
Answering on behalf of the government, Lord Stewart of Dirleton, the Advocate General for Scotland, said that the Wessex Area of the CPS had "undertaken a post-case review and acknowledges that the statement was inappropriate".
He added that the statement "was not intended to and does not represent a change to published CPS Policy".
"It is not indicative of a general approach by the CPS to cases involving the right to freedom of thought, conscience and religion, and the right to freedom of expression," he said.
The Christian Legal Centre said the statement was "a win for Christian freedoms" and "a clear acknowledgement that the CPS was wrong to suggest that quoting the Bible could be a criminal offence".
Andrea Williams, chief executive of the Christian Legal Centre, said, "This was a clear example of overreach from the CPS and demonstrated the extent to which they misunderstand the Bible and freedom of speech.
"Their arguments in this case were so steeped in the prevailing secular political orthodoxy, it was chilling.
"The Bible and its teachings have historically been the foundation of our society and provided many of the freedoms and protections that we still enjoy today.
"It is extraordinary that the prosecution, speaking on behalf of the state, could say that the Bible contains abusive words which, when spoken in public, constitute a criminal offence.
"Scraping the barrel in this way to try and convict a Christian preacher does not just waste a lot of people's time, but consistently creates wider and far-reaching implications for Christians across the UK who believe in Jesus and for society in general.
"This recognition from the government that what happened was inappropriate represents good news for UK Christians, and such arguments must never be made by the CPS again."
Responding to the news, Mr Dunn said: "I am pleased that the government has recognised that the arguments made against me by the CPS were wrong.
"I faced the prospect of criminal conviction for over two years on these grounds. I hope what has happened will protect other Christians who find themselves unfairly on the wrong side of the law for speaking biblical truth."
