He wrote: 'As a lifelong beneficiary of the freedom to make jokes about religion, I do think that Boris Johnson's joke about wearers of the burka resembling letterboxes is a pretty good one'.
'You should really only apologise for a bad joke.
Duncan Smith is the latest of a string of right-wing Conservatives who have come out in defence of Johnson since it emerged that complaints about his comments had been referred to an investigator. "On that basis, no apology is required", he wrote, in a letter to the Times.
He added that Theresa May was "clearly wrong to have asked him to have apologised".
"The Prime Minister is of course entitled to expect ministers to stick to a government line but that does not apply to backbench MPs".
The women also stressed that "contrary to what you may have been told by sections of the media and columnists who profess to know what is best for us", Muslim women are not forced to wear the niqab or burqa, nor are they oppressed. What about metaphor or simile?
Kanye West: I wasn't stumped by Kimmel's Trump question
Kanye West took to Twitter to defend his inability at responding to a question comparing Donald Trump and George W. Kanye West stopped by Jimmy Kimmel Live! this week, and gave one of his more calm and concise interviews of late.
The Daily Mail said it received one of its largest postbags ever from hundreds of readers, with 19 backing Mr Johnson to every one who was critical.
The Tory MP for Sutton Coldfield then suggested if Lord Sheikh handed the comments in to the police, he may find they were "actionable".
Former chief whip Andrew Mitchell said there was no need for the former Cabinet minister to apologise for his "colourful" language.
Mr Mitchell, who was chief whip in 2012 and global development secretary before that, told BBC Two's Newsnight: "I don't think he should apologise".
"The code of conduct process is strictly confidential", a Conservative party spokesperson said, refusing to elaborate on the procedure. "But he hasn't committed any offence". "Nevertheless we do so because we believe it is a means to get closer to God", the women said in a letter.
Some Brexiteer MPs claimed the complaints were politically driven by those who disagree with Mr Johnson over the European Union, while others warned that pursuing him would tear the party apart. You're in a free society, there are a lot of things that go on that we may not personally approve of or wouldn't do ourselves, but that's the benefit of living in a free and open democracy.