An officer and nine soldiers died or were wounded in the incident, Iraq`s joint operations command said.
Iraqi forces have been battling militants in Anbar province.
The air strike during close combat near Fallujah caused casualties on both sides, Iraq`s joint operations command said.
Iraqi army units had been advancing on IS positions near Fallujah but could not be supported by the Iraqi air force due to bad weather, the command said.
Coalition air forces carried out two strikes in response to an Iraqi request for support, it said, enabling Iraqi soldiers to advance rapidly and engage the jihadists in close combat.
"A third coalition strike carried out without taking into account the distance that had been covered resulted in injuries among our forces too," the command said in a statement.
The US military said it was the first reported incident of so-called "friendly fire" during Operation Inherent Resolve, which aims to drive IS militants out of Anbar province.
More about: