Iran's supreme leader Ayatollah Khamenei led prayers and at one point was seen weeping.
Iran has vowed severe revenge for the death of Soleimani and on Sunday pulled back from the 2015 nuclear accord.
Soleimani, 62, spearheaded Iranian military operations in the Middle East, and was regarded as a terrorist by the US.
President Trump said Soleimani was "plotting imminent and sinister attacks" on US diplomats and military personnel in Iraq and elsewhere in the region.
State television showed large crowds in Tehran for Soleimani's funeral.
People could be seen crying while others clutched pictures of the late commander. "Death to America" chants were heard.
His daughter Zeinab Soleimani warned the US it faces a "dark day" for the killing. "Crazy Trump, don't think that everything is over with my father's martyrdom," she said.
Following Monday's funeral, the general's remains will then be taken to Qom, one of the centres of Shia Islam, for a ceremony ahead of a funeral in his hometown of Kerman on Tuesday.
When visiting Soleimani's family members at their house in Tehran, President Rouhani said: "The Americans really did not realise what grave error they have committed.
"Revenge for his blood will be exacted on that day when the filthy hands of America will be cut off forever from the region."
Over the weekend, Mr Trump said that the US would strike back at Iran in the event of retaliation for Soleimani's death, "perhaps in a disproportionate manner".
He said the US was ready to strike 52 Iranian sites and "will strike very fast and very hard" if Tehran attacks Americans or US assets.
More about: Iran
