The work is taking place at the airport in the central city of Danang.
The US sprayed millions of gallons of the toxic defoliant over jungle areas to destroy enemy cover.
The US sprayed Vietnam's jungles to deprive the enemy of places to hide
Vietnam says several million people have been affected by Agent Orange, including 150,000 children born with severe birth defects.
'Injustice'
Agent Orange Victims Association Vice Chairman Tran Xuan Thu told the BBC that although the clean-up activities were "a little late", they were "greatly appreciated".
"They show that the US government now is taking the responsibility to assist us. I hope these efforts will be multiplied in future," he said.
"However we consider that the clean-up is separate from the issue of compensating Vietnamese Agent Orange victims, who are still suffering from injustice. These victims will carry on with their lawsuits, no matter what."
A lawsuit brought by a group of Vietnamese nationals against US manufacturers was dismissed in 2007.
On Thursday, a ceremony was held at the Danang airport where the defoliant was stored before being sprayed over forests hiding fighters from the Viet Cong, guerrillas backed by the Communist government of North Vietnam.
The US government is providing $41m (£26m) to the clean-up project, which is being carried out by two American companies in co-operation with the Vietnamese defence ministry.
The US has in the past helped fund some social services in Vietnam, but this is its first direct involvement in clean-up work.
The contaminated soil and sediment is to be excavated and then heated to a high temperature to destroy the dioxins, a US embassy statement said.
Frank Donovan of USAID told Radio Australia the project would last until 2016.
"We expect it will be cleaned up to rid the contaminated areas of dioxins down to harmless levels that are accepted both by the government of the US and the government of Vietnam, and so safe for industrial, commercial or residential use," he said.
There are dozens of other contamination hotspots where the defoliant was stored, including two more airports.
The US and Vietnam resumed full diplomatic ties in 1995 and have grown closer in recent years amid concerns over China's assertiveness over disputed territories in the South China Sea.
The US compensates its veterans exposed to the defoliant, but does not compensate Vietnamese nationals.
Nga Pham, of the BBC's Vietnamese service, says Agent Orange is a very bitter legacy of the war, and most Vietnamese think the US should do more to help.
BBC news