Pedro Hernandez of Maple Shade, New Jersey, told police he choked Etan Patz to death and left his body in a bag in an alley, New York Police Commissioner Raymond Kelly told reporters.
Etan vanished while walking to a school bus stop on his own for the first time.
He became one of the first to appear on milk cartons asking for information about missing children.
Mr Hernandez, 51, is the first suspect to be arrested in connection with the case. He worked in a convenience shop near the Patz family home in Manhattan, New York.
"He was remorseful, and I think the detectives thought that it was a feeling of relief on his part," Mr Kelly said. "We believe that this is the individual responsible for the crime."
Mr Kelly added that Mr Hernandez had lured the boy "with the promise of a soda". After leading the boy into the basement he "choked him there and disposed of the body by placing him in a plastic bag and placing it in the trash".
No body or bag was ever recovered.
Breakthrough?
Mr Kelly told reporters that police took Mr Hernandez back to the scene of the crime, which is now a shop selling spectacles. When the incident took place, Mr Hernandez had been stacking shelves at the small grocery shop for about a month.
He added that police had already informed Etan's family of the development in the case.
"We can only hope that these developments bring some measure of peace to the family," he said.
US media said that police had been tipped off by someone Mr Hernandez had confided in, possibly a family member.
Continue reading the main story
Investigators last month searched a handyman's former workshop near the Patz family home.
In an apparent breakthrough for the decades-old investigation, the Manhattan basement flat was excavated over four days. But no obvious human remains and little forensic evidence were found.
Several containers of rubble and sand from the property were hauled away and preserved in case officials need to revisit the excavation.
The handyman, Othniel Miller, has been questioned by detectives over the past year. But he denies having anything to do with his disappearance.
Etan vanished on 25 May 1979, near his home in New York's SoHo district. Friday is the anniversary his disappearance.
In 1983, President Ronald Reagan declared 25 May National Missing Children's Day.
Etan's parents, Stanley and Julie Patz, became outspoken advocates for missing children in the years after their son's disappearance.
Etan's father Stanley Patz and mother Julie live in the same SoHo apartment as they did in 1979
The Patzes have not moved since his disappearance and for years refused to change their phone number, hoping that Etan was alive.
In 2001, however, the family obtained a court order declaring Etan dead as part of a lawsuit in which a convicted child molester was held responsible for their son's death.
A judge ordered Jose Antonio Ramos, who knew Etan's babysitter, to pay $2m (£1.3m) to the Patzes.
Ramos is serving a 20-year prison sentence for a different case, and is scheduled to be released this year.
Analysis
Laura Trevelyan - BBC News, New York
The image of Etan Patz's smiling face, framed by his fair hair, haunted Americans in the 1980s as they saw it on their milk cartons nationwide.
Etan's disappearance helped launch a national missing children's movement. The harrowing story of how a six-year-old vanished while walking to the school bus stop for the very first time prompted American parents to curb their children's independence.
Today it's almost impossible to think of a six-year-old being given such freedom, but back in 1979, it was common.
BBC news