Thirty-two cars were operating when the incident occurred, the public transportation authorities for the city in North-Rhine-Westphalia state said.
It is not yet known what caused the car to derail, but there was strong wind at the time of the accident.
German fire crews lowered people to safety from the cable cars in a dramatic recovery operation, with children seen clinging to parents as they hung 40 metres above the river.
Rescue teams in Cologne had practised for an incident on the gondola just a week ago, German media reported.
A pregnant woman and a man were slightly injured in the rescue mission where mobile cranes were used to bring the stranded passengers to safety.
No other injuries were reported.
Martina and Hans-Peter Rieger, who were the first to be rescued after being trapped for about an hour, were celebrating their 41st wedding anniversary.
“That will always be in our memory,” Mr Rieger told German website, KTSA.
“Fortunately, we were also given direct hand signals to tell us we should remain calm and there was nothing bad.”
The mayor of Cologne, Henriette Reker, said, “I would like to thank the Cologne fire brigade for their great dedication. This meant that the passengers could be rescued quickly.
"My thoughts were with the children and adults, who had to wait courageously and patiently for many hours.”
Cable car manager Thomas Miebach apologized to riders and said an investigation was underway.
It was the first accident of its kind in 50 years, he added.
As a popular means of transportation, the cable cars connect the zoo to the Rheinpark across the river.
The system remained close on Sunday evening where officials worked to establish the cause.
