Baltimore lifted a curfew Sunday that was imposed across the US East Coast city following widespread riots, as thousands of National Guard troops began to pull out.

Mayor Stephanie Rawlings-Blake had faced growing calls for the curfew to be scrapped, particularly from store and restaurant owners who said the 10:00 pm to 5:00 am nightly restriction was wrecking business.

"My goal has always been to not have the curfew in place a single day longer than was necessary," Rawlings-Blake wrote on Twitter. "I believe we have reached that point today."

Hundreds of people meanwhile rallied again at City Hall plaza, summoned by local religious leaders.

Across the city, people cheered the return to relative calm with impromptu celebrations, a far cry from the riots that saw angry protesters torch cars, pelt police with stones and ransack stores on April 27.

The riots stemmed from protests over the death of Freddie Gray, 25, who suffered a serious spinal injury while in the back of a police van on April 12.

He died a week later.

A total of 113 officers were wounded and 486 protesters were arrested, including 46 overnight into Sunday, Baltimore police spokesman Captain Eric Kowalczyk told reporters.

The curb on nighttime activity was initially supposed to remain in place until Monday.

National Guard troops a rarity

To help prevent repeat violence, authorities sent about 3,000 Maryland National Guard troops into Baltimore.

Yet "the sense of siege will persist as long as Governor Hogan leaves military forces in the city -- we urge him to remove them now," warned Deborah Jeon, legal director of the Maryland chapter of the American Civil Liberties Union.

Soon after the curfew ended, Governor Larry Hogan said the troops had already started to leave the city.

"It's going to take a couple of days to get everybody out," Hogan cautioned. "We had to build an entire city to save the city."

Hogan called for "healing" and prayer in Baltimore and spoke of the "devastating" financial ramifications of the riots and curfew.

The governor, who is Roman Catholic, attended mass with the city's Archbishop William Lori.

About 200 businesses, most minority-owned, were lost in the rioting on April 27, according to Hogan. He estimated the losses in the hundreds of millions of dollars.

Prosecutors say Gray should never even have been arrested and had committed no crime.

Six Baltimore police officers, three white and three black, were charged Friday with multiple counts, including second-degree murder and manslaughter in connection with the death.

Baltimore's police union has condemned what it calls "an egregious rush to judgement," as it defended the officers and expressed confidence they would be cleared.

Music icon Prince has recorded a tribute to Baltimore that addresses the unrest as well as "the slew of killings of young black men," a spokeswoman told The Baltimore Sun.

He plans to host a "Dance Rally 4 Peace" in Minneapolis on Saturday night, asking all to wear gray in memory of Freddie Gray, the report said.

Growing roster of deaths

Gray's case is the latest in a growing roster of high-profile deaths of unarmed black men at the hands of police, and has rekindled national debate and simmering tensions about whether police are too hasty to use deadly violence when dealing with African American or minority suspects.

One of the most prominent cases is that of Michael Brown, an 18-year-old who was fatally shot in Ferguson, Missouri.

The death and subsequent lack of legal action against the police officer who shot him prompted widespread riots in the St Louis suburb.

Similarly, the death of father-of-six Eric Garner, 43, sparked broad protests. Garner died after being held in a police chokehold while being arrested for illegally selling cigarettes in New York.

Many residents in Baltimore, a port city of 620,000 people, reacted jubilantly to news of the six officers being charged, and demonstrations since the April 27 riots were generally peaceful.