Hostages were being held inside a cafe in central Sydney Monday with a siege underway and an Islamic flag held up to a window, witnesses and reports said.

Martin Place in the central business district was shut down as scores of police surrounded the Lindt Chocolat Cafe, with TV pictures showing a flag, black with white Arabic writing, held to a window.

Reports said as many as 20 people were in the cafe and that there were at least two gunmen, although police had no immediate comment, only confirming that an operation was underway.

Witnesses reported hearing loud bangs that sounded like gun shots.

Patrick Byrne, a producer at Channel Seven whose newsroom is opposite the cafe, said staff at the television station watched the situation unfold.

"We raced to the window and saw the shocking and chilling sight of people putting their hands up against the panes of glass at the cafe," he told the Australian Broadcasting Corporation.

"This was just extraordinary."