Former premier Muhyiddin Yassin took an early close second, while Prime Minister Ismail Sabri Yaakob's ruling coalition put up the weakest showing among the three main contenders, losing ground in traditional strongholds, the results showed.
A record number of Malaysians voted on Saturday to end a recent spate of political uncertainty that has seen three prime ministers lead the country in as many years amidst an economic slowdown and the COVID-19 pandemic.
But the election looked unlikely to produce a clear winner as none of the three main contenders came close to the majority needed to form a government.
Of the 177 of 222 parliamentary seats counted, the Election Commission said Anwar's multi-ethnic Pakatan Harapan coalition was leading with 61 seats, while Muhyiddin's Perikatan Nasional alliance won 60 seats. Ismail's Barisan coalition got 24.
If Anwar clinches the top job, it would cap a remarkable journey for a politician who in 25 years has gone from heir apparent, to the premiership, to a prisoner convicted of sodomy, to the country's leading opposition figure.
Muhyiddin's alliance, which includes a Malay-centric conservative party and an Islamist party that has touted shariah law, is a junior partner in Ismail's ruling coalition and is contesting the national election for the first time.
Malaysia's veteran leader Mahathir Mohamad meanwhile was dealt his first election defeat in 53 years in a blow that could mark the end of a seven-decade political career, losing his seat to Muhyiddin's alliance.
Barisan said it accepted the people's decision, but stopped short of conceding defeat. The coalition said in a statement it remains committed to contributing to the formation of a stable government.
Failure by the main parties to win a majority means rival coalitions would have to work together to form a government, and Barisan could still be part of that winning bloc.
PROGRESSIVE VS CONSERVATIVE
Oh Ei Sun, a senior fellow with Singapore's Institute of International Affairs, said the early results showed a polarisation between progressive and conservative forces.
Race and religion are divisive issues in Malaysia, where the Muslim ethnic-Malay population make up the majority and ethnic Chinese and Indians the minorities.
"In order to form the next government, Anwar must win an outright majority. Otherwise, even if he wins the most number of seats, the Malay-Muslim coalitions will come together to form the next government," Oh said.
Perikatan and Barisan both prioritise the interests of Muslim Malays.
Barisan, dominated by the United Malays National Organisation, governed for 60 years from independence until 2018, while Perikatan is a new bloc that has emerged as a strong third force with Malay voters' backing.
Anwar was released from prison in 2018 after joining with old foe Mahathir and Muhyiddin to defeat Barisan for the first time in Malaysia's history, amid public anger at the government over the multibillion-dollar 1MDB scandal.
That coalition collapsed after 22 months in power due to infighting over a promise by Mahathir to hand the premiership to Anwar. Muhyiddin briefly became premier, but his administration collapsed last year, paving the way for Barisan's return to power with Ismail at the helm.
Earlier, Malaysians voted amid concerns over the economy, inflation, political instability and corruption. Several Barisan leaders face graft accusations.
"I hope there's a change in the government," Ismat Abdul Rauf, a 64-year-old retiree, told Reuters. "There are many issues that need to be addressed - the economy, the wealth of the country, the people (guilty of) wrongdoing who are not being prosecuted."
