US Secretary of State John Kerry said progress had been made during extended talks with President Hamid Karzai in Kabul on Saturday to hammer out a deal on the future of US forces in Afghanistan.

Kerry had been due to fly to Paris in the morning, but all-day negotiations ran into the evening to try to finalise the Bilateral Security Agreement (BSA), which would allow some US troops to stay in Afghanistan after 2014.

"We are going to try to see if we can make a little more progress, which is what we have been doing all day long," Kerry said at the US embassy before returning the presidential palace for further meetings.

"If this thing can come together, this will put the Taliban (insurgents) on their heels," he added.

A US official said Kerry wanted "to leave Kabul with as many issues resolved as possible".

Karzai recently threatened to abandon the security deal, which the US wants signed within weeks to enable the NATO military coalition to plan its withdrawal of 87,000 combat troops from Afghanistan by December 2014.

"There is still some work to do on the document. Things are not yet finalised," Karzai's spokesman told reporters before the start of the third round of talks since Kerry arrived on Friday.

The Afghan government has previously said its sticking points over the BSA were US demands for the right to conduct unilateral military operations against militants, and how the US would pledge to protect Afghanistan.

The lengthy negotiations between Kerry and Karzai came as the US said it had captured a senior leader of the Pakistani Taliban, Latif Mehsud, who is being held in Afghanistan, according to officials.

"I can confirm that US forces did capture... terrorist leader Latif Mehsud in a military operation," State Department deputy spokeswoman Marie Harf said, describing him as a senior commander in the Tehreek-e-Taliban Pakistan (TTP).

The Washington Post reported Mehsud was seen by Afghanistan as a possible go-between in the struggling peace efforts between Kabul and the Afghan Taliban, and that Karzai had been angered by his capture.

Karzai officially suspended BSA talks in June in a furious reaction to the Afghan Taliban opening a liaison office in Qatar that was presented as an embassy for a government in waiting.

He has said he refuses to be rushed into signing the deal, and would first seek approval from a traditional grand assembly of tribal leaders to be convened in about month's time.

Any agreement would see between 5,000 and 10,000 US troops remain in Afghanistan to train local forces and target Al-Qaeda remnants, but months of wrangling have tested both sides' patience.

The collapse of a similar agreement with Iraq in 2011 led to the US pulling its troops out of the country, which is currently suffering its worst sectarian violence since 2008.

But Afghan officials dismiss the possibility that the US may enact the "zero option" of a complete pull-out after its soldiers have fought the Taliban since 2001.

Karzai has had a tempestuous relationship with the US and other foreign allies since he came to power in 2001, often sparking outrage with his criticism of international military efforts to thwart the Taliban insurgents.

"The entire NATO exercise was one that caused Afghanistan a lot of suffering, a lot of loss of life, and no gains," he said this week.

A credible election to choose Karzai's successor next year is seen as the key test of Afghan stability as NATO troops withdraw. Kerry was also due to discuss issues such as voter registration and security with the president.

The Taliban regime was driven from power by a US-led coalition in 2001 for sheltering the Al-Qaeda leaders behind the 9/11 attacks.

Since then, the Islamist rebels have fought a bloody insurgency, and the US and Afghan governments now back peace talks to end the conflict.

Kerry had been due to travel to London after Paris.