This is one of the main arguments he will bring to talks with Israeli Prime Minister Benjamin Netanyahu later on Sunday during his whirlwind trip to the region, he said.
The hastily arranged talks come after Israel on Friday approved a plan to attack the city on the southern edge of the shattered Palestinian enclave where more than half of its 2.3 million residents are sheltering after five months of war.
"Right now, it is about ensuring we come to a long-lasting ceasefire," Scholz said after talks with Abdullah at his private residence in the Jordanian Red Sea port of Aqaba.
"That would enable us to prevent such a ground offensive from taking place."
Asked if he was prepared to exert pressure on Netanyahu to stop such an assault, Scholz said it was "very clear we must do everything so the situation does not get worse than it already is."
"Israel has every right to protect itself .. At the same time, it cannot be that those in Gaza who fled to Rafah are directly threatened by whatever military actions and operations are undertaken there."
Israel says Rafah is one of the last strongholds of Hamas, which it has pledged to eliminate, and that residents will be evacuated.
Scholz did not directly answer a question about whether Germany would react to a large-scale Rafah offensive, for example by restricting German weapons exports to Israel.
Germany has been one of Israel's staunchest allies alongside the United States, consistently supporting its right to defend itself, underscoring its duty to stand by the country's side in atonement for its perpetration of the Nazi Holocaust in which 6 million Jews died.
But the government has faced accusations - including from prominent Jewish residents in Germany - of allowing guilt to blinker its response to Israel's retaliation.
Malaysian Prime Minister Anwar Ibrahim accused Europe during a visit to Berlin last week of being "hypocritical" and selective in its concern for different peoples, overlooking decades of widespread Palestinian suffering.
Still, German government officials have stressed in recent months the need for Israel to adhere to international law in its response to the Hamas attacks and called upon Israel to allow more humanitarian aid into Gaza. They have also denounced the violence of Jewish settlers in the West Bank.
Scholz said the talks with King Abdullah had made clear once again how important it was for all negotiations to touch on the long-term prospects for a possible peaceful co-existence between Israel and a Palestinian state.
