Nicolas Otamendi broke the deadlock after 67 minutes at the Etihad Stadium, powering home Kevin De Bruyne's corner for his first goal for the club, but Norwich were level seven minutes from time when Hart spilt a routine take from Robbie Brady's cross to leave Cameron Jerome with a tap-in.
The mistake ultimately did not prove costly, as Yaya Toure converted an 89th-minute penalty after Russell Martin was sent off for blocking Raheem Sterling's goalbound shot with his arm to keep Pellegrini's team ahead of Arsenal at the summit on goal difference.
Hart, one of City's most consistent performers this season, partially redeemed himself with a vital save from Martin Olsson when the score was 2-1 and he retained his manager's backing after the match.
Pellegrini told reporters: "It was unfortunate. Sometimes, these things happen. Joe is human and he can make mistakes.
"Apart from that error, there were only two shots at him in the game that he saved because he is a very good goalkeeper."
Following the blow of conceding an equaliser and the prospect of being dislodged from top spot, Pellegrini was delighted with the character shown by his players in securing a win that could have been by a greater margin - only for Aleksandar Kolarov to miss from 12 yards after Sterling won a second penalty deep into stoppage time.
He said: "Norwich played very well. They defended the whole game with 100 per-cent concentration and the chances that we created, we missed.
"After we scored the goal from Nicolas Otamendi it opened up the game but, unfortunately, Joe allowed them to draw.
"The last 10 minutes was the most important 10 minutes from my team. We had the character, the personality and the trust to go again for the game.
"We were not finished in the last minutes because we always felt that we could score another goal.
"It is a relief because it is important not to drop points here at home."