The 37-year-old will line up in the first row with Spaniard Maverick Vinales (Suzuki), who will be his teammate at Yamaha next year, and fellow Italian Andrea Iannone (Ducati).
It was a return to pole for Rossi in Italy for the first time since 2008, when he achieved his last victory on the Tuscan circuit.
MUST READ: 5 facts about Moto3 #GPArgentina winner, Khairul Idham Pawi
"Here it's crucial to start on the front row and the atmosphere generated by all this crowd is always fantastic," said Rossi.
"I'm very, very happy because it has been a long time since I last had pole here in Mugello, in the last years I was always struggling a lot."
It will be the Yamaha rider's 53rd pole in MotoGP and 63rd of his career, the same as his teammate Jorge Lorenzo, winner of the French Grand Prix, who starts from the second row.
Spaniard Lorenzo, the championship leader, will start behind compatriot Vinales, who as at the Qatar MotoGP will again start on the front row.
"In all the practice sessions, for some reason, we couldn't take advantage of using a new tyre like the other riders and in qualifying I did my best over three laps but there is still something that's not good on the bike that we need to discover for tomorrow, especially on braking," said world champion Lorenzo.
Spaniard Marc Marquez, second in the championship, five points behind Lorenzo, clocked the fourth fastest time and will start just behind the Italian.
"It's a shame that on the last lap I made a mistake when I was improving my time, so tomorrow I'll try to have a very good race in order to finish on the podium. It won't be easy," said Marquez.
In Moto 2, Britain's Sam Lowes of Kalex, second in the championship, is on pole for the sixth time in his career and third this season, ahead of Japan's Takaaki Nakagami and Lorenzo Baldassari of Italy.
Italy's Romano Fenati of KTM will start from pole in the Moto3 category, the third of his career beside compatriot Andrea Migno and Malaysian 17-year-old Khairul Idham Pawi, surprise winner this season in Argentina.
