Roma are ninth in the league after Sunday's 3-1 loss to AC Milan, and have struggled to overcome the loss of key players due to injury, including striker Paulo Dybala.
They are 22 points behind league leaders Inter Milan.
"AS Roma can confirm that Jose Mourinho and his coaching staff will leave the club with immediate effect," the club said in a statement.
"Further updates on the new first team coaching staff will follow imminently."
Roma have taken just four points from their last five league matches, and were eliminated from the Coppa Italia last week in a 1-0 defeat to rivals Lazio.
Following that defeat, Mourinho had likened himself to fictional wizard Harry Potter, saying his position at the club was raising fans' expectations for success.
He also hit back at critics, labelling himself as the perfect example of professionalism despite his team suffering recent poor results.
"We would like to thank Jose on behalf of all of us at AS Roma for his passion and efforts since his arrival at the club," club owners Dan and Ryan Friedkin said in a statement.
"We will always have great memories of his tenure at Roma, but we believe that an immediate change is in the best interests of the club.
"We wish Jose and his assistants all the best in their future endeavours."
Mourinho, who has coached a host of top European clubs including Porto, Chelsea, Inter Milan, Real Madrid and Manchester United, was appointed coach of Roma in 2021.
Under the Portuguese manager, Roma won the third-tier Europa Conference League in 2022. He also led them to the Europa League final in 2023, where they were beaten on penalties by Sevilla.
Roma next host 18th-placed Verona on Saturday.
