Quite simply, Dwane Casey could not advance past the Cavs in the playoffs. Most franchises get impatient when the next step in the process to achieve a certain goal, in this case, a finals appearance by the Raptors, is not achieved. Casey engineered back-to-back great regular seasons but they got swept by Lebron James and the Cavs in back-to-back years in the Eastern Conference semifinals not to mention also losing to them in the Eastern Conference finals in 2016. 

<p> [2016 Eastern Conference Finals][3]
<p> [2017 Eastern Conference Semis][1]
<p> [2018 Eastern Conference Semis][2]
<p>[Here][4]'s an article from ESPN concerning his firing with comments from the team president.

  [1]: https://www.basketball-reference.com/playoffs/2017-nba-eastern-conference-semifinals-raptors-vs-cavaliers.html
  [2]: https://www.basketball-reference.com/playoffs/2018-nba-eastern-conference-semifinals-cavaliers-vs-raptors.html
  [3]: https://www.basketball-reference.com/playoffs/2016-nba-eastern-conference-finals-raptors-vs-cavaliers.html
  [4]: http://www.espn.com/nba/story/_/id/23471720/toronto-raptors-fire-head-coach-dwane-casey-7-seasons