Quite simply, Dwane Casey could not advance past the Cavs in the playoffs. Most franchises get impatient when the next step in the process to achieve a certain goal, in this case, a finals appearance by the Raptors, is not achieved. Casey engineered back-to-back great regular seasons but they got swept by Lebron James and the Cavs in back-to-back years in the Eastern Conference semifinals not to mention also losing to them in the Eastern Conference finals in 2016. The Raptors felt like they need something to get them over the Cavaliers, and firing Dwane was the right decision.
[2016 Eastern Conference Finals][3]
[2017 Eastern Conference Semis][1]
[2018 Eastern Conference Semis][2]
[Here][4]'s an article from ESPN concerning his firing with comments from the team president.