I would say whenever the drums are completely wrong and don't fit the song.
Either poorly recorded or if it's a hard hitting rock song and the drummer played with tooth-picks.
And no matter how I EQ or compress doesn't fix it then I deff. consider replacing them. Anything that will make them fit the context of the song.
Depending on the source I either use Trigger from Slate for individual drums or Addictive Drums for the whole kit.