As soon as Daphne found out about Niles' feelings for her. That was always Frasiers trump card, a great running joke they could fall back on. As soon as they erased it they took some of the spark away from the show.
Where it was once demanding and undemanding in equal measure it soon came down to the level of Will & Grace or Everybody Loves Raymond. Not a good place to be.
It still remains one of my favourite sitcoms tho, but I feel it went on a bit too long in the end.