Start off with the shows problems:
The problem with ENT was it got tied up in the time paradox to much. Unlike the other Star Treks it didn't improve the characters much. It focused too much on Trip, when this happens it destroys any series. (Voyager season 5 and 6 was all about Seven of Nine and Stargate Atlantis became all about Col. Sheppard and the Wrath). Other characters were just boring Reed and Mayweather.
Another problem was the Star Trek Universe at this time was just not well defined Technologically, Socially and Politically.
Fan problems with this show:
The one thing I did like about this Star Trek ENT is the it wasn't this unrealistic utopia that the TNG was, until season 6 when they had the Maquis. Every races had and age old enemy and weren't these lovely dovey races that most were in TNG and humans were still up to their old tricks. Much like the humans today lessoning to rock music, into sports (Archor with his basketball), like to be in good physical shape (Mayweather, Trip and Archor) and money was still around.
I think this, more than anything is why ENT didn't last. It showed too much of the true. On ENT the crew only knew what their particular field was. Archor was the captain and was put their form his leadership skills not his technical skills and didn't have much of, for that he relied on Trip and T'Pol. Mayweather was the pilot and only knew how to fly the ship or shuttle pod and not much else. Trip was the Engineer and was there for his tech skills, but was also brave and knew how to fight.
This crew was not made up of these super science nerds that new everything! When we and if at all do encounter an Alien race we'll find out that they're going to act very much like us and this is what tuned a lot of Trek fan out, the fact that ENT showed it wasn't a perfect universe and Humans like every other race aren't perfect ether and never going to be.