I forget, which "mistake" was talked about in the episode? The guy cheating on his wife? Or Taub's moment of weakness and taking the DNA samples.
I agree that "mistake" is kind of a fuzzy concept..
I can have the chicken or the pork dish for dinner. I chose the chicken which tastes good, but everyone else chose pork and they're all very impressed with it. I regret not choosing their dish since they appear to be enjoying their meal more than I am, but did I make a mistake? The chicken was still good and it's not making me sick... I just regret the decision.
In my interpreation, mistake has to mean an actual error. Choosing the wrong answer in a multiple choice test, making an error in a math problem or work-project early on that has negative ramifications later, making a choice that hurts a person's feelings, making a choice that gets you fired over something stupid, etc.
Meanwhile, when forced to pick between 2 choices... after choosing a choice with some negative ramifications will still "stand by their decision" since it might've been made with (A) good intentions (B) as much knowledge available at the time (C) the good outweighed the bad. In their opinion, there was no mistake.