Now with the luxury of having established its kooky characters at Middleton Interactive Learning Centre, they are unleashing more indulgent, but fun, situations.
"I'm pretty sure that other people found it funny last time, but this time we wanted to write jokes, not just the situations and characters which I think is what a lot of modern comedy does," says Butler, who plays Frances O'Brien.
"We really thought 'We just want to make people laugh.' Of course, we want to make people think and be moved and all that but we really want to make people laugh. So it's been, I think, really funny as a result."
Wayne Hope, who plays Terry O'Brien, says the ABC has always been supportive of the series.
"When we got together towards the end of the first series it's always been on the basis that they've gone 'What do you want to do?' And we've always said 'What else do we want to say? Is there anything else that we like?'
"But the great storyline we hadn't tapped is 'Who the hell made Frances O'Brien?'
"We thought it was time her mother, who just featured in flashbacks in the first two series, was brought into it.
"So we asked Victoria Eagger, who played her in the flashbacks, to play her in the present day, and she does a fantastic job."
"The primary thing for Frances is that her mother needs care and is kind of going insane. She's a dreadful, dreadful person so it's pretty horrible but hilarious," says Butler.
Hope agrees: "There's a chance that if you cast her at the correct age, because Vicki's not the age of Pearl, she's too vulnerable. The audience worry. If she's a mean character and she's of age it's too hard to watch.
"You don't want to repel people. Sometimes the material is coarse enough. What we think is black plays, but other people baulk from it. We want it to be confronting sometimes but we don't to scare people away. So we want to ride that line as delicately as we can."
The major plot in this series sees Angus Sampson playing a government minister looking for the library to pay its way.
"The big picture is that the library has been forced to make a profit," says Butler. "It's being partly privatised, which means the librarians have to make money, which is ridiculous and as it turns out impossible.
"But they give it a red hot go!"
The Librarians airs 8:30pm Wednesdays on ABC1.