After wrapping up their latest mystery Velma gets a call from her mother asking her to check on her younger sister Madelyn, where rumors of a giant griffin haunting the college are flourishing.
The gang agrees to help and head off to the Whirlen Merlin Magic Academy - as Madelyn has been studying stage magic - where apparently there has been some trouble. After a bumpy ride (thanks to Fred's new GPS system), they arrive at the college, where they meet Madelyn and the owner and namesake of the school, Whirlen Merlin, along with his brother Marlen, who acts as cook and butler in the school, and Crystal, Whirlen's former stage assistant, whom Fred quickly gains a crush on, much to Daphne's annoyance. From them, the gang learns how sightings of a giant griffin have been scaring all the students and teaching staff away. The gang smells trouble and decides to investigate.
Later that evening, Madelyn (who has had a crush on Shaggy since she was younger) decides to take Shaggy on a romantic walk to show where the ancient sun-dial of the magician O'Flannery is located. The Griffin appears and a chase breaks out. As Shaggy, Madelyn, and Scooby flee to safety, a student takes a picture of the griffin on her cell-phone, which then appears on the morning newspaper the next day. The rest of the students leave, so in order to better understand the case, the gang starts taking classes. Meanwhile, Calvin Curdles, a powerful vegetable ice-cream guru, continually offers to buy Whirlen's castle from him.
After classes, Velma catches Daphne practicing ballet so as to conquer her danger prone ways and perhaps be more graceful, in order to capture Fred's attention. The two then stumble on a secret passageway leading to the attic of the school, where they discover (through an old scrapbook) that Alma Rumblebuns, the school's head custodian/maid, used to date Calvin Curdles. Meanwhile, after running away from the 'griffin' in the castle (which turns out to be a lion and some birds), Shaggy and Scooby accidentally discover the secret effects room of the castle, where they are discovered by Ms. Rumblebuns, knock over some containers of chemicals to the floor, and create fog. The gang then coallesces again, and learns how many magic tricks in Whirlen's performances are substituted with holograms and flashy puppets. A little later, Madelyn goes to the library and finds a book about the ancient staff of O'Flannery, the magician who brought over the castle, which is now the school, from Ireland, that controls the Griffin so they go looking for it. But when they find it on the island in Shadow Lake, they are found by a Banshee. Intially not posing a threat to the gang, she attaks them and tries to destroy them. ortunately, they manage to make a quick escape thanks to Shaggy.
Afterwards, Shaggy and Madelyn sit down and talk, and Shaggy discovers that he likes Madelyn in the same way that she likes him. Not long after, another chase breaks out with the Griffin. As the gang runs into the castle, Shaggy and Scooby bump into the groundskeeper and misplace the staff. When Madelyn runs outside to get the real staff, she is kidnapped by the griffin and taken to the Griffin's Roost, an ancient dilapidated tower on the campus, which is off limits to everyone. The gang goes after her, while Calvin Curdles tries to convince Whirlen to sign a deal that would give the castle to Calvin.
Meanwhile, the gang is split up after another meeting with the griffin. Shaggy and Scooby rescue Madelyn, and when trying to escape the griffin, they manage to end up in its clutches as it flies around the castle. Shaggy tries to get the griffin to stop by throwing the staff at it, but nothing happens, as the staff has no real powers. Suddenly, the griffin starts losing altitude and crashes in to the ground, where the rest of the gang discover that the griffin was actually a giant puppet, controlled from a giant blimp hidden by a fog machine. They end up discovering that Whirlen's brother Marlen was the one behind the griffin, wanting to make his brother fail because Marlen was tired of doing all of teh work for the shows and getting none of the credit for them. He then apologizes for trying to make Whirlen fail, saying that he now knew just how much the school meant to him and didn't want to take away his dream. Whirlen forgives Marlin, and all is well. Calvin Curdles then reveals how he wanted to buy the castle to win back Alma's heart after having left her to make something of himself so that he would be worthy of her love, and they become a couple again.
The Merlin brothers decide to make Madelyn their apprentice, and soon after, Calvin Curdles sponsers the reopening of the Merlin Brothers' Academy of Magic, where Madelyn performs (Daphne acting as her assistant), and Scooby 'tames' a lion; in reality, the Real Scooby is controlling a puppet to tame the lion with. The movie ends with him saying 'Scooby Abracadabra Doo!'