The more I think about The Prisoner of Azkaban, the more convinced I am that this is where the series truly begins. It becomes darker, more fleshed out - more mature, basically. And that isn't a bad thing, although it's admittedly a little intimidating when you realise Azkaban is the last slim volume in the series.
So, with The Prisoner of Azkaban, it feels like Rowling has finally got some more solid direction for the series. We go from Harry basically bumbling around and somehow managing to thwart two attempts by Voldemort to gain power over wizarding kind - two stories I kinda feel aren't entirely believable, and it's nothing to do with the magic in it - to a string of events where Harry really has to come into his own, and this is where it really begins. On top of that, this third book really fleshes out the world Harry is in, and it helps the story feel more alive. It's not just Hogwarts and the mentioned-but-not-really-explained Ministry of Magic, we now know about the horrors of Azkaban, the Dementors, omens, people surrounding Voldemort, and so on.
I don't really know if I found any flaws as such - bar a certain awkward scene I elaborate on later - but I think the story isn't as "magic" and as compelling as the first book. It doesn't have that same sense of "By all that is unholy this is good", instead you just think "This is good", which is a little bit of a shame. It's still really good and still draws you in, but stepping back allows you to sometimes see the mechanisms and the strings that Rowling has employed to make this work, and the whole Harry Potter Is Special thing does drag a little here.
Summary: The Prisoner of Azkaban takes a step backwards to allow the reader to see the wider picture. Whilst Voldemort is still very much a 'thing', we're spared an appearance and instead we get to see some of the back story to various characters. This expansion to the plot also vaguely breaks up the Stuff Only Happens When Harry Is At Hogwarts And Harry Stops It formula we've had from the first two books, because I think in this one he's very much a vehicle for the plot, rather than the driving force.
Favourite Moment: Probably Harry winning the final Quidditch match. And Professor Lupin, who is an excellent character. And Harry smart-talking Malfoy into silence. There's quite a few 'favourite' moments here.
Least Favourite Moment: The over-drawn scene in Hogsmeade with the Hogwarts staff, which is essentially the sledgehammer approach to plot exposition. It is long, it is drawn out, and it is surprisingly clumsy. Rowling doesn't shy from slightly awkward moments to elaborate on plot points, but this goes a little too far towards being the characters turning to the reader and explaining things.
Improvements From Earlier Book(s): Again, Rowling further increases the range of voices in the book, Cho Chang in particular being a welcome example. The teachers, or at least some of them, become more human too, as we begin to see facets of them in situations other than being teachers.