Definitely looks like calculus to me but considering that most of the writing is literally layered over itself and that she allegedly doesn't actually understand it, yeah no I don't think Miss Circle is actually qualified.
Coming from a current honors Pre-Calculus student, my teacher makes sure that people can understand what the hell is going on. Believe it or not, but I have never actually somewhat enjoyed a math class like his before. That's just how good of a teacher he is.
That being said, I have a feeling the youngest they could possibly be is middle school, considering the fandom wiki page on Ruby talks about her being much taller than everyone else and "based on the things written on Miss Bloomie's board in her classroom, we can also assume that the students are in high school, making Ruby's height even more astonishing."
Again, coming from a personal standpoint on this, I have known SEVERAL people at my school who are actually just MOUNTAINS of men - two graduated already and one is currently a sophomore.
Before you mention Miss Sasha, either, it could be possible that they have something at that school for the really young kiddos. Once again, at my own, there is something there specifically for the really young children (granted I am pretty sure it is special education, but my point still stands - there's like two or three total faculty members responsible for that part of the school). So it isn't entirely out of the question imo. Moreover, Zip is most definitely A LOT older than her younger brother Chip. So the high school thing would make sense.
To top it off, the characters' appearances alone would definitely check out. Paper School being some kind of private high school would likely put Claire somewhere between 13-15 years old, and she does look to be somewhere within that age range. The youngest one actually at the school, by my opinion, would have to be Riley.