Teaching first Bayesian or frequentist statistics

Both Bayesian statistics and frequentist statistics are based on probability theory, but I'd say that the former relies more heavily on the theory from the start. On the other hand, surely the concept of a credible interval is more intuitive than that of a confidence interval, once the student has a good understanding of the concept of probability. So, whatever you choose, I advocate first of all strengthening their grasp of probability concepts, with all those examples based on dice, cards, roulette, Monty Hall paradox, etc..

I would choose one approach or the other based on a purely utilitarian approach: are they more likely to study frequentist or Bayesian statistics at school? In my country, they would definitely learn the frequentist framework first (and last: never heard of high school students being taught Bayesian stats, the only chance is either at university or afterwards, by self-study). Maybe in yours it's different. Keep in mind that if they need to deal with NHST (Null Hypothesis Significance Testing), that more naturally arises in the context of frequentist statistics, IMO. Of course you can test hypotheses also in the Bayesian framework, but there are many leading Bayesian statisticians who advocate not using Bayesian NHST (for example, Andrew Gelman from Columbia University).

Finally, I don't know about the level of high school students in your country, but in mine it would be really difficult for a student to successfully assimilate (the basics of) probability theory and integral calculus at the same time. So, if you decide to go with Bayesian statistics, I'd really avoid the continuous random variable case, and stick to discrete random variables.

This question risks being opinion-based, so I'll try to be really brief with my opinion, then give you a book suggestion. Sometimes it's worth taking a particular approach because it's the approach that a particularly good book takes.

I would agree that Bayesian statistics are more intuitive. The Confidence Interval versus Credible Interval distinction pretty much sums it up: people naturally think in terms of "what is the chance that..." rather than the Confidence Interval approach. The Confidence Interval approach sounds a lot like it's saying the same thing as the Credible Interval except on general principle you can't take the last step from "95% of the time" to "95% chance", which seems very frequentist but you can't do it. It's not inconsistent, just not intuitive.

Balancing that out is the fact that most college courses they will take will use the less-intuitive frequentist approach.

That said I really like the book Statistical Rethinking: A Bayesian Course with Examples in R and Stan by Richard McElreath. It's not cheap, so please read about it and poke around in it on Amazon before you buy. I find it a particularly intuitive approach that takes advantage of the Bayesian approach, and is very hands-on. (And since R and Stan are excellent tools for Bayesian statistics and they're free, it's practical learning.)

EDIT: A couple of comments have mentioned that the book is probably beyond a High Schooler, even with an experienced tutor. So I'll have to place an even bigger caveat: it has a simple approach at the beginning, but ramps up quickly. It's an amazing book, but you really, really would have to poke through it on Amazon to get a feel for its initial assumptions and how quickly it ramps up. Beautiful analogies, great hands-on work in R, incredible flow and organization, but maybe not useful to you.

It assumes a basic knowledge of programming and R (free statistical package), and some exposure to the basics of probability and statistics. It's not random-access and each chapter builds on prior chapters. It starts out very simple, though the difficulty does ramp up in the middle -- it ends on multi-level regression. So you might want to preview some of it at Amazon, and decide if you can easily cover the basics or if it jumps in a bit too far down the road.

EDIT 2: The bottom line of my contribution here and attempt to turn it from pure opinion is that a good textbook may decide which approach you take. I'd prefer a Bayesian approach, and this book does that well, but perhaps at too fast a pace.

Bayesian and frequentist ask different questions. Bayesian asks what parameter values are credible, given the observed data. Frequentist asks about the probability of imaginary simulated data if some hypothetical parameter values were true. Frequentist decisions are motivated by controlling errors, Bayesian decisions are motivated by uncertainty in model descriptions.

So which should you teach first? Well, if one or the other of those questions is what you want to ask first, that's your answer. But in terms of approachability and pedagogy, I think that Bayesian is much easier to understand and is far more intuitive. The basic idea of Bayesian analysis is re-allocation of credibility across possibilities, just like Sherlock Holmes famously said, and which millions of readers have intuitively understood. But the basic idea of frequentist analysis is very challenging: The space of all possible sets of data that might have happened if a particular hypothesis were true, and the proportion of those imaginary data sets that have a summary statistic as or more extreme than the summary statistic that was actually observed.

A free introductory chapter about Bayesian ideas is here. A article that sets frequentist and Bayesian concepts side by side is here. The article explains frequentist and Bayesian approaches to hypothesis testing and to estimation (and a lot of other stuff). The framework of the article might be especially useful to beginners trying to get a view of the landscape.

I have been taught the frequentist approach first, then the Bayesian one. I am not a professional statistician.

I have to admit I didn't found my prior knowledge of the frequentist approach to be decisively useful in understanding the Bayesian approach.

I would dare to say it depends on what concrete applications you will be showing your pupils next, and how much time and effort you will be spending on them.

Having said this, I would start with Bayes.

Are you teaching for fun and insight or for practical use? If it's about teaching and understanding, I'd go Bayes. If for practical purposes, I'd definitely go Frequentist.

In many fields -and I suppose most fields- of natural sciences, people are used to publish their papers with a p-value. Your "boys" will have to read other people's papers before they come to writing their own. To read other people's papers, at least in my field, they need to understand null hypotheses an p-values, no matter how stupid they may appear after Bayesian studies. And even when they are ready to publish their first paper, they will probably have some senior scientist leading the team and chances are, they prefer Frequentism.

That being said, I would like to concur with @Wayne , in that Statistical rethinking shows a very clear way towards Bayesian statistics as a first approach and not based on existing knowledge about Frequentism. It is great how this book does not try to convince you in a fight of the better or worse statistics. The stated argument of the author for Bayes is (IIRC) that he has been teaching both kinds and Bayes was easier to teach.