Bayesian evaluation for the likelihood of Christ's resurrection (edit 60)

I'm continuing to work on editing Bayesian evaluation for the likelihood of Christ's resurrection.

Things are edited down to "For example, this is what the tail end of the distribution looks like with various shape parameters", and the plot titled "Generalized Pareto Distribution (tail shapes)" has been updated.

You may next want to read:
Questions and answers: my career change to data science
I am a sinner.
Another post, from the table of contents

A presentation on Bayesian hierarchical modeling

This is a slide deck that I put together for my day job. It explains Bayesian hierarchical modeling.

Link to the slide deck

It's given in the context of A/B testing and multiple comparisons, and assumes some knowledge of Beta distributions - but even if you don't have that background, it should still be useful as an intuitive, conceptual understanding of Bayesian logic and hierarchical modeling.

In particular, there's one part in my Bayesian evaluation for the likelihood of Christ's resurrection where I mention Bayesian hierarchical modeling in passing. This presentation actually came out of me thinking through that part. So I thought I'd make a post out of it.

You may next want to read:
Sherlock Bayes, logical detective: a murder mystery game
A record of the total solar eclipse
Another post, from the table of contents

A small addendum to the Bayesian evaluation of the resurrection

When faced with the argument presented in Bayesian evaluation for the likelihood of Christ's resurrection, a common tactic is to try to push the prior lower. After all, this is one of the very few options available when one has no evidence on their side.

But as I argued in the post itself, this will not work. The constraints of empirical reasoning will simply not allow it. The opponents of the resurrection will want the prior to be 1e-40, or 1e-100 instead of 1e-11. Well, there's simply no empirical basis for choosing such numbers. So they have to be made up, like the evidence they don't have.

Some will argue for some kind of theoretical justification, and push for things like the Beta(0, 0) distribution, which is clearly absurd in our situation, as it's immediately falsified upon any kind of contact with reality. Another will want to impose some kind of hyperprior on the prior, hoping for a different kind of conclusion.

Well, the most common hyperprior on the Beta distribution is the uninformative prior where the pdf is proportional to (a + b) ** -2.5. I thought I'd investigate this hyperprior. What if we draw our Beta distributions from this hyperprior, and then add 1e11 negative samples to it? What kind of distributions would we get?

The following is a histogram of 1e6 Beta distribution samples drawn from the (a + b) ** -2.5 hyperprior, after we added the 1e11 negative samples. The x axis is the mean of each of the Beta distributions (that is, a/(a+b)), on a log scale.

The salient features are fairly obvious: this distribution is centered close to 1e-11, and even at its most extreme (i.e. even at a 1e-6 chance) it never goes beyond 1e-17. If this were the prior distribution we used instead of 1e-11, essentially nothing would change.

Of course, the skeptic can continue to propose alternate hyperpriors or hyper-hyperpriors. But every layer on this theoretical tower represents an exponential decrease in likelihood, so they would be even harder pressed to justify their choice empirically. In the end, the Beta(0.5, 0.5) prior remains the simplest and most correct one to use for our problem: it has all of the desirable characteristics and any attempt to significantly deviate from it results in immediate falsification or a divorce from empiricism.

You may next want to read:
Bayesian evaluation for the likelihood of Christ's resurrection
Basic Bayesian reasoning: a better way to think (Part 1)
Another post, from the table of contents