The Amelioration of Uncertainty

## Was “Statistical zealots” copied from an old letter?

Recently Jeff Leek over at Simple Statistics posted what I though was an original discussion about Statistical Zealotry. But then I saw this newly unearthed letter from an unknown European Professor to a colleague. It’s at least 300 years old, but the similarity with Leek’s post are so strong it makes one wonder if it weren’t plagiarized. It’s reproduced below for your comparison: (more…)

November 28, 2013

## What’s your connection to the distant stars?

Many have thought motion somehow required the presence of other objects. Just like gravity requires two masses to exist, it seems like motion is meaningless in a universe with just one mass. This suggests inertia itself somehow requires other masses and perhaps results from an interaction with them. (more…)

November 26, 2013

## Hidden dangers of a sloppy understanding of probabilities

Andrew Gelman’s new favorite example of the hidden dangers of noninformative priors is the following. If we observe data y ~ N(theta,1) and get y=1, then this is consistent with being pure noise, but the posterior probability for theta>0 is .84. Gelman thinks this is an example of priors gone wild, but I claim this prior works perfectly. Here’s why. (more…)

November 24, 2013

## Laplace didn’t have a time machine

Dr. Mayo responded to criticism of the Severity Principle here. The main points are (A) if SEV differs from Bayes it doesn’t mean SEV’s bad (B) you shouldn’t compare SEV and Bayes because they do different things (C) A prior can always be invented which allows Bayes to replicate Frequentists success, but that’s a meaningless attempt by Bayesians to save face, (D) it’s the philosophy of the Severity Principle which matters more than the numbers. (more…)

November 15, 2013

## Classical Statistics really is screwed.

It’s believed the crises in science will abate if we only educate everyone on the correct interpretation of p-values and confidence intervals. I explained before in this long post why this isn’t true. Below is a summary. (more…)

November 12, 2013

## Data Compression and the Nobel in Economics

Consider the following data compression problem. Suppose we have a large data set we wish to transmit. They’re too many to send directly but luckily the precise values aren’t important. Slightly different values would work as long as the data retained it’s overall character. So how do we avoid transmitting n raw data points? (more…)

November 11, 2013

## If we observed p_hat = .46, why do we use p=.5?

I aim to commit statistical sin. I’m going to accept the null hypothesis for no other reason than because I “failing to reject it”. Having tarnished my reputation with that, I’ll finish by ignoring the only data available and base everything on non-informative priors which prominent authorities assure us don’t even exist. Let the debauchery begin. (more…)

November 6, 2013

## Bayes Theorem and Quantum Mechanics

I was reminded of this old post by Andrew Gelman about whether Quantum Mechanics requires a change in the axioms of probability theory. Without weighing in I’ll just point out that Bayesian Statistics is more general than Bayes Theorem and this affects the controversy. (more…)

November 2, 2013

## Max Planck and the Foundations of Statistics

Statistics is full of old and difficult ideas. It’s time for something new and simple. Well, it’s not actually new, but it will seem that way to most. The story begins with the physicist Max Planck over a century ago. (more…)

October 30, 2013

## Mother Nature makes fools of Statisticians

The most dispiriting thing I’ve read in a while was this post by John Cook involving coin flips. The discussants are certain they have all the puzzle pieces and all that remains is to arrange them with the right prose. There’s not even a hint of awareness that they might be missing something. (more…)

October 28, 2013

## What the data doesn’t tell you

Andrew Gelman recently commented on the difficulties of measuring or interpreting just about anything, and gave an example about sexual harassment in the Marine Corps. I wanted to relay a story. There is no general conclusion to be drawn that I can see; it’s merely offered to sheltered academics to show that people are a hell of a lot more interesting than data. (more…)

October 8, 2013

## The most interesting thing you’ll hear about Fisher today

Nothing brings out the silliness in smart people like Quantum Mechanics; a subject I always associate with … R. A. Fisher. I confess to liking Fisher more than Bayesians should. Unlike the forgettable p-value conjurers I’ve known in person, Fisher’s writing portrays a thinking, creative scientist and feels oddly familiar. So what does Fisher have to do with Quantum Mechanics? (more…)

September 25, 2013

## Integrity and the Jaynesian/Frequentist divide.

The heart of the Jaynesian/Frequentist divide can be found in the post “The Definition of a Frequentist”. A Frequentist views weather prediction as one of getting the frequency of weather events right. A Jaynesian’s goal is to pin down the one-off true weather sequence as tightly as possible. I illustrated this difference in a way so clear it’s easy to see which is better. I got this example from Jaynes: (more…)

September 23, 2013

## Cox’s Theorem and Mayo’s Error Statistics

This post will take a different tack. Rather than criticize the Severity Principle, I will attempt to patch it up. But as we try to fix problems with SEV, we’ll run up against Cox’s Theorem:

A measure of evidence like SEV will either be equivalent to using probabilities or have serious problems.

The mathematics of Cox’s Theorem isn’t in doubt, but it’s not clear the conditions of the theorem apply to SEV. So this makes for an interesting struggle between two philosophies put to mathematics. (more…)

September 19, 2013

## The Wages of Philosophical Sin are Research Death

The last post about Mayo’s Severity Principle got me thinking about that xkcd cartoon which generated so much hate-and-discontent among Frequentists. I didn’t care for it because all Frequentists can say in response is “we’re not that dumb”; which is a reasonable point and instantly sends the debate in worthless directions. (more…)

September 17, 2013

## Does Mayo’s “Error Statistics” fix classical statistics?

Taking a break from Statistical Mechanics I noticed Corey Yanofsky, whom I respect a great deal, is starting a blog. Corey plans to explore Dr. Mayo’s Severity Principle, which he describes as the “strongest defense of frequentism I’ve ever encountered.” A similarly great geometer, Cosma Shalizi, is even more effusive.

I believe Dr. Mayo misunderstands error distributions and the basic facts concerning them (see here and here), but philosophy can be argued endlessly. It’s more productive to examine Corey’s (and Mayo’s) claim that “the severity principle scotches many common criticisms of frequentism”. (more…)

September 12, 2013

## The odds of choking from atmospheric fluctuations

This is the first of a two part series about Bayes Theorem and the Second Law of Thermodynamics. It begins with the question:

What are the odds that one hour from now, the diffuse air in the sealed and insulated room I’m in will position itself in the half where I’m not, thereby causing me to choke?

September 10, 2013

## Bayes Theorem and the Forward Arrow of Time.

Shalizi wrote a paper purporting to show Statistical Mechanics couldn’t be an application of Bayesian Statistics because Bayesian updating implies Entropy is decreasing. Such confusion arises because empirical entropies computed from frequencies are confused with computed from probabilities. This mix-up is natural for a Frequentist like Shalizi, but since it’s at the heart of every misunderstanding about Statistical Mechanics, it’s worth looking at an example so simple anyone can see what and mean and why they sometimes move in different directions. (more…)

August 29, 2013

## The Intellectual oubliette of p-values and Confidence Intervals.

Criticizing P-values and Confidence Intervals (CI’s) is nothing new, but my aim is different. My goal is to tear down, shred, burn and destroy them using arguments you probably haven’t seen before, aided by Entropy and it’s various mathematical properties. Let the fun begin. (more…)

August 27, 2013

## When did statistics jump the shark?

Statistics jumped the shark the moment they adopted the following definition, (Gelman & Hill, page 13):

A probability distribution corresponds to an urn with a potentially infinite number of balls inside. When a ball is drawn at random, the “random variable” is what is written on the ball.

for situations where no such “urn” or “population” existed. To explain why requires an answer to the question: when will data appear to be drawn from a frequency distribution ? (more…)

August 20, 2013

## IID doesn’t mean what you think it does

The post “What do we need to Model?” showed what our goal in modeling errors should be. This one shows how it’s achieved. Assigning a distribution to the fixed parameters is like finding a prior ; it’s successful whenever the ‘s high probability region accurately describes where is located in space. (more…)

August 14, 2013

## What do we need to model?

Part of the communication difficulty between Bayesians and Frequentists is that they’re modeling different things using similar mathematics. So it’s worth looking closely at a simple example to see what each is hoping to achieve with their methods. (more…)

August 9, 2013

## All Probabilites are for One-off Events

Most Statisticians think of sampling distributions as a kind of physical model for an infinite sequence of events. They view priors on the other hand as something different since they can be assigned to one-off hypothesis like “Republicans will win the 2016 election” or “A meteor wiped out the dinosaurs”. This conventional wisdom gets it wrong however, since all distributions describe the probability of one-off events. (more…)

August 8, 2013

## The Definition of a Frequentist

The illustrious Dr. Mayo recently reminded me of Larry Wasserman’s take on what is a Frequentist. In Larry’s words:

[Nate Silver] “One of the most important tests of a forecast — I would argue that it is the single most important one — is called calibration. Out of all the times you said there was a 40 percent chance of rain, how often did rain actually occur? If over the long run, it really did rain about 40 percent of the time, that means your forecasts were well calibrated.”

[Wasserman] It does not get much more frequentist than that. And if using Bayes’ theorem helps you achieve long run frequency calibration, great. If it didn’t, I have no doubt he would have used something else.

Well I have major doubts; especially since non-calibrated forecasts are often far superior to calibrated ones. (more…)

August 5, 2013

## The Decline of Science and Science (Magazine)

According to an article in Science magazine titled “Quantifying the Influence of Climate on Human Conflict” global warming may result in “amplified rates of human conflict could represent a large and critical impact of anthropogenic climate change”. The article is gated, so I can’t look at the details buts it’s worth doing a sanity check on the main conclusion. (more…)

August 2, 2013

## Data Science is inherently limited

A previous post showed how diffuse distributions on a space can be used to estimate functions which aren’t sensitive to . This is the essence of statistics as evidence by the ‘s found in coin tossing, election prediction, Statistical Mechanics, and error statistics. But while being able to easily predict is handy, sometimes we’d rather observe and learn something about . Unfortunately these goals are in tension, which is why “data science” is inherently limited. (more…)

July 25, 2013

## The Higgs Boson and Bayesian Statistics

When the discovery of the Higgs Boson was announced there was controversy over the use of classical Hypothesis Testing and the lack of Bayes in particle physics. Some reactions, with useful links, can be found here, here, here, and here. In this post, I’ll point out what particle physics has to lose by ignoring Bayesians. (more…)

July 23, 2013

## Noninformative Priors

The esteemed Dr. Wasserman claimed “This is a general problem with noninformative priors. If is somehow noninformative for , it may still be highly informative for sub-parameters, that is for functions where and .”

Not only is it not a problem, but it’s the key to Statistics and fundamental to the philosophy of science. (more…)

July 21, 2013

## The Law of Trading Edges

Andrew Gelman recently ran a post title “Why waste time philosophizing?” My answer is that different philosophies dramatically affect how, and how well, we get answers from probabilities. This is illustrated with an example from Finance.

February 16, 2013

## Finance, Cauchy, and Statistical Power

Nominally this post is about the importance of Statistical power. In reality it answers the immortal, but often neglected question: “why everyone in Quantitative Finance should be a Bayesian and love the Cauchy Distribution?” (more…)

March 7, 2012

## Are Probability Distributions in Physics Correct?

In some cases, no. For example, 95% Confidence Intervals for physical constants are known to have coverage rates noticeably less than 95%. But this is far from the only use of statistics in physics. There are the various Ensembles of Statistical Mechanics which yield accurate predictions. Are these correct in a Frequentist sense? (more…)

March 3, 2012

## A Challenge for Frequentists

Here is a simple challenge for the Frequentists out there. How do you include information about the range of a mean when estimating it from a series of measurements? (more…)

February 16, 2012

## Free Energy and Statistics

There is a long and sordid history of trying to use thermodynamic analogies in Economics. Everyone senses thermodynamics is relevant somehow, but no one’s able to make the notion specific, true, and useful. The latest effort comes from Andrew Gelman, who speculates on “Free Energy” in Economics. Ironically, “Free Energy” occurs all the time in Gelman’s field of Statistics; a fact which gives clues to its profitable use in Economics.
(more…)

November 15, 2011

Comment (1)

## How Much Did Maxwell’s Equations Cost?

Tyler Cowen’s The Great Stagnation claims progress is slowing leading to a kind of stagnation. Cowen’s pamphlet spells out the what and the why, while many have added debate. My favorite anecdote is the graph of per capita GDP growth, which shows a decline impervious to taxes, spending, elections, and everything else that matters. It certainly suggests diminishing returns from some prior successes, whatever they were: (more…)

October 11, 2011

## Non-random Metropolis-Hastings Algorithms

It appears to be a quite general principle that, whenever there is a randomized way of doing something, then there is a non-randomized way that delivers better performance but requires more thought.

This remark by E. T. Jaynes will be illustrated with the Metropolis-Hastings algorithm. (more…)

September 29, 2011

## Randomization Doesn’t Solve the Problem

Imagine a medical researcher is conducting a drug trial on 1000 people and wishes to compare it to a placebo. The researcher randomly assigns 500 to the placebo group and the rest to the treatment group. The hope is that if there is some unknown variable influencing the efficacy of the drug, then it will evenly split between the two groups, allowing the statistician to say if the drug worked. Unfortunately, the belief that randomization removes the problem of unknown factors or influences is false. (more…)

September 19, 2011

## Why Conspiracy Theories are Bunk

The peculiar philosopher John Derbyshire once wrote :

Practically all conspiracy theories are false … You will likely get through life with your mind’s serenity undisturbed if you dismiss without investigation every conspiracy theory that comes to your attention. I do so reflexively.

September 16, 2011

## Models so Nice they Melt Ice

How to choose probability models to avoid over fitting data? Below I show how for Maximum Entropy models, but the results are more general than this special case. There is even a connection to melting ice, but you have to read to the end to see it.

August 6, 2011

## Finance Theory Buffetted by Practice

Warren Buffett brazenly discounts cash flows using risk free rates. This effrontery is excused by claiming his predictions are so good as to be without risk; a view hard to square with Buffett’s occasional losses. This post outlines a better, or at least more useful, explanation using financial economics. (more…)

June 15, 2011

## Simple Insights into Insurgencies

Much blood, sweat, and beer has been spilled understanding insurgencies.  Having none of these in excess, however, I’m forced to use simpler methods.  What follows is low tech, but it gets results. (more…)

June 8, 2011

Comment (1)

## You’re Calculating Volatility Wrong

You’re a student of volatility.  You wrote the book on volatility.  You probably invented volatility and would rather find out you’re adopted than be accused of unlawful calculation.  Nevertheless, you’re getting it wrong. (more…)

June 8, 2011