On zero probability in the continuous variable case

Here is a quote from a SIGGRAPH course by Welch and Bishop:

In the case of continuous random variables, the probability of any single discrete event A is in fact 0.

The same quote could be taken from many introductory texts on probability theory. It seemed absurd to me the first time when I read it. Back then, I got over it, attributing the feeling to my own inexperience. Well, after some years and I dare say improved understanding, I know that it is in fact an absurd - or at least an uncomfortably sloppy - statement. Moreover, I can explain why and get rid of the confusion.

There are two main reasons for the intuitively perceived absurdity:

  • Zero probability is synonymous with "impossible event". If the quoted statement was true, it would follow that, regardless what value of the random variable you choose, it is impossible (and I really mean any value). Yet we know from experience, which our model is supposed to reflect, that the random variable does assume some value in reality.
  • The positive probability of a value falling in a given interval arises from summing probabilities (integration) of all discrete values within that interval. However, adding together zeros - even in an infinite loop - yields zero.

Of course, one could ask: if the probability P(A=x) is not 0 in the continuous case, then how big is this probability? The simple answer to that is: there is no continuous case, it is a figment of a mathematician's imagination, a model primarily intended to ease calculations, rather than a representation of reality. The zero probability "exists" in the same sense as a mathematical point "exists". On the other hand, when we talk about "possible" and "impossible" events, we talk about [our perceptions of] reality. We'd also like the connection to reality to remain intact when we use the notion of probability, continuous or not. Of course, if the continuous case is discretized (and you can choose to do it using as many discrete events as you desire), the "paradox" of possible zero-probability events is resolved at once.

Where do the idea and the bold assertions about P(A=x) = 0 come from, then? They are but a sloppy description of the limiting process, of increasing the number of events without bounds. That is, a way to say that "the more equiprobable events we have, the smaller the per-event probability". It is correct to say that we approach zero probability, which is quite a different thing from saying that we (ever) reach this value. In all practical thinking, we may safely ignore infinite processes and infinite "things" a mathematician is so fond of, or better yet, accept them as a convenient approximation of our discrete reality to which our actual reasoning applies.


krukow said...

I agree with your reasoning completely. Once again, probability theory would be much simpler if taught as "an extension to logic".

Keep it up!
- Karl

Anonymous said...

Your first assertion that zero probability is synonymous with impossibility is incorrect. Learn some measure theory.

jpl said...

Anonymous: Terms should be defined in a way which makes them both useful and consistent with what has been already defined and well understood. Not making zero probability synonymous with impossibility just leads to the sort of linguistic sloppiness that this post is about.

There are also other occassions on which mathematicians confuse themselves and innocent bystanders with careless language. For example, when trying to wrap their heads around the notion of infinite sets when they really mean generating processes with comfortably finite descriptions. It's a sort of "ontological poison" for the mind. Some even go mad through overdosing it.