Conditional Probability and Bayes
#Bayesian #Bayes's Theorem #Conditional Probability
The Bayesian view of probability is quite objective and also more general than the frequentist’s view. It doesn’t rely on repeatition of events.
one of the most important concepts in all of probability theory — that of conditional probability.
– Sheldon M. Ross
Topics
- Association Rules
- Conditional Probability
- Naive Bayes
Association Rules
- A measure of co-occurrence
Prize in Boxes
Three boxes M, N, O:
- Only one of them refers to a prize.
- The participant claims one of them, e.g., M.
- The host removes one of the empty boxes (N) from the other two boxes (N, O).
- Now we have only two candidate boxes for the participants, M, O.
- The participant is asked to reclaim a box.
Question:
- Should the participant switch?
Frequentist vs Bayesian
Frequentists:
- Probability is based on the repetition of events.
- Without repetition, the probability is unknown.
- Events are random.
- Make predictions based on probabilities.
- Relies on NHST to validate our models.
Bayesian:
- Probability is objective (educated guess). It is a conceptual tool to describe our degree of certainty.
- Probability is not necessarily a one-to-one map of occurrences of events.
- Data (such as a previous reoccurring event) is used to update our beliefs.
- Parameters of models are random.
Joint Probability
Joint Probability
Examples of joint probabilities:
: Event and event are so different that they will never happen together. In this case, . : and are independent of each other. .
Conditional Probability and Bayes’s Theorem
- It’s a rescaling/(re)normailization of
: ; - We didn’t specify
and : also holds; - Apply
:
In
: prior : posterior : likelihood
Solutions to the Prize in Boxes Problem
: The participant made a wrong choice at the first attempt, then switched. : We will solve this.
Applying Bayes’ theorem,
We update our probability perception when we have new data.
Rare Disease
We are about to test for a rare disease:
- Prevalence:
- Test method:
- Test method:
The positive rate for a random person is
If a test is positive, the probability of the person being test has the disease is
Naive Bayes
- Chapter 16 of Introduction to Probability and Statistics for Engineers and Scientists (6th ed.) by Sheldon Ross.
- Naive Bayes
Planted:
by LM;
: 2020-11-28T14:30:00 - 2020-11-28T16:00:00 (CET)
: More details on page
Conditional Probability Estimation
Table of Contents
References:
Current Ref:
- cpe/01.conditional-probability-and-bayes.md