Back to book
This page will give you the means for performing simple Bayesian analyses.
For a practical tutorial see:
Dienes, Z. (2014). Using Bayes to get the most out of non-significant results. Frontiers in Psycholology, 5: 781. doi: 10.3389/fpsyg.2014.00781
Dienes, Z., & McLatchie, N. (submitted). Four reasons to prefer Bayesian over orthodox statistical analyses. Psychonomic Bulletin & Review,
The Bayes factor tells you how strongly data support one theory (e.g. your pet scientific theory under test) over another (e.g. the null hypothesis). It is a simple intuitive way of performing the Bayesian equivalence of significance testing, telling you the sort of answer which many people mistakenly think they obtain from significance testing, but cannot. A "null" result in significance testing, for example, does not automatically mean you should reduce your confidence in the theory under test; often you should actually increase your confidence. A non-significant p-value does not tell you whether you have evidence for the null or no evidence for any conlusion at all (or indeed evidence against the null). Yet people routinely take a non-significant result as indicating they should reduce their confidence in a theory that predicts a difference.
The Bayes factor needs two types of input: 1) a summary of the data and 2) a specification of what the theories predict. In total you will only need to enter about four numbers!
1) In a situation where you could do a t-test, the data summary is exactly the same as would be used for a t-test:
a) the sample mean difference between conditions, or between a mean and a baseline, call this meandiff; and
b) the standard error of the difference, call this SE.
Note that t = meandiff/SE. Thus, if you know t and the mean differnce between conditions, you can get the relevant SE from SE = meandiff/t. This applies for any type of t-test.
(Note more generally: a) could be any sample statistic, such as a regression coefficient, and b) is its standard error, so long as a) is distributed roughly normally.)
In sum: For the first step, enter the difference between conditions in the "sample mean" box and its standard error in the "standard error" box.
2) Next you specify the theory you are testing against the null hypthesis. Specifying the theory means saying what range of effects are consistent with the theory and if any are particularly likely. The calculator calls the plot of different plausibilities of population effects given the theory "p(population effect|theory)", and asks if this is uniform. A simple rule is that if you can say what the maximum plausible effect is, say "yes"; otherwise say "no".
a) If you can specify a plausible maximum effect, use a uniform from 0 to that maximum. Enter "0" in the lower limit box and the maximum in the upper limit box. Then click "Go!"
b) If you can specify a plausible predicted effect size P (e.g. based on a previous study or meta-analysis), say "No" to a uniform. Three new boxes will come up, asking for the mean, standard deviation and number of tails (of a normal). A simple rule: If your theory allows effects to go in either direction,set the mean to 0, the standard deviation to P, and the tails to 2. If the theory makes a directional prediction, set mean to 0, SD to P and tails to 1. Then click "Go".
In sum: For the second step, you enter two numbers to describe your theory.
Note: A Bayes factor that is not based on the predicitons of your theory will be irrelevant to your theory.
A Bayes factor of 3 or more can be taken as substantial evidence for your theory (and against the null) and of 1/3 or less as evidence for the null (and against your theory). Bayes factors between 1/3 and 3 show the data do not provide much evidence to distinguish your theory from the null: The data are insensitive.
A 5-minute short instructional on using the calculator
Now click here to calculate your Bayes factor!
For those who use Matlab, here is Matlab code for calculating Bayes factor in the same way as the flash program above. Baguley and Kaye (2010) provide equivalent R code. John Christie has also provided R code for the calculator, modified so that one can adjust the quality of the estimation of area under the curve; John claims greater accurcy for his calculator!
For information on how the Rouder et al (2009) calculator differs from the Dienes (2008) one. This page also provides R code for a different calculator for more flexibility in specifying H1.
For Bayes factor calculators for the binomial situation see here for two groups and here for one group .
Five minute Bayes:
The weakness of power
How many participants might I need?
How to analyze a 2X2 contingency table
As well as a Bayes factor, it is usualy useful to determine what the most plausible set of population mean differences are, given your data and other constraints.
You start with prior beleifs about the population parameter. Assume you can represent your prior by a normal distribution (without grave misrepresentation) and also that your data are normal. Once you have determined the mean and standard deviation of your prior, collected data and hence found the mean and standard deviation of your likelihood (i.e. the mean difference in your data and its standard error), use this flash program to determine the mean and standard deviation of your posterior and look at graphs of the prior, likelihood and posterior distributions. If your prior is quite vague, the posterior is largely determined by the data. Thus your new prior before looking at the next study will be a normal distribution with a mean equal to the mean of Study 1 and a standard deviation equal to the standard error of the mean from Study 1.
Thus you can meta-analytically combine evidence across a series of studies with the same DV in the following way: For the mean of the prior enter the mean (mean difference etc) of Study 1 and for the standard deviaton of the prior enter the standard error of the mean difference. (This distributon represents the rational beleifs to have about the population paraemter value after seeing Study 1, given a vague prior beforehand.) Enter the mean difference for Study 2 as the mean of the likelihood and the standard error of Study 2 as the standard deviation of the likelihood. The posterior then indicated by the program gives the best estimate of the populaton parameter and its uncertainty in the light of both Studies 1 and 2. This could form the new prior to combining with a Study 3, and so on iteratively. If you have several studies with the same DV, this procedure can be followed to obtain an overall estimate of the mean difference and its standard error, which can be used in a Bayes factor calculator to determine the overall strength of evidence for H1 versus H0, or to evaluate credibiltiy intervals overall (see Dienes, 2014, for the principles of inference by interval).
Example: a previous study found that asking people to peform maths problems for 5 minutes a day increased their self discipline generally so they ended up doing the washing up two extra days each week. You replicate with different will-power interventions in three studies, finding the following increases in number of days of doing washing up each week: 0.5 (SE = 1.2), 2 (SE = 0.9), -0.5 (SE = 1.5). After Study 1, and before Study 2, one's prior would have a mean of 0.5 days and a SD of 1.2 (assuming before study 1 one had a very vague prior). After Study 2 and before Study 3, one's posterior from Study 2, and hence one's prior for Study 3, now has mean 1.46 (SD = 0.72). Finally, after Study 3, one's posterior has mean 1.09 (SD = 0.65). To perform a Bayes factor on the three studies as a whole, enter 1.09 as the sample mean and 0.65 as the standard error. Using a half-normal with SD = 2 days (the effect size from the original study), B = 2.08, indicating the theory that practicing will-power increases washing up episodes does not have substanital evidence either for or against provided by the three studies. (It might be worth exactly replicating Study 2 or the original study to see how that affects the overall evidence.)
To test your intuitions concerning Bayesian versus Orthodox statistics try this QUIZ.
For other practical examples of using Bayes factors:
Dienes, Z (2015). How Bayesian statistics are needed to determine whether mental states are unconscious. In M. Overgaard (Ed.), Behavioural Methods in Consciousness Research. Oxford: Oxford University Press.
For a discussion of Bayes and the credibility crisis in Psychology:
Dienes, Z. (2016) How Bayes factors change scientific practice. Journal of Mathematical Psychology,
For a discussion of conceptual issues:
Dienes, Z. (2011). Bayesian versus Orthodox statistics: Which side are you on? Perspectives on Psychological Sciences, 6(3), 274-290.
A talk on how to use Bayes given at Lancaster earlier in 2015.
This is a lecture I gave on Bayes to Masters students at University of Sussex in 2014.
A talk delivered October 2014 on the relation of Bayes to the credibility crisis.
An essay I set students is: "Perform a Bayesian analysis on a part of the data from your project or from a paper published this year (consider an interesting question tested by a t-test – one test will do). Compare and contrast the conclusions from your analysis with those that follow from an analysis using Neyman-Pearson (orthodox) statistics. "
See also this assessment of several topics from the book.
For papers using a Bayes factor for every inferential test ("a B for every p"):
Ling, X., Li, F., Qiao, F., Guo, X., & Dienes, Z. (2016). Fluency Expresses Implicit Knowledge of Tonal Symmetry. Frontiers in Psycholology, 7, 57, doi: 10.3389/fpsyg.2016.00057
Ziori, E., & Dienes, Z. (2015). Facial beauty affects implicit and explicit learning of men and women differently. Frontiers in Psycholology, 6, 1124. doi: 10.3389/fpsyg.2015.01124
For papers using Bayes factors for certain key tests:
Greve, A., Cooper, E., & Henson, R. N. (2014). No evidence that ‘fast-mapping’ beneﬁts novel learning in healthy older adults. Neuropsychologia, 60, 52–59
Allen, C. P. G. , Dunkley, B. T., Muthukumaraswamy, S. D., Edden, R., Evans, C. J., et al. (2014) Enhanced Awareness Followed Reversible Inhibition of Human Visual Cortex: A Combined TMS, MRS and MEG Study. PLoS ONE 9(6): e100350. doi:10.1371/journal.pone.0100350
Allen, C. P. G., Sumner, P., & Chambers, C. D. (in press). The Timing and Neuroanatomy of Conscious Vision as Revealed by TMS-induced Blindsight. Journal of Cognitive Neuroscience
Anderson, H. P., Seth, A. K., Dienes, Z., & Ward, J. (2014). Can Grapheme-Colour Synaesthesia be Induced by Hypnosis? Frontiers in Human Neuroscience, 8:220. doi: 10.3389/fnhum.2014.00220
Armstrong, A. M., & Dienes, Z. (2014). Subliminal Understanding of Active vs. Passive Sentences. Psychology of Consciousness: Theory, Research, and Practice, 1, 32-50.
Eitam, B, Glass, R. S., Hillel, A., Dienes, Z., Shoval, R., & Higgins, E. T. (2014). Are Task Irrelevant Faces Unintentionally Processed? Implicit Learning as a Test Case. Journal of Experimental Psychology: Human Perception and Performance, 40(5), 1741-1747
Mealor, A., Dienes, Z., & Scott, R. B. (in press). Unconscious sources of familiarity can be strategically excluded in support of conscious task demands. Psychology of Consciousness: Theory, Research, and Practice,
Tan, L. F., Dienes, Z., Jansari, A., & Goh, S. Y. (2014). Effect of Mindfulness Meditation on Brain-Computer Interface Performance. Consciousness and Cognition, 23(1), 12-21.
Armstrong, A. M., & Dienes, Z. (2013). Subliminal Understanding of Negation: Unconscious Control by Subliminal Processing of Word Pairs. Consciousness & Cognition, 22 (3), 1022-1040. (see footntoe 1, and results for experiments 3, 4, and 5)
Dienes, Z, & Hutton, S. (2013). Understanding hypnosis metacognitively: rTMS applied to left DLPFC increases hypnotic suggestibility. Cortex, 49, 386-392 (see footnotes 2 and 3 - footnote 3.)
Fu, Q., Bin, G., Dienes, Z., Fu, X., Gao, X. (2013). Learning without consciously knowing: Evidence from event-related potentials in sequence learning. Consciousness and Cognition, 22 (1), 22-34, (see page 30)
Fu, Q., Dienes, Z., Shang, J., & Fu, X. (2013). Who learns more? Cultural differences in implicit sequence learning. PLoS ONE 8(8): e71625. doi:10.1371/journal.pone.0071625
Guo, X., Jiang, S., Wang, H., Zhu, L., Tang, J., Dienes, Z., & Yang, Z. (2013). Unconsciously Learning Task-irrelevant Perceptual Sequences. Consciousness and Cognition, 22(1), 203–211.(see pages 206-207 for using Bayes Factors to interpret a null result using the Zero Correlation Criterion of unconscious knowledge; see also footnote 6)
Guo, X., Li, F., Yang, Z., & Dienes, Z. (2013). Bidirectional transfer between metaphorical related domains in Implicit learning of form-meaning connections. PLoS ONE, 8(7): e68100. doi:10.1371/journal.pone.0068100 (Bayes is used in the norming study in Materials and throughout the Results)
Li, F., Guo, X., Zhu, L., Yang, Z., & Dienes, Z. (2013). Implicit learning of mappings between forms and metaphorical meanings. Consciousness & Cognition,22 (1), 174-183..(see page 177-178 for using Bayes Factors to interpret a null result using the Zero Correlation Criterion of unconscious knowledge.)
Mealor, A. D., & Dienes, Z. (2013). The speed of metacognition: Taking time to get to know one’s structural knowledge. Consciousness & Cognition, 22 (1), 123-136. (see page 131)
Mealor, A. D., & Dienes, Z. (2013). Explicit feedback maintains implicit knowledge. Consciousness & Cognition, 22 (3), 822-832. (See section 3.4)
Parris, B. A., & Dienes, Z. (2013). Hypnotic suggestibility predicts the magnitude of the imaginative word blindness suggestion effect in a non-hypnotic context. Consciousness & Cognition, 22 (3), 868-874. (Bayesian trend analysis)
Parris, B. A., Dienes, Z., Bate, S., & Gothard, S. (2013). Oxytocin impedes the effect of the word blindness posthypnotic suggestion on Stroop task performance. Social Cognitive and Affective Neuroscience, doi: 10.1093/scan/nst063 (see Results)
Parris, B. A., Dienes, Z., & Hodgson, T. L. (2013). Application of the ex-Gaussian function to the effect of the word blindness suggestion on Stroop task performance. Frontiers in Cognition, 4, 647. doi: 10.3389/fpsyg.2013.00647
Semmens-Wheeler, R., Dienes, Z., & Duka, T. (2013). Alcohol Increases Hypnotic Susceptibility. Consciousness & Cognition, 22 (3), 1082–1091.(A new Bayesian approach to mediation)
Shang, J., Fu, Q., Dienes, Z., Shao, C. &, Fu, X. (2013). Negative affect reduces performance in implicit sequence learning.PLoS ONE 8(1): e54693. doi:10.1371/journal.pone.0054693. . (See results sections for using Bayes Factors to allow conclusions from using non-signfiicant recognition performance to infer unconscious knowledge.)
Verbruggen F, Adams RC, van ‘t Wout F, Stevens T, McLaren IPL, et al. (2013) Are the Effects of Response Inhibition on Gambling Long-Lasting? PLoS ONE 8(7): e70155. doi:10.1371/journal.pone.0070155 (see Data nalysis section)
Dienes, Z., Baddeley, R. J., & Jansari, A. (2012). Rapidly Measuring The Speed Of Unconscious Learning: Amnesics Learn Quickly And Happy People Slowly. PLoS ONE 7(3): e33400. doi:10.1371/journal.pone.0033400 (see results for experiment 4)
Jiang, S., Zhu, L., Guo, X., Ma, W., Yang, Z., & Dienes, Z. (2012). Unconscious structural knowledge of tonal symmetry: Tang poetry redefines limits of implicit learning. Consciousness & Cognition, 21,476-486. (see bottom 481 - top 482)
Mealor, A. D., & Dienes, Z. (2012). Conscious and unconscious thought in artificial grammar learning. Consciousness & Cognition, 21, 865-874. (see bottom page 870- top page 871)
Parris, B. A., Dienes, Z., & Hodgson, T. L. (2012). Temporal constraints of the word blindness post-hypnotic suggestion on Stroop task performance. Journal of Experimental Psychology: Human Perception and Performance 38(4), 833-837. (see foootnote 3)
Rohrmeier, M., Fu, Q., & Dienes, Z. (2012).Implicit learning of recursive, hierarchical grammatical structures. PLoS ONE 7(10): e45885. doi:10.1371/journal.pone.0045885 (see just below Table 3 where a significant result is actually shown to be evidence for the null relative to our theory that predicted a difference)
Newell, B. R., & Rakow, T. (2011). Revising beliefs about the merit of unconscious thought: Evidence in favour of the null hypothesis? Social Cognition, 29,
Verdonschot, R.G., Kiyama, S., Tamaoka, K., Kinoshita, S., La Heij, W., & Schiller, N.O. (2011). The Functional Unit of Japanese Word Naming: Evidence From Masked Priming. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37 ( 6), 1458–1473.
Thanks to Online Experiments for providing the Flash conversions of my Matlab. Use Online Experiments for convenient programming of all your experiments!