Repeated measures ANOVA is a common task for the data analyst.
There are (at least) two ways of performing “repeated measures ANOVA” using R but none is really trivial, and each way has it’s own complication/pitfalls (explanation/solution to which I was usually able to find through searching in the R-help mailing list).
So for future reference, I am starting this page to document links I find to tutorials, explanations (and troubleshooting) of “repeated measure ANOVA” done with R
Functions and packages
(I suggest using the tutorials supplied bellow for how to use these functions)
- aov {stats} – offers SS type I repeated measures anova, by a call to lm for each stratum. A short example is given in the ?aov help file
- Anova {car} – Calculates type-II or type-III analysis-of-variance tables for model objects produced by lm, and for various other object. The ?Anova help file offers an example for how to use this for repeated measures
- ezANOVA {ez} – This function provides easy analysis of data from factorial experiments, including purely within-Ss designs (a.k.a. “repeated measures”), purely between-Ss designs, and mixed within-and-between-Ss designs, yielding ANOVA results and assumption checks. It is a wrapper of the Anova {car} function, and is easier to use. The ez package also offers the functions ezPlot and ezStats to give plot and statistics of the ANOVA analysis. The ?ezANOVA help file gives a good demonstration for the functions use (My thanks goes to Matthew Finkbe for letting me know about this cool package)
- friedman.test {stats} – Performs a Friedman rank sum test with unreplicated blocked data. That is, a non-parametric one-way repeated measures anova. I also wrote a wrapper function to perform and plot a post-hoc analysis on the friedman test results
- Non parametric multi way repeated measures anova – I believe such a function could be developed based on the Proportional Odds Model, maybe using the {repolr} or the {ordinal} packages. But I still didn’t come across any function that implements these models (if you do – please let me know in the comments).
- Repeated measures, non-parametric, multivariate analysis of variance – as far as I know, such a method is not currently available in R. There is, however, the Analysis of similarities (ANOSIM) analysis which provides a way to test statistically whether there is a significantdifference between two or more groups of sampling units. Is is available in the {vegan} package through the “anosim” function. There is also a tutorial and a relevant published paper.
Good Tutorials
- A basic tutorial about ANOVA with R (only the last bit holds some example of repeated measures) on personality-project
- A thorough tutorial on motor control lab
- A thorough tutorial on UCLA seminar page
- Another good tutorial by
Jonathan Baron and Yuelin Li on “Notes on the use of R for psychology experiments and questionnaires”
Troubelshooting
Unbalanced design
Unbalanced design doesn’t work when doing repeated measures ANOVA with aov, it just doesn’t. This situation occurs if there are missing values in the data or that the data is not from a fully balanced design. The way this will show up in your output is that you will see the between subject section showing withing subject variables.
A solution for this might be to use the Anova function from library car with parameter type=”III”. But before doing that, first make sure you understand the difference between SS type I, II and III. Here is a good tutorial for helping you out with that.
By the way, these links are also useful in case you want to do a simple two way ANOVA for unbalanced design
I will “later” add R-help mailing list discussions that I found helpful on the subject.
If you come across good resources, please let me know about them in the comments.
Howdy,
I am going to get a reputation as the disgruntled nay sayer. Two comments:
1) Type III sums of squares should be avoided for the very compelling reasons stated in the following document, which should be required reading for all Splus/R users
http://www.stats.ox.ac.uk/pub/MASS3/Exegeses.pdf
2) Repeated measures are handled much better within the linear mixed model framework. Unbalanced designs are handled better, with the note that for small samples the variances may not be estimated with any precision. But this would bias classical ANOVA in the unbalanced case as well.
–The grumpy old man
Hello Nicholas,
As always – thank you for a very informative reply.
I am joining Patrick with asking for any suggestions for good resources for performing repeated measures analysis with linear mixed models.
Best,
Tal
Thanks for posting this Tal, I’ve tried to poke around several times to find resources on repeated measure analysis with R and haven’t had much luck.
Nicholas, would you have any links to a good resource/tutorial on performing repeated measure analysis with the linear mixed model framework?
Cheers,
Pat
Thanks Tal for addressing this important (to me, at least 🙂 and yet still somewhat tricky (at least to the less savvy R user).
I’ve found the r-tutorial of Jonathan Baron & Yuelin Li helpful: http://www.psych.upenn.edu/~baron/rpsych/rpsych.html
Section 6.9 addresses repeated measure anova.
And there is the (unresolved?) issue of simple-main-effects. for example: http://tinyurl.com/ygkfrty
All the best,
dror
Thanks Dror,
I am glad this post was of help to you 🙂
I didn’t know about that resource you mentioned – I added it to the list of links – thanks.
Regarding Or’s question and the issue of “simple-main-effects”, I remember meeting him in the Israeli R tutorial session about a month ago. He asked me about it, I remember understanding better what he wanted, and invited him to e-mail me with a description of it so I might help him. He didn’t do so, so I dropped the subject.
Cheers,
Tal
These links are great! But have a look at the ez package; it’s all I use for repeated measures anymore.
https://cran.r-project.org/web/packages/ez/index.html
Hi Matthew,
Thank you so much for the heads up on this cool package.
Your comment inspired me to extend my post to include reference to it (while also giving you due credit).
Best,
Tal
Thanks for posting these resources. I’ve been reading the section in Agresti & Finlay’s statistics textbook on using repeated measures ANOVA (they use SAS and SPSS). But, I only have access to R and need some guidance. This is very helpful, including all the comments.
Thank you!
Very helpful thank you
great man!!!! I just added this notes to my archive
Thank you for the links, here is another simpler piece which made me go “aaah”.http://www.stat.umn.edu/geyer/5102/examp/dummy.html
I have tested differences between treatments using ANOSIM in R. But, how can I get the results of multiply comparison? Thanks a lot.
Tal, I’ve followed the link form StackExchange’s CrossValidated to here – don’t have a working solution to offer, so I’m not going to comment there.
However: this might be interesting.
Oliver-Rodríguez, J. C. and Wang, X. T. (2013), Non-parametric three-way
mixed ANOVA with aligned rank tests. British Journal of Mathematical
and Statistical Psychology. doi:10.1111/bmsp.12031
(Not implemented in R, currently, I guess. Also seen no discussions about it.)
I also tried out this, which comes with handy R code:
Herberich E., Sikorski J., Hothorn T. (2010), A Robust Procedure for
Comparing Multiple Means under Heteroscedasticity in Unbalanced Designs.
PLoS ONE 5(3): e9788. doi:10.1371/journal.pone.0009788
Still, I am not really convinced. Seems like there’s quite some demand for
something like this, but it’s mathematically quite challenging. Out of
my league, sadly.
FTR, as I often see people recommend adonis{vegan} for this type of analysis: these are not the droids we are looking for. Check out Kay Cichini’s blog for repeated measures and permutation trouble:
http://thebiobucket.blogspot.de/2011/04/repeat-measure-adonis-lately-i-had-to.html
– again, for complicated designs, this is a pain and really challenging.
Thank you for the detailed reply Probiers.
Hi,
The aligned rank transform has been implemented for R.
https://depts.washington.edu/aimgroup/proj/art/
Wobbrock, J.O., Findlater, L., Gergle, D. and Higgins, J.J. (2011). The Aligned Rank Transform for nonparametric factorial analyses using only ANOVA procedures. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI ’11). Vancouver, British Columbia (May 7-12, 2011). New York: ACM Press, pp. 143-146.
Hope this helps.
Hi, thanks for your helpful post! What exactly do you mean by unbalanced design? Is it unbalanced if the groups are not the same size?
Thank you!
Hi all,
You may find the welchADF package useful. It is aimed at conducting non-parametric omnibus tests and pairwise comparisons in arbitrary mixed models with any number of between and within subjects factors (allowing nesting for the latter) when the design is unbalanced and/or non-normal. It is an implementation of Welch ADF (approximate degrees of freedom) test, with the option of using trimmed means and Winsorized variances if desired.
https://cran.r-project.org/web/packages/welchADF/