Continuing my thoughts on random experiments and what can go wrong:
Another common problem is Attrition, especially situations where the attrition rate is not randomly distributed (if the attrition rate is randomly distributed then you have lost power in your study).
After random assignment and administration of treatment/program/test, some subjects will not be available and so you won’t have information about the impact of the treatment/program/test.
Bias to the treatment effect estimate can be either positive or negative:
There is no perfect statistical solution for differential attrition rates though there some options including.
– One can estimate “bounds” on the resulting bias
– One can focus on analyzing just the complete cases using non-random design methods including modeling the probability of attrition and develop weighted, matched or other regression-adjusted methods
Also, it is important to explore why there is differential attrition and to see what can be done to adjust the design so as to reduce the attrition in future studies.
