[R-sig-ME] SE and CI for ICC

Discussion:

Mohd Masood

2014-09-12 16:20:34 UTC

I am using random intercept logistic model (in lme4) to calculated
Intraclass correlation coefficient (ICC). lme4 only provides point
estimates and standard deviation (not standard errors) of variance
estimates.These
point estimates can be used to calculated point estimates for ICC. The
problem is how can I calculate standard error and confidence interval for
ICC. I couldn't find any literature showing formula to calculate confidence
interval around ICC. Or is it not possible to calculate SE and CI for ICC
due to skewed sampling distribution (Please see PMCID: PMC3426610).

Thanks
Masood

[[alternative HTML version deleted]]

Ben Bolker

2014-09-12 19:57:06 UTC

Permalink

Post by Mohd Masood
I am using random intercept logistic model (in lme4) to calculated
Intraclass correlation coefficient (ICC). lme4 only provides point
estimates and standard deviation (not standard errors) of variance
estimates.These
point estimates can be used to calculated point estimates for ICC. The
problem is how can I calculate standard error and confidence interval for
ICC. I couldn't find any literature showing formula to
calculate confidence
interval around ICC. Or is it not possible to calculate
SE and CI for ICC
due to skewed sampling distribution (Please see PMCID: PMC3426610).
Thanks
Masood

Some possibilities:

* If you want the standard deviations of the variance estimates
(keeping the strong caveats about non-Normal sampling distributions
in mind), you could adapt the approach shown in
http://rpubs.com/bbolker/varwald (presumably formulas
for confidence intervals
of the ICC based on the variances of the estimates of the RE variances
are using the delta method? I don't know this literature)

* as suggested in a previous e-mail, it might be possible to
compute profile confidence intervals on the ICCs by using
nloptr::auglag or some other optimization framework that allows
nonlinear equality constraints.

* or via parametric bootstrap/bootMer ...

David Atkins

2014-09-12 22:18:07 UTC

Permalink

Masood--

If memory serves, the following article might be helpful (and, I believe
had associated R code):

http://www.ncbi.nlm.nih.gov/pubmed/20569253

Biol Rev Camb Philos Soc. 2010 Nov;85(4):935-56. doi:
10.1111/j.1469-185X.2010.00141.x.

Repeatability for Gaussian and non-Gaussian data: a practical guide for
biologists.

Nakagawa S1, Schielzeth H.

[Note that repeatability = ICC]

Hope that helps.

cheers, Dave
--
Dave Atkins, PhD

Research Professor
Department of Psychiatry and Behavioral Science
University of Washington
datkins at u.washington.edu
http://depts.washington.edu/cshrb/david-atkins/#more-48

"You can never solve a problem on the level on which it was created."
(attributed to) Albert Einstein

Michael Cone

2014-09-13 07:59:12 UTC

Permalink

Dave,
I am trying to learn more about repeatability and method comparisons
right now, and this article was helpful for me. Thanks.
Do you happen to know of a similar review on modeling method comparison
studies when reference values are known?

Michael

Post by David Atkins
Masood--
If memory serves, the following article might be helpful (and, I
http://www.ncbi.nlm.nih.gov/pubmed/20569253
10.1111/j.1469-185X.2010.00141.x.
Repeatability for Gaussian and non-Gaussian data: a practical guide
for biologists.
Nakagawa S1, Schielzeth H.
[Note that repeatability = ICC]
Hope that helps.
cheers, Dave
--
Dave Atkins, PhD
Research Professor
Department of Psychiatry and Behavioral Science
University of Washington
datkins at u.washington.edu
http://depts.washington.edu/cshrb/david-atkins/#more-48
"You can never solve a problem on the level on which it was created."
(attributed to) Albert Einstein

* If you want the standard deviations of the variance estimates
(keeping the strong caveats about non-Normal sampling distributions
in mind), you could adapt the approach shown in
http://rpubs.com/bbolker/varwald (presumably formulas
for confidence intervals
of the ICC based on the variances of the estimates of the RE variances
are using the delta method? I don't know this literature)
* as suggested in a previous e-mail, it might be possible to
compute profile confidence intervals on the ICCs by using
nloptr::auglag or some other optimization framework that allows
nonlinear equality constraints.
* or via parametric bootstrap/bootMer ...
_______________________________________________
R-sig-mixed-models at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

Mark Lai

2014-09-13 06:36:44 UTC

Permalink

Hi Masood,

I think you can do a parametric bootstrap for the ICC with the bootMer
method in lme4. You will need to define the function for obtaining the
ICC, and then from the bootstrap resamples you can construct your
confidence intervals (e.g, by taking the 2.5 and the 97.5 percentiles).

Hope this helps,

Mark

Post by Mohd Masood
I am using random intercept logistic model (in lme4) to calculated
Intraclass correlation coefficient (ICC). lme4 only provides point
estimates and standard deviation (not standard errors) of variance
estimates.These
point estimates can be used to calculated point estimates for ICC. The
problem is how can I calculate standard error and confidence interval for
ICC. I couldn't find any literature showing formula to calculate confidence
interval around ICC. Or is it not possible to calculate SE and CI for ICC
due to skewed sampling distribution (Please see PMCID: PMC3426610).
Thanks
Masood
[[alternative HTML version deleted]]
_______________________________________________
R-sig-mixed-models at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

Martin Maechler

2014-09-13 14:32:17 UTC

Permalink

Post by Mohd Masood

Mohd Masood <drmasoodmohd at gmail.com>

Linear mixed model fit by REML ['lmerMod']
Formula: Reaction ~ Days + (Days | Subject)
Data: sleepstudy

REML criterion at convergence: 1743.6

Scaled residuals:
Min 1Q Median 3Q Max
-3.9536 -0.4634 0.0231 0.4634 5.1793

Random effects:
Groups Name Variance Std.Dev. Corr
Subject (Intercept) 612.09 24.740
Days 35.07 5.922 0.07
Residual 654.94 25.592
Number of obs: 180, groups: Subject, 18

Fixed effects:
Estimate Std. Error t value
(Intercept) 251.405 6.825 36.84
Days 10.467 1.546 6.77

Correlation of Fixed Effects:
(Intr)
Days -0.138

Post by Mohd Masood
confint(pfm1)

2.5 % 97.5 %
.sig01 14.3814761 37.715996
.sig02 -0.4815007 0.684986
.sig03 3.8011641 8.753383
.sigma 22.8982669 28.857997
(Intercept) 237.6806955 265.129515
Days 7.3586533 13.575919

Post by Mohd Masood
confint(pfm1, level = 0.99)

0.5 % 99.5 %
.sig01 11.697963 43.9249121
.sig02 -0.611183 0.8620229
.sig03 3.313161 10.1306220
.sigma 22.149064 30.0282535
(Intercept) 232.619539 270.1906680
Days 6.212281 14.7222899
does provide confidence intervals (CI) also for all variance
parameters, not just the fixed effects

and you should really also look at

require(lattice)
xyplot(pfm1)

which shows you confidence intervals to a couple of levels simultaneously,
and notably also visualizes how (un)reasonable a Gaussian
approximation to the sigma's would be.

Martin Maechler, ETH Zurich

Post by Mohd Masood
These point estimates can be used to calculated point estimates for ICC. The
problem is how can I calculate standard error and confidence interval for
ICC. I couldn't find any literature showing formula to calculate confidence
interval around ICC. Or is it not possible to calculate SE and CI for ICC
due to skewed sampling distribution (Please see PMCID: PMC3426610).
Thanks
Masood
[[alternative HTML version deleted]]
_______________________________________________
R-sig-mixed-models at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models