Discussion:
[R-sig-ME] post hoc tests for glmmTMB
Aoibheann Gaughran
2018-11-09 12:24:47 UTC
Permalink
Dear list,

I am trying to perform post hoc tests on a glmmTMB model. I would normally
use car::Anova and mulgcomp::glht on my glmms (lmers). However, these
functions do not appear to be working for glmmTMB (when I run the model as
an lmer they work fine). I have also tried lsmeans and emmeans but they do
not appear to support glmmTMB either (although it appears they used to). I
have found various treads online suggesting that these functions should
work with TMB but they date back a few months.

I am using the most up to date versions of R (3.5.1) and have updated all
of my packages e.g. glmmTMB 0.2.2.0, lsmeans 2.30-0
subscript out of bound
dimensions of coefficients and covariance matrix don't match
Can't handle an object of class “glmmTMB”
Use help("models", package = "emmeans") for information on supported
models.> rw.emm.s <- emmeans(topmodTRFETE, "roadworks")Error in
ref_grid(object, ...) :
Can't handle an object of class “glmmTMB”
Use help("models", package = "emmeans") for information on supported models.


Can any point me in the direction of a workaround for performing posthocs
on my glmmTMB model?

Many thanks,
--
Aoibheann Gaughran

Behavioural and Evolutionary Ecology Research Group
Zoology Building
School of Natural Sciences
Trinity College Dublin
Dublin 2
Ireland
Phone: +353 (86) 3812615

[[alternative HTML version deleted]]
Aoibheann Gaughran
2018-11-09 14:24:16 UTC
Permalink
Update: lsmeans works if I use an older version of lsmeans (2.27-62) - can
I rely on the results?

Many thanks, Aoibheann
Post by Aoibheann Gaughran
Dear list,
I am trying to perform post hoc tests on a glmmTMB model. I would normally
use car::Anova and mulgcomp::glht on my glmms (lmers). However, these
functions do not appear to be working for glmmTMB (when I run the model as
an lmer they work fine). I have also tried lsmeans and emmeans but they do
not appear to support glmmTMB either (although it appears they used to). I
have found various treads online suggesting that these functions should
work with TMB but they date back a few months.
I am using the most up to date versions of R (3.5.1) and have updated all
of my packages e.g. glmmTMB 0.2.2.0, lsmeans 2.30-0
subscript out of bound
dimensions of coefficients and covariance matrix don't match
Can't handle an object of class “glmmTMB”
Can't handle an object of class “glmmTMB”
Use help("models", package = "emmeans") for information on supported models.
Can any point me in the direction of a workaround for performing posthocs
on my glmmTMB model?
Many thanks,
--
Aoibheann Gaughran
Behavioural and Evolutionary Ecology Research Group
Zoology Building
School of Natural Sciences
Trinity College Dublin
Dublin 2
Ireland
Phone: +353 (86) 3812615
--
Aoibheann Gaughran

Behavioural and Evolutionary Ecology Research Group
Zoology Building
School of Natural Sciences
Trinity College Dublin
Dublin 2
Ireland
Phone: +353 (86) 3812615

[[alternative HTML version deleted]]
Guillaume Adeux
2018-11-09 14:42:54 UTC
Permalink
Hi Aoibheann,

I think that anova on glmmTMB objects only produce Wald tests, which don't
seem to be very reliable.
You might want to look at the monet package (or its little brother afex)
that can produce LRT tests or parametric bootstrap.

Moreover, emmeans should work fine with glmmTMB but I remember having a
similar problem.

Maybe this thread and the following discussion can help you out:
https://stackoverflow.com/questions/48609432/error-message-lsmeans-for-beta-mixed-regression-model-with-glmmtmb

GA2
Post by Aoibheann Gaughran
Update: lsmeans works if I use an older version of lsmeans (2.27-62) - can
I rely on the results?
Many thanks, Aoibheann
Post by Aoibheann Gaughran
Dear list,
I am trying to perform post hoc tests on a glmmTMB model. I would
normally
Post by Aoibheann Gaughran
use car::Anova and mulgcomp::glht on my glmms (lmers). However, these
functions do not appear to be working for glmmTMB (when I run the model
as
Post by Aoibheann Gaughran
an lmer they work fine). I have also tried lsmeans and emmeans but they
do
Post by Aoibheann Gaughran
not appear to support glmmTMB either (although it appears they used to).
I
Post by Aoibheann Gaughran
have found various treads online suggesting that these functions should
work with TMB but they date back a few months.
I am using the most up to date versions of R (3.5.1) and have updated all
of my packages e.g. glmmTMB 0.2.2.0, lsmeans 2.30-0
Anova(topmodTRFETE, type = 2)Error in I.p[c(subs.relatives,
subscript out of bound
summary(glht(topmodTRFETE, linfct = mcp(roadworks = "Tukey")), test =
dimensions of coefficients and covariance matrix don't match
source(system.file("other_methods","lsmeans_methods.R",package="glmmTMB"))>
lsmeans(topmodTRFETE, pairwise ~ roadworks, adjustSigma = TRUE, adjust =
Post by Aoibheann Gaughran
Can't handle an object of class “glmmTMB”
Use help("models", package = "emmeans") for information on supported
models.> rw.emm.s <- emmeans(topmodTRFETE, "roadworks")Error in
Post by Aoibheann Gaughran
Can't handle an object of class “glmmTMB”
Use help("models", package = "emmeans") for information on supported
models.
Post by Aoibheann Gaughran
Can any point me in the direction of a workaround for performing posthocs
on my glmmTMB model?
Many thanks,
--
Aoibheann Gaughran
Behavioural and Evolutionary Ecology Research Group
Zoology Building
School of Natural Sciences
Trinity College Dublin
Dublin 2
Ireland
Phone: +353 (86) 3812615
--
Aoibheann Gaughran
Behavioural and Evolutionary Ecology Research Group
Zoology Building
School of Natural Sciences
Trinity College Dublin
Dublin 2
Ireland
Phone: +353 (86) 3812615
[[alternative HTML version deleted]]
_______________________________________________
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
[[alternative HTML version deleted]]
Aoibheann Gaughran
2018-11-09 15:07:40 UTC
Permalink
Thank you, I have managed to get lsmeans to run using a slightly older
version of the package.

However, the problem with glmmTMB seems to be with the most up to date
version of lsmeans (which, as is noted, now now relies primarily on code in
the 'emmeans' package. 'lsmeans' will be archived in the near future).
emmeans is definitely *not *working with glmmTMB currently, even with the
like of code supplied by Ben Bolker:
source(system.file("other_methods","lsmeans_methods.R",package="glmmTMB").
Post by Guillaume Adeux
Hi Aoibheann,
I think that anova on glmmTMB objects only produce Wald tests, which don't
seem to be very reliable.
You might want to look at the monet package (or its little brother afex)
that can produce LRT tests or parametric bootstrap.
Moreover, emmeans should work fine with glmmTMB but I remember having a
similar problem.
https://stackoverflow.com/questions/48609432/error-message-lsmeans-for-beta-mixed-regression-model-with-glmmtmb
GA2
Post by Aoibheann Gaughran
Update: lsmeans works if I use an older version of lsmeans (2.27-62) - can
I rely on the results?
Many thanks, Aoibheann
Post by Aoibheann Gaughran
Dear list,
I am trying to perform post hoc tests on a glmmTMB model. I would
normally
Post by Aoibheann Gaughran
use car::Anova and mulgcomp::glht on my glmms (lmers). However, these
functions do not appear to be working for glmmTMB (when I run the model
as
Post by Aoibheann Gaughran
an lmer they work fine). I have also tried lsmeans and emmeans but they
do
Post by Aoibheann Gaughran
not appear to support glmmTMB either (although it appears they used
to). I
Post by Aoibheann Gaughran
have found various treads online suggesting that these functions should
work with TMB but they date back a few months.
I am using the most up to date versions of R (3.5.1) and have updated
all
Post by Aoibheann Gaughran
of my packages e.g. glmmTMB 0.2.2.0, lsmeans 2.30-0
Anova(topmodTRFETE, type = 2)Error in I.p[c(subs.relatives,
subscript out of bound
summary(glht(topmodTRFETE, linfct = mcp(roadworks = "Tukey")), test =
dimensions of coefficients and covariance matrix don't match
source(system.file("other_methods","lsmeans_methods.R",package="glmmTMB"))>
lsmeans(topmodTRFETE, pairwise ~ roadworks, adjustSigma = TRUE, adjust =
Post by Aoibheann Gaughran
Can't handle an object of class “glmmTMB”
Use help("models", package = "emmeans") for information on supported
models.> rw.emm.s <- emmeans(topmodTRFETE, "roadworks")Error in
Post by Aoibheann Gaughran
Can't handle an object of class “glmmTMB”
Use help("models", package = "emmeans") for information on supported
models.
Post by Aoibheann Gaughran
Can any point me in the direction of a workaround for performing
posthocs
Post by Aoibheann Gaughran
on my glmmTMB model?
Many thanks,
--
Aoibheann Gaughran
Behavioural and Evolutionary Ecology Research Group
Zoology Building
School of Natural Sciences
Trinity College Dublin
Dublin 2
Ireland
Phone: +353 (86) 3812615
--
Aoibheann Gaughran
Behavioural and Evolutionary Ecology Research Group
Zoology Building
School of Natural Sciences
Trinity College Dublin
Dublin 2
Ireland
Phone: +353 (86) 3812615
[[alternative HTML version deleted]]
_______________________________________________
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
--
Aoibheann Gaughran

Behavioural and Evolutionary Ecology Research Group
Zoology Building
School of Natural Sciences
Trinity College Dublin
Dublin 2
Ireland
Phone: +353 (86) 3812615

[[alternative HTML version deleted]]
Ben Bolker
2018-11-09 15:07:06 UTC
Permalink
The point about Wald tests is correct, although their reliability
depends very much on context (they should be pretty good for tests of
fixed effects when the data set is reasonably large and predicted
probabilities/counts are not too extreme, i.e. not too close to zero
(counts) or 0/1 (probabilities)).

There are a lot of improvements in the development version with
respect to post hoc tests etc. on glmmTMB fits, as documented here:
<https://github.com/glmmTMB/glmmTMB/blob/master/glmmTMB/vignettes/model_evaluation.rmd>

If you can install the development version (see
https://github.com/glmmTMB/glmmTMB/blob/master/README.md), that should
help a lot.

If you can't, most of these improvements will probably get to CRAN in
the next week or two; we're planning a new release soon.

In any case, I think that running the following code should make
multcomp work with glmmTMB objects (not quite sure what that first
function is doing ... ???)

glht_glmmTMB <- function (model, ..., component="cond") {
glht(model, ...,
coef. = function(x) fixef(x)[[component]],
vcov. = function(x) vcov(x)[[component]],
df = NULL)
}
modelparm.glmmTMB <- function (model, coef. = function(x)
fixef(x)[[component]],
vcov. = function(x) vcov(x)[[component]],
df = NULL, component="cond", ...) {
multcomp:::modelparm.default(model, coef. = coef., vcov. = vcov.,
df = df, ...)
}

## example
g1 <- glht(cbpp_b1, linfct = mcp(period = "Tukey"))
Post by Guillaume Adeux
Hi Aoibheann,
I think that anova on glmmTMB objects only produce Wald tests, which don't
seem to be very reliable.
You might want to look at the monet package (or its little brother afex)
that can produce LRT tests or parametric bootstrap.
Moreover, emmeans should work fine with glmmTMB but I remember having a
similar problem.
https://stackoverflow.com/questions/48609432/error-message-lsmeans-for-beta-mixed-regression-model-with-glmmtmb
GA2
Post by Aoibheann Gaughran
Update: lsmeans works if I use an older version of lsmeans (2.27-62) - can
I rely on the results?
Many thanks, Aoibheann
Post by Aoibheann Gaughran
Dear list,
I am trying to perform post hoc tests on a glmmTMB model. I would
normally
Post by Aoibheann Gaughran
use car::Anova and mulgcomp::glht on my glmms (lmers). However, these
functions do not appear to be working for glmmTMB (when I run the model
as
Post by Aoibheann Gaughran
an lmer they work fine). I have also tried lsmeans and emmeans but they
do
Post by Aoibheann Gaughran
not appear to support glmmTMB either (although it appears they used to).
I
Post by Aoibheann Gaughran
have found various treads online suggesting that these functions should
work with TMB but they date back a few months.
I am using the most up to date versions of R (3.5.1) and have updated all
of my packages e.g. glmmTMB 0.2.2.0, lsmeans 2.30-0
Anova(topmodTRFETE, type = 2)Error in I.p[c(subs.relatives,
subscript out of bound
summary(glht(topmodTRFETE, linfct = mcp(roadworks = "Tukey")), test =
dimensions of coefficients and covariance matrix don't match
source(system.file("other_methods","lsmeans_methods.R",package="glmmTMB"))>
lsmeans(topmodTRFETE, pairwise ~ roadworks, adjustSigma = TRUE, adjust =
Post by Aoibheann Gaughran
Can't handle an object of class “glmmTMB”
Use help("models", package = "emmeans") for information on supported
models.> rw.emm.s <- emmeans(topmodTRFETE, "roadworks")Error in
Post by Aoibheann Gaughran
Can't handle an object of class “glmmTMB”
Use help("models", package = "emmeans") for information on supported
models.
Post by Aoibheann Gaughran
Can any point me in the direction of a workaround for performing posthocs
on my glmmTMB model?
Many thanks,
--
Aoibheann Gaughran
Behavioural and Evolutionary Ecology Research Group
Zoology Building
School of Natural Sciences
Trinity College Dublin
Dublin 2
Ireland
Phone: +353 (86) 3812615
--
Aoibheann Gaughran
Behavioural and Evolutionary Ecology Research Group
Zoology Building
School of Natural Sciences
Trinity College Dublin
Dublin 2
Ireland
Phone: +353 (86) 3812615
[[alternative HTML version deleted]]
_______________________________________________
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
[[alternative HTML version deleted]]
_______________________________________________
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
Aoibheann Gaughran
2018-11-09 15:22:36 UTC
Permalink
Great, thank you - I will keep plugging on. I will reconsider my use of the
Anova function in car as well!
Post by Ben Bolker
The point about Wald tests is correct, although their reliability
depends very much on context (they should be pretty good for tests of
fixed effects when the data set is reasonably large and predicted
probabilities/counts are not too extreme, i.e. not too close to zero
(counts) or 0/1 (probabilities)).
There are a lot of improvements in the development version with
<
https://github.com/glmmTMB/glmmTMB/blob/master/glmmTMB/vignettes/model_evaluation.rmd
If you can install the development version (see
https://github.com/glmmTMB/glmmTMB/blob/master/README.md), that should
help a lot.
If you can't, most of these improvements will probably get to CRAN in
the next week or two; we're planning a new release soon.
In any case, I think that running the following code should make
multcomp work with glmmTMB objects (not quite sure what that first
function is doing ... ???)
glht_glmmTMB <- function (model, ..., component="cond") {
glht(model, ...,
coef. = function(x) fixef(x)[[component]],
vcov. = function(x) vcov(x)[[component]],
df = NULL)
}
modelparm.glmmTMB <- function (model, coef. = function(x)
fixef(x)[[component]],
vcov. = function(x) vcov(x)[[component]],
df = NULL, component="cond", ...) {
multcomp:::modelparm.default(model, coef. = coef., vcov. = vcov.,
df = df, ...)
}
## example
g1 <- glht(cbpp_b1, linfct = mcp(period = "Tukey"))
Post by Guillaume Adeux
Hi Aoibheann,
I think that anova on glmmTMB objects only produce Wald tests, which
don't
Post by Guillaume Adeux
seem to be very reliable.
You might want to look at the monet package (or its little brother afex)
that can produce LRT tests or parametric bootstrap.
Moreover, emmeans should work fine with glmmTMB but I remember having a
similar problem.
https://stackoverflow.com/questions/48609432/error-message-lsmeans-for-beta-mixed-regression-model-with-glmmtmb
Post by Guillaume Adeux
GA2
Post by Aoibheann Gaughran
Update: lsmeans works if I use an older version of lsmeans (2.27-62) -
can
Post by Guillaume Adeux
Post by Aoibheann Gaughran
I rely on the results?
Many thanks, Aoibheann
Post by Aoibheann Gaughran
Dear list,
I am trying to perform post hoc tests on a glmmTMB model. I would
normally
Post by Aoibheann Gaughran
use car::Anova and mulgcomp::glht on my glmms (lmers). However, these
functions do not appear to be working for glmmTMB (when I run the model
as
Post by Aoibheann Gaughran
an lmer they work fine). I have also tried lsmeans and emmeans but they
do
Post by Aoibheann Gaughran
not appear to support glmmTMB either (although it appears they used
to).
Post by Guillaume Adeux
Post by Aoibheann Gaughran
I
Post by Aoibheann Gaughran
have found various treads online suggesting that these functions should
work with TMB but they date back a few months.
I am using the most up to date versions of R (3.5.1) and have updated
all
Post by Guillaume Adeux
Post by Aoibheann Gaughran
Post by Aoibheann Gaughran
of my packages e.g. glmmTMB 0.2.2.0, lsmeans 2.30-0
Anova(topmodTRFETE, type = 2)Error in I.p[c(subs.relatives,
subscript out of bound
summary(glht(topmodTRFETE, linfct = mcp(roadworks = "Tukey")), test =
dimensions of coefficients and covariance matrix don't match
source(system.file("other_methods","lsmeans_methods.R",package="glmmTMB"))>
Post by Guillaume Adeux
Post by Aoibheann Gaughran
lsmeans(topmodTRFETE, pairwise ~ roadworks, adjustSigma = TRUE, adjust =
Post by Aoibheann Gaughran
Can't handle an object of class “glmmTMB”
Use help("models", package = "emmeans") for information on supported
models.> rw.emm.s <- emmeans(topmodTRFETE, "roadworks")Error in
Post by Aoibheann Gaughran
Can't handle an object of class “glmmTMB”
Use help("models", package = "emmeans") for information on supported
models.
Post by Aoibheann Gaughran
Can any point me in the direction of a workaround for performing
posthocs
Post by Guillaume Adeux
Post by Aoibheann Gaughran
Post by Aoibheann Gaughran
on my glmmTMB model?
Many thanks,
--
Aoibheann Gaughran
Behavioural and Evolutionary Ecology Research Group
Zoology Building
School of Natural Sciences
Trinity College Dublin
Dublin 2
Ireland
Phone: +353 (86) 3812615
--
Aoibheann Gaughran
Behavioural and Evolutionary Ecology Research Group
Zoology Building
School of Natural Sciences
Trinity College Dublin
Dublin 2
Ireland
Phone: +353 (86) 3812615
[[alternative HTML version deleted]]
_______________________________________________
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
[[alternative HTML version deleted]]
_______________________________________________
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
_______________________________________________
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
--
Aoibheann Gaughran

Behavioural and Evolutionary Ecology Research Group
Zoology Building
School of Natural Sciences
Trinity College Dublin
Dublin 2
Ireland
Phone: +353 (86) 3812615

[[alternative HTML version deleted]]
Fox, John
2018-11-09 15:37:43 UTC
Permalink
Dear Aoibheann,

There is no specific Anova() method for "glmmTMB" objects, so the default method is invoked. This won't work because of the structure of "glmmTMB" models. I think that ideally one would want two tables of Wald tests of fixed effects, one for the conditional nonzero part of the model and one for the zero-inflation part of the model.

This seems to me of sufficient interest that I'll look into writing "glmmTMB" methods for Anova() and for the linearHypothesis() function in the car package, on which Anova() depends. I can't promise, however, when I'll get to this.

Best,
John

--------------------------------------
John Fox, Professor Emeritus
McMaster University
Hamilton, Ontario, Canada
Web: socialsciences.mcmaster.ca/jfox/
-----Original Message-----
project.org] On Behalf Of Aoibheann Gaughran
Sent: Friday, November 9, 2018 10:23 AM
Subject: Re: [R-sig-ME] post hoc tests for glmmTMB
Great, thank you - I will keep plugging on. I will reconsider my use of
the Anova function in car as well!
Post by Ben Bolker
The point about Wald tests is correct, although their reliability
depends very much on context (they should be pretty good for tests of
fixed effects when the data set is reasonably large and predicted
probabilities/counts are not too extreme, i.e. not too close to zero
(counts) or 0/1 (probabilities)).
There are a lot of improvements in the development version with
<
https://github.com/glmmTMB/glmmTMB/blob/master/glmmTMB/vignettes/model
_evaluation.rmd
If you can install the development version (see
https://github.com/glmmTMB/glmmTMB/blob/master/README.md), that should
help a lot.
If you can't, most of these improvements will probably get to CRAN in
the next week or two; we're planning a new release soon.
In any case, I think that running the following code should make
multcomp work with glmmTMB objects (not quite sure what that first
function is doing ... ???)
glht_glmmTMB <- function (model, ..., component="cond") {
glht(model, ...,
coef. = function(x) fixef(x)[[component]],
vcov. = function(x) vcov(x)[[component]],
df = NULL)
}
modelparm.glmmTMB <- function (model, coef. = function(x)
fixef(x)[[component]],
vcov. = function(x)
vcov(x)[[component]],
Post by Ben Bolker
df = NULL, component="cond", ...) {
multcomp:::modelparm.default(model, coef. = coef., vcov. = vcov.,
df = df, ...)
}
## example
g1 <- glht(cbpp_b1, linfct = mcp(period = "Tukey"))
Post by Guillaume Adeux
Hi Aoibheann,
I think that anova on glmmTMB objects only produce Wald tests, which
don't
Post by Guillaume Adeux
seem to be very reliable.
You might want to look at the monet package (or its little brother
afex) that can produce LRT tests or parametric bootstrap.
Moreover, emmeans should work fine with glmmTMB but I remember
having a similar problem.
https://stackoverflow.com/questions/48609432/error-message-lsmeans-for
-beta-mixed-regression-model-with-glmmtmb
Post by Guillaume Adeux
GA2
Post by Aoibheann Gaughran
Update: lsmeans works if I use an older version of lsmeans
(2.27-62) -
can
Post by Guillaume Adeux
Post by Aoibheann Gaughran
I rely on the results?
Many thanks, Aoibheann
Post by Aoibheann Gaughran
Dear list,
I am trying to perform post hoc tests on a glmmTMB model. I would
normally
Post by Aoibheann Gaughran
use car::Anova and mulgcomp::glht on my glmms (lmers). However,
these functions do not appear to be working for glmmTMB (when I
run the model
as
Post by Aoibheann Gaughran
an lmer they work fine). I have also tried lsmeans and emmeans but they
do
Post by Aoibheann Gaughran
not appear to support glmmTMB either (although it appears they
used
to).
Post by Guillaume Adeux
Post by Aoibheann Gaughran
I
Post by Aoibheann Gaughran
have found various treads online suggesting that these functions
should work with TMB but they date back a few months.
I am using the most up to date versions of R (3.5.1) and have
updated
all
Post by Guillaume Adeux
Post by Aoibheann Gaughran
Post by Aoibheann Gaughran
of my packages e.g. glmmTMB 0.2.2.0, lsmeans 2.30-0
Anova(topmodTRFETE, type = 2)Error in I.p[c(subs.relatives,
subscript out of bound
summary(glht(topmodTRFETE, linfct = mcp(roadworks = "Tukey")), test =
dimensions of coefficients and covariance matrix don't match
source(system.file("other_methods","lsmeans_methods.R",package="glmmTM
B"))>
Post by Guillaume Adeux
Post by Aoibheann Gaughran
lsmeans(topmodTRFETE, pairwise ~ roadworks, adjustSigma = TRUE,
Post by Aoibheann Gaughran
Can't handle an object of class “glmmTMB”
Use help("models", package = "emmeans") for information on supported
models.> rw.emm.s <- emmeans(topmodTRFETE, "roadworks")Error in
Post by Aoibheann Gaughran
Can't handle an object of class “glmmTMB”
Use help("models", package = "emmeans") for information on supported
models.
Post by Aoibheann Gaughran
Can any point me in the direction of a workaround for performing
posthocs
Post by Guillaume Adeux
Post by Aoibheann Gaughran
Post by Aoibheann Gaughran
on my glmmTMB model?
Many thanks,
--
Aoibheann Gaughran
Behavioural and Evolutionary Ecology Research Group Zoology
Building School of Natural Sciences Trinity College Dublin Dublin
2 Ireland
Phone: +353 (86) 3812615
--
Aoibheann Gaughran
Behavioural and Evolutionary Ecology Research Group Zoology
Building School of Natural Sciences Trinity College Dublin Dublin 2
Ireland
Phone: +353 (86) 3812615
[[alternative HTML version deleted]]
_______________________________________________
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
[[alternative HTML version deleted]]
_______________________________________________
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
_______________________________________________
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
--
Aoibheann Gaughran
Behavioural and Evolutionary Ecology Research Group Zoology Building
School of Natural Sciences Trinity College Dublin Dublin 2 Ireland
Phone: +353 (86) 3812615
[[alternative HTML version deleted]]
_______________________________________________
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
Ben Bolker
2018-11-09 15:27:57 UTC
Permalink
The devel version of glmmTMB contains Anova methods for glmmTMB :

https://github.com/glmmTMB/glmmTMB/blob/master/glmmTMB/R/Anova.R

cheers
Ben
Post by Fox, John
Dear Aoibheann,
There is no specific Anova() method for "glmmTMB" objects, so the default method is invoked. This won't work because of the structure of "glmmTMB" models. I think that ideally one would want two tables of Wald tests of fixed effects, one for the conditional nonzero part of the model and one for the zero-inflation part of the model.
This seems to me of sufficient interest that I'll look into writing "glmmTMB" methods for Anova() and for the linearHypothesis() function in the car package, on which Anova() depends. I can't promise, however, when I'll get to this.
Best,
John
--------------------------------------
John Fox, Professor Emeritus
McMaster University
Hamilton, Ontario, Canada
Web: socialsciences.mcmaster.ca/jfox/
-----Original Message-----
project.org] On Behalf Of Aoibheann Gaughran
Sent: Friday, November 9, 2018 10:23 AM
Subject: Re: [R-sig-ME] post hoc tests for glmmTMB
Great, thank you - I will keep plugging on. I will reconsider my use of
the Anova function in car as well!
Post by Ben Bolker
The point about Wald tests is correct, although their reliability
depends very much on context (they should be pretty good for tests of
fixed effects when the data set is reasonably large and predicted
probabilities/counts are not too extreme, i.e. not too close to zero
(counts) or 0/1 (probabilities)).
There are a lot of improvements in the development version with
<
https://github.com/glmmTMB/glmmTMB/blob/master/glmmTMB/vignettes/model
_evaluation.rmd
If you can install the development version (see
https://github.com/glmmTMB/glmmTMB/blob/master/README.md), that should
help a lot.
If you can't, most of these improvements will probably get to CRAN in
the next week or two; we're planning a new release soon.
In any case, I think that running the following code should make
multcomp work with glmmTMB objects (not quite sure what that first
function is doing ... ???)
glht_glmmTMB <- function (model, ..., component="cond") {
glht(model, ...,
coef. = function(x) fixef(x)[[component]],
vcov. = function(x) vcov(x)[[component]],
df = NULL)
}
modelparm.glmmTMB <- function (model, coef. = function(x)
fixef(x)[[component]],
vcov. = function(x)
vcov(x)[[component]],
Post by Ben Bolker
df = NULL, component="cond", ...) {
multcomp:::modelparm.default(model, coef. = coef., vcov. = vcov.,
df = df, ...)
}
## example
g1 <- glht(cbpp_b1, linfct = mcp(period = "Tukey"))
Post by Guillaume Adeux
Hi Aoibheann,
I think that anova on glmmTMB objects only produce Wald tests, which
don't
Post by Guillaume Adeux
seem to be very reliable.
You might want to look at the monet package (or its little brother
afex) that can produce LRT tests or parametric bootstrap.
Moreover, emmeans should work fine with glmmTMB but I remember
having a similar problem.
https://stackoverflow.com/questions/48609432/error-message-lsmeans-for
-beta-mixed-regression-model-with-glmmtmb
Post by Guillaume Adeux
GA2
Post by Aoibheann Gaughran
Update: lsmeans works if I use an older version of lsmeans
(2.27-62) -
can
Post by Guillaume Adeux
Post by Aoibheann Gaughran
I rely on the results?
Many thanks, Aoibheann
Post by Aoibheann Gaughran
Dear list,
I am trying to perform post hoc tests on a glmmTMB model. I would
normally
Post by Aoibheann Gaughran
use car::Anova and mulgcomp::glht on my glmms (lmers). However,
these functions do not appear to be working for glmmTMB (when I
run the model
as
Post by Aoibheann Gaughran
an lmer they work fine). I have also tried lsmeans and emmeans but they
do
Post by Aoibheann Gaughran
not appear to support glmmTMB either (although it appears they
used
to).
Post by Guillaume Adeux
Post by Aoibheann Gaughran
I
Post by Aoibheann Gaughran
have found various treads online suggesting that these functions
should work with TMB but they date back a few months.
I am using the most up to date versions of R (3.5.1) and have
updated
all
Post by Guillaume Adeux
Post by Aoibheann Gaughran
Post by Aoibheann Gaughran
of my packages e.g. glmmTMB 0.2.2.0, lsmeans 2.30-0
Anova(topmodTRFETE, type = 2)Error in I.p[c(subs.relatives,
subscript out of bound
summary(glht(topmodTRFETE, linfct = mcp(roadworks = "Tukey")), test =
dimensions of coefficients and covariance matrix don't match
source(system.file("other_methods","lsmeans_methods.R",package="glmmTM
B"))>
Post by Guillaume Adeux
Post by Aoibheann Gaughran
lsmeans(topmodTRFETE, pairwise ~ roadworks, adjustSigma = TRUE,
Post by Aoibheann Gaughran
Can't handle an object of class “glmmTMB”
Use help("models", package = "emmeans") for information on supported
models.> rw.emm.s <- emmeans(topmodTRFETE, "roadworks")Error in
Post by Aoibheann Gaughran
Can't handle an object of class “glmmTMB”
Use help("models", package = "emmeans") for information on supported
models.
Post by Aoibheann Gaughran
Can any point me in the direction of a workaround for performing
posthocs
Post by Guillaume Adeux
Post by Aoibheann Gaughran
Post by Aoibheann Gaughran
on my glmmTMB model?
Many thanks,
--
Aoibheann Gaughran
Behavioural and Evolutionary Ecology Research Group Zoology
Building School of Natural Sciences Trinity College Dublin Dublin
2 Ireland
Phone: +353 (86) 3812615
--
Aoibheann Gaughran
Behavioural and Evolutionary Ecology Research Group Zoology
Building School of Natural Sciences Trinity College Dublin Dublin 2
Ireland
Phone: +353 (86) 3812615
[[alternative HTML version deleted]]
_______________________________________________
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
[[alternative HTML version deleted]]
_______________________________________________
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
_______________________________________________
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
--
Aoibheann Gaughran
Behavioural and Evolutionary Ecology Research Group Zoology Building
School of Natural Sciences Trinity College Dublin Dublin 2 Ireland
Phone: +353 (86) 3812615
[[alternative HTML version deleted]]
_______________________________________________
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
Fox, John
2018-11-09 16:32:24 UTC
Permalink
Hi Ben,

My apologies if you told me about this before and I forgot (because it does ring a bell).

I've taken a look at what you did and have some questions/comments:

(1) I see that you have a component argument to determine what to test -- the conditional part of the model, the zero-inflated part, or the dispersion part. I'd allow multiple choices, like component=c("cond", "zi"), and consider making c("cond", "zi") the default. It should be easy to do this via a recursive call.

(2) I also see that you don't export a linearHypothesis.glmmTMB() method. I think that's a pity since it would be useful and shouldn't involve much more work.

Best,
John

--------------------------------------
John Fox, Professor Emeritus
McMaster University
Hamilton, Ontario, Canada
Web: socialsciences.mcmaster.ca/jfox/
-----Original Message-----
Sent: Friday, November 9, 2018 10:28 AM
Subject: Re: [R-sig-ME] post hoc tests for glmmTMB
https://github.com/glmmTMB/glmmTMB/blob/master/glmmTMB/R/Anova.R
cheers
Ben
Post by Fox, John
Dear Aoibheann,
There is no specific Anova() method for "glmmTMB" objects, so the
default method is invoked. This won't work because of the structure of
"glmmTMB" models. I think that ideally one would want two tables of Wald
tests of fixed effects, one for the conditional nonzero part of the
model and one for the zero-inflation part of the model.
Post by Fox, John
This seems to me of sufficient interest that I'll look into writing
"glmmTMB" methods for Anova() and for the linearHypothesis() function in
the car package, on which Anova() depends. I can't promise, however,
when I'll get to this.
Post by Fox, John
Best,
John
--------------------------------------
John Fox, Professor Emeritus
McMaster University
Hamilton, Ontario, Canada
Web: socialsciences.mcmaster.ca/jfox/
-----Original Message-----
project.org] On Behalf Of Aoibheann Gaughran
Sent: Friday, November 9, 2018 10:23 AM
Subject: Re: [R-sig-ME] post hoc tests for glmmTMB
Great, thank you - I will keep plugging on. I will reconsider my use
of the Anova function in car as well!
Post by Ben Bolker
The point about Wald tests is correct, although their reliability
depends very much on context (they should be pretty good for tests
of fixed effects when the data set is reasonably large and predicted
probabilities/counts are not too extreme, i.e. not too close to zero
(counts) or 0/1 (probabilities)).
There are a lot of improvements in the development version with
<
https://github.com/glmmTMB/glmmTMB/blob/master/glmmTMB/vignettes/mod
el
_evaluation.rmd
If you can install the development version (see
https://github.com/glmmTMB/glmmTMB/blob/master/README.md), that
should help a lot.
If you can't, most of these improvements will probably get to CRAN
in the next week or two; we're planning a new release soon.
In any case, I think that running the following code should make
multcomp work with glmmTMB objects (not quite sure what that first
function is doing ... ???)
glht_glmmTMB <- function (model, ..., component="cond") {
glht(model, ...,
coef. = function(x) fixef(x)[[component]],
vcov. = function(x) vcov(x)[[component]],
df = NULL)
}
modelparm.glmmTMB <- function (model, coef. = function(x)
fixef(x)[[component]],
vcov. = function(x)
vcov(x)[[component]],
Post by Ben Bolker
df = NULL, component="cond", ...) {
multcomp:::modelparm.default(model, coef. = coef., vcov. =
vcov.,
Post by Fox, John
Post by Ben Bolker
df = df, ...) }
## example
g1 <- glht(cbpp_b1, linfct = mcp(period = "Tukey"))
Post by Guillaume Adeux
Hi Aoibheann,
I think that anova on glmmTMB objects only produce Wald tests, which
don't
Post by Guillaume Adeux
seem to be very reliable.
You might want to look at the monet package (or its little brother
afex) that can produce LRT tests or parametric bootstrap.
Moreover, emmeans should work fine with glmmTMB but I remember
having a similar problem.
https://stackoverflow.com/questions/48609432/error-message-lsmeans-f
or -beta-mixed-regression-model-with-glmmtmb
Post by Guillaume Adeux
GA2
Post by Aoibheann Gaughran
Update: lsmeans works if I use an older version of lsmeans
(2.27-62) -
can
Post by Guillaume Adeux
Post by Aoibheann Gaughran
I rely on the results?
Many thanks, Aoibheann
Post by Aoibheann Gaughran
Dear list,
I am trying to perform post hoc tests on a glmmTMB model. I would
normally
Post by Aoibheann Gaughran
use car::Anova and mulgcomp::glht on my glmms (lmers). However,
these functions do not appear to be working for glmmTMB (when I
run the model
as
Post by Aoibheann Gaughran
an lmer they work fine). I have also tried lsmeans and emmeans but they
do
Post by Aoibheann Gaughran
not appear to support glmmTMB either (although it appears they
used
to).
Post by Guillaume Adeux
Post by Aoibheann Gaughran
I
Post by Aoibheann Gaughran
have found various treads online suggesting that these functions
should work with TMB but they date back a few months.
I am using the most up to date versions of R (3.5.1) and have
updated
all
Post by Guillaume Adeux
Post by Aoibheann Gaughran
Post by Aoibheann Gaughran
of my packages e.g. glmmTMB 0.2.2.0, lsmeans 2.30-0
Anova(topmodTRFETE, type = 2)Error in I.p[c(subs.relatives,
subscript out of bound
summary(glht(topmodTRFETE, linfct = mcp(roadworks = "Tukey")), test =
dimensions of coefficients and covariance matrix don't match
source(system.file("other_methods","lsmeans_methods.R",package="glmm
TM
B"))>
Post by Guillaume Adeux
Post by Aoibheann Gaughran
lsmeans(topmodTRFETE, pairwise ~ roadworks, adjustSigma = TRUE,
Post by Aoibheann Gaughran
Can't handle an object of class “glmmTMB”
Use help("models", package = "emmeans") for information on supported
models.> rw.emm.s <- emmeans(topmodTRFETE, "roadworks")Error in
Post by Aoibheann Gaughran
Can't handle an object of class “glmmTMB”
Use help("models", package = "emmeans") for information on supported
models.
Post by Aoibheann Gaughran
Can any point me in the direction of a workaround for performing
posthocs
Post by Guillaume Adeux
Post by Aoibheann Gaughran
Post by Aoibheann Gaughran
on my glmmTMB model?
Many thanks,
--
Aoibheann Gaughran
Behavioural and Evolutionary Ecology Research Group Zoology
Building School of Natural Sciences Trinity College Dublin Dublin
2 Ireland
Phone: +353 (86) 3812615
--
Aoibheann Gaughran
Behavioural and Evolutionary Ecology Research Group Zoology
Building School of Natural Sciences Trinity College Dublin Dublin
2 Ireland
Phone: +353 (86) 3812615
[[alternative HTML version deleted]]
_______________________________________________
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
[[alternative HTML version deleted]]
_______________________________________________
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
_______________________________________________
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
--
Aoibheann Gaughran
Behavioural and Evolutionary Ecology Research Group Zoology Building
School of Natural Sciences Trinity College Dublin Dublin 2 Ireland
Phone: +353 (86) 3812615
[[alternative HTML version deleted]]
_______________________________________________
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
Lenth, Russell V
2018-11-09 20:46:02 UTC
Permalink
With emmeans/lsmeans, it turns out the mistake was in sourcing that obsolete lsmeans-methods.R code. Please erase all functions in your workspace that begin with "recover.data" or "lsm.basis" -- then you can obtain EMMs without problems, because emmeans support is built-in to the cirrent glmmTMB package. For example:

require(glmmTMB)
require(emmeans)

example("glmmTMB")
# ... quite a few lines excluded here

emmeans(m2, "spp")
# spp emmean SE df lower.CL upper.CL
# GP 0.10429796 0.2629172 626 -0.4120086 0.6206045
# PR -0.85940395 0.6723753 626 -2.1797882 0.4609803
# DM 0.27498137 0.2456147 626 -0.2073472 0.7573099
# EC-A -0.28275838 0.3610020 626 -0.9916800 0.4261633
# EC-L 0.59224612 0.2581265 626 0.0853474 1.0991448
# DES-L 0.69378481 0.2410053 626 0.2205080 1.1670616
# DF -0.00896838 0.2496003 626 -0.4991237 0.4811869

Russ

-----Original Message-----
From: Lenth, Russell V
Sent: Friday, November 9, 2018 10:22 AM
To: r-sig-mixed-***@r-project.org; 'Aoibheann Gaughran' <***@tcd.ie>
Subject: RE: R-sig-mixed-models Digest, Vol 143, Issue 12

I screwed up -- in a kind of brown-paper-bag way...

I think it may work with emmeans if you do this:

recover.data <- function(object, ...) UseMethod("recover.data")
lsm.basis <- function(object, ...) UseMethod("lsm.basis")
recover.data.default <- function(object, ...) emmeans::recover_data(object, ...)
lsm.basis.default <- function(object, ...) emmeans::emm_basis(object, ...)

Hope this helps.

Russ Lenth

Russell V. Lenth  -  Professor Emeritus
Department of Statistics and Actuarial Science The University of Iowa  -  Iowa City, IA 52242  USA Voice (319)335-0712 (Dept. office)  -  FAX (319)335-3017



-----Original Message-----

Message: 1
Date: Fri, 9 Nov 2018 12:24:47 +0000
From: Aoibheann Gaughran <***@tcd.ie>
To: r-sig-mixed-***@r-project.org
Subject: [R-sig-ME] post hoc tests for glmmTMB
Message-ID:
<CAN=0SEmQE6f=OAofUeSg_e2f-***@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Dear list,

I am trying to perform post hoc tests on a glmmTMB model. I would normally use car::Anova and mulgcomp::glht on my glmms (lmers). However, these functions do not appear to be working for glmmTMB (when I run the model as an lmer they work fine). I have also tried lsmeans and emmeans but they do not appear to support glmmTMB either (although it appears they used to). I have found various treads online suggesting that these functions should work with TMB but they date back a few months.

I am using the most up to date versions of R (3.5.1) and have updated all of my packages e.g. glmmTMB 0.2.2.0, lsmeans 2.30-0
subscript out of bound
dimensions of coefficients and covariance matrix don't match
Can't handle an object of class “glmmTMB”
Use help("models", package = "emmeans") for information on supported models.> rw.emm.s <- emmeans(topmodTRFETE, "roadworks")Error in ref_grid(object, ...) :
Can't handle an object of class “glmmTMB”
Use help("models", package = "emmeans") for information on supported models.


Can any point me in the direction of a workaround for performing posthocs on my glmmTMB model?

Many thanks,
--
Aoibheann Gaughran

Behavioural and Evolutionary Ecology Research Group Zoology Building School of Natural Sciences Trinity College Dublin Dublin 2 Ireland
Phone: +353 (86) 3812615
Loading...