Discussion:
[R-sig-ME] Anova (type III-tests) table based on LRT for glmmTMB models
Guillaume Adeux
2018-07-30 08:59:46 UTC
Permalink
Hello everyone,

I'm looking for a method/function in order to produce an Anova table based
on Likelihood Ratio Tests (LRT) for a glmmTMB model (R software). In my
case it is with a beta distribution and log link. My response is a ratio
(%) in a repeated measures design.

-

the function Anova() from the {car} package doesn't not run on single
models (i.e. Anova(mod)). It only allows comparison of two models (i.e.
Anova(mod,mod1)).
-

for glmer models, I was used to using the mixed() function from the
{afex} packages which produced Anova tables (type III tests) based on LRT
(or parametric bootstrap) for glmms.

Could anyone shed their on light on a function like mixed() which would run
on glmmTMB objects or on a procedure to do this by hand?

I suppose if only one fixed predictor was present in the model, this would
be simple by comparing it to a null model but my model contains an
interaction. Hence, I am incable of comparing a model A+B+B:C with a model
containing A+B:C.

Thanks for your interest.

Guillaume ADEUX

[[alternative HTML version deleted]]
Mollie Brooks
2018-07-30 09:04:58 UTC
Permalink
Hi Guillaume,

I’m not very experienced with Anova, but I know the development version of glmmTMB supports car::Anova. Ben recently added this functionality.

To install the development version try devtools::install_github("glmmTMB/glmmTMB/glmmTMB")
or see further instructions here https://github.com/glmmTMB/glmmTMB

After installing, check out vignette("model_evaluation")

cheers,
Mollie

> On 30Jul 2018, at 10:59, Guillaume Adeux <***@gmail.com> wrote:
>
> Hello everyone,
>
> I'm looking for a method/function in order to produce an Anova table based
> on Likelihood Ratio Tests (LRT) for a glmmTMB model (R software). In my
> case it is with a beta distribution and log link. My response is a ratio
> (%) in a repeated measures design.
>
> -
>
> the function Anova() from the {car} package doesn't not run on single
> models (i.e. Anova(mod)). It only allows comparison of two models (i.e.
> Anova(mod,mod1)).
> -
>
> for glmer models, I was used to using the mixed() function from the
> {afex} packages which produced Anova tables (type III tests) based on LRT
> (or parametric bootstrap) for glmms.
>
> Could anyone shed their on light on a function like mixed() which would run
> on glmmTMB objects or on a procedure to do this by hand?
>
> I suppose if only one fixed predictor was present in the model, this would
> be simple by comparing it to a null model but my model contains an
> interaction. Hence, I am incable of comparing a model A+B+B:C with a model
> containing A+B:C.
>
> Thanks for your interest.
>
> Guillaume ADEUX
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-***@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models


[[alternative HTML version deleted]]
Ben Bolker
2018-07-30 21:40:23 UTC
Permalink
car::Anova only does Wald tests.

I don't know whether drop1() works, although if not it would be worth
seeing what it would take to make it work.

"Type 3" ANOVA is tricky in R, even with drop1() constructs, because
it sometimes requires constructing model matrices that R won't easily
provide. For example: if A and B are both factors, then ~A:B as
constructed by model.matrix() will still include the main effects.
The only way I know of to do it is to use ~1+A+B+A:B (or equivalently
~A*B) and then drop or zero-out the unwanted columns of the model
matrix.

afex::mixed has some nice type-3 constructs, I don't remember exactly
how they work.

On Mon, Jul 30, 2018 at 5:05 AM Mollie Brooks <***@gmail.com> wrote:
>
> Hi Guillaume,
>
> I’m not very experienced with Anova, but I know the development version of glmmTMB supports car::Anova. Ben recently added this functionality.
>
> To install the development version try devtools::install_github("glmmTMB/glmmTMB/glmmTMB")
> or see further instructions here https://github.com/glmmTMB/glmmTMB
>
> After installing, check out vignette("model_evaluation")
>
> cheers,
> Mollie
>
> > On 30Jul 2018, at 10:59, Guillaume Adeux <***@gmail.com> wrote:
> >
> > Hello everyone,
> >
> > I'm looking for a method/function in order to produce an Anova table based
> > on Likelihood Ratio Tests (LRT) for a glmmTMB model (R software). In my
> > case it is with a beta distribution and log link. My response is a ratio
> > (%) in a repeated measures design.
> >
> > -
> >
> > the function Anova() from the {car} package doesn't not run on single
> > models (i.e. Anova(mod)). It only allows comparison of two models (i.e.
> > Anova(mod,mod1)).
> > -
> >
> > for glmer models, I was used to using the mixed() function from the
> > {afex} packages which produced Anova tables (type III tests) based on LRT
> > (or parametric bootstrap) for glmms.
> >
> > Could anyone shed their on light on a function like mixed() which would run
> > on glmmTMB objects or on a procedure to do this by hand?
> >
> > I suppose if only one fixed predictor was present in the model, this would
> > be simple by comparing it to a null model but my model contains an
> > interaction. Hence, I am incable of comparing a model A+B+B:C with a model
> > containing A+B:C.
> >
> > Thanks for your interest.
> >
> > Guillaume ADEUX
> >
> > [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > R-sig-mixed-***@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-***@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
John Maindonald
2018-07-30 22:44:36 UTC
Permalink
On Type III tests, I note that pp.24-26 of the document give a comprehensive
account.
afex::introduction-mixed-models<http://127.0.0.1:11966/help/library/afex/doc/introduction-mixed-models.pdf>
NB that in suitably ‘balanced’ designs, type III sums of squares are not in
contention.

The complication is that a term A:B, when A and B do not appear in the model
formula, results in general in fitted values and in a contribution to the anova
sum of squares (this gets more complicated in multi-level models) that depend
on how A and B are parameterized (‘treatment’, or ‘sum’, or … contrasts).
One needs to think very carefully about the specific meaning that might attach
to any specific type 3 test and/or to the estimates that remain when lower order
terms are left out. In models with several factors and/or covariates, the
complications of interpretation that result will often be just one more unhelpful
source of confusion.

Any model selection process changes the meaning that should be attached to
the p-values in the model that remains, with the extent of the change cumulating
as more terms are omitted. The result may readily be that p-values that appear
‘highly significant’, in the model that results and with no account taken of
selection effects, should really suggest ‘not at all significant’.


John Maindonald email: ***@anu.edu.au<mailto:***@anu.edu.au>

On 31/07/2018, at 09:40, Ben Bolker <***@gmail.com<mailto:***@gmail.com>> wrote:

car::Anova only does Wald tests.

I don't know whether drop1() works, although if not it would be worth
seeing what it would take to make it work.

"Type 3" ANOVA is tricky in R, even with drop1() constructs, because
it sometimes requires constructing model matrices that R won't easily
provide. For example: if A and B are both factors, then ~A:B as
constructed by model.matrix() will still include the main effects.
The only way I know of to do it is to use ~1+A+B+A:B (or equivalently
~A*B) and then drop or zero-out the unwanted columns of the model
matrix.

afex::mixed has some nice type-3 constructs, I don't remember exactly
how they work.

On Mon, Jul 30, 2018 at 5:05 AM Mollie Brooks <***@gmail.com<mailto:***@gmail.com>> wrote:

Hi Guillaume,

I’m not very experienced with Anova, but I know the development version of glmmTMB supports car::Anova. Ben recently added this functionality.

To install the development version try devtools::install_github("glmmTMB/glmmTMB/glmmTMB")
or see further instructions here https://github.com/glmmTMB/glmmTMB

After installing, check out vignette("model_evaluation")

cheers,
Mollie

On 30Jul 2018, at 10:59, Guillaume Adeux <***@gmail.com<mailto:***@gmail.com>> wrote:

Hello everyone,

I'm looking for a method/function in order to produce an Anova table based
on Likelihood Ratio Tests (LRT) for a glmmTMB model (R software). In my
case it is with a beta distribution and log link. My response is a ratio
(%) in a repeated measures design.

-

the function Anova() from the {car} package doesn't not run on single
models (i.e. Anova(mod)). It only allows comparison of two models (i.e.
Anova(mod,mod1)).
-

for glmer models, I was used to using the mixed() function from the
{afex} packages which produced Anova tables (type III tests) based on LRT
(or parametric bootstrap) for glmms.

Could anyone shed their on light on a function like mixed() which would run
on glmmTMB objects or on a procedure to do this by hand?

I suppose if only one fixed predictor was present in the model, this would
be simple by comparing it to a null model but my model contains an
interaction. Hence, I am incable of comparing a model A+B+B:C with a model
containing A+B:C.

Thanks for your interest.

Guillaume ADEUX

[[alternative HTML version deleted]]

_______________________________________________
R-sig-mixed-***@r-project.org<mailto:R-sig-mixed-***@r-project.org> mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models


[[alternative HTML version deleted]]

_______________________________________________
R-sig-mixed-***@r-project.org<mailto:R-sig-mixed-***@r-project.org> mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

_______________________________________________
R-sig-mixed-***@r-project.org<mailto:R-sig-mixed-***@r-project.org> mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models


[[alternative HTML version deleted]]
Sorkin, John
2018-07-31 01:46:24 UTC
Permalink
A model that contains an interaction without the main effects that are included in the interaction is unusual. Before I would run, or publish, such a model, I would think long and hard.

John


John David Sorkin M.D., Ph.D.
Professor of Medicine
Chief, Biostatistics and Informatics
University of Maryland School of Medicine Division of Gerontology and Geriatric Medicine
Baltimore VA Medical Center
10 North Greene Street
GRECC (BT/18/GR)
Baltimore, MD 21201-1524
(Phone) 410-605-7119
(Fax) 410-605-7913 (Please call phone number above prior to faxing)



________________________________
From: R-sig-mixed-models <r-sig-mixed-models-***@r-project.org> on behalf of John Maindonald <***@anu.edu.au>
Sent: Monday, July 30, 2018 6:44 PM
To: Ben Bolker
Cc: R SIG Mixed Models
Subject: Re: [R-sig-ME] Anova (type III-tests) table based on LRT for glmmTMB models

CAUTION: This message originated from a non UMB, UMSOM, FPI, or UMMS email system. Whether the sender is known or not known, hover over any links before clicking and use caution opening attachments.



On Type III tests, I note that pp.24-26 of the document give a comprehensive
account.
afex::introduction-mixed-models<http://127.0.0.1:11966/help/library/afex/doc/introduction-mixed-models.pdf>
NB that in suitably ‘balanced’ designs, type III sums of squares are not in
contention.

The complication is that a term A:B, when A and B do not appear in the model
formula, results in general in fitted values and in a contribution to the anova
sum of squares (this gets more complicated in multi-level models) that depend
on how A and B are parameterized (‘treatment’, or ‘sum’, or … contrasts).
One needs to think very carefully about the specific meaning that might attach
to any specific type 3 test and/or to the estimates that remain when lower order
terms are left out. In models with several factors and/or covariates, the
complications of interpretation that result will often be just one more unhelpful
source of confusion.

Any model selection process changes the meaning that should be attached to
the p-values in the model that remains, with the extent of the change cumulating
as more terms are omitted. The result may readily be that p-values that appear
‘highly significant’, in the model that results and with no account taken of
selection effects, should really suggest ‘not at all significant’.


John Maindonald email: ***@anu.edu.au<mailto:***@anu.edu.au>

On 31/07/2018, at 09:40, Ben Bolker <***@gmail.com<mailto:***@gmail.com>> wrote:

car::Anova only does Wald tests.

I don't know whether drop1() works, although if not it would be worth
seeing what it would take to make it work.

"Type 3" ANOVA is tricky in R, even with drop1() constructs, because
it sometimes requires constructing model matrices that R won't easily
provide. For example: if A and B are both factors, then ~A:B as
constructed by model.matrix() will still include the main effects.
The only way I know of to do it is to use ~1+A+B+A:B (or equivalently
~A*B) and then drop or zero-out the unwanted columns of the model
matrix.

afex::mixed has some nice type-3 constructs, I don't remember exactly
how they work.

On Mon, Jul 30, 2018 at 5:05 AM Mollie Brooks <***@gmail.com<mailto:***@gmail.com>> wrote:

Hi Guillaume,

I’m not very experienced with Anova, but I know the development version of glmmTMB supports car::Anova. Ben recently added this functionality.

To install the development version try devtools::install_github("glmmTMB/glmmTMB/glmmTMB")
or see further instructions here https://github.com/glmmTMB/glmmTMB

After installing, check out vignette("model_evaluation")

cheers,
Mollie

On 30Jul 2018, at 10:59, Guillaume Adeux <***@gmail.com<mailto:***@gmail.com>> wrote:

Hello everyone,

I'm looking for a method/function in order to produce an Anova table based
on Likelihood Ratio Tests (LRT) for a glmmTMB model (R software). In my
case it is with a beta distribution and log link. My response is a ratio
(%) in a repeated measures design.

-

the function Anova() from the {car} package doesn't not run on single
models (i.e. Anova(mod)). It only allows comparison of two models (i.e.
Anova(mod,mod1)).
-

for glmer models, I was used to using the mixed() function from the
{afex} packages which produced Anova tables (type III tests) based on LRT
(or parametric bootstrap) for glmms.

Could anyone shed their on light on a function like mixed() which would run
on glmmTMB objects or on a procedure to do this by hand?

I suppose if only one fixed predictor was present in the model, this would
be simple by comparing it to a null model but my model contains an
interaction. Hence, I am incable of comparing a model A+B+B:C with a model
containing A+B:C.

Thanks for your interest.

Guillaume ADEUX

[[alternative HTML version deleted]]

_______________________________________________
R-sig-mixed-***@r-project.org<mailto:R-sig-mixed-***@r-project.org> mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models


[[alternative HTML version deleted]]

_______________________________________________
R-sig-mixed-***@r-project.org<mailto:R-sig-mixed-***@r-project.org> mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

_______________________________________________
R-sig-mixed-***@r-project.org<mailto:R-sig-mixed-***@r-project.org> mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models


[[alternative HTML version deleted]]
Sorkin, John
2018-07-30 15:58:08 UTC
Permalink
Guillaume,

Although not a perfect answer to your question, and I am not certain it will work with glmmTMB, I suggest you look at the drop1 function. It may give you an answer that you can use.

John


John David Sorkin M.D., Ph.D.
Professor of Medicine
Chief, Biostatistics and Informatics
University of Maryland School of Medicine Division of Gerontology and Geriatric Medicine
Baltimore VA Medical Center
10 North Greene Street
GRECC (BT/18/GR)
Baltimore, MD 21201-1524
(Phone) 410-605-7119
(Fax) 410-605-7913 (Please call phone number above prior to faxing)



________________________________
From: R-sig-mixed-models <r-sig-mixed-models-***@r-project.org> on behalf of Guillaume Adeux <***@gmail.com>
Sent: Monday, July 30, 2018 4:59 AM
To: r-sig-mixed-***@r-project.org
Subject: [R-sig-ME] Anova (type III-tests) table based on LRT for glmmTMB models

CAUTION: This message originated from a non UMB, UMSOM, FPI, or UMMS email system. Whether the sender is known or not known, hover over any links before clicking and use caution opening attachments.



Hello everyone,

I'm looking for a method/function in order to produce an Anova table based
on Likelihood Ratio Tests (LRT) for a glmmTMB model (R software). In my
case it is with a beta distribution and log link. My response is a ratio
(%) in a repeated measures design.

-

the function Anova() from the {car} package doesn't not run on single
models (i.e. Anova(mod)). It only allows comparison of two models (i.e.
Anova(mod,mod1)).
-

for glmer models, I was used to using the mixed() function from the
{afex} packages which produced Anova tables (type III tests) based on LRT
(or parametric bootstrap) for glmms.

Could anyone shed their on light on a function like mixed() which would run
on glmmTMB objects or on a procedure to do this by hand?

I suppose if only one fixed predictor was present in the model, this would
be simple by comparing it to a null model but my model contains an
interaction. Hence, I am incable of comparing a model A+B+B:C with a model
containing A+B:C.

Thanks for your interest.

Guillaume ADEUX

[[alternative HTML version deleted]]

_______________________________________________
R-sig-mixed-***@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

[[alternative HTML version deleted]]
Loading...