[R-sig-ME] prevent for-loop from stopping after an lmer warning/error

[Aleksander Adam Glowka]

Hi all,

I'm fitting mixed-effects regression models to bootstrap samples in a for-loop and writing a subset of model results to file. I diverted the stream from the console to a text file so I can keep track any warnings or errors. For some samples the model does not converge and a warning is issued. For other samples there is an error in calculation of the Satterthwaite's approximation. The problem is that my loop iteration is aborted for some reason after a warning is issued. Do you know why this might be happening and how I can make the loops continue to the next iteration after an error or a warning?

Below I've included an abridged version of my script and the warnings and errors I get. I'd be grateful for any advice you may have!

Thank you,

Aleksander Glowka
PhD Candidate
Department of Linguistics
Stanford University

#packages
require(lme4)
require(lmerTest)

setwd(path0)
source("lmer-data-extract-boot-fnc.R") #selected lmer results extractor

# divert messages stream to file, so you can log warnings and errors
options(warn=1)
wngs=file("warnings_log.txt",open="w+",blocking=TRUE)
sink(wngs,type="message")

for(iter in 1:2000){

setwd(path1)

data = read.csv(paste("sample", iter,".csv", sep=""), header=TRUE)

#log model number to file so that any potential warnings/errors appear underneath
message(paste("holistic model #", iter, sep=""))

mod = lmer(y ~ x1 +
x2 +
x3 +
(1|ranef1) +
(x1|ranef2),
data = data,
REML = FALSE)

#write model results to file
setwd(path2)
write.csv(lmer.data.extract.boot(mod, iter), paste("mod.fixef_", iter, ".csv", sep=""), row.names=TRUE) #fixed effects
write.csv(lmer.ranef.data.extract.boot(holistic.mod, iter), paste("mod.ranef_", iter, ".csv", sep=""), row.names=FALSE)
write.csv(lmer.optim.data.extract.boot(holistic.mod, iter), paste("mod.optim_", iter, ".csv", sep=""), row.names=TRUE)
write.csv(as.data.frame(***@optinfo$derivs$Hessian), paste("mod.hessian_", iter, ".csv", sep=""))
write.csv(summary(holistic.mod)$resid, paste("mod.resid_", iter, ".csv", sep=""), row.names=FALSE)
write(unlist(***@optinfo$conv$lme4$messages), paste("mod.warnings_", iter, ".txt", sep=""))

cat("Iteration", iter, "completed!\n")

}

#close log file & restore warnings stream to console
closeAllConnections()

Errors:

Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
unable to evaluate scaled gradient
Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
Model failed to converge: degenerate Hessian with 1 negative eigenvalues

Error in calculation of the Satterthwaite's approximation. The output of lme4 package is returned
summary from lme4 is returned
some computational error has occurred in lmerTest




[[alternative HTML version deleted]]

[Diogo Melo]

You can use the try() or a try-catch block.
See here: http://adv-r.had.co.nz/Exceptions-Debugging.html


On Thu, Mar 22, 2018 at 4:18 PM, Aleksander Adam Glowka <
***@stanford.edu> wrote:

> Hi all,
>
> I'm fitting mixed-effects regression models to bootstrap samples in a
> for-loop and writing a subset of model results to file. I diverted the
> stream from the console to a text file so I can keep track any warnings or
> errors. For some samples the model does not converge and a warning is
> issued. For other samples there is an error in calculation of the
> Satterthwaite's approximation. The problem is that my loop iteration is
> aborted for some reason after a warning is issued. Do you know why this
> might be happening and how I can make the loops continue to the next
> iteration after an error or a warning?
>
> Below I've included an abridged version of my script and the warnings and
> errors I get. I'd be grateful for any advice you may have!
>
> Thank you,
>
> Aleksander Glowka
> PhD Candidate
> Department of Linguistics
> Stanford University
>
> #packages
> require(lme4)
> require(lmerTest)
>
> setwd(path0)
> source("lmer-data-extract-boot-fnc.R") #selected lmer results extractor
>
> # divert messages stream to file, so you can log warnings and errors
> options(warn=1)
> wngs=file("warnings_log.txt",open="w+",blocking=TRUE)
> sink(wngs,type="message")
>
> for(iter in 1:2000){
>
> setwd(path1)
>
> data = read.csv(paste("sample", iter,".csv", sep=""), header=TRUE)
>
> #log model number to file so that any potential warnings/errors appear
> underneath
> message(paste("holistic model #", iter, sep=""))
>
> mod = lmer(y ~ x1 +
> x2 +
> x3 +
> (1|ranef1) +
> (x1|ranef2),
> data = data,
> REML = FALSE)
>
> #write model results to file
> setwd(path2)
> write.csv(lmer.data.extract.boot(mod, iter), paste("mod.fixef_", iter,
> ".csv", sep=""), row.names=TRUE) #fixed effects
> write.csv(lmer.ranef.data.extract.boot(holistic.mod, iter),
> paste("mod.ranef_", iter, ".csv", sep=""), row.names=FALSE)
> write.csv(lmer.optim.data.extract.boot(holistic.mod, iter),
> paste("mod.optim_", iter, ".csv", sep=""), row.names=TRUE)
> write.csv(as.data.frame(***@optinfo$derivs$Hessian),
> paste("mod.hessian_", iter, ".csv", sep=""))
> write.csv(summary(holistic.mod)$resid, paste("mod.resid_", iter,
> ".csv", sep=""), row.names=FALSE)
> write(unlist(***@optinfo$conv$lme4$messages),
> paste("mod.warnings_", iter, ".txt", sep=""))
>
> cat("Iteration", iter, "completed!\n")
>
> }
>
> #close log file & restore warnings stream to console
> closeAllConnections()
>
> Errors:
>
> Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl =
> control$checkConv, :
> unable to evaluate scaled gradient
> Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl =
> control$checkConv, :
> Model failed to converge: degenerate Hessian with 1 negative eigenvalues
>
> Error in calculation of the Satterthwaite's approximation. The output of
> lme4 package is returned
> summary from lme4 is returned
> some computational error has occurred in lmerTest
>
>
>
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-***@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>

[[alternative HTML version deleted]]

[Andrew Robinson]

?try

?tryCatch

Cheers,

Andrew


On 23 March 2018 at 06:18, Aleksander Adam Glowka <***@stanford.edu>
wrote:

> Hi all,
>
> I'm fitting mixed-effects regression models to bootstrap samples in a
> for-loop and writing a subset of model results to file. I diverted the
> stream from the console to a text file so I can keep track any warnings or
> errors. For some samples the model does not converge and a warning is
> issued. For other samples there is an error in calculation of the
> Satterthwaite's approximation. The problem is that my loop iteration is
> aborted for some reason after a warning is issued. Do you know why this
> might be happening and how I can make the loops continue to the next
> iteration after an error or a warning?
>
> Below I've included an abridged version of my script and the warnings and
> errors I get. I'd be grateful for any advice you may have!
>
> Thank you,
>
> Aleksander Glowka
> PhD Candidate
> Department of Linguistics
> Stanford University
>
> #packages
> require(lme4)
> require(lmerTest)
>
> setwd(path0)
> source("lmer-data-extract-boot-fnc.R") #selected lmer results extractor
>
> # divert messages stream to file, so you can log warnings and errors
> options(warn=1)
> wngs=file("warnings_log.txt",open="w+",blocking=TRUE)
> sink(wngs,type="message")
>
> for(iter in 1:2000){
>
> setwd(path1)
>
> data = read.csv(paste("sample", iter,".csv", sep=""), header=TRUE)
>
> #log model number to file so that any potential warnings/errors appear
> underneath
> message(paste("holistic model #", iter, sep=""))
>
> mod = lmer(y ~ x1 +
> x2 +
> x3 +
> (1|ranef1) +
> (x1|ranef2),
> data = data,
> REML = FALSE)
>
> #write model results to file
> setwd(path2)
> write.csv(lmer.data.extract.boot(mod, iter), paste("mod.fixef_", iter,
> ".csv", sep=""), row.names=TRUE) #fixed effects
> write.csv(lmer.ranef.data.extract.boot(holistic.mod, iter),
> paste("mod.ranef_", iter, ".csv", sep=""), row.names=FALSE)
> write.csv(lmer.optim.data.extract.boot(holistic.mod, iter),
> paste("mod.optim_", iter, ".csv", sep=""), row.names=TRUE)
> write.csv(as.data.frame(***@optinfo$derivs$Hessian),
> paste("mod.hessian_", iter, ".csv", sep=""))
> write.csv(summary(holistic.mod)$resid, paste("mod.resid_", iter,
> ".csv", sep=""), row.names=FALSE)
> write(unlist(***@optinfo$conv$lme4$messages),
> paste("mod.warnings_", iter, ".txt", sep=""))
>
> cat("Iteration", iter, "completed!\n")
>
> }
>
> #close log file & restore warnings stream to console
> closeAllConnections()
>
> Errors:
>
> Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl =
> control$checkConv, :
> unable to evaluate scaled gradient
> Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl =
> control$checkConv, :
> Model failed to converge: degenerate Hessian with 1 negative eigenvalues
>
> Error in calculation of the Satterthwaite's approximation. The output of
> lme4 package is returned
> summary from lme4 is returned
> some computational error has occurred in lmerTest
>
>
>
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-***@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>
>


--
Andrew Robinson
Director, CEBRA, School of BioSciences
Reader & Associate Professor in Applied Statistics Tel: [phone]
School of Mathematics and Statistics Fax: (+61) 03
8344 4599
University of Melbourne, VIC 3010 Australia
Email: ***@unimelb.edu.au
Website: http://www.ms.unimelb.edu.au/~andrewpr

[[alternative HTML version deleted]]

[Ben Bolker]

Is your loop really stopping after warnings (that would be
surprising), not just after errors?

On Thu, Mar 22, 2018 at 3:42 PM, Andrew Robinson <***@unimelb.edu.au> wrote:
> ?try
>
> ?tryCatch
>
> Cheers,
>
> Andrew
>
>
> On 23 March 2018 at 06:18, Aleksander Adam Glowka <***@stanford.edu>
> wrote:
>
>> Hi all,
>>
>> I'm fitting mixed-effects regression models to bootstrap samples in a
>> for-loop and writing a subset of model results to file. I diverted the
>> stream from the console to a text file so I can keep track any warnings or
>> errors. For some samples the model does not converge and a warning is
>> issued. For other samples there is an error in calculation of the
>> Satterthwaite's approximation. The problem is that my loop iteration is
>> aborted for some reason after a warning is issued. Do you know why this
>> might be happening and how I can make the loops continue to the next
>> iteration after an error or a warning?
>>
>> Below I've included an abridged version of my script and the warnings and
>> errors I get. I'd be grateful for any advice you may have!
>>
>> Thank you,
>>
>> Aleksander Glowka
>> PhD Candidate
>> Department of Linguistics
>> Stanford University
>>
>> #packages
>> require(lme4)
>> require(lmerTest)
>>
>> setwd(path0)
>> source("lmer-data-extract-boot-fnc.R") #selected lmer results extractor
>>
>> # divert messages stream to file, so you can log warnings and errors
>> options(warn=1)
>> wngs=file("warnings_log.txt",open="w+",blocking=TRUE)
>> sink(wngs,type="message")
>>
>> for(iter in 1:2000){
>>
>> setwd(path1)
>>
>> data = read.csv(paste("sample", iter,".csv", sep=""), header=TRUE)
>>
>> #log model number to file so that any potential warnings/errors appear
>> underneath
>> message(paste("holistic model #", iter, sep=""))
>>
>> mod = lmer(y ~ x1 +
>> x2 +
>> x3 +
>> (1|ranef1) +
>> (x1|ranef2),
>> data = data,
>> REML = FALSE)
>>
>> #write model results to file
>> setwd(path2)
>> write.csv(lmer.data.extract.boot(mod, iter), paste("mod.fixef_", iter,
>> ".csv", sep=""), row.names=TRUE) #fixed effects
>> write.csv(lmer.ranef.data.extract.boot(holistic.mod, iter),
>> paste("mod.ranef_", iter, ".csv", sep=""), row.names=FALSE)
>> write.csv(lmer.optim.data.extract.boot(holistic.mod, iter),
>> paste("mod.optim_", iter, ".csv", sep=""), row.names=TRUE)
>> write.csv(as.data.frame(***@optinfo$derivs$Hessian),
>> paste("mod.hessian_", iter, ".csv", sep=""))
>> write.csv(summary(holistic.mod)$resid, paste("mod.resid_", iter,
>> ".csv", sep=""), row.names=FALSE)
>> write(unlist(***@optinfo$conv$lme4$messages),
>> paste("mod.warnings_", iter, ".txt", sep=""))
>>
>> cat("Iteration", iter, "completed!\n")
>>
>> }
>>
>> #close log file & restore warnings stream to console
>> closeAllConnections()
>>
>> Errors:
>>
>> Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl =
>> control$checkConv, :
>> unable to evaluate scaled gradient
>> Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl =
>> control$checkConv, :
>> Model failed to converge: degenerate Hessian with 1 negative eigenvalues
>>
>> Error in calculation of the Satterthwaite's approximation. The output of
>> lme4 package is returned
>> summary from lme4 is returned
>> some computational error has occurred in lmerTest
>>
>>
>>
>>
>> [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-mixed-***@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>
>>
>
>
> --
> Andrew Robinson
> Director, CEBRA, School of BioSciences
> Reader & Associate Professor in Applied Statistics Tel: [phone]
> School of Mathematics and Statistics Fax: (+61) 03
> 8344 4599
> University of Melbourne, VIC 3010 Australia
> Email: ***@unimelb.edu.au
> Website: http://www.ms.unimelb.edu.au/~andrewpr
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-***@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

[Aleksander Adam Glowka]

Hi Ben,


Thank you for pointing this out. Indeed the loops only stops after errors, not warnings.


By the way, I am still stuck with my issue. I've made some progress with using tryCatch() as suggested, but still have not resolved my issue. As far as I can tell, currently there are two issues with my code. First, when I query the returned exception in `tryCatch()`, e.g. `poss.err.mod1` I get the following error:

<simpleError in tryCatchList(expr, classes, parentenv, handlers): argument "expr" is missing, with no default>

Still, the warnings are correctly written to file, which is puzzling. Second, when no error is detected, the code seems to be ignored (i.e. no model summaries are written to file). So the loop is running vacuously, skipping errors but apparently handling them and doing nothing else. If anyone has any advice on how to debug this, please let me know!

# divert console stream to file
options(warn=1)
wngs=file("warnings_log.txt",open="w+",blocking=TRUE)
sink(wngs,type="message")

for(iter in 1:1000){

data = read.csv(paste("sample", iter,".csv", sep=""), header=TRUE)

#error handling case #1
poss.err.mod1 = tryCatch(

lmer(y ~ x1 + (1|ranef), data = data, REML = FALSE), #try part
error=function(e) e #catch part
)

#error handling case #2
poss.err.mod2 = tryCatch(

lmer(y ~ x1 + x2 + (1|ranef), data = data, REML = FALSE), #try part
error=function(e) e #catch part
)

#real work #1
if(!inherits(poss.err.mod1, "error")){

mod1 = lmer(y ~ x1 + (1|ranef1), data = data, REML = FALSE)
write.csv(summmary$mod2, paste("mod2.sum_", iter, ".csv", sep="")

}else{

write(unlist(***@optinfo$conv$lme4$messages), paste("mod1.errors_", iter, ".txt", sep=""))
}

#real work #2
if(!inherits(poss.err.mod2, "error")){

mod2 = lmer(y ~ x1 + x2 + (1|ranef1), data = data, REML = FALSE)
write.csv(summmary$mod2, paste("mod2.sum_", iter, ".csv",)

}else{

write(unlist(***@optinfo$conv$lme4$messages), paste("mod1.errors_", iter, ".txt", sep=""))
}

cat("Iteration", iter, "completed!\n")
}

#close log file & restore warnings stream to console
closeAllConnections()

________________________________
From: Ben Bolker <***@gmail.com>
Sent: Friday, March 23, 2018 10:42:47 AM
To: Andrew Robinson
Cc: Aleksander Adam Glowka; Help Mixed Models
Subject: Re: [R-sig-ME] prevent for-loop from stopping after an lmer warning/error

Is your loop really stopping after warnings (that would be
surprising), not just after errors?

On Thu, Mar 22, 2018 at 3:42 PM, Andrew Robinson <***@unimelb.edu.au> wrote:
> ?try
>
> ?tryCatch
>
> Cheers,
>
> Andrew
>
>
> On 23 March 2018 at 06:18, Aleksander Adam Glowka <***@stanford.edu>
> wrote:
>
>> Hi all,
>>
>> I'm fitting mixed-effects regression models to bootstrap samples in a
>> for-loop and writing a subset of model results to file. I diverted the
>> stream from the console to a text file so I can keep track any warnings or
>> errors. For some samples the model does not converge and a warning is
>> issued. For other samples there is an error in calculation of the
>> Satterthwaite's approximation. The problem is that my loop iteration is
>> aborted for some reason after a warning is issued. Do you know why this
>> might be happening and how I can make the loops continue to the next
>> iteration after an error or a warning?
>>
>> Below I've included an abridged version of my script and the warnings and
>> errors I get. I'd be grateful for any advice you may have!
>>
>> Thank you,
>>
>> Aleksander Glowka
>> PhD Candidate
>> Department of Linguistics
>> Stanford University
>>
>> #packages
>> require(lme4)
>> require(lmerTest)
>>
>> setwd(path0)
>> source("lmer-data-extract-boot-fnc.R") #selected lmer results extractor
>>
>> # divert messages stream to file, so you can log warnings and errors
>> options(warn=1)
>> wngs=file("warnings_log.txt",open="w+",blocking=TRUE)
>> sink(wngs,type="message")
>>
>> for(iter in 1:2000){
>>
>> setwd(path1)
>>
>> data = read.csv(paste("sample", iter,".csv", sep=""), header=TRUE)
>>
>> #log model number to file so that any potential warnings/errors appear
>> underneath
>> message(paste("holistic model #", iter, sep=""))
>>
>> mod = lmer(y ~ x1 +
>> x2 +
>> x3 +
>> (1|ranef1) +
>> (x1|ranef2),
>> data = data,
>> REML = FALSE)
>>
>> #write model results to file
>> setwd(path2)
>> write.csv(lmer.data.extract.boot(mod, iter), paste("mod.fixef_", iter,
>> ".csv", sep=""), row.names=TRUE) #fixed effects
>> write.csv(lmer.ranef.data.extract.boot(holistic.mod, iter),
>> paste("mod.ranef_", iter, ".csv", sep=""), row.names=FALSE)
>> write.csv(lmer.optim.data.extract.boot(holistic.mod, iter),
>> paste("mod.optim_", iter, ".csv", sep=""), row.names=TRUE)
>> write.csv(as.data.frame(***@optinfo$derivs$Hessian),
>> paste("mod.hessian_", iter, ".csv", sep=""))
>> write.csv(summary(holistic.mod)$resid, paste("mod.resid_", iter,
>> ".csv", sep=""), row.names=FALSE)
>> write(unlist(***@optinfo$conv$lme4$messages),
>> paste("mod.warnings_", iter, ".txt", sep=""))
>>
>> cat("Iteration", iter, "completed!\n")
>>
>> }
>>
>> #close log file & restore warnings stream to console
>> closeAllConnections()
>>
>> Errors:
>>
>> Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl =
>> control$checkConv, :
>> unable to evaluate scaled gradient
>> Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl =
>> control$checkConv, :
>> Model failed to converge: degenerate Hessian with 1 negative eigenvalues
>>
>> Error in calculation of the Satterthwaite's approximation. The output of
>> lme4 package is returned
>> summary from lme4 is returned
>> some computational error has occurred in lmerTest
>>
>>
>>
>>
>> [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-mixed-***@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>
>>
>
>
> --
> Andrew Robinson
> Director, CEBRA, School of BioSciences
> Reader & Associate Professor in Applied Statistics Tel: [phone]
> School of Mathematics and Statistics Fax: (+61) 03
> 8344 4599
> University of Melbourne, VIC 3010 Australia
> Email: ***@unimelb.edu.au
> Website: http://www.ms.unimelb.edu.au/~andrewpr
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-***@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

[[alternative HTML version deleted]]

[Ben Bolker]

It's possible that lmerTest is doing slightly weird things with trying
to catch the errors itself.

Rune Haubo is working on a revamped version of lmerTest
(https://rdrr.io/github/runehaubo/lmerTestR/); it *might* be worth
taking a try at installing that to see if it behaves better?

On Fri, Mar 23, 2018 at 5:57 PM, Aleksander Adam Glowka
<***@stanford.edu> wrote:
> Hi Ben,
>
>
> Thank you for pointing this out. Indeed the loops only stops after errors, not warnings.
>
>
> By the way, I am still stuck with my issue. I've made some progress with using tryCatch() as suggested, but still have not resolved my issue. As far as I can tell, currently there are two issues with my code. First, when I query the returned exception in `tryCatch()`, e.g. `poss.err.mod1` I get the following error:
>
> <simpleError in tryCatchList(expr, classes, parentenv, handlers): argument "expr" is missing, with no default>
>
> Still, the warnings are correctly written to file, which is puzzling. Second, when no error is detected, the code seems to be ignored (i.e. no model summaries are written to file). So the loop is running vacuously, skipping errors but apparently handling them and doing nothing else. If anyone has any advice on how to debug this, please let me know!
>
> # divert console stream to file
> options(warn=1)
> wngs=file("warnings_log.txt",open="w+",blocking=TRUE)
> sink(wngs,type="message")
>
> for(iter in 1:1000){
>
> data = read.csv(paste("sample", iter,".csv", sep=""), header=TRUE)
>
> #error handling case #1
> poss.err.mod1 = tryCatch(
>
> lmer(y ~ x1 + (1|ranef), data = data, REML = FALSE), #try part
> error=function(e) e #catch part
> )
>
> #error handling case #2
> poss.err.mod2 = tryCatch(
>
> lmer(y ~ x1 + x2 + (1|ranef), data = data, REML = FALSE), #try part
> error=function(e) e #catch part
> )
>
> #real work #1
> if(!inherits(poss.err.mod1, "error")){
>
> mod1 = lmer(y ~ x1 + (1|ranef1), data = data, REML = FALSE)
> write.csv(summmary$mod2, paste("mod2.sum_", iter, ".csv", sep="")
>
> }else{
>
> write(unlist(***@optinfo$conv$lme4$messages), paste("mod1.errors_", iter, ".txt", sep=""))
> }
>
> #real work #2
> if(!inherits(poss.err.mod2, "error")){
>
> mod2 = lmer(y ~ x1 + x2 + (1|ranef1), data = data, REML = FALSE)
> write.csv(summmary$mod2, paste("mod2.sum_", iter, ".csv",)
>
> }else{
>
> write(unlist(***@optinfo$conv$lme4$messages), paste("mod1.errors_", iter, ".txt", sep=""))
> }
>
> cat("Iteration", iter, "completed!\n")
> }
>
> #close log file & restore warnings stream to console
> closeAllConnections()
>
> ________________________________
> From: Ben Bolker <***@gmail.com>
> Sent: Friday, March 23, 2018 10:42:47 AM
> To: Andrew Robinson
> Cc: Aleksander Adam Glowka; Help Mixed Models
> Subject: Re: [R-sig-ME] prevent for-loop from stopping after an lmer warning/error
>
> Is your loop really stopping after warnings (that would be
> surprising), not just after errors?
>
> On Thu, Mar 22, 2018 at 3:42 PM, Andrew Robinson <***@unimelb.edu.au> wrote:
>> ?try
>>
>> ?tryCatch
>>
>> Cheers,
>>
>> Andrew
>>
>>
>> On 23 March 2018 at 06:18, Aleksander Adam Glowka <***@stanford.edu>
>> wrote:
>>
>>> Hi all,
>>>
>>> I'm fitting mixed-effects regression models to bootstrap samples in a
>>> for-loop and writing a subset of model results to file. I diverted the
>>> stream from the console to a text file so I can keep track any warnings or
>>> errors. For some samples the model does not converge and a warning is
>>> issued. For other samples there is an error in calculation of the
>>> Satterthwaite's approximation. The problem is that my loop iteration is
>>> aborted for some reason after a warning is issued. Do you know why this
>>> might be happening and how I can make the loops continue to the next
>>> iteration after an error or a warning?
>>>
>>> Below I've included an abridged version of my script and the warnings and
>>> errors I get. I'd be grateful for any advice you may have!
>>>
>>> Thank you,
>>>
>>> Aleksander Glowka
>>> PhD Candidate
>>> Department of Linguistics
>>> Stanford University
>>>
>>> #packages
>>> require(lme4)
>>> require(lmerTest)
>>>
>>> setwd(path0)
>>> source("lmer-data-extract-boot-fnc.R") #selected lmer results extractor
>>>
>>> # divert messages stream to file, so you can log warnings and errors
>>> options(warn=1)
>>> wngs=file("warnings_log.txt",open="w+",blocking=TRUE)
>>> sink(wngs,type="message")
>>>
>>> for(iter in 1:2000){
>>>
>>> setwd(path1)
>>>
>>> data = read.csv(paste("sample", iter,".csv", sep=""), header=TRUE)
>>>
>>> #log model number to file so that any potential warnings/errors appear
>>> underneath
>>> message(paste("holistic model #", iter, sep=""))
>>>
>>> mod = lmer(y ~ x1 +
>>> x2 +
>>> x3 +
>>> (1|ranef1) +
>>> (x1|ranef2),
>>> data = data,
>>> REML = FALSE)
>>>
>>> #write model results to file
>>> setwd(path2)
>>> write.csv(lmer.data.extract.boot(mod, iter), paste("mod.fixef_", iter,
>>> ".csv", sep=""), row.names=TRUE) #fixed effects
>>> write.csv(lmer.ranef.data.extract.boot(holistic.mod, iter),
>>> paste("mod.ranef_", iter, ".csv", sep=""), row.names=FALSE)
>>> write.csv(lmer.optim.data.extract.boot(holistic.mod, iter),
>>> paste("mod.optim_", iter, ".csv", sep=""), row.names=TRUE)
>>> write.csv(as.data.frame(***@optinfo$derivs$Hessian),
>>> paste("mod.hessian_", iter, ".csv", sep=""))
>>> write.csv(summary(holistic.mod)$resid, paste("mod.resid_", iter,
>>> ".csv", sep=""), row.names=FALSE)
>>> write(unlist(***@optinfo$conv$lme4$messages),
>>> paste("mod.warnings_", iter, ".txt", sep=""))
>>>
>>> cat("Iteration", iter, "completed!\n")
>>>
>>> }
>>>
>>> #close log file & restore warnings stream to console
>>> closeAllConnections()
>>>
>>> Errors:
>>>
>>> Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl =
>>> control$checkConv, :
>>> unable to evaluate scaled gradient
>>> Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl =
>>> control$checkConv, :
>>> Model failed to converge: degenerate Hessian with 1 negative eigenvalues
>>>
>>> Error in calculation of the Satterthwaite's approximation. The output of
>>> lme4 package is returned
>>> summary from lme4 is returned
>>> some computational error has occurred in lmerTest
>>>
>>>
>>>
>>>
>>> [[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> R-sig-mixed-***@r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>
>>>
>>
>>
>> --
>> Andrew Robinson
>> Director, CEBRA, School of BioSciences
>> Reader & Associate Professor in Applied Statistics Tel: [phone]
>> School of Mathematics and Statistics Fax: (+61) 03
>> 8344 4599
>> University of Melbourne, VIC 3010 Australia
>> Email: ***@unimelb.edu.au
>> Website: http://www.ms.unimelb.edu.au/~andrewpr
>>
>> [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-mixed-***@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-***@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models