We are getting a issue on our site getting down with following message in mod_jk.log
[Tue Sep 03 16:22:34 2013][22149:1585183040] [info] service::jk_lb_worker.c (1448): All tomcat instances failed, no more workers left for recovery (attempt=0, retry=0)
[Tue Sep 03 16:22:34 2013][22149:1585183040] [info] service::jk_lb_worker.c (1457): All tomcat instances failed, no more workers left (attempt=1, retry=0)
[Tue Sep 03 16:22:34 2013][22149:1501264192] [info] service::jk_lb_worker.c (1448): All tomcat instances failed, no more workers left for recovery (attempt=0, retry=0)
[Tue Sep 03 16:22:34 2013][22149:1501264192] [info] service::jk_lb_worker.c (1457): All tomcat instances failed, no more workers left (attempt=1, retry=0)
[Tue Sep 03 16:22:34 2013][22149:1616652608] [info] service::jk_lb_worker.c (1457): All tomcat instances failed, no more workers left (attempt=0, retry=1)
[Tue Sep 03 16:22:34 2013][22149:1616652608] [info] service::jk_lb_worker.c (1457): All tomcat instances failed, no more workers left (attempt=1, retry=1)
[Tue Sep 03 16:22:34 2013][22149:1616652608] [info] service::jk_lb_worker.c (1468): All tomcat instances are busy or in error state
[Tue Sep 03 16:22:34 2013][22149:1616652608] [error] service::jk_lb_worker.c (1473): All tomcat instances failed, no more workers left
[Tue Sep 03 16:22:34 2013]loadbalancer - naf.timesjobs.com 0 0.100549 POST /naf/log.html 200
[Tue Sep 03 16:22:34 2013][22149:1616652608] [info] jk_handler::mod_jk.c (2627): Service error=0 for worker=loadbalancer
[Tue Sep 03 16:22:34 2013][22149:1606162752] [info] service::jk_lb_worker.c (1457): All tomcat instances failed, no more workers left (attempt=0, retry=1)
[Tue Sep 03 16:22:34 2013][22149:1606162752] [info] service::jk_lb_worker.c (1457): All tomcat instances failed, no more workers left (attempt=1, retry=1)
[Tue Sep 03 16:22:34 2013][22149:1606162752] [info] service::jk_lb_worker.c (1468): All tomcat instances are busy or in error state
[Tue Sep 03 16:22:34 2013][22149:1606162752] [error] service::jk_lb_worker.c (1473): All tomcat instances failed, no more workers left
[Tue Sep 03 16:22:34 2013]loadbalancer - naf.timesjobs.com 0 0.100469 POST /naf/log.html 200
[Tue Sep 03 16:22:34 2013][22149:1606162752] [info] jk_handler::mod_jk.c (2627): Service error=0 for worker=loadbalancer
[Tue Sep 03 16:22:34 2013][22149:1690081600] [info] service::jk_lb_worker.c (1448): All tomcat instances failed, no more workers left for recovery (attempt=0, retry=0)
[Tue Sep 03 16:22:34 2013][22149:1690081600] [info] service::jk_lb_worker.c (1457): All tomcat instances failed, no more workers left (attempt=1, retry=0)
[Tue Sep 03 16:22:34 2013][22149:1501264192] [info] service::jk_lb_worker.c (1448): All tomcat instances failed, no more workers left for recovery (attempt=0, retry=0)
[Tue Sep 03 16:22:34 2013][22149:1501264192] [info] service::jk_lb_worker.c (1457): All tomcat instances failed, no more workers left (attempt=1, retry=0)
[Tue Sep 03 16:22:34 2013][22149:1616652608] [info] service::jk_lb_worker.c (1457): All tomcat instances failed, no more workers left (attempt=0, retry=1)
[Tue Sep 03 16:22:34 2013][22149:1616652608] [info] service::jk_lb_worker.c (1457): All tomcat instances failed, no more workers left (attempt=1, retry=1)
[Tue Sep 03 16:22:34 2013][22149:1616652608] [info] service::jk_lb_worker.c (1468): All tomcat instances are busy or in error state
[Tue Sep 03 16:22:34 2013][22149:1616652608] [error] service::jk_lb_worker.c (1473): All tomcat instances failed, no more workers left
[Tue Sep 03 16:22:34 2013]loadbalancer - naf.timesjobs.com 0 0.100549 POST /naf/log.html 200
[Tue Sep 03 16:22:34 2013][22149:1616652608] [info] jk_handler::mod_jk.c (2627): Service error=0 for worker=loadbalancer
[Tue Sep 03 16:22:34 2013][22149:1606162752] [info] service::jk_lb_worker.c (1457): All tomcat instances failed, no more workers left (attempt=0, retry=1)
[Tue Sep 03 16:22:34 2013][22149:1606162752] [info] service::jk_lb_worker.c (1457): All tomcat instances failed, no more workers left (attempt=1, retry=1)
[Tue Sep 03 16:22:34 2013][22149:1606162752] [info] service::jk_lb_worker.c (1468): All tomcat instances are busy or in error state
[Tue Sep 03 16:22:34 2013][22149:1606162752] [error] service::jk_lb_worker.c (1473): All tomcat instances failed, no more workers left
[Tue Sep 03 16:22:34 2013]loadbalancer - naf.timesjobs.com 0 0.100469 POST /naf/log.html 200
[Tue Sep 03 16:22:34 2013][22149:1606162752] [info] jk_handler::mod_jk.c (2627): Service error=0 for worker=loadbalancer
[Tue Sep 03 16:22:34 2013][22149:1690081600] [info] service::jk_lb_worker.c (1448): All tomcat instances failed, no more workers left for recovery (attempt=0, retry=0)
[Tue Sep 03 16:22:34 2013][22149:1690081600] [info] service::jk_lb_worker.c (1457): All tomcat instances failed, no more workers left (attempt=1, retry=0)
mod-jk.conf
<IfModule mpm_worker_module>
ThreadLimit 60
ServerLimit 10
StartServers 3
MinSpareThreads 5
MaxSpareThreads 20
MaxClients 600
ThreadsPerChild 60
MaxRequestsPerChild 0
</IfModule>
While in tomcat server.xml <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" maxThreads="400" acceptCount="100"/>
Due to which the above error is coming at peak load when apache 600 request coming while tomcat is able to serve only 400.
We changed the setting <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" maxThreads="600" acceptCount="100"/>
and the problem got resolved.
No comments:
Post a Comment