(Migrated) ASK::All Server Is down after rebooting the VM

(This message has been automatically imported from the retired mailing list)

Dear Zato,

// (sorry for re-openting the topic, lets just make the topic clear)
My problem the server says its all down
the problem cause i think because i wasn’t run the "./zato-qs-stop.sh"
before rebooting the VM

here is the cronology,
i run the "./zato-qs-start.sh"
i reboot the VM
i tried to enter the zato webadmin
and it says

ClusterAddresses quickstart-381899

Created by zato@esb on 2014-12-03T10:58:52.478737 (UTC)

All servers are DOWNHTTP: http://127.0.0.1:11223 http://127.0.0.1:11223/
LB XML-RPC agent: https://localhost:20151/RPC2
https://localhost:20151/RPC2
LB stats: http://localhost:11223/zato-lb-stats
http://localhost:11223/zato-lb-stats
LB HTTP health-check: http://127.0.0.1:11223/zato-lb-aliveEditDelete
Load-balancer
http://192.168.1.107:8183/zato/load-balancer/manage/cluster/1/Servers
http://192.168.1.107:8183/zato/cluster/servers/?cluster=1
Here is what the logs says:

2014-12-03 19:00:26,358 - INFO - 2367:Dummy-5 - root:111 - Shutting down
thread pool
2014-12-03 19:00:26,495 - INFO - 2343:Dummy-1 - gunicorn.error:22 -
Shutting down: Master
2014-12-03 19:00:49,741 - INFO - 2466:Dummy-1 - gunicorn.error:22 -
Starting gunicorn 0.16.1
2014-12-03 19:00:49,749 - INFO - 2466:Dummy-1 - gunicorn.error:22 -
Listening at: http://127.0.0.1:17010 (2466)
2014-12-03 19:00:49,749 - INFO - 2466:Dummy-1 - gunicorn.error:22 - Using
worker: gevent
2014-12-03 19:00:49,757 - INFO - 2474:Dummy-1 - gunicorn.error:22 - Booting
worker with pid: 2474
2014-12-03 19:01:00,805 - INFO - 2474:Dummy-3 - zato.broker.client:22 -
Starting broker client, host:[localhost], port:[6379], name:[parallel-
K113618630153458089892620258310947400441], topics:[[’/zato/to-parallel/all’,
’/zato/to-parallel/any’, ‘/zato/to-singleton’]]
2014-12-03 19:01:01,306 - INFO - 2474:Thread-2 - SingletonServer:22 -
Pickup notifier starting
2014-12-03 19:01:01,308 - INFO - 2474:Dummy-1 - apscheduler.threadpool:22 -
Started thread pool with 0 core threads and 20 maximum threads
2014-12-03 19:01:01,308 - INFO - 2474:APScheduler -
apscheduler.scheduler:22 - Scheduler started
2014-12-03 19:01:01,333 - INFO - 2474:Dummy-1 - zato.server.odb:22 - Server
id:[1], name:[server1] is now a connector server for cluster id:[1],
name:[quickstart-381899]
2014-12-03 19:01:01,340 - INFO - 2474:Dummy-1 - apscheduler.scheduler:22 -
Added job “zato.server.cluster-wide-singleton-keep-alive (trigger:
interval[0:00:30], next run at: 2014-12-03 11:01:31.340579)” to job store
"default"
2014-12-03 19:01:01,341 - INFO - 2474:Dummy-1 - zato.server.scheduler:22 -
Interval-based job [zato.server.cluster-wide-singleton-keep-alive] scheduled
2014-12-03 19:01:01,341 - INFO - 2474:Dummy-1 -
zato.server.base.parallel:22 - Initializing connectors
2014-12-03 19:01:01,345 - INFO - 2474:Dummy-1 -
zato.server.base.parallel:22 - No AMQP channels to start
2014-12-03 19:01:01,355 - INFO - 2474:Dummy-1 -
zato.server.base.parallel:22 - No AMQP outgoing connections to start
2014-12-03 19:01:01,365 - INFO - 2474:Dummy-1 -
zato.server.base.parallel:22 - No JMS WebSphere MQ channels to start
2014-12-03 19:01:01,369 - INFO - 2474:Dummy-1 -
zato.server.base.parallel:22 - No JMS WebSphere MQ outgoing connections to
start
2014-12-03 19:01:01,379 - INFO - 2474:Dummy-1 -
zato.server.base.parallel:22 - No Zero MQ channels to start
2014-12-03 19:01:01,388 - INFO - 2474:Dummy-1 -
zato.server.base.parallel:22 - No Zero MQ outgoing connections to start
2014-12-03 19:01:01,405 - INFO - 2474:Dummy-1 - apscheduler.scheduler:22 -
Added job “zato.stats.aggregate-by-day (trigger: interval[1:00:00], next
run at: 2014-12-03 12:00:23)” to job store "default"
2014-12-03 19:01:01,405 - INFO - 2474:Dummy-1 - zato.server.scheduler:22 -
Interval-based job [zato.stats.aggregate-by-day] scheduled
2014-12-03 19:01:01,406 - INFO - 2474:Dummy-1 - apscheduler.scheduler:22 -
Added job “zato.stats.aggregate-by-hour (trigger: interval[0:10:00], next
run at: 2014-12-03 11:10:23)” to job store "default"
2014-12-03 19:01:01,406 - INFO - 2474:Dummy-1 - zato.server.scheduler:22 -
Interval-based job [zato.stats.aggregate-by-hour] scheduled
2014-12-03 19:01:01,406 - INFO - 2474:Dummy-1 - apscheduler.scheduler:22 -
Added job “zato.stats.aggregate-by-minute (trigger: interval[0:01:00], next
run at: 2014-12-03 11:01:23)” to job store "default"
2014-12-03 19:01:01,407 - INFO - 2474:Dummy-1 - zato.server.scheduler:22 -
Interval-based job [zato.stats.aggregate-by-minute] scheduled
2014-12-03 19:01:01,408 - INFO - 2474:Dummy-1 - apscheduler.scheduler:22 -
Added job “zato.stats.aggregate-by-month (trigger: interval[1:00:00], next
run at: 2014-12-03 12:00:23)” to job store "default"
2014-12-03 19:01:01,415 - INFO - 2474:Dummy-1 - zato.server.scheduler:22 -
Interval-based job [zato.stats.aggregate-by-month] scheduled
2014-12-03 19:01:01,416 - INFO - 2474:Dummy-1 - apscheduler.scheduler:22 -
Added job “zato.stats.process-raw-times (trigger: interval[0:01:30], next
run at: 2014-12-03 11:01:53)” to job store "default"
2014-12-03 19:01:01,416 - INFO - 2474:Dummy-1 - zato.server.scheduler:22 -
Interval-based job [zato.stats.process-raw-times] scheduled
2014-12-03 19:01:01,417 - INFO - 2474:Dummy-1 - apscheduler.scheduler:22 -
Added job “zato.stats.summary.create-summary-by-day (trigger:
interval[0:10:00], next run at: 2014-12-03 11:10:23)” to job store "default"
2014-12-03 19:01:01,417 - INFO - 2474:Dummy-1 - zato.server.scheduler:22 -
Interval-based job [zato.stats.summary.create-summary-by-day] scheduled
2014-12-03 19:01:01,418 - INFO - 2474:Dummy-1 - apscheduler.scheduler:22 -
Added job “zato.stats.summary.create-summary-by-month (trigger:
interval[1:00:00], next run at: 2014-12-03 12:00:23)” to job store "default"
2014-12-03 19:01:01,418 - INFO - 2474:Dummy-1 - zato.server.scheduler:22 -
Interval-based job [zato.stats.summary.create-summary-by-month] scheduled
2014-12-03 19:01:01,418 - INFO - 2474:Dummy-1 - apscheduler.scheduler:22 -
Added job “zato.stats.summary.create-summary-by-week (trigger:
interval[0:10:00], next run at: 2014-12-03 11:10:23)” to job store "default"
2014-12-03 19:01:01,418 - INFO - 2474:Dummy-1 - zato.server.scheduler:22 -
Interval-based job [zato.stats.summary.create-summary-by-week] scheduled
2014-12-03 19:01:01,419 - INFO - 2474:Dummy-1 - apscheduler.scheduler:22 -
Added job “zato.stats.summary.create-summary-by-year (trigger:
interval[1:00:00], next run at: 2014-12-03 12:00:23)” to job store "default"
2014-12-03 19:01:01,420 - INFO - 2474:Dummy-1 - zato.server.scheduler:22 -
Interval-based job [zato.stats.summary.create-summary-by-year] scheduled
2014-12-03 19:01:14,480 - INFO - 2474:Dummy-3 -
zato.server.ensure-cluster-wide-singleton:22

  • cid:[K061349808730559658767184062360546851156],
    name:[zato.server.ensure-cluster-wide-singleton],
    SIO request:[{}]
    2014-12-03 19:01:14,480 - INFO - 2474:Dummy-3 -
    zato.server.ensure-cluster-wide-singleton:22
  • cid:[K061349808730559658767184062360546851156],
    name:[zato.server.ensure-cluster-wide-singleton],
    response:[]

Thank You,

On 03/12/14 19:34, Rafed Ramzi wrote:

2014-12-03 19:00:49,749 - INFO - 2466:Dummy-1 - gunicorn.error:22 -
Listening at: http://127.0.0.1:17010 http://127.0.0.1:17010/ (2466)

Right - so the server is started, that’s good.

But the load balancer doesn’t notice it - that’s less than good and
let’s focus on that one now.

What happens if you issue the command below?

$ curl localhost:17010/zato/ping

On 03/12/14 19:34, Rafed Ramzi wrote:

2014-12-03 19:00:49,749 - INFO - 2466:Dummy-1 - gunicorn.error:22 -
Listening at: http://127.0.0.1:17010 http://127.0.0.1:17010/ (2466)

Right - so the server is started, that’s good.

But the load balancer doesn’t notice it - that’s less than good and
let’s focus on that one now.

What happens if you issue the command below?

$ curl localhost:17010/zato/ping