«

»

Sep 01

Mysql error followed by Openfire XMPP server halt

I faced an issue as my IM messenger suddenly got disconnected from the openfire xmpp server. I checked the internet connection to my computer and it was still looking fine. So I hop into openfire server and tried to manually start it. It showed as openfire service starting, but it went off suddenly. I checked some of its logs and could see the issue was with connectivity to mysql service running in same box. Following was the step by step methods I tried to pin point the issue and to sort it out.

First I did ssh the server and got into bash shell.

Tried to manually start openfire service
host # /etc/init.d/openfire start
Starting openfire:

It started running
host # /etc/init.d/openfire status
openfire is running

Suddenly went off
host # /etc/init.d/openfire status
openfire is not running

Showed a pid already exists for openfire when I tried to start openfire again
host # /etc/init.d/openfire start
Openfire is already running. Remove /var/run/openfire.pid if you know this to be untrue.

Removed existing pid
host # rm /var/run/openfire.pid

Starting service
host # /etc/init.d/openfire start
Starting openfire:

It started once again
host # /etc/init.d/openfire status
openfire is running

Went off again
host # /etc/init.d/openfire status
openfire is not running

Checked one of openfire’s log. After referencing some threads, I could see this is due to connectivity issues with the database.
host # tail -f /opt/openfire/logs/info.log
at org.jivesoftware.openfire.XMPPServer.<init>(XMPPServer.java:216)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at java.lang.Class.newInstance0(Unknown Source)
at java.lang.Class.newInstance(Unknown Source)
at org.jivesoftware.openfire.starter.ServerStarter.start(ServerStarter.java:113)
at org.jivesoftware.openfire.starter.ServerStarter.main(ServerStarter.java:58)
2015.08.31 11:49:23 org.jivesoftware.openfire.XMPPServer – Server halted

I checked status of mysql and found it was down
host # /etc/init.d/mysqld status
mysqld is stopped

Failed over and over while attempting on starting it.
host # /etc/init.d/mysqld start
MySQL Daemon failed to start.
Starting mysqld: [FAILED]

Found the issue with broken binary index file
host # tail -f /var/log/mysqld.log
150831 11:42:10 [ERROR] Failed to open log (file ‘/var/lib/mysql/mysql-bin.000078’, errno 2)
150831 11:42:10 [ERROR] Could not open log file
150831 11:42:10 [ERROR] Can’t init tc log
150831 11:42:10 [ERROR] Aborting

150831 11:42:10 InnoDB: Starting shutdown…
150831 11:42:11 InnoDB: Shutdown completed; log sequence number 97468154
150831 11:42:11 [Note] /usr/libexec/mysqld: Shutdown complete

150831 11:42:11 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended

Removed current index file
host # rm /var/lib/mysql/mysql-bin.index

Started mysql server. Fresh Index file will automatically recreate.
host # /etc/init.d/mysqld start
Starting mysqld: [ OK ]

Found it was still running fine
host # /etc/init.d/mysqld status
mysqld (pid 10838) is running…

Finally fixed openfire issue as well.
host # /etc/init.d/openfire start
Starting openfire:

host # /etc/init.d/openfire status
openfire is running

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>