Outage - 26th January 2018

We are currently investigating problems, since approx 10.46am, with several of our servers located at the Cheltenham data centre.

Email service appears off-line and there is intermittent service on web servers.


UPDATES

2.30pm All servers back on-line

3.30pm email servers off line again and all web servers off line

4.30pm Service restored on mailbox server only

Saturday 27th January 2018

We are unable to contact support staff associated with the servers at Cheltenham. Mailscanner and web servers at Cheltenham still off line but mailbox server on-line.

8.30am Started to alter dns settings for individual clients so that if mailscanner server is not available mail is sent directly to mailbox server

9.30am Started to move websites to a server at a different data centre

10.30am most email clients are now receiving mail directly to their mailboxes, but may receive more spam mail.

Sunday 28th January 2018

9.00am Cloned copies of all web servers at Cheltenham now on-line at new data centre but all IP addresses in hosts files need to be changed. Also DNS records for all domains need to be changed to point to new servers.

10.00am All host files now edited. Started to change DNS records. Individual websites will come on-line as DNS is changed.

2.34pm Mailscanner server now on-line at new data centre. Mail starting to come through with reduced spam content.

7.27pm Finally completed changing all DNS records - all websites now back on-line.

Monday 29th January 2018

Incoming mail is not being transferred from the mailscanner to the mailbox server
11.00am in attempting to fix the mail transfer problem the mailbox server has not come back on-line after a re-boot.

12.15pm We have asked technicians at Cheltenham to restart the server and are awaiting their response.

2.10pm A technician has started working directly on the server at Cheltenham.

Tuesdayday 30th January 2018

mailbox server is still off-line
2.00pm A new mailbox server has been put on-line and we have started migrating email accounts to the new server, but please be aware this will take several days to complete

Wednesdayday 31st January 2018

Continuing to migrate email accounts to new mailbox server.
7.09pm old mailbox server has unexpectedly come back on line.