Gallery Blog
« Idiot Drivers
Unbelievable »

Some organisations never learn

Oh, surprise surprise. Warwick University’s IT facilities are broken again.

From http://www2.warwick.ac.uk/services/its/aboutus/message/

A major power failure affecting much of campus occurred at about 4:30 am on 9th November and power was off for over an hour. It affected both University House and the IT Services building where the University’s two main computer rooms are housed.

[snip – the UPS generator failed despite being tested last week]

The sudden loss of power to the servers in the ITS machine room impacted most University IT facilities and caused both physical and data corruption damage. IT Services were alerted to the situation at 6:15am by Security and initiated their Major Incident procedure. By soon after 7:00am it had been established that while many servers were operating, there were substantial failures. The network had been affected but was in process of recovery.

Sounds remarkably familiar, no? Indeed – a very similar thing happened earlier this year.

5 days later and email is not fully restored. I know that the problems with Groupwise were one of the reasons for moving to Exchange, but still – 5 days without being able to recover these servers is a new record, methinks.

This entry was posted on Tuesday, November 14th, 2006 at 14:12 and is filed under General. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

2 Responses to “Some organisations never learn”

  1. Mat Mannion Says:
    November 14th, 2006 at 16:57

    The GroupWise server has been in “On fire” mode for some time, unfortunately, it’s an unmaintainable piece of shit when it gets over 1TB of data, and I think Warwick’s is between 20 and 50 times that now, hence it being unrecoverable. It’s unfortunate that this has happened so close to the Exchange switchover, but still, since the generator was tested last week (obviously incorrectly) and the UPS’s did their job I don’t see how it could have really been avoided at the ITS end of things.

  2. Max Hammond Says:
    November 14th, 2006 at 17:44

    but still, since the generator was tested last week (obviously incorrectly) and the UPS’s did their job I don’t see how it could have really been avoided at the ITS end of things.

    Every incident is preventable, it just comes down to how much an organisation is willing to spend to mitigate any given threat.

    Evidently, Warwick’s power supply and UPS are not really up to task; the failure to provide these systems appropriately is one of management, not technology.

Leave a Reply