May 2005 Archives

Recall that mathserv2 crashed early Saturday morning before the Friday backups were complete; the new mathserv was brought up with backups from early Friday morning.

I've been able to mount the crashed file system and I've found no evidence of any damage to the mail, home-directory and web-site files created after the Friday am backups and before the Saturday am crash. These files are now available to you.

People have encountered a number of problems in the wake of the recovery from the weekend server crash; there are listed here in order of descending priority.

You will sometimes see references not just to mathserv but to mathserv1, mathserv2, mathserv3 and mathserv4. What does this mean?

Mathserv crashed at 4:07 am on Saturday May 21st. The new fail-over server is now running as mathserv and all services are up as of Monday afternoon. Some email and data files are missing temporarily; I expect to have any missing data restored before Thursday. We may have to schedule some off-hours downtime in the coming weeks to complete the installation of the new fail-over configuration.

Cisco has released a new version of their VPN Client; this version works with Tiger. UTS has not posted the version on their web site yet; email sysadmin@math.mcmaster.ca if you'd like a copy.

Update 2005/06/01: UTS has the new Tiger-friendly client available for download.

Mathserv crashed late at night on Saturday, May 21st, while I was away at UBC. The new fail-over server has been put into service in its place.

You may well a message like the following when you try to ssh to mathserv:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!

That is because the new mathserv has a new ssh host-identification key. You can remove the existing keys from your ssh known-hosts file on a unix system (including OS X) like so:

cp -p ~/.ssh/known_hosts{,.bak}
perl -i -ne "print unless (/^(mathserv|www\.math)/);" \ ~/.ssh/known_hosts

You can copy-and-paste the three lines above into a terminal window.

The will be a brief power interruption in Hamilton Hall on Thursday, May 12th at 6:30 am.

The msprime stations and other managed unix and Macintosh systems will be shut down automatically at 6:00 am. If you manage your own system - which is the case for all Windows PCs - I suggest that you power it off before you leave for the weekend.

The servers will not be affected by the power outage directly, though they will be inaccessible while the network has no power.