Recently in Performance Note Category

February 18, 2011

Mail Restrictions Off

Mail (via imap clients) is no longer restricted.

Mail Restricted for a Bit

In order to reduce the load on the storage server while to digests the replacement disk, I've turned of IMAP access to mail boxes.

You can read your mail via pine or http://mathmail.mcmaster.ca.

I'll be turning things on periodically to test the ability of the storage server to accept the mail load. Access from on-campus locations will be turned on before access from off campus.

Server & Workstations Still Sluggish Due to Storage Rebuild

The main server (ms) is still sluggish; consequently, the workstations and web sites are sluggish and mail delivery is slow. The storage array is still working to incorporate the replacement disk and so its performance is degraded - and that affects everything which stores data there.

There may be brief service interruptions (of up to five minutes) if I decide to shift load to another server.

February 17, 2011

Server Problems - sort of

The ms server and the workstations have been agonizingly slow (at best) since about 8:15 this morning. A disk on our main storage array failed and the array was hobbled (in "degraded mode", for those who follow these sorts of things). We do not yet know why performance was as miserable as it was - it should have been poor, not horrible.

There were three interruptions of five to ten minutes as I sought the cause of the problem - working on the invalid assumption that it was our server again.

The disk has been replaced and the storage array is rebuilding itself. Performance is going to be poor until the rebuild is complete.

Workstations may need to be rebooted if they have got confused over the state of the links to the home directories (though I have forced a refresh remotely on all systems which were responding).

February 15, 2011

Sluggishness Resolved

The sluggishness was resolved at 3:30 this afternoon.

Server/System Sluggishness Tuesday Afternoon

The server and most of the workstations (which access home directories on the server) are sluggish this afternoon due to an as-yet unidentified cause.

Some services (e.g. web, email) may go down for a few seconds at a time and workstations may freeze for up to half a minutes while I do some poking around.

November 29, 2010

VPN + pine = slow

We've had complaints of pine (and other command-line applications) being painfully slow in the past few weeks. It appears that this is almost always the result of logging in from off campus via VPN. Our own testing has shown that the slowness disappears when one shuts off VPN and connects directly to ms.mcmaster.ca.

We don't know yet whether this is a result of a change in the UTS VPN service or load or whether it might be due to an interaction between the new server and the UTS service ... though the former seems far more likely than the latter.

UTS is aware of the problem.

September 10, 2010

Server Problems & Service Interruptions

Our main server blew a disk this morning and is struggling while a spare is built into the main storage array. In order to allow the array to rebuild more quickly, I will be turning off mail services for up to half an hour at a time. Other services (workstations, Windows file sharing) may also be interrupted.

I will probably leave the interface at mail.math.mcmaster.ca up all the while, though.

August 25, 2009

Systems Slow Late Tuesday Evening

Mathserv and dependent workstations and servers are a little slow right now due to some work I'm doing on the main file system. Things should be limber again by 9:45 pm.

June 8, 2009

System Performance Spotty

The server and ms workstations are still sluggish at times as one server is still doing the bulk of the heavy service. This should be rectified by Tuesday.

April 24, 2009

Systems Sluggish Late Friday Afternoon

The ms workstations became very slow for ten minutes at around 4 pm Friday - that was me running some updates which hammered the server harder than expected.

March 31, 2009

Systems Sluggish Tuesday Morning

The server and workstations were a bit sluggish until ca. 9:45 am due to the nightly backups running later than usual (related to the power outage).

August 4, 2006

Mathserv Slowdown

Mathserv and the msprime workstations were running sluggishly for about half an hour this afternoon due to an I/O problem on freesurface.

February 28, 2006

Systems Slow Tuesday AM

Mathserv was bogged down for about two hours late Tuesday morning; email access and the msprime stations felt sluggish as a consequence. We are working to pinpoint the cause and reduce the likelihood of recurrence.

January 26, 2006

Campus Email Sluggish

The campus mail gateway appears to be bogged down. Mail which passes through that gateway is taking about an hour to be delivered - and pretty much any mail which is not sent internally to mathserv (i.e. from one mathserv user to another) goes through the gateway. Nothing that we can do but be patient, I'm afraid.

November 11, 2005

Mail Backup & System Lag III

We are no longer backing up mail hourly during the day time: as often as not, the backups would slow down the server and msprime workstations noticably.

Continue reading Mail Backup & System Lag III.

November 8, 2005

Server Down Briefly

Mathserv was down for about five minutes at 9:50 am Tuesday due to a (theoretically invisible) performance tweak gone wrong; dependent systems should not require rebooting.

October 27, 2005

Mail Backup & System Lag II

The problem of mathserv (and linux workstation) lag during the hourly mail backups has been reduced to the point that one should rarely even notice it.

October 26, 2005

Mail Backup & System Lag

You may notice that mathserv and the linux workstations feel a bit laggy for approximately ten minutes at the top of each hour. This is because we have begun mirroring mail folders to the fail-over server every hour instead of once a day. We will be working on reducing the impact of these backups on system performance. Web and mail-client users shouldn't feel much of a difference; pine and graphical file browsing will be most strongly affected.

Continue reading Mail Backup & System Lag.

April 27, 2005

Systems Suggish

Mathserv and the msprime felt sluggish much of this morning and yesterday afternoon, the reason being that mathserv was receiving a large amount of data from a computational job on bayes (the Stats server). The systems are loping along again as of ca. 12:30 pm today.

March 17, 2005

Server Crash - the Good News

While there is no sliver lining to the dark cloud of yesterday's half-day server outage, there is some good news to report. The first is that no data was lost: all incoming mail was queued for delivery and the data from the dead disk was reconstructed when the RAID array rebuilt itself. The second is that more than 90% of the network boot and standalone linux workstations survived the outage without a reboot - in theory, no program runs would have crashed or outputs written to /scratch been effected by the outage.

March 15, 2005

Backups Back to Normal

The restored back up server caught up over the weekend and is no longer copying files from mathserv (which resulted in mathserv feeling a little logy). Backups of mathserv and all other departmental servers and workstations are being completed by 6:00 am each day so the performance impact is minimal.

March 11, 2005

Server Load Spikes Resolved

While the new primary server - formerly the fail-over server - generally performs better the the former primary server did, there have been one- to two-minutes stretches a few times a day when the server was nearly brought to its knees: msprime workstations would start stuttering and web and email responses would be very slow. The source of the problem - a bad table in a research database - was fixed on Tuesday and no more load spikes have been observed.
Continue reading Server Load Spikes Resolved.

Systems Logy from Backups

Our backup server is back online after having disks replaced. Because the primary file system was lost, we are having to prime the backups from mathserv: the I/O activity is keeping mathserv's disks and the HH network rather busy, so the msprime stations will feel a bit sluggish at times (e.g. two to three second delays when opening an application or window), probably until some time Saturday.

About this Archive

This page is an archive of recent entries in the Performance Note category.

OS X is the previous category.

Printers is the next category.

Find recent content on the main index or look in the archives to find all content.