Recently in Campus Systems Category

UTS reports that univmail was unavailable for receiving or reading mail for several hours this weekend.   Details follow.

Our primary Internet link remains down as of 10:30 pm. Rogers (formerly Atria Networks) are actively investigating the problem and provided an update at about 10:00 pm (they provide the network path between the University and the Cogent POP in Hamilton). The problem appears to lie between the University and the John St. POP downtown Hamilton. I reminded them that the Downtown Center network fibre also terminates at the John St. POP in case the two problems are related. They don't appear to be but it is unusual to be having two problems at one time.

As I mentioned our backup Internet provider is only 100 Mbps and has been maxed out most of the day. Internet sites that don't route through ORION will be slower than usual and may timeout periodically.

We've contacted UTS and confirmed that the McMaster VPN service is down as of some time on Thursday.

This morning's campus internet issues are resolved.

UTS tells us that our connection to the internet at large via Orion is iffy right now. We'll just have to wait it out.

We've experience more than our share of computer woes this week: in the department, across campus, and beyond. Most things have settled down as of late Thursday evening.

In Math & Stats

The HH-303 printer has gone from bad to worse: I disabled printing on Tuesday while I get HP to deal with this properly. Scanning still works just fine. The HH-403 printer is the main alternative.

Campus-wide and Beyond

Tuesday saw problems with some UTS systems such as univmail, resulting (as I understand it) from the strain of catching up after the Internet outage.

Some email messages delivered following the end of yesterday's network interruption will have confusing timestamps: the messages are stamped with the date and time of their arrival on our servers, not the date and time at which they were sent. And even though our network connection has been up since 7:30 a.m., some messages will arrive up to several hours later as the upstream servers only attempt delivery periodically.

McMaster's internet connection was down from mid-Monday afternoon until 7:30 this morning. Apparently a fibre-optic cable belonging to out upstream provider was cut.

Email sent to your @math.mcmaster.ca or @mcmaster.ca addresses during the downtime will either have made it through when the network came up or will have been bounced back to the sender with a warning message.

We had spotty/wacky internet access between campus and the rest of the world from about 3:30 pm to 9:30 pm today - spotty in that sometimes one could get out from on campus, wacky in that some web sites and servers were accessible from off-campus the whole time ... unfortunately, www.math.mcmaster.ca and mathmail.mcmaster.ca were not among the latter*.

* Interestingly, one could ssh to mathserv2 the whole while, and from there to ms in order to ready mail via pine.

UTS says ...

Our primary ISP Cogent is down, are aware of the problem and are working on it (no estimate yet). There is some traffic through our backup supply, but that has only ~20% of primary capacity.

UTS has confirmed that McMaster's link to the Internet is down. Do not panic. I repeat, DO NOT PANIC. Grab a coffee; read a book; call a friend. IT WILL BE OK.

UTS announced that mail sent to @mcmaster.ca addresses yesterday morning between 8:00 and 10:00 may not have been delivered. Mail sent to directly to @math.mcmaster.ca addresses were not affected. Mail redirected to @math.mcmaster.ca from @mcmaster.ca may well have been affected.

Here is the original UTS announcement ...

University Technology Services has confirmed that MacConnect wireless is having troubles - the main symptom is a general slowness.

The connection between the campus network and the rest of the internet is slow. UTS tells me that our up-stream provider is working on it. Enjoy that mid-90s feel.

UTS has announced that access to the internet from campus and vice versa will be spotty on Friday, June 26th while Orion does some maintenance.

The main McMaster web site (and some related sites) is down. Nothing to do with the departmental systems, but you might want to know that it's not just you.

UTS advises that "Cogent, our primary Commercial Internet provider is experiencing routing issues for customers in the Toronto area. Sites normally reached via ORION (such as other universities, google.com, yahoo.ca, cnn.com, etc) are still reachable."

UTS has announced that they will be doing network upgrades to Hamilton Hall between 8 am and noon this Sunday; assume that internal and external network access and workstations will be down for this period.

The power outage scheduled for Friday evening will now take place between 3 pm and 5 pm on Saturday afternoon. Mail and web services will be accessible from off campus during the outage. Compute servers and workstations will be shut down at 2:30 pm.

Workstations should restart automatically.

I'm told that we will lose power in Hamilton Hall and BSB on Friday evening between 8 pm and 10 pm, with a chance that the outage will last until some time on Saturday.

I will be shutting off the compute servers and workstations this evening at 7:45 pm. If the outage extends to Saturday, I will have to shut off the web/email/file servers as well. Full service may not resume before Monday morning in that case.

The internet connection between McMaster and the rest of the world will be down from 7 am to 9 am this Sunday morning. In-bound mail will be queued for delivery once the connection is resumed.

The power is back on in Hamilton Hall but the server and systems are still off-line while I complete some opportunistic maintenance and upgrade work.

The power will be out in Hamilton Hall until 5 pm today. I am going to be taking everything but the web server off line shortly.

The power outage persists in Hamilton Hall and BSB and is likely to do so until 12:30 or 1:00 pm. I will leave mathserv running on battery backup as long as possible, but I may have to shut it down before the power returns.

There was a brief, unscheduled power interruption at ca. 8:20 this morning. Mathserv and other servers on UPS weren't effected, but most workstations and some other servers went down and will need to be restarted.

Network traffic was interrupted for a few minutes just after 4 pm today due to the crash of a central campus network switch. The most obvious effect was that the linux workstations seized up and traffic in general (email, web, ssh) went sluggish.

UTS has announced that MacConnect (wired and wireless) will be down Tuesday evening from 5:00 pm to 6:30 pm for upgrades.

The network name servers were down intermittently for a few minutes at a time early this afternoon, causing web, email and system problems.

The network problem at noon time lasted about ten minutes but appear to be over.

It appears that we are having some intermittent problems connecting to off-campus network addresses - the problem is broader than just Hamilton Hall, so there's nothing we can do but wait it out.

Workstations and servers will be down or unavailable to some degree four times in the next week:

• Thursday from 5:00 am to 8:30 am due to a power shutdown by Facility Services
  
• Friday from 3:00 pm to 5:00 pm while I do fail-over testing
  
• Sunday morning from 7:00 am to 8:00 am due to network work by UTS
  
• Sunday evening / Monday morning from 6:00 pm to 8:30 am due to a power shutdown by Facility Services

UTS and FS, like RHPCS, have scheduled disruptive work for the end of the exam period, it appears.

UTS has announced that network access to and from campus will be down for ca. ten minutes on Sunday, May 4th from 7:00 am to 8:00 am. MacConnect wireless will be down from 8:00 am to 11:00 am.

UTS informs us that the network connection between McMaster and the rest of the Internet will be down this Sunday between 7am and 11am.

UTS reports that we are experiencing problems connecting to parts of the outside Internet; the source of the problems are outside the McMaster network and ORION staff are working on things.

UTS tells me that the battery backup unit for the network link to Hamilton Hall failed again on Saturday and so the website and email were inaccessible between 2:30 am and 7:30 am. The unit is to be serviced on Monday.

UTS has announced that the campus connection to the Internet will be down from midnight on Friday the 17th to 6am on Saturday the 18th due to a network upgrade by ORION.

The network connection between Hamilton Hall and the rest of campus was down for part of Saturday morning - email and web sites were inaccessible for that period. UTS got things working again very quickly once the problem was reported.

The problems with DNS names which began Wednesday evening are continuing due to a problem at the CIRA end. So there may be some problems with connections to mathserv or email bouncing today.
I'm told that the fix should have worked its way through the Internet name servers later on today.

There were DNS problems on Wednesday evening and Thursday morning which caused problems with web, mail and ssh connections:

Date: Thu, 11 Oct 2007 09:24:30 -0400
From: Colin Wu
Subject: Name resolutioni from off-campus

While attempting to configure an external secondary name server for the mcmaster.ca domain a misconfiguration resulted in some off-campus sites not being able to resolve names correctly. I have corrected the problem but due to caching it may take up to an hour for the correct information to propagate.

We will lose access to the Internet for ca. two hours on Wednesday morning, some time between 9 am and 1 pm.

See the announcement from UTS for details (said details include nothing about the outage on Monday, oddly).

The good news is that the broken connection between Hamilton and Oakville has been repaired. The bad news is that subsequent "underground explosion in Stoney Creek" leaves completely disconnected, nevertheless

We have no access to the off-campus Internet due to "a fibre break between Oakville and Hamilton"; UTS has posted details on the daily news site.

You should have no problems with on-campus web sites, email, printers, etc.

UTS has announced that univmail and the main campus web site will be unavailable for most of Saturday, June 30th due to a planned power outage.

The usenet news server run by UTS crashed over the holiday and apparently hasn't come up yet; I'm told that usenet news will be unavailable until further notice.