The Grumpy IT Admin

Well no upgrades happening this weekend due to compatibilty issues and trying to schedule down time for servers. Due to a lack of foresight by the powers that be, we do not have a scheduled period where we can take servers down for maintenance, to my mind this needs to be set in stone in the following way "between x time and y time on this day you need to expect your server to be restarted", this gives us a windows when we don't need to worry about whether data is be imported or worked on and there is no other real activity of note being carried out. Just because it is there doesn't mean we will use it, but it is nice to have the opportunity without jumping through hoops trying to get permission from a bunch of system owners to update drivers and reboot a server. This time it was both VMware and EqualLogics updates that needed to reboot servers. Some of our servers have direct attached iSCSI drives on the EqualLogic, these run the HIT kit and since version 5.0.2...

Hopefully if plans go well I will be doing a bunch of upgrades this weekend. Firstly I will be doing the EqualLogic firmware, which, due to some rather buttock clenching moments, I have got different versions on a couple of our systems so I need to get them all in line. I will probably go for 5.0.5 rather than the latest 5.0.7 as it has been out for a while and seems to be relatively safe and as I get older safest seems more bearable than latest. This also means I need to get all the iSCSI connected servers on to the latest version of the HIT kit which is always a worry as most of these are physical servers so you can't just snapshot and then roll back if there is a problem, I will probably pull one of the mirrored system drives before updating to give a quick recovery if it all goes horribly wrong. The other is an update to do is VMware up to 4.1, this is a straight forward thing for the most part, however it is made more problematic by the need to update the VMware tools on ...

There is nothing like doing patching or firmware upgrades to really get those buttocks clenching. After so many that have gone wrong it has become one of those jobs I'd rather go on holiday when they are due and leave to someone else who is younger and more care free and has yet to be scarred. The latest round of EqualLogic patching at one of our sites went a bit tom tit. Upgrading from 4.3.7 to 5.0.4 should have been a fairly simple thing, though previously I did experience a small problem with one of the controllers restarting about 15 minutes after the upgrade which was a bit weird I wasn't expecting any issues. However this went completely bonkers after the upgrade. Lots of errors which were triggered by cloning a replicated drive. Replicating would start to fail, drives would disappear and then reappear, the only solution that Dell could come up with was to move to version 5.0.5. as 5.0.4. This was weird as I had been told previously that I should move to 5.0.4 as it w...

Another one of those weird things that happen to us admins. This time it was our HP C7000 chassis. On and off we have been getting some errors where the Chassis reports some problem then 1 millisecond later all is well again, also we have had VMware reporting loss of redundancy on the iSCSI connections and packet floods logged on the chassis. Whilst I was off enjoy the sunshine this went completely ballistic to the extent I was getting emails at the rate of 1 every 1 to 10 minutes. I returned to work to find 4000+ emails, not a bad score for 4 days off. A quick Google and the best advice I could find mentioned reboots can make it go away for month but best to do a firmware update, though these mentioned old versions we were on 3.11 and the latest was 3.30 so maybe worth a shot. Given my experiences with EqualLogics (did I mention that another site upgraded their EQ to 5.0.4 and lost all their replication, barely managed to make it through the day with drive intact and had to upda...