The Grumpy IT Admin

After spending some time wading through a lot of stats and figures and checking the configuration on all the switches we have partly resolved the problem at our most badly affected site. This always seemed to have extra issues as the problems seemed sporadic and didn't follow the same pattern we saw at other sites. We traced to the problem to two ports that were connected in to the second newer EqualLogic unit. These were showing a number of errors on the switch end. Shutting down these two ports stopped all the latency emails we were getting and now we are just getting the normal pattern were a number of servers all email about latency at the same time rather than random emails from different servers. I have yet to visit the site and ascertain whether it is a problem with the switch, controller or the patch lead. But it just goes to show that if you start at one end and work your way through the path, make sure every setting is what it should be and above all never assume any...

We have had an ongoing issue with our EqualLogics systems where they have a massive disk latency problem. We have narrowed this down to 11 second periods where there is no network activity on the controllers. We have passed all the information to Dell but are still without a fix. This problem has been with us from the start across all the firmware revisions. The most affected system is the system that receives the most replication traffic, had has the fewest live servers on it. The only thing we have got back so far is that there is something going on with replication even though the periods we are seeing this are not confined to when replication is happening. Dell obviously want us to upgrade to the latest firmware but this is not a good idea at the moment as a replication bug has been found that appears to throttle replication to around 25MB/s which is not desirable. Dell are not acknowledging this but our suppliers are definitely seeing it and have advised us to not update to ...

Here's some more weirdness related to Exchange. Basically we saw a lot of slowness, not all the time and not on all sites. This slowness would manifest itself in response times from the CAS/HUB server that sit behind a loadbalancer which comes from Kemp. The CAS/HUB servers use the loadbalancer as the default gateway, now for all our traffic this worked extremely well, the only issue we got was this slow down. Even connecting to the server via RDP would be very slow and unresponsive, the cure was to vMotion the loadbalancer. More of a workaround than a fix but it helped. After much troubleshooting and packet sniffing and a conversation with Kemp it was ascertained that if we change the NIC type from a VMXNET3 to a E1000 all our problems would disappear. Surprisingly this actually worked. Now the CAS/HUB servers are much more responsive and don't fall in to an apathetic well of despair. Would I build an Exchange system that was fronted by a loadbalaner again? Probably n...