The Grumpy IT Admin

VMware used to be very good at releasing stable mature and sensible upgrades, though of late that seems to have gone a bit wonky. The latest vSphere release 5/1 is a bit of a headache to install due to the inclusion of RSAs SSO module. RSA is another part of EMC, the ultimate owner of VMware. Now the cynical may see this as a way of getting RSA product in to companies. if it is the case then it will back fire spectacularly as RSA products have generally been a bit clunky, unintuitive to use and difficult/pernickety to install and configure. I remember needing to the time on our RSA server to not use Daylight Saving Time as the RSA product couldn't cope with it. Unfortunately the SSO module in the current 5.1 install of Virtual Centre seems to have its own problems. Despite its best efforts it fails miserably to find AD, it suggest that maybe AD is not installed, running or some other problem with the environment, clearly indicating the short fall in performance is nothing to do...

I finally got around to to moving over our bigger VMs to our larger BL 690 blades after I had completed upgrading the RAM on them to get them up to 160GB each to give us some decent space for running some large SQL boxes with 32GB of RAM specified. The kicker for this was that one Sunday morning around 3am we lost power to teh server room. Now we have two UPS, one for site and one for the server room. The server room one is about 4 years old and had been serviced about two weeks previously, to give you some perspective this is a steel box the size of half a shipping container that you see on the decks of those big ships. However, at the appointed time everything in the control panel and all the capacitors went bang, we know not why and our other UPS was all happy to carry on. But basically we had a power blip that caused all the storage and all the host to crash. As all our server are set to remain off in this situation to avoid multiple crashes and allow us to bring everything up...

Everyone hates it but it is a necessary evil that we have to go through at least once a year, some twice others, god help their minds, several times. In previous years we have had a truck turn up with a roof that went up to create a two story office, a bit like a low rent Formala 1 paddock truck with a bad paint job. You had the office upstairs and a bunch of racks with a HP EVA SAN and a shed load of servers of your choice and some tape libraries. We would get our backup tapes and recover as many systems as possible pretending we were doing a good job and if it came to it we could recover all our systems within a week, whilst knowing full well we wouldn't have a hope in hell. Now my life is made a lot simpler. We have visualized as many of our systems as possible, these are all stored on some cheap EqualLogics (compared to EVA and all the licenses) and replicated to a remote site. I now sit at my desk and bring online replicated copies of our VMFS partitions, present to our ho...

Did you know that if you snapshot a server in VMware the snap files are kept with the server? So what you say. Well if you have some tiered storage and your nice super-duper SQL box has system drive on some slow SATA storage and your Logs and Data are nice fast SAS or FC storage when you snap all the writes will go to the slow storage and your SQL box will run like a complete dog, not a greyhound but three legged pug with a bad hip. You can if you so wish specify where VMware stores it’s snap files but you will always see performance problems as it switches between locations to provide the latest data. VMware snapshots should be seen as a very short term thing, i.e. snapshot, do update. does it look ok, remove snapshot or revert, anything more than a couple of hours and you are starting to go in to dodgy performance territory particularly on SQL boxes and I would guess nything else running a database like Exchange or Oracle. One useful tool to have in your armoury is RVTools w...

I seem to have a mental aberration every month or so, something to do with full moons I think, when I do something rather rash, often preceded by doing too many things at once and time constraints. During the last one I decided to set the wrong drive offline. This caused a couple of servers to disappear, no VMware is really quite resilient but losing a drive seemed to cause it a few problems. I quickly realised my error and set the drive online again, however, the two ESXi host servers that had running guest servers on the offline volume had locked up and were unresponsive, though other guest servers on the two host servers will still up and running though the VC was showing the ESXi servers as disconnected and the other servers as unavailable. I started shutting down the servers using RDP to connect with the intention of forcing a reboot of the ESXi servers. I cleared one off and restarted it in the meantime the second one had managed to recover its composure. All the servers wer...