Maybe you are lamenting the fact that ESXi currently does not support USB. Maybe you tell it to someone or adjust your thinking because of it a few times a week. A great many people are making a great deal of noise about it, and since the competition is so fierce in the virtualization market right now, I suspect it will be a non-issue very soon. I dearly hope so anyway. However, for the time being we keep running into potential deal-busters because of this very irritating issue.

We’ve heard about the Digi AnywhereUSB device – but hey man, that thing’s expensive, and I shouldn’t have to buy one or force my customer to buy one, should I?

I thought these devices were $350. Today I learned that their LIST price may be in that area, but their retail channel pricing is much lower. These devices can be had for as little as $250. External USB fax modems are cheap. In the same vein, older battery backup devices or things that use serial ports can be connected to over the network with a device that costs $110.

You can find them here:
Digi AnywhereUSB
Moxa NPort 5110 (multi-port models are also available)

Sure, this is inconvenient as hell, but let’s do a reality check. In order to connect a SCSI tape drive to a server, you need a SCSI controller card. We regularly sell SCSI cards to customers for $250 and nobody blinks. It’s just a device you need to make things work. Period. Nobody whines about it, because it is just the way it goes.

So I am here to say that we should not be so high-strung about the ESXi USB issue. We should instead treat it as one of those little necessities. Instead of asking in a hesitant, fearful tone whether there will be any local USB or serial devices involved, we should just mention that if they will need to do this, they will need another device to make it work. If the topic seems to cause consternation, shrug it off and minimize it. We should say “Yeah, but you can pick one of those up only $250.” like it is nothing – because, really, it isn’t a big deal.

Let’s put things into perspective. Many IT professionals charge their customers anywhere in the range of $100 to $150 per hour for labor. Even if the customer is unwilling to spend that money on hardware (even though they spend much more on labor), the IT professional could easily eat the cost of buying the device, write it off as an expense, and bill their customer for the installation and configuration of the device. Or they could just let it be absorbed in the bigger picture as time progresses. A good customer will accept that it is necessary, and buy it. A good IT professional will explain it to a customer in a way that will facilitate this, or just provide the device and be sure to bill enough to cover the expense. An IT “professional” who is not very good at what they do will make a big deal out of it and raise the customer’s fears and resistance to virtualization. Let’s set an example for them.

When someone wrings his/her hands and makes strangling noises about a little $110 device or even a $250 device to go along with a new server and a migration project, that person is being short-sighted, overly dramatic, or is looking for objections to Virtualization because it is outside their comfort zone – or all of the above. Those people need to shut up and get on with it. And stop whining.

Tags: anywhereusb, digi, ESXi, USB

The StorCenter ix2 is a very versatile device. It is quick and easy to set up. You can access data via SMB, FTP, NFS, and HTTPS.

You can copy files to it via Windows, then attach it as a datastore to an ESXi server to use those files. Of course, it makes a great backup device. I got nearly 700MB/min over a gigabit network.

However, there are limits to the NFS protocol, and more specifically ESXi’s implementation of it, that severely limit its usefulness for backups as a directly-attached hard disk in a VM.

For one thing, it’s terribly slow. I got less than 100MB/min over the same network, from the same VM, using the same backup software. The only difference was that one test was performed using WBAdmin to do the backup to a UNC path to the StorCenter, and the other was done with the NFS share mounted as a datastore and a VMDK allocated to the VM as a hard disk for Windows Server Backup to use as its backup device.

Another limitation is that you cannot change out the StorCenter for another one easily when it is mounted via NFS. In order to change out your backup device, you’d have to shut down the VM, remove the datastore, change out the StorCenter, mount the new one as a datastore, add a new VMDK to the VM, and boot Windows again. At least you don’t have to reboot ESXi, and you could do it remotely, but it still stinks.

If you need an NFS server in a pinch, or want to back up to a network device using a UNC path with a wbadmin command (perhaps scripted and scheduled), the StorCenter is great.  However, you lose the ability to do a full/incremental schedule with the built-in backup software in Windows 2008. If you have third-party software that can do UNC paths, it’s all good.

If you want to use Windows Server Backup, and you have something else available for offsite backup, then I recommend an eSATA disk for Windows 2008/SBS2008 backups, NOT a StorCenter.

Tags: backup, Iomega, NAS, StorCenter

You’re familiar, of course with Windows software mirroring: A poor man’s RAID.

In the traditional hardware world, you added two disks, loaded Windows on one of them, added the second one as a mirror set, optionally added a line in the boot.ini with the second ARC path, and you were off to the races.

ESXi has a fairly specific list of RAID controllers that it is fully compatible with, and sometimes either you don’t have one to work with, or the one you have doesn’t work right. In such cases, if you want fault tolerance, you have to get creative about.

This, then is a creative plan:

1. Establishing Software Mirroring in ESXi (or Hyper-V, for that matter)
2. Add one physical disk to your server.
3. Install ESXi on the disk.
4. Disconnect the disk.
5. Connect a second disk.
6. Install ESXi on the second disk.
7. Connect both disks and boot. Verify that ESXi can see both datastores.
8. Create your VM, and add two virtual hard disks of identical size to it. One virtual disk should be placed in each datastore.
9. Load Windows, and add the second virtual disk as a mirror set.

It’s quite simple, really, and it’s the same concept – the only differences are that you have to load your base OS (ESXi) twice, and that if you have a failure on your primary boot disk, you’re going to have to create your VM (using the existing virtual hard disk) on the second OS on order to boot it. That’s not such a big deal, really. A bit of a manual process, but not bad.

Fair warning: Switching back and forth between ESXi instances in this configuration had some interesting and undesirable results in the area of snapshots, and virtual disk files, so your best bet is to do this only when you have to becauseof disk failure. If you have an active snapshot when your primary disk fails, you’re going to forcibly be reverted to the snapshot. It could get very interesting. I recommend very short-lived snapshots in this scenario.

Come to think of it, I don’t really recommend this scenario – but if you’re broke/desperate, or don’t have much to lose…

Tags: ESXi, Hyper-V, RAID, Software Mirroring

A customer was kind enough to let me play mad scientist with one of his servers today, and I learned the following:

The HP Proliant DL320 G3 is (mostly) compatible with ESXi.

What I mean by “mostly” is that while ESXi will recognize and use the hard disks, it doesn’t really like the ICH6R RAID controller.

• When you disable the RAID controller in the BIOS, and put two disks in the server, ESXi will see two disks.
• When you enable the RAID controller, and create a RAID1 mirror from two disks, ESXi will see two disks.
• If you load ESXi on one of the two disks it see when when they are mirrored, it will fail to boot.

So, your best bet is to turn off RAID, and use software mirroring if you want fault tolerance.

ESXi will detect and use the network interfaces (HP NC7782).

This server is 32-bit so your OS options are limited.

I also discovered by experimentation that this server will recognize a 1TB SATA disk, so that’s useful.

Tags: compatibility, ESXi, Proliant DL320 G3

Today, after finishing up a test migration from SBS 2003 to SBS 2008, I deleted the SBS 2008 VM from the ESXi server to free up space for a real migration – but first, I used the built-in Windows Backup to make a one-time backup to a shared folder on my workstation via UNC path.

Then I got to thinking: This is as good a time as any to check out restoring SBS 2008, so let’s give it a go.

There are plenty of walkthroughs with screen shots on the web, so I won’t bother with that, I just want to comment on the process and give a brief overview for those who don’t need the screenshots and would rather just go through it with a few pointers.

First, I created a new VM with the same virtual disks, processors, and memory as the original. I suppose these could have been different sizes, as long as they were enough to cover it – but I didn’t test that.

Then, I attached the SBS 2008 DVD ISO image to the VM, set it to boot into the BIOS on first boot, and fired it up. Why did I set it to boot into the BIOS? Well, for two reasons actually. First, the time and date on the freshly created VM is often incorrect. In this case, the time was 1:30pm on the 24th. When I fired up the VM, it thought the time was 9:30pm. 8 hours wrong. SBS gets annoyed about things like that. The second reason was so that I could adjust the boot sequence.

Everything properly adjusted, I rebooted the VM and it started the SBS setup sequence.

At the proper junction, I told it to run a repair. I further told it to restore from backup. Here is where it got a little strange. It wanted me to select an operating system to repair, but there was none. I clicked the Next button and was rewarded with the option to do a Complete Windows PC Restore. Clicked that, and it looked around for a local backup device, didn’t find one. the available buttons were Retry, Cancel. At this point I started to despair, thinking MS had made an assumption about the backup device being local. Still, I clicked the Retry button first (more of the same), then the Cancel button on the third go-round.

The Cancel button turned out to be the right answer, because then there was an option for “Restore from a different backup”. Aha. Now we’re getting somewhere. Clicked the Next button. Was presented with a listof available backups. An empty list.

Man, if I had a lot of customer data and billable time invested in this, I would be on a serious emotional rollercoaster by now. MS could have presented this better. Still, there was a ray of hope: An Advanced button. I clicked it.

In the Advanced screen, there was an option to “Search for a backup on the network”. Bingo. Clicked that, confirmed the security warning, and Windows fired up the network stack.

Windows then prompted me for the location of the backup. I expected some trouble here, because the storage location was a UNC path to a workstation in a different subnet, on an XP machine attached to a non-trusting domain. However, my pessimism was unwarranted. MS got this one right. I was able to specify the UNC path to the machine (I used the IP address because I had no name resolution mechanism in place). It popped up an authentication dialog box and I was able to supply domain credentials, but it failed to connect to the workstation. I tried this a couple of times, just in case I had typed something wrong, but then I realized that I had fired up the VM in a network that had no DHCP server online. Well, that won’t work out really well, will it? I edited the VM settings and swapped it over to the network subnet that did have a DHCP server running, rebooted, and tried again. It worked much better that time. How about that? Anyway, once I got over that little hump, I was able to specify the full path to the storage location, and it found the backup.

From there it was pretty uneventful, which in itself is saying something positive for the process. I went and did some other things while waiting for the restore process to finish. Came back after a while and it was done. Rebooted the machine, and I had a fully functional SBS 2008, complete with users, data, configurations, etc. All was good. Didn’t have to re-activate, didn’t have to reboot to finish installing hardware.

Overall, not a bad process. Way better than loading a complete OS, booting into Directory Services Restore Mode, and restoring the backup right over the top of the freshly loaded OS that took so long to build.

Maybe there’s something to this SBS 2008 and Windows Backup thing.

Update: I ran this again with smaller disks (still larger than the 60gb requirement for SBS2008, but smaller than the original disks by 5gb). Way above the actual data size. It failed. Apparently it can’t resize on the fly, like Symantec Ghost has been able to do since the turn of the century. Sigh. Well, you can’t have everything. I guess this is likely to be a side effect of block level backups. Still, Storagecraft ShadowProtect can do this as of version 3.3, and they’re using the same VSS and block-level backup technology. Their restore operation is way different though.

Tags: backup, ESXi, SBS 2008