Today’s lesson is that when you are using bootable media in the drive of a host, and attaching it to one of the guest VMs, remember to remove the media (or change the boot settings) when you remote reboot the host after updates… 🙂  This led to thinking about disaster recovery and business continuity.

We make use of a lot of virtual machines in our test environment in order to develop new products and test things out for clients.

The World’s Most Powerful Mini Linux Workstation?

We also find bootable “live OSs” like Puppy Linux to be a great way to quickly boot machines and get to files, or manage drives and suchlike…

My nonchalant remote reboot of the development host server this morning from my PC reminded me of the importance of making sure your systems are not just correctly configured, but tested, and kept the way they are meant to be (like we do with the live environment, which came up just fine, thank you very much! :).

So instead of ending up with a Hyper-V host with 5 test VMs happily running on it, I wound up with what may be the world’s most powerful workstation for the PuppyLinux cut-down Linux distribution (fits on a 256MB SD card – on a machine with Xeon CPUs, many GB RAM and RAID array) patiently waiting for someone to plug a mouse in…

As I was in the office today, once I’d figured out what had happened, it was a simple matter of removing the PuppyLinux CD and rebooting the server.

I think I might also tweak the PowerShell management scripts to warn me if there is a CD/DVD in the server before it reboots. Hmm.

So Why Am I Telling You This?

So, I was stimulated into posting a blog entry now, because with Christmas coming up, this might be an ideal time for your business to do a “DR Test” – Disaster Recovery, or Business Continuity if you fancy a more positive spin on the unexpected. Obviously I’m making this an assumption that this is a quiet time for you – retailers, please look away now…

If you have a formal DR or Business Continuity plan, with tests, why not schedule one soon, when you have perhaps more chance to deal with any unexpected fallout?

If you don’t have a formal plan, you could use the Christmas period to draw up a checklist of what you’d do if you came in one day to find a server wouldn’t boot up, or a critical PC you use if you are a smaller business.

What would you do?

How long would you spend on the technical problem before working out what the impact might be, how you might keep your business going, and contacting customers to manage expectations?

How much would you tell customers?

When it comes to doing real tests, unless you’ve thought about it beforehand, there is an element of risk in deciding to have your own test disaster. You may just find out what you are missing when you do it for real. So whilst I wouldn’t advocate just switching off the power to a server to start a test recovery scenario, you could think about what you’d do to get by if it did happen.

Make a list of actions and things you’d need.

Do you have them all? Could you quickly get them?

Do you know how to use them? Does everyone else in the business?

Could you use “cloud-based services” to help you?

Yes, using “the cloud” can entail very grand “solutions”, but it can also be as simple as making sure you have a webmail account with your internet service provider so that you can still send and receive emails to/from your customers and suppliers without resorting to personal accounts like fluffy_bunny42@hotmail.com (apologies if this made-up email address is yours… :), which don’t give the best impression of your business.

Think also about how you would inform your staff of the situation if our notional disaster was happening. How would you speak to them, what would they need to know? To do?

The Bottom Line?

There’s a huge amount you could consider, document in your processes, and practise with your people when it comes to preparing for the unexpected. You can make business continuity/DR exercises as simple or complex as you like, but I’d say it’s definitely worth an hour or two of quality thinking time to get started.

Maybe over the next few weeks?