Data Protection & Disaster Recovery in Windows 8 Server Hyper-V 3.0

The news coming in from the Build Windows conference is awesome. The speculation of the last months is being validated by what is being told and on top of that more goodness is thrown at us Hyper-V techies.

On the data protection and disaster recovery front some great new weapons are at our disposal. Let’s take a look at some of them.

Live Migration & Storage Live Migration.

Among the goodies are the improvements in Live Migration and the introduction of Storage Live Migration.  Hyper-V 3.0 supports multiple concurrent Live Migrations now, which combined with adequate bandwidth will provide for fast evacuation of problematic hosts. Storage Live Migration means you can move a VM (configuration, VHD & snapshots) to different storage while the guest remains on line so the users are not hindered by this. I’m trying to find out if they will support multiple networks / NICs  with this.

Now to make this shine even more MSFT has another ace up it’s sleeve. You can do Live Migration and Storage Live Migration without the requirement of shared storage on the backend. This combination is a big one. This is means “shared nothing” high availability. Even now when prices for entry level shard storage has plummeted we see SMB being weary of SAN technology. It’s foreign to them and the fact they haven’t yet gained any confidence with the technology makes them hesitant. Also the real or perceived complexity might hold ‘m back. For that segment of the market it is now possible to have high availability anyway with the combo Live Migration / Storage Migration.  Add to this that Hyper-V now supports running virtual machines on a file share and you can see the possibilities of NAS appliances in this space of the market for achieving some very nice solutions.

Replication to complete the picture

To top this of you have replication built in, meaning we have the possibility to provide reasonably fast disaster recovery. It might not be real time data center fail over but a lot of clients don’t need that. However, they do need easy recoverability and here it is. To give you even more options, especially  if you only have one location, you can replicate to the cloud.

So now I start dreaming Smile We have shared nothing Live & Storage Live  Migration, we have replication. What could achieve with this? Do synchronous replication locally over a 10Gbps for example and use that to build something like continuous availability. There we go, we already have requirements for “Windows 8 Server R2”!

NIC Teaming in the OS

No more worries about third party NIC teaming woes. It has arrived in the OS (finally!) and it will support load balancing & failover. I welcome this, again it makes this a lot more feasible for the SMB shops.

IP Virtualization / Address Mobility

Another thing that will aid with any kind of of site  disaster recovery / high availability is IP address Mobility. You have an IP for the hosting of the VM and one for internal use by VM. That means you can migrate to other environments (cloud, remote site) with other addresses as the VM can change the hosted IP address, while the internal IP address remains the same.  Just imagine the flexibility this gives us during maintenance, recovery, trouble shooting network infrastructure issues and all this without impacting the users who depend on the VM to get their job done.

Conclusion

Everything we described is out of the box with Windows 8 Server Hyper-V. To a lot of business this can  mean a  huge improvement in their current  availability and disaster recovery situation. More than ever there is now no more reason for any company to go down or even out of business due to catastrophic data loss as all this technology is available on site, in hybrid scenarios and in the cloud with the providers.

Build Windows Key Note 2011/09/13

Updated as we follow the key note

After the talk about Windows 8 being even better and greater for all form factors (hardware people, the ARM architecture, it will be fun to see how the competition responds) I want to dive into Windows Server 8. Yes I’m here for the server side. But as the Hyper-V is now brought to the client there is a lot to say about Hyper-V here as well. No problem. But not yet, not yet.

First, mobile devices. Lots of touch, looks all very cool with the Metro UI. As I live in the country with the most expensive smart phones & mobile data subscriptions in the world I’m not a heavy user. It’s a great market, it’s cool, it’s important, but it’s not my primary theater of operation so to speak. But I might need to get me some of those devices to play with Smile It really looks cool. It looks all very fast & fluid. And the resource hogging should be reduced. Bring it on I say Smile But don’t worry if you’re a “Grand Pa Box” keyboard & mouse jockey. Windows 8 works just as well for you. the idea is Windows 8 everywhere on every device & form factor.

Now they first need to talk about all the developers will be writing applications for Windows 8. Here comes Metro Style applications development. The bold WinRT API bet (yet another one). The languages used are the one we all know, love or hate Smile. No worries you’re coding skills have not been dumped into the toilet. Oh yes, Silverlight is not dead. An no .NET is not dead either. Really? Even COM+ is not dead yet. But Metro style development is the way ahead. But please dump the hyped drama and o continue coding on your current projects Winking smile They promised everything that runs on Windows 7 today will run on Windows 8. There you go Smile with tongue out You might say with less drama that Win Forms & co will be less dominant. Nothing that new. New form factors & mobility ask for new tools. But guess what you’ll be coding those apps in? Metro Style apps will be written in C, C++, C#, Visual Basic, HTML5/JavaScript and/or using XAML. XAML is for “Jupiter,”which is the XAML/UI layer on top of Windows 8 needed for Silverlight and Windows Presentation Foundation (WPF) apps to work on the platform.

They are now coding on stage. Perhaps not the best use of time during a keynote but hey, we’ll get to the good stuff eventually. Once again we see the impression launched you can write apps in a couple of minutes with no knowledge at all. Take that devies! We IT Pro’s are not the only ones facing unemployment (cloud) Open-mouthed smile we’re all going to be replaced by a very small easy script with drag & drop. I know some hard core consultants/developers who are now buying stocks in their own company to cash in on the fixing of all that Smile

We’re treated to some very impressive hardware demos. Really impressive. Mobile device OS people we have met your competition and it is called Microsoft. The crowd goes wild when they are told they are getting a Samsung slate machine. Hmmm, why am I working instead of being at Build? My priorities are wrong I guess Sad smile

We’re shown deep freeze, the new task manager that look pretty neat.  The command line  to set a base line for your machine refresh is very appealing to me. At a point you have your machine just right => grab it for refresh if/when needed.

Metro over RDP looks awesome remote charms, virtual keyboard and of cause touch! I bet the VDI crow is going a little wild dreaming of the possibilities straight out of the box.

Hyper-V on Windows 8 client! We’re there Winking smile. The guy is storming through the features. He’s on the clock. We arrived at the business crowd. A lot of stuff for the desktop is also improved. Multiple Monitor support, control of Metro & desktop with shortcuts within the monitors. The UP button should be a good alternative to select delete in Window explorer paths. Lots of stuff to explore.

Windows Live integration with Windows 8 is extensive. The SkyDrive examples are impressive. Windows 8 will be the first mesh /hybrid / integrated OS. WinRT API exposes this so you can use that cloud extensibility in your Metro style apps!

Sorry if all this reads hectic, but it’s kind of hard to keep up. This is a tsunami of information! Keynote is wrapping up. The Hyper-V Windows 8 Server stuff will be for another day.

In the end a call to action for developers. Get the preview and get ahead of the pack delivering Metro style apps to a billion potential users. Up and at them developers!

The Dilbert® Life Series: Mental Hygiene Is Counter Productive

There are times that IT people need to vent. Usually they do that amongst their peers. Sometimes they disagree with each other and they express that. Why? Well most of them are straight shooters, not politicians or diplomats.  Now don’t get me wrong. I do understand the benefits of politics & diplomacy and I most definitely see the need for it. They can achieve things more often than conflict or direct orders can. Mainly because they make the people think it was their own decision and/or choice. The drawback with politics is that it takes time and in some situations, unfortunately, you don’t have that luxury. Don’t forget IT Pros work in sometimes rather stressful crisis situations. The bad part about politics is that it can also be perceived as “shady dealings” but this is actually not true. This is a negative connotation due to the often very poor quality of politicians. But I digress. I actually love diplomacy. It’s the process that delivers me either the desired result or buys me enough time to for my sniper to get the range . Either way, politics and diplomacy gets the job done, when you fulfill one prerequisite and that is to have professional diplomats around. As you might have already guessed, that wouldn’t exactly be me Winking smile. Politics however is not the same as “political correctness” run amok. Don’t be afraid of people speaking their mind. Don’t let the fear of others hearing some strong language or an unpopular issue being discussed guide you. That alone will not kill a reputation or wreck a well-oiled team.

Reputations have a major flaw. They take a life time to build and only a second to destroy. Are you telling me your approach to protecting a reputation is making sure no one ever hears a bad word out of the mouth of an employee who’s ranting to blow off steam? Guess what? You’re doomed to fail. Don’t we need to protect people from being offended? Yes, but don’t take it to far. Chances are that the offence is both ways. So don’t restrict free speech & open communication too much. But perception is reality right? Good lord, get a grip and grow a pair. People need to vent, express themselves and be allowed to do that in an not overly politically correct way amongst their peers. These people are in the trenches together, they deal with all the shit and stress. They shouldn’t be worried or stressed about using the proper diplomatic approach to everything they say. Political correctness can be taken too far. It makes for a very hypocritical, bottled up with frustration, unhealthy work environment. Amongst comrades you need not have to worry about that. And for crying out loud, I really do hope that humanities only hope for decent behavior is the fact that things are forbidden or regulated.

One shouldn’t judge IT managers or team leaders by the fact none of their team members ever curses or vents. Let alone some silly dress code. No, that T-shirts saying “You’ve read my T-shirt. That’s enough social interaction for one day” will not ruin professional relationships. Acting on those things remind me of micro managers. Meaning they focus on small issues for all kinds of reasons, non of which have anything to do with them being good managers. Do you want to know what you IT teams are worth? Look at the members. Do they stick up for each other? Are they not afraid to stand up and speak up about issues that are “threatening” one of them or their boss? Do they get the job done? No I don’t mean that they wear a tie, are in the office at 08:30 or never ever vent, I mean do they get the job done. Even at night, during those wee hours of the morning when needed or just even when is more convenient for the business? That should tell you a lot. That’s their PR without the glossy brochures.

Next to that it also has some other negatives associated with it.

  • First of all you lose your eyes and ears. Trust me, your IT people are your boots on the ground. They see, hear & know a lot as they deal with the entire organization. No matter how many tests, technology and reports you got at your disposal your people are a very valuable resource of information on what’s going on in the company. IT  as an bio indicator so to speak. From problems with vendors, storage issues, dysfunctional project managers to insane analysis and architects who’ve become a bit to enamored with the esoteric part of their job. In other words, if you want to know what going on let your IT staff speak their minds without fear. Create an environment where they can do that. Otherwise they’ll shut up even when they better open their mouths.
  • You’re flushing the morale of your troops down the drain. When people feel frustrated they need to vent, not be censored. That leads to unhappy employees and instead of having “undesired” verbal statements about a situation you’ll be hearing some very unsettling complaints about your stupid company. You might not like those either but you’d better listen and learn from them instead of saying that such talk “ist verboten”.
  • Don’t block the vents on a steam engine. They are there for a very important reason. Their proper functioning is to assure that the pressure doesn’t build up to high, thereby preventing the engine from blowing up. Same thing here, speaking their minds relieves pressure , stress and prevents frustrations. That’s a good thing as human beings under high pressure tend not to become diamonds even if they are bio carbon life forms. Chances are they’ll explode out of proportion when it really shouldn’t happen. A bit counter productive don’t you think?

Now this doesn’t mean you should stand for an all-out negative culture where all is piss and vinegar. Some venting is good, being a full time complaining sourpuss is not. Lead by example. By all means avoid e-mailing vents and frustrations. Words are volatile and dissipate. E-mail is very persistent. Maintain professional courtesy whenever possible.  While I think that respect needs to be earned, politeness and correctness can and should indeed be given. It goes along way when dealing with people. And the beauty is that by allowing people to vent and speak their minds you help achieve this. All you have to to do is maintain balance and don’t let the morale and the culture go south. So forget about dress codes, punch clocks, “mental hygiene” measures. They indicate another much worse problem. Management failure. Sure you can blame the issues on that T-shirt or someone’s venting. Perhaps you can even fool yourself into believing it. Perhaps it even helps you sleep at night. But it sure will not help you improve your business. For that you’ll need to put the good managers, diplomats & politicians in the right place instead of trying to rely on never needing those particular skills.

Assigning Large Memory To Virtual Machine Fails: Event ID 3320 & 3050

We had a kind reminder recently that we shouldn’t forget to complete all steps in a Hyper-V cluster node upgrade process. The proof of a plan lies in the execution Smile. We needed to configure a virtual machine with a whooping 50GB of memory for an experiment. No sweat, we have plenty of memory in those new cluster nodes. But when trying to do so it failed with a rather obscure error in System Center Virtual Machine Manager 2008 R2

Error (12711)

VMM cannot complete the WMI operation on server hypervhost01.lab.test because of error: [MSCluster_Resource.Name="Virtual Machine MYSERVER"] The group or resource is not in the correct state to perform the requested operation.

(The group or resource is not in the correct state to perform the requested operation (0x139F))

Recommended Action

Resolve the issue and then try the operation again.

image

One option we considered was that SCVMM2008R2 didn’t want to assign that much memory as one of the old host was still a member of the cluster and “only” has 48GB of RAM. But nothing that advanced was going on here. Looking at the logs found the culprit pretty fast: lack of disk space.

We saw following errors in the Microsoft-Windows-Hyper-V-Worker-Admin event log:

Log Name:      Microsoft-Windows-Hyper-V-Worker-Admin
Source:        Microsoft-Windows-Hyper-V-Worker
Date:          17/08/2011 10:30:36
Event ID:      3050
Task Category: None
Level:         Error
Keywords:     
User:          NETWORK SERVICE
Computer:      hypervhost01.lab.test
Description:
‘MYSERVER’ could not initialize memory: There is not enough space on the disk. (0x80070070). (Virtual machine ID DEDEFFD1-7A32-4654-835D-ACE32EEB60EE)

Log Name:      Microsoft-Windows-Hyper-V-Worker-Admin
Source:        Microsoft-Windows-Hyper-V-Worker
Date:          17/08/2011 10:30:36
Event ID:      3320
Task Category: None
Level:         Error
Keywords:     
User:          NETWORK SERVICE
Computer:      hypervhost01.lab.test
Description:
‘MYSERVER’ failed to create memory contents file ‘C:ClusterStorageVolume1MYSERVERVirtual MachinesDEDEFFD1-7A32-4654-835D-ACE32EEB60EEDEDEFFD1-7A32-4654-835D-ACE32EEB60EE.bin’ of size 50003 MB. (Virtual machine ID DEDEFFD1-7A32-4654-835D-ACE32EEB60EE)

Sure enough a smaller amount of memory, 40GB, less than the remaining disk space on the CSV did work. That made me remember we still needed to expand the LUNS on the SAN to provide for the storage space to store the large BIN files associated with these kinds of large memory configurations. Can you say "luxury problems"? The BIN file contains the memory of a virtual machine or snapshot that is in a saved state. Now you need to know that the BIN file actually requires the same disk space as the amount of physical memory assigned to a virtual machine. That means it can require a lot of room. Under "normal" conditions these don’t get this big and we provide a reasonable buffer of free space on the LUNS anyway for performance reasons, growth etc. But this was a bit more than that buffer could master.

As it was stated in the planning that we needed to expand the LUNS a bit to be able to deal with this kind of memory hogs this meant that the storage to do so was available and the LUN wasn’t maxed out yet. If not, we would have been in a bit of a pickle.

So there you go a real life example of what Aidan Finn warns about when using dynamic memory. Also see KB 2504962 “Dynamic Memory allocation in a Virtual Machine does not change although there is available memory on the host” which discusses the scenario where dynamic memory allocation seems not to work due to lack of disk space. Don’t forget about your disk space requirements for the bin files when using virtual machines with this much memory assigned. They tend to consume considerable chunks of your storage space. And even if you don’t forget about it in your planning, please don’t forget the execute every step of the plan Winking smile