Event ID: 11 From Microsoft-Windows-RPC-Events Are Indicating Possible Memory Leaks With MMC

After finishing putting some brand new servers in place with Windows 2008 R2, installing its rolls and leaving a happy client I’m usually very happy about a job well done. That feeling can last for a while when doing the paperwork involved with the project. It can also go away blazingly fast when you get a call that there is an “RPC memory leak or something no right” on the servers.  Not good. So you remotely access the server and start looking. Luckily for me this was to be a non issue. The event logged was the following:

Log Name:      Application

Source:        Microsoft-Windows-RPC-Events

Date:          06/01/2011 22:26:18

Event ID:      11

Task Category: None

Level:         Warning

Keywords:     

User:          BIGBillyTheServerAdmin

Computer:      infra01.big.corp

Description:

Possible Memory Leak.  Application ("C:Windowssystem32mmc.exe" "C:Windowssystem32dhcpmgmt.msc" ) (PID: 5000) has passed a non-NULL pointer to RPC for an [out] parameter marked [allocate(all_nodes)].  [allocate(all_nodes)] parameters are always reallocated; if the original pointer contained the address of valid memory, that memory will be leaked.  The call originated on the interface with UUID ({6bffd098-a112-3610-9833-46c3f874532d}), Method number (2).  User Action: Contact your application vendor for an updated version of the application.

If you do a search for this you’ll find several unresolved news group and support site questions but also a Microsoft knowledge base article http://support.microsoft.com/kb/974814. It states that when you run the Server Manager Snap-in (servermanager.msc) for extended periods of time, the application event log warning as seen above is logged. It also says it only happens on DHCP servers, which is exactly a roll these servers have and the warning entry we see in the application even log. As long as the UUID is {6bffd098-a112-3610-9833-46c3f874532d} and you have no other indications of a memory leak you’re good to go. Armed with the link we quickly put the owners mind at easy and all is well again. Back to the paperwork.

DHCP Behavioral Change in 2008 R2

Well today we got bitten by an unexpected functional change in Windows 2008 R2 DHCP. A colleague of mine needed to replace a defect printer that had a reservation in DHCP so it would always get the same reserved IP configuration. This can be handy for some “light security” reasons or for when a repair men resets the configuration to default settings or so.

So he set out to replace the original reservation by deleting the existing one and replacing it with a new reservation with the same IP address but using the MAC address of the NIC in the new printer. At least that was the intention. He swiftly received an error: The specified DHCP Client is not a reserved client.

So what does that mean? Well it turns out that something that was quiet possible up until Windows 2008 is no longer allowed in Windows 2008 R2. Consider the following:

For example if you have a 10.100.0.0/16 subnet and about 600 client devices max on your LAN you could make your DHCP severs fault tolerant by setting them up as follows:

DHCP Server 1 with a Scope lease range of 10.100.20.0 – 10.100.23.255

DHCP Server 2 with a Scope lease range of 10.100.24.0 – 10.100.26.255

You can lose a DHCP server that way the remaining one still has more then enough IP addresses in it’s range handing out to all potential client devices. This is about the easiest and simplistic way of providing DHCP redundancy. You might not like it (not really the 80/20 rule for a split scope according to the book) but it is very widely used, simple and it works in environments where IP addresses are plentiful.

Now before Windows 2008 R2 you could add reservations for IP address that are in the subnet of the scope but are outside in the lease range. For example if you agreed internally to put all printers on 10.100.50.x you could add reservation for them, even if they fall outside the lease ranges of DHCP Server 1 and 2 because they where in the same subnet of the scope. That now no longer works and gives you the above error.

Why did we never notice this before? The existing reservations just keep working, you just can’t create new ones. And yes, this was the first time it was needed to be done after the upgrade to Windows 2008 R2.

The Solution (no impact on current IP addressing schemes already in use).

My colleague (great guy, keen eye for trouble shooting) was well on his way to the solution and finally we implemented the following. We set up a split scope with a lease range that included the IP addresses used by the printers, and than added exclusions for the ranges that are used for clients on the other DHCP server. More like the split scope DHCP concept by the rules but without 80/20 rule.

When you do this turn of the printer or other device and delete the lease it might already have for its MAC address. You can than work without any further issues because of the IP or hardware address already having a lease. Oh yeah, when you delete a lease in DHCP, refresh the MMC tree manually or you might see the result of your deletion 🙂

Et voila. We’re done and back in business. This helpdesk call on replacing a printer turned out to be a rather expensive one 🙂

Conclusion

DHCP got some really neat extra’s (DHCP allow & deny filtering with scripting support for automation, the split scope wizard, …) in Windows 2008 R2 but this little change is going to bite a lot of people when they migrate or upgrade from previous versions, as split DHCP scopes are de facto standard in a lot of DHPC implementations. Why it provides for simple and easy DHCP service redundancy and it used to let you let you define reserve IP address for special uses in a range that was not even handed to other clients (Printers, scanners, Wireless Access Points, …). Well now you have to work around it (or be a bit more by the book), but as you have seen you can still get it function again. So beware of this when you make the move to DHCP on Windows 2008 R2 and implement a solution accordingly.