DELL XPS Laptops and seemingly random Blue Screens of Death

Introduction

Over the last years, I have endured some blue screens on my personal travel laptop and those of some buddies. The common factors were various types of DELL XPS laptops and seemingly random Blue Screens of Death (BSOD) with irql_not_less_or_equal. The problem and cause are however not limited to these devices or brand. The cause, I found, very often was the Intel RST driver.

Trust me, there is nothing I find more annoying than a BSOD while I’m on the move and working in a train, airplane, attending a conference or in the speaker room of conference finishing my slides. Heaven forbid it happens while I am presenting. It is always annoying, but in my home office I have other options. While I am traveling, I have no other options.

Diagnose the problem

To fix the issue we need to stop guessing and randomly upgrading or downgrading drivers in a trial and error fashion. Diagnose the problem properly. So what does a seasoned IT Pro do? The IT Pro copies the memory dump from the system and feeds it into WinDbg (download it here).

Be sure to work on a copy of the MEMORY.DMP if you run WinDbg on the problematic machine itself to prevent any permissions issues. Open it via the “File” menu and the option “Open Crash Dump”.

Select your copy of the MEMORY.DMP file.

Let it run and have some patience as it does its work. Pretty soon you’ll see output like below.

Probably caused by : iaStorAVC.sys ( iaStorAVC!Wcdl::Allocator::freeContiguous+20 )

When you run analyze -v you’ll get extra details but you already know what you need to know.

The outcome over the past years, in many (most) cases, pointed to iaStorAVC.sys more than anything else. This is the Intel Rapid Storage Technology (intel RST) driver. Your mileage may vary but I have seen the same cause on multiple XPS systems I figured I would blog it and save my future self (and maybe some of you) a ton of headaches. That said, when you get a BSOD you really must investigate the MEMORY.DMP file from that system yourself to see what driver is the culprit. But with an XPS, chances are iaStorAVC.sys is a reasonable candidate suspect. It has been a common issue over the past few years it seems.

Fine now we know what to fix. How to fix it is what we’ll look at next. I just wrote this blog for my own reference.

Fixing iaStorAVC.sys related BSOD

You have two possible approaches to fixing iaStorAVC.sys related BSODs.

  1. Get an updated Intel Rapid Storage Technology driver (this port)
  2. Move from RAID to AHCI without reinstalling
  3. Move from AHCI to RAID without reinstalling(handy when newer RST drivers are available and your want to test your luck).

We will look at all 3 options, staring with option 1 below.

For these operations make sure you have a (local) account with admin rights. These procedures should work with BitLocker enabled (my laptop has) but make sure you have your recovery keys at hand somewhere. Also, when using a PIN this won’t work in safe mode you know your username/password. The Barney Bear essentials for sysadmins who’ve been around the block a few times.

Get an updated Intel Rapid Storage Technology driver

This is pretty straightforward. Making sure all drivers are up to date is the 1st response to driver issues. No need to explain this in detail, just install them. If DELLEMC has one on their support site that’s great. If they don’t head over to the Intel site and look over there. DELLEMC can lag behind a few months which is why SuoortAssist or you won’t find anything. I have used the more recent Intel downloads with success in the past (https://downloadcenter.intel.com/download/28255/Intel-Rapid-Storage-Technology-Intel-RST-User-Interface-and-Driver?product=55005) to keep Intel RST update.

As RAID mode is how the systems ships by default this is the easiest way to resolve the issue with DELL XPS Laptops and seemingly random Blue Screens of Death. Now if this doesn’t fix the issue or there is no newer driver to be found move on to option 2 and eventually maybe even 3. Those are other blog posts.

New-SMBMapping -UseWriteThrough $True

New-SMBMapping -UseWriteThrough $True

As my readers probably know I like networking technologies. I experiment a lot find the best solutions for the challenges at hand. The aim is to make sure what I finally deploy in production is a very solid design. This avoids issues. Details can be very important in all this.

One such details is that in Windows Server 2019 (1809) and Windows 10 1809 mapping an SMB share has gotten a new parameter. For specific use cases, where caching is undesirable, you can now leverage: New-SMBMapping -UseWriteThrough $True

Some software requires and is dependent on the absolute guarantee data is consistent and persisted at all times. If so, the standard behavior of the OS in relation to a SMB 3 share is not enough to achieve this. Backups are such a use case. I wrote an article on this that was published on the StarWind Software blog: 
Windows Server 2019 introduces a new SMB Mapping Option UseWriteThrough

Go have a look and I hope you find it useful. You will find more info on the potential issue it fixes inWhen using file shares as backup targets you should leverage continuous available SMB 3 file shares

Attending the Global MVP Summit 2019

Attending the Global MVP Summit 2019

This week I registered for attending the Global MVP Summit 2019. My flights are booked, as is the hotel. The train tickets to and from the airport can still wait a bit. I am already looking forward to it. Getting out of the office to interact with so many people with various back grounds and roles is a great way to evaluate where you stand, what you do and where you are going. That’s why I also attend and present at conferences and I am happy to contribute to community programs like Microsoft Most Valuable Professional, Veeam Vanguard, DELL Rockstar etc. It is time and money well spent.

Being a Microsoft MVP is a great experience. You get to give and receive feedback on their technology and gain a better understanding about the why.

The experience

I for one can say that arriving in Bellevue and at the Redmond campus has always been an educational, stimulating experience. There is always a bit of a buzz and a focus. They’ll gladly pull you in a room with a few program managers to discuss issues, ideas or concepts you brought up and, in some cases, you’ll see the results of that discussion later that year. That’s kind of cool and satisfying to help make something a little bit better with our contributions.

This doesn’t mean we agree on everything, far from, but discussion is good as challenging things help find the strengths and weaknesses. There is a reason for the word “real-world knowledge, independent,expertise and passion to describe what MVPs do and are. As one manager puts it “MVPs help keep us honest”. It is one way of prevent tunnel vision and I think this might be the most important part for Microsoft. They don’t need a couple of fan boys showing up but people with feedback and opinions, and real world, hands on experience.

I realize that I do not I get to set the course at Microsoft or determine their business plans and strategies. Far from. But you are a valued contributor, a real-world bio indicator of how well they’re are doing in the global IT ecosystem. In that week we work together and learn from each other. I feel appreciated.

Even when I drop in to their offices when passing by Redmond, I always got a warm welcome. Many of my own employers or managers have never made me feel that appreciated and welcome. We can learn something from that. Appreciate people who dare tell you what you need to hear, not what you want to hear. Whatever you do after that feedback is fine, but at least genuinely listen and evaluate it.

Conclusion

I think it is all the above that keeps me coming back to the MVP Summit. Sure, it costs me: vacation days, travel expenses, out of pocket expenses. All costs that I cannot declare or recuperate. I’m traveling for 20 hours inbound and outbound but, with or without my employers support, I am going.  It is an investment in myself and you cannot let others decide on everything you learn or do. Remain valuable, remain independent and don’t be afraid to do the right thing. With or without a sponsor or your employers consent, go! Go and be all you can be.

Kemp LoadMaster template import issue: Command serverinit needs 1 parameter(s)

Introduction

The error Kemp LoadMaster template import issue: Command serverinit needs 1 parameter(s) was encounter during a migration project I was involved in recnetly. Recently. The job was to migrate the virtual services of an aging Kemp LoadMaster HA solution (LM-2200, 32 bit) to newer and more capable versions (running 7.2.43, X64 bit). Even more important, migrate to a version that is still in support. The LM-2200 series are like many older ones End of Life (EOL). Kemp still delivers important fixes for critical security issues like this year (7.1.35.5 and 7.1.35.6) but that’s it in long term support (LTS). This is not bad, these have been supported for a very long time but the aging 32-bit hardware has reached the point where replacing the is the only right thing to do.

With Kemp we can select new hardware, virtual machines or appliances in the cloud. So on-premises, hybrid and public cloud needs can be taken care off with the same familiar load balancer / Application delivery controller (ADC).

One technique to migrate Kemp workloads is to export the virtual services to templates and use these to configure the new solution. It is during that process we ran into an error: Kemp LoadMaster template import issue: Command serverinit needs 1 parameter(s)

Kemp LoadMaster template import issueCommand serverinit needs 1 parameter(s)

One of the applications is a mission critical one that has many different virtual services. FTP/FTPS/HTTP/HTTPS and also a bunch of TCP/UDP connectivity over ports that are very application/industry specific. A dozen virtual services, all on the same IP address with different ports and configuration needs.

We exported all of them to templates which we then used to recreate them on the replacement ADC. While doing so we used a different IP address for these newly created virtual services for testing purposes. This test IP address is changesd to the original production IP when ready to make the move and after disabling the old virtual services. That also allows for a quick exit / roll back if needed.

The process of creating a new virtual service requires a little work to be done still such as defining a gateway, SSL certificates, adding real servers, … but the bulk of the work is done for you.

But with certain templates we got an error: Syntax error detected in template file: Command serverinit needs 1 parameter(s)

clip_image001

When looking at the template in a text editor we see the following:

clip_image002

Clearly “servinit” does not have a parameter set. So, it probably needs one, to specify which type of the Server Initiating Protocol needs to be set for our generic service.

clip_image003

Finding a solution

We need to find out what parameter goes with our setting and add that the template. That value is found is easy enough. Via educated trial and error. On a test virtual service on the current version ADC we set the value for the Server Initiating Protocols to our value (other server initiating) and then export this virtual service to a template. We look at the template and note the number. That way we map what server initiating protocol corresponds to what value. In our case other server initiating is “3”.

clip_image004

We edit our exported templates to have the correct parameter value set that we find out this way and try to import it again.

clip_image005

That’s all that we needed to do to get these exported templates to be imported with no further issues.

clip_image006

clip_image008

All that’s left to do is to finish configuring the virtual service and continue our migration.

Conclusion

Don’t panic. Many problems you encounter have a solution, workaround or fix. Maybe this will help someone out there. Take care and until next time.3