Category Archives: IT Pro

DELL XPS Laptops and seemingly random Blue Screens of Death

Posted on by
Reply

Introduction

Over the last years, I have endured some blue screens on my personal travel laptop and those of some buddies. The common factors were various types of DELL XPS laptops and seemingly random Blue Screens of Death (BSOD) with irql_not_less_or_equal. The problem and cause are however not limited to these devices or brand. The cause, I found, very often was the Intel RST driver.

Trust me, there is nothing I find more annoying than a BSOD while I’m on the move and working in a train, airplane, attending a conference or in the speaker room of conference finishing my slides. Heaven forbid it happens while I am presenting. It is always annoying, but in my home office I have other options. While I am traveling, I have no other options.

Diagnose the problem

To fix the issue we need to stop guessing and randomly upgrading or downgrading drivers in a trial and error fashion. Diagnose the problem properly. So what does a seasoned IT Pro do? The IT Pro copies the memory dump from the system and feeds it into WinDbg (download it here).

Be sure to work on a copy of the MEMORY.DMP if you run WinDbg on the problematic machine itself to prevent any permissions issues. Open it via the “File” menu and the option “Open Crash Dump”.

Select your copy of the MEMORY.DMP file.

Let it run and have some patience as it does its work. Pretty soon you’ll see output like below.

Probably caused by : iaStorAVC.sys ( iaStorAVC!Wcdl::Allocator::freeContiguous+20 )

When you run analyze -v you’ll get extra details but you already know what you need to know.

The outcome over the past years, in many (most) cases, pointed to iaStorAVC.sys more than anything else. This is the Intel Rapid Storage Technology (intel RST) driver. Your mileage may vary but I have seen the same cause on multiple XPS systems I figured I would blog it and save my future self (and maybe some of you) a ton of headaches. That said, when you get a BSOD you really must investigate the MEMORY.DMP file from that system yourself to see what driver is the culprit. But with an XPS, chances are iaStorAVC.sys is a reasonable candidate suspect. It has been a common issue over the past few years it seems.

Fine now we know what to fix. How to fix it is what we’ll look at next. I just wrote this blog for my own reference.

Fixing iaStorAVC.sys related BSOD

You have two possible approaches to fixing iaStorAVC.sys related BSODs.

  1. Get an updated Intel Rapid Storage Technology driver (this port)
  2. Move from RAID to AHCI without reinstalling
  3. Move from AHCI to RAID without reinstalling(handy when newer RST drivers are available and your want to test your luck).

We will look at all 3 options, staring with option 1 below.

For these operations make sure you have a (local) account with admin rights. These procedures should work with BitLocker enabled (my laptop has) but make sure you have your recovery keys at hand somewhere. Also, when using a PIN this won’t work in safe mode you know your username/password. The Barney Bear essentials for sysadmins who’ve been around the block a few times.

Get an updated Intel Rapid Storage Technology driver

This is pretty straightforward. Making sure all drivers are up to date is the 1st response to driver issues. No need to explain this in detail, just install them. If DELLEMC has one on their support site that’s great. If they don’t head over to the Intel site and look over there. DELLEMC can lag behind a few months which is why SuoortAssist or you won’t find anything. I have used the more recent Intel downloads with success in the past (https://downloadcenter.intel.com/download/28255/Intel-Rapid-Storage-Technology-Intel-RST-User-Interface-and-Driver?product=55005) to keep Intel RST update.

As RAID mode is how the systems ships by default this is the easiest way to resolve the issue with DELL XPS Laptops and seemingly random Blue Screens of Death. Now if this doesn’t fix the issue or there is no newer driver to be found move on to option 2 and eventually maybe even 3. Those are other blog posts.

New-SMBMapping -UseWriteThrough $True

Posted on by
2

New-SMBMapping -UseWriteThrough $True

As my readers probably know I like networking technologies. I experiment a lot find the best solutions for the challenges at hand. The aim is to make sure what I finally deploy in production is a very solid design. This avoids issues. Details can be very important in all this.

One such details is that in Windows Server 2019 (1809) and Windows 10 1809 mapping an SMB share has gotten a new parameter. For specific use cases, where caching is undesirable, you can now leverage: New-SMBMapping -UseWriteThrough $True

Some software requires and is dependent on the absolute guarantee data is consistent and persisted at all times. If so, the standard behavior of the OS in relation to a SMB 3 share is not enough to achieve this. Backups are such a use case. I wrote an article on this that was published on the StarWind Software blog: 
Windows Server 2019 introduces a new SMB Mapping Option UseWriteThrough

Go have a look and I hope you find it useful. You will find more info on the potential issue it fixes inWhen using file shares as backup targets you should leverage continuous available SMB 3 file shares

Attending the Global MVP Summit 2019

Posted on by
Reply

Attending the Global MVP Summit 2019

This week I registered for attending the Global MVP Summit 2019. My flights are booked, as is the hotel. The train tickets to and from the airport can still wait a bit. I am already looking forward to it. Getting out of the office to interact with so many people with various back grounds and roles is a great way to evaluate where you stand, what you do and where you are going. That’s why I also attend and present at conferences and I am happy to contribute to community programs like Microsoft Most Valuable Professional, Veeam Vanguard, DELL Rockstar etc. It is time and money well spent.

Being a Microsoft MVP is a great experience. You get to give and receive feedback on their technology and gain a better understanding about the why.

The experience

I for one can say that arriving in Bellevue and at the Redmond campus has always been an educational, stimulating experience. There is always a bit of a buzz and a focus. They’ll gladly pull you in a room with a few program managers to discuss issues, ideas or concepts you brought up and, in some cases, you’ll see the results of that discussion later that year. That’s kind of cool and satisfying to help make something a little bit better with our contributions.

This doesn’t mean we agree on everything, far from, but discussion is good as challenging things help find the strengths and weaknesses. There is a reason for the word “real-world knowledge, independent,expertise and passion to describe what MVPs do and are. As one manager puts it “MVPs help keep us honest”. It is one way of prevent tunnel vision and I think this might be the most important part for Microsoft. They don’t need a couple of fan boys showing up but people with feedback and opinions, and real world, hands on experience.

I realize that I do not I get to set the course at Microsoft or determine their business plans and strategies. Far from. But you are a valued contributor, a real-world bio indicator of how well they’re are doing in the global IT ecosystem. In that week we work together and learn from each other. I feel appreciated.

Even when I drop in to their offices when passing by Redmond, I always got a warm welcome. Many of my own employers or managers have never made me feel that appreciated and welcome. We can learn something from that. Appreciate people who dare tell you what you need to hear, not what you want to hear. Whatever you do after that feedback is fine, but at least genuinely listen and evaluate it.

Conclusion

I think it is all the above that keeps me coming back to the MVP Summit. Sure, it costs me: vacation days, travel expenses, out of pocket expenses. All costs that I cannot declare or recuperate. I’m traveling for 20 hours inbound and outbound but, with or without my employers support, I am going.  It is an investment in myself and you cannot let others decide on everything you learn or do. Remain valuable, remain independent and don’t be afraid to do the right thing. With or without a sponsor or your employers consent, go! Go and be all you can be.

Nuclear data waste

Posted on by
Reply

Introduction

Nowadays everyone seems to be heading to the hills to try and cash in on the new gold rush. Data! You have all heard that data is the new gold. Some call it the new oil, as that is their favorite fantasy, that’s all good. But there are drawbacks to this.

The cost of data and data waste

Like any resource you mine it does come with associated costs. A cost that has to be covered by the value you derive from it. That value has to exceed the money spend to gather, process and consume it. That can be an expensive business.

On top of that you have to deal with “the waste” it creates as a byproduct. Waste can be toxic. Mining data tends to produce nuclear data waste, the bad kind where “safe levels” are hard to determine. In the rush to grab the gold many forget a couple of important lessons from history. We should know by now we need to act proactively to avoid waste. That is the cheapest option in the long run and mitigates many of the risks. We should also know that not all data is gold, some of it just glitters, but it isn’t valuable. Fool’s data, like fool’s gold, is essentially worthless no matter how much money you have spent. Even worse, it’s still produces nuclear data waste asset than can get you into (legal) trouble.

Data storages and backups

How much data to you need to get to the gold and at what cost. Storage capabilities as well as storage capacity grows fast and cost seems under control for now. But will this last forever? And even if so, what’s the ratio of data gold versus raw data stored? Can we improve this ratio? Because even when things are cheap, why even do it if it is not needed?

Protecting the data and the waste

And then there is the cost with protecting that data as well as the governance around it.  The sad reality with data is that once you have it the probability that it will get you into trouble is real. Data, sooner or latter will get lost, misplaced, sold, hacked, leaked, … it’s almost guaranteed. Ask any real InfoSec professional (not the standard issue, policy quoting security officers, those are just windows dressing) and they will open your eyes to the reality of the risks. It’s very sobering.

The gold rush

As with any hype or gold rush we can avoid costly mistakes buy looking at history. Think about who benefited and who lost out. Think about why this happened and how. Can you see any parallels?

  • Many people are drawn to the data gold fields. Very few strike a gold vein.
  • There is a lot of money to be made selling the tools, supplies and gear to mine the data, process it, store and protect it.
  • Gathering raw data and processing it can be highly toxic
  • Storing and protecting the gold is expensive and hard.

Let’s dive a bit deeper into these issues, what these mean and how they materialize. They all have one thing in common for sure and that is that the fear of missing out is one of the driving factors.

Gold diggers

The reality is that many people that now become “data scientists” are not all highly skilled mathematicians and experts at statistics analysis. It’s a new hype, just like OLAP tools and data mining where before. We now have BI, big data and data science. That’s where the gold can be found so that’s where gold diggers flock to. Some have the skills, abilities and the luck to derive wealth from that. Most will just have the job digging.

Pick Axe

There is a lot more data than there is science in the hype created around data scientists. Data scientists should be great at math and statistics. Those are not very fields of human endeavor that do not scale well. They are not even popular. Attaching “scientist” to something doesn’t make it a science. Be sure of the quality of your gold and make sure it is not fool’s gold. But the gold rush is on. There’s money to be made. In an era where science is viewed by many as “an opinion” the urge to derive some credibility from adding “science” to any endeavor is a paradox. It is on the rise. Clearly, this shows the value of real science even when some only seem to like it when it suits their agenda.

But the field is exploding as companies want people working on all the raw data they collect. As one statistician stated: “I used to be a boring, underpaid geek with glasses, now that I’m a data scientist I’m cool, in demand and paid very well even if the work is less scientific.” That’s the nature of the beast. Her employer got a real statistician, but as the mines require a lot more bodies, many will make due with less.

Merchants

The “sure” money is in the supply chain. Storage, networking, compute … no matter where it is (cloud, fog or on-premises computing). There is money to be made with tools to process the data, protect it (backups) and secure it against unwanted prying eyes and theft. If you’re selling any of those business is a booming.

Everyone seems obsessed with collecting data. Luckily storage costs are down per GB and we can store ever more. We also need to protect more. But whoever deletes data? Who dares push the button? A lot of data is collected “just in case”. We might find gold in there later and if we don’t have it we cannot for sure. The fear of missing out in action. That is great if you’re selling stuff. Data lakes, data ponds, storage blobs or tables, Mongo DB or SQL PaaS, storage arrays, data processing technology and data protection. These can be products or services, it doesn’t matter, there is money to be made. And while you’re selling you’re not asking the buyers if the really need it. You don’t question them, you praise their insights and help them protect their investment. Everyone is doing it, so must you. The copy/paste strategy in action.

Nuclear data waste

While the vast growth in data is spectacular. A lot of it is crap. But there is very little effort put into  being selective. It’s too cheap right now to collect and store it. No one want to say “we don’t need it” and be the one to blame if you don’t have it.

But in the age of data leaks, hackers, privacy concerns and ever more legislation around data protection it’s worth making sure you don’t store data just because you can. Storing data holds inherent risks. Risk of losing it, corrupting it, deriving faulty information from it, leaking it, have it stolen or abused. It

In the age of GDPR and many other rightful privacy and data protection concerns collecting data should be treated like nuclear power. The value it brings is undeniable. But you don’t need vast amounts of nuclear fuel to deliver that value. You do need very good processes, fail safes, regulation, capable people and technology.

We should start looking at data as nuclear fuel and as such, after use and processing, part of it is left as toxic nuclear data waste. It’s a long-term toxic by product of the process of deriving information form data. Minimize the collection, storage and of data to achieve your goals at minimum cost and risk. Luckily, we have a very good solution for toxic data waste. You can delete it and wipe it securely.

We have to stop thinking that more is better when much of it is junk. The overhead of caring for that junk is ridiculous. We might have to do so for nuclear waste out of need. But for data there are alternatives. Destroy it if you don’t need it. It’s the safest way to handle the legal and reputation risks related to it. That will take a conscious effort.

Efforts and costs

Critical thinking about collecting data is lacking. That is understandable. There is a lot of the money to be made in data mining is in providing the tools to collect, process, store and protect the data. Even with many people that warn us of the security issues and legal responsibilities around it is often about selling services and products. For many all this might turn out to be a lot like the other gold rushes. There were a lot more suppliers of the tools that got rich than actual finders of profitable gold mines. This means there is also a lot of pressure and incentive to feed the “data is the new gold” beast.

Where a SQL database or a data warehouse at least meant you had to put effort into collecting the data, the rise of unstructured data technologies means way too often we don’t care and we’ll figure it out later. Imagine doing that to nuclear fuel! For now, the technical advances in storage and data technologies has allowed us to act without too much deliberation on the sanity of our choices. That might change, it might be wise to avoid the cold shower when it does and benefit from minimizing toxic data risks today.

Conclusion

Now true data gold is very valuable, but make sure you can recognize it. Just going through the motions and buying the tools, copying “in the know” statements from the internet isn’t going to cut it. That is called pretending. Sure, it’s fun. It is also a very dangerous and costly mistake when things get real. At best you look like an idiot with money. Many sales people will separate you from your money very efficiently.

The smarter organizations already have a data strategy that includes waste avoidance, reduction and management. Many don’t unfortunately. Collecting data for those is the main goal, driven by the tyranny of action over strategies. You have to be seen acting and being in charge. The buzz words have to be present and you have to come across as a “can do sir, yes sir” person. Well that is what will kill you. The late Norman Schwarzkopf knew this all too well.

Take care of your weaknesses, figure them out before they hurt you and before they destroy your ability to exploit your strengths. That people is a strategy exercise. I can do that for you and it will cost you a lot of money. But remember, strategies are not products you can buy, they are not commodities and as such buying them is a paradox. A strategy is what will give you the edge over your competitors.If you have others determine your strategy, your competitors will pay them more to find out . So, roll up your sleeves and put in the effort yourself. In the end, it’s all about common sense and this is true for data-mining, AI and BI as well.