Category Archives: Active Directory

Active Directory Replication Status Tool’s rise, fall, and rebirth

Posted on by
2

Active Directory Replication Status Tool’s rise, fall, and rebirth

For many years the Active Directory Replication Status Tool has been a trusty companion for many IT Pro. That includes seasoned systems engineers as well as accidental Active Directory administrators. It was an easy way to get a quick and good idea of the replication health of your Active Directory forest or domain.

Sure, repladm is our friend and keeps doing its job with us in the trenches. But I would say that the Active Directory Replication Status Tool is a lot less scary for people. Especially those who incidentally need to find out what issues to address where. It is a non-scarry, kind wrapper to visualize the results and see where we need to pay attention.

It had some quirks, like the ridiculous need to download it again when the license (certificate) expired (I never liked the Sirona DLL hack in a production environment). If other issues arose, I blogged on how to work around them, like in Microsoft Active Directory Replication Status Tool won’t upgrade – Working Hard In ITWorking Hard In IT. But that was minor compared to the sage of the last 18 months. Let’s quickly look at the Active Directory Replication Status Tool’s rise, fall, and rebirth.

The fall

Once in a while, an update to Windows or .NET broke the application. Normally, a fix would follow soon, and everyone was happy again. Last year in the spring or early summer of 2022, the tool was broken for months. Finally, somewhere in August 2022, we got a new version that worked, as far as my experience goes, for about 4 to 6 weeks, It then broke again, and all it ever did after was crash. The cert also expired again but who cared? It was no longer functional anyway,

Well, I cared, and I gave a lot of feedback via Twitter and to Microsoft via e-mail.

Active Directory Replication Status Tool's rise, fall, and rebirth

That went on for about 10 months without any progress! Then suddenly, I notice this: How to get and use the Active Directory Replication Status Tool – Windows Server | Microsoft Learn.

Active Directory Replication Status Tool's rise, fall, and rebirth

What!!??? Is this how it ends? Abandoned in a ditch somewhere?

Important

As of June 2nd, 2023, the Active Directory Replication Status Tool is no longer available for download. The following article is provided for historical purposes only.

Are you kidding me? There went another e-mail right to Redmond! Was this the sad end? I got a reply that not all was lost with a link.

The rebirth

The link I got is this one GitHub – ryanries/ADReplStatus: AD Replication Status Tool. People meet Ryan Ries, an Escalation Engineer at Microsoft who has some very useful and handy private projects to share with the world. ADReplStatus is one of the more recent ones.

I downloaded it and started testing it in the lab. That went well and has already found its way to two production environments. Here is a screenshot from my lab environment!

Active Directory Replication Status Tool's rise, fall, and rebirth

Thank you, Ryan Ries, for helping your customers in your free time with your private projects. You have made my IT Pro existence a bit easier again instead of more difficult. It is appreciated! Thank you, Ned Pyle, for bringing this GitHub repo to my attention. Download your copy here Release v1.3.1 · ryanries/ADReplStatus · GitHub.

What I did find interesting was that the cert expiration time bomb was also an internal issue. That and the fact that the application had no maintainers. But that was obvious to us all.

The old version of the tool had a time bomb in it – an expiring SSL certificate – that rendered the app unusable sometime around September 2022. Only through great effort internally were we able to periodically renew this certificate and republish the app, and the app was architected in such a way that excising the signing certificate check was more work than just rewriting the whole thing. (I know about the Sirona DLL hack, but still.)
The old tool had no active maintainers and no one who was still around was familiar with the app’s internals or source code.

Things are tough all over, it seems.

PowerShell script to maintain Azure Public DNS zone conditional forwarders

Posted on by
2

Introduction

I recently wrote a PowerShell script to maintain Azure Public DNS zone conditional forwarders. If you look at the list is quite long. Adding these manually is tedious and error-prone. Sure you might only need a few, but hey, I think and prepare long term.

Some background on DNS and private endpoints

When using private endpoints in Azure correct DNS name resolution is essential. While Azure can do a lot of things for you under the hood it is important to wrap your head around name resolution in Azure, for all your public, private, and custom DNS requirements. In the end, you need a DNS solution that is maintainable and works for current and future use cases. Your peaceful IT existence will fall apart fast without the ability to correctly resolve the private endpoint IP addresses to their fully qualified domain name (FQDN).

That in itself is a big subject I will not dive into right now. I will say that host files (if applicable) are OK for testing but not a maintainable solution, except for the smallest environments. In Azure, you can link virtual networks to Private DNS zones to resolve DNS queries for private endpoints. As an alternative, you can use your own custom DNS Server(s) with a forwarder to Azure’s VIP 168.63.129.16 and, at least on-premises conditional forwarders.

The latter is a requirement to resolve DNS queries for Azure resources with private endpoints for on-premises. At least until Azure DNS Private Resolver becomes generally available. That will be the way forward in the future if you otherwise have no need for custom DNS servers.

Please note that name resolution for private endpoints uses the public DNS zones. This allows existing Microsoft Azure services with DNS configurations for a public endpoint to keep functioning when accessed from the internet. Azure will intercept queries that originate from Azure or connected on-premises locations and reply with the private IP address of private endpoints. This configuration must be overridden to connect using your private endpoint.

On-premises DNS Servers

While your custom DNS servers in Azure can forward queries they are not authoritative for to the Azure VIP 168.63.129.16, on-premises servers cannot reach that IP address. They need to send the DNS queries for private Azure resources to a custom DSN Server in Azure via conditional forwarding. The Azure custom DNS server will forward the query to 168.63.129.16.

Example on-prem / Azure ADDS environment with Azure FW DNS proxy

PowerShell script to maintain Azure Public DNS zone conditional forwarders

Below you will find the code for the script. I created a CSV file with all the Azure public DNS Conditional forwarder zones. Zones with placeholders for regions, partitions, or SQL instances will be generated. For that, you need to provide the correct parameters. if not these are ignored.

PowerShell script to maintain Azure Public DNS zone conditional forwarders
Adding all Azure public DNS conditional forwarders to an on-premises DNS server

Another attention point is the fact that you can opt to store the zones in Active Directory or not. If so you can specify in what builtin or custom partition.

There are examples in the script TestAzurePublicDNSZoneForwardersScript.ps1 on how to use it. You will need at least one playground DNS server or better, 2 AD integrated DC/DNS servers for testing.

You can find the script at WorkingHardInIT/AzurePublicDnsZoneForwarders (github.com)

Conclusion

That’s it. I can extend the PowerShell script to maintain Azure Public DNS zone conditional forwarders with extra options when adding or updating conditional forwarders. Right now, for its current role, it does what I need. I do not plan to add an option to update the “store this conditional forwarder in Active Directory” setting as this has a bug.

See Bug when changing the “store this conditional forwarder in active directory” setting for more info. The gist is that it makes changing the setting causes DNS queries for the conditional forwarder to fail. We avoid that issue by removing and adding the conditional forwarders again. In many (most?) use cases so far, the default setting of not storing the conditional forwarder in Active Directory is what I need, so the script has no option to change that default setting until I might need it.

Code

Bug when changing the “store this conditional forwarder in active directory” setting

Posted on by
4

Bug when changing the “store this conditional forwarder in active directory” setting

Recently I encountered a bug when changing the “store this conditional forwarder in active directory” setting. I have been doing quite some active directory extensions to Azure lately. Part of that, post-process, is making sure that DNS name resolution from on-premises to Azure and vice versa is working optimally. When it comes to resolving Azure private endpoints and other private DNS zones from on-premises we need to add the conditional forwarders for the respective Azure DNS zones.

As we have different needs for this configuration on-premises versus in Azure we disable “Store this conditional forwarder in Active Directory, and replicate as follows” for all zones. This is the defaultm when you add a conditional forwarder.

However, you will also need to do this, in certain cases for other conditional forwarders depending on the DNS infrastructure between Azure and on-premises. I tend to change those non-Azure resource conditional forwarders before I add the one needed for Azure.

Bug when changing the "store this conditional forwarder in active directory" setting
The “store this conditional forwarder in active directory” setting

While that sounds easy enough, you can easily get into a pickle. When you change this, while the configuration seems perfectly fine, the name resolution for those zones where you change this stops working. That is bad. No bueno!

That can break a lot of services and applications leading to support calls, causing upset application owners, and lost revenue while leaving you scrambling to find a fix.

So how do we fix this?

Well, the only solution is to remove each and every conditional forwarder involved and add them again, While re-adding it you might get an “unknown error” in the GUI, but ignore it. Just go ahead. When your reverse lookup zones are in order it will resolve to the FQDN and name resolution will start working again. You can also use PowerShell or the command line. It is worth checking if changing the setting via PowerShell or the command line triggers the bug or not.

Please note that, as your are not replication the conditional forwarders in Active Directory, you must do that on all DNS servers on-premises involved in resolving Azure resources.

Is this a known bug?

Well, it looks like it, but I have yet to find a knowledge base article about it. There are mentions of other people running into the issue. This is not per se Azure-related. Take a look here DNS Conditional Forwarder stops working as soon as it’s Domain Replicated – Microsoft Q&A and AD Integrating conditional DNS forwarders stops them working (microsoft.com).

Note that this bug when changing the “store this conditional forwarder in active directory” setting will appear when you either enable or disable it.

This bug has existed for many years and over many versions of Windows DNS. The last encounters I had was with Windows Server 2019 and 2022. But beware with Windows Server 2016 and 2012 (R2) as well.

Linux AD computer object operating system values

Posted on by
Reply

Introduction

So, why am I dealing with Linux AD computer object operating system values? OK, here is some background. In geographic services, engineering, etc. people often run GIS and CAD software from various big-name vendors on Windows Servers. But it also has a rich and varied open source ecosystem driven by academic efforts. Often a lot of these handy tools only run in Linux.

The Windows Linux Subsystem might be an option for client-based or interactive tools. But when running a service I tend to use Ubuntu. It is the most approachable for me and, you can buy support for it in an enterprise setting if so desired or required.

To keep things as easy as possible and try to safeguard the concept of single sign-on we join these Ubuntu servers to Active Directory (AD) so they can log with their AD credentials.

Pre-staging computer objects

When joining an Ubuntu server to AD it partially fills out the Operating System values.

Not too detailed and only partially filled out.

However, we tend to pre-stage the computer accounts in the correct OU and not create them automatically in the default Computer OU when joining. In that case, the Operating System values seem to be left all blank. We can fix that with PowerShell.

Don’t worry, the screenshot is from my lab with my fictitious Active Directory forest/domain. You also have a lab right?

Linux AD computer object operating system values
Fill out the operating system info for pre-staged computer objects of Active Directory joined Ubuntu servers

Actually we need PowerShell Core

Now, this all very good and well, but how do we find out the values for the operating system. During deployment, we know, but over time they will update and upgrade. So it would be nice to figure out those values automatically and remotely.

PowerShell Core to the rescue! With PowerShell Core, we can do PowerShell Remoting Over SSH to run a remote session on our Linux server over SSH and get all the information we need. To make this automation-friendly you must certificate bases authentication for your SSH connection. Setting that up can be a bit tricky, especially on Windows. That is a subject for a future blog post I hope. You can also use the SecretStore to securely store the AD automation account credentials. Note that I also use a dedicated automation account on all my Linux systems for this purpose. Here is a “quick & dirty” code snippet to give you some inspiration on how to do that for Ubuntu.

#Grab the AD automation account credentials - please don't use a domain admin for this.
#Use a dedicated account with just enough privileges to get the job done.
$Creds = Get-Credential -UserName 'DATAWISETECH\dwtautomationaccount'
 
#Connect to a remote PowerShell session on our Linux server using certificate authentication.
#Setting this up is beyond the scope of this article but I will try to post a blog post on this later.
#Note you need to configure all Linux servers and desktops with the $public cert and allow the user to authenticate with it.
#We use a cert as that is very automation friendly! You will not get #prompted for a password for the Linux host.
$RemoteSession = New-PSSession -Hostname GRIZZLY -UserName autusrli
 
#Grab the OS information. Note that $PSVersionTable.OS only exist on PowerShell Core.
#which is OK as that is the version that is available for Linux.
 
$OS = Invoke-command -Session $RemoteSession { $PSVersionTable.OS }
 
#Grab the OSVersion.VersionString.
$VersionString = Invoke-command -Session $RemoteSession { [System.environment]::OSversion.VersionString }
 
#Clean up, we no longer need the remote session.
Remove-PSSession $RemoteSession
 
#Sanitize the strings for filling out the Active Directory computer object operating system values.
$UbuntuVersionFull = ($OS | Select-String -pattern '(\d+\.)(\d+\.)(\d)-Ubuntu').Matches.Value
$OperatingSystem = $UbuntuVersionFull.Split('-')[1] + " " + (($UbuntuVersionFull.Split('-')[0])).Substring(0, 5)
 
#Grab the Active Directory computer object and fill out the operating system values.
$Instance = (Get-AdComputer -Credential $Creds -Identity GRIZZLY -Server datawisetech.corp)
$Instance.OperatingSystem = $OperatingSystem
$Instance.OperatingSystemVersion = $VersionString
$Instance.OperatingSystemServicePack = $UbuntuVersionFull
Set-AdComputer -Instance $Instance

That’s it! Pretty cool huh?!

Conclusion

While you cannot edit the Linux AD computer object operating system values in the GUI you can do this via PowerShell. With Windows, this is not needed. This is handled for you. When joining Ubuntu to Active Directory this only gets set if you do not pre-stage the computer accounts. When you do pre-stage them, these are left blank. I showed you a way of adding that info via PowerShell. The drawback is that you need to maintain this and as such you will want to automate it further by querying those computers and updating the values as you update or upgrade these Ubuntu servers. Remote PowerShell over SSH and PowerShell Core on Linux are your friends for this. Good luck!