If you are interested in Microsoft and QUIC I have some good news for you. Recently a new article, SMB over QUIC Technology | StarWind Blog (starwindsoftware.com) was published. It is the first in a series about what Microsoft is working on in regards to QUIC. While not without some controversy, QUIC does a lot for a number of issues connectivity over “the internet at large” has been dealing with.
It leverages UDP.
TLS 1.3 is built into the protocol.
Reduces RTT during connection & encryption setup.
Handles and optimizes flow control and loss recovery.
QUIC reduces the round trips during the TLS handshake significantly
Over the internet, with mobile clients, this is a big deal. Since it is secure by default people really started thinking about where this can be used to improve things for all involved.
I think QUIC is going to be more and more important in the future and this article positions QUIC in the Microsoft ecosystem. So, head over there, read it, and let me know what you think.
TLS 1.3, QUIC, HTTP/3, and SMB 3.1.1 are shaking up things a bit by challenging TCP. Microsoft dropped QUIC into Windows Server 2022 Azure edition. That went into public preview last week and I dove in to the lab to figure out what I can do with it.
As a technologist, I am having a lot of fun testing this out in the lab. Last weekend I was busy with SMB over QUIC and QUIC in IIS. I learned a lot and have made up my mind I can use this in the real world to solve requirements. I will share my findings and musing with you in near the future. But today, start with an introduction in SMB over QUIC Technology | StarWind Blog (starwindsoftware.com).
Recently I was implementing a high available Kemp LoadMaster X15 system. I prepared everything, documented the switch and LM-X15 configuration, and created a VISIO to visualize it all. That, together with the migration and rollback scenario was all presented to the team lead and the engineer who was going to work on this with us. I told the team lead that all would go smoothly if my preparations were good and I did not make any mistakes in the configuration. Well, guess what, I made a mistake somewhere and had to solve a Kemp LoadMaster ad digest – md2=[31084da3…] md=[20dcd914…] – Check vhid, password and virtual IP address log entry.
Check vhid, password and virtual IP address
As, while all was working well, we saw the following entry inundate the system message file log:
<date> <LoadMasterHostName> ucarp[2193]: Bad digest – md2=[xxxxx…] md=[xxxxx…] – Check vhid, password and virtual IP address
every second …
Wait a minute, as far as I know all was OK. The VHID was unique for the HA pair and we did not have duplicate IP addresses set anywhere on other network appliances. So what was this about?
Figuring out the cause
Well, we have a bond0 on eth0 and eth2 for the appliance management. We also have eth1 which is a special interface used for L1 health checks between the Loadmasters. We don’t use a direct link (different racks) so we configure them with an IP on a separate dedicated subnet. Then we have the bonds with the VLAN for the actual workloads via Virtual Services.
We have heartbeat health checks on bond0, eth1 and on at least one VLAN per bonds for the workloads.
Confirm that Promiscuous mode and PortFast are enabled. Check! HA is configured for multicast traffic in our setup so we confirm that the switch allows multicast traffic. Check!
Make sure that switch configurations that block multicast traffic, such as ‘IGMP snooping’, are disabled on the switch/switch ports as needed. Check!
Now let’s look at possible causes and check our confguration:
So what else? The documentation states as possible other causes the following:
There is another device on the network with the same HA Virtual ID. The LoadMasters in a HA pair should have the same HA Virtual ID. It is possible that a third device could be interfering with these units. As of LoadMaster firmware version 7.2.36, the LoadMaster selects a HA Virtual ID based on the shared IP address of the first configured interface (the last 8 bits). You can change the value to whatever number you want (in the range 1 – 255), or you can keep it at the value already selected. Check!
An interface used for HA checks is receiving a packet from a different interface/appliance. If the LoadMaster has two interfaces connecting to the same switch, with Use for HA checks enabled, this can also cause these error messages. Disable the Use for HA checks option on one of the interfaces to confirm the issue. If confirmed, either leave the option disabled or move the interface to a separate switch.
I am sure there is no interference from another appliance. Check! As we had checked every other possible cause the line in red caught my attention. Could it be?
Time for some packet captures
So we took a TCP dump on bond0 and looked at it in Wireshark. You can make a TCP dump via debug options under System Log Files.
Debug Options, once there find TCP dump.
Select your interface, click start, after 10 seconds or so click stop and download the dump
TCP dump
Do note that Wireshark identifies this as VRRP, but the LoadMaster uses CARP (open source) do set it to decode as CARP, that way you’ll see more interesting information in Info
No, not proprietary VRRP but CARP
Also filter on ip.dst == 244.0.0.18 (multicast address). What we get here is that on eth0 we see multicasts from eth1. That is the case described in the documentation. Aha!
Aha, we see CARP multicasts from eth1 on eth0, that is what we call a clue!
So now what, do we need to move eth1 to another switch to solve this? Or disable the HA check? No, luckily not. Read on.
The fix for Check vhid, password and virtual IP address
No, I did not use one or more separate switches just to plug in the heartbeat HA interfaces on the LoadMasters. What I did is create a separate VLAN for the eth0 HA heartbeat uplink interfaces on the switches. This way I ensure that they are in a separate unicast group from the management interface uplinks on the switches
By selecting a different VLAN for the MGNT and Heartbeat interface uplink they are in different TV VLAN groups by default.
By default the Multicast TV VLAN Membership is per VLAN. The reason the actual workload interfaces did not cause an issue when we enabled HA checks is that these were trunk ports with a number of allowed VLANs, different from the management VLAN, which prevents this error being logged in the first place.
That this works was confirmed in the packet trace from the LM-X15 after making the change.
No more packets received from a different interface. Mission accomplished.
So that was it. The error was gone and we could move along with the project.
Conclusion
Well, I should have know as normally I do put those networks not just in a separate subnet but also make sure they are on different VLANs. This goes to show that no matter how experienced you are and how well you prepare you will still make mistakes. That’s normal and that’s OK, it means you are actually doing something. Key is how you deal with a mistake and that why I wrote this. To share how I found out the root cause of the issue and how I fixed it. Mistakes are a learning opportunity, use them as such. I know many organizations frown upon a mistake but really, these should grow up and don’t act this silly.
Last week, around August 26-27th 2020 Custom Route Tables in Azure Virtual WAN lit up in my Azure Tenants. Awesome news. Normally this should have happened the week of the 3d of August 2020. However, some delay happened. Now it is here is has come in silence. Which I find odd. This is a major capability that offers so much of what we need to make Azure Virtual WAN shine. But it is here, ready to shine at Microsoft Ignite
Watch my video on custom route tables in Azure virtual WAN
Custom Route Tables in Azure Virtual WAN
What do we have now? You can read up on Azure Virtual WAN route tables over here. I have made a video about all this which you can find on my blog and on my Vimeo channel. Please take a look for some walkthroughs and links to some other blog posts by me on Azure Virtual WAN.
LabELs
First of all, let’s discuss the labels. Labels logically group route tables. These are very helpful when propagating routes from connections to multiple route tables. The Default Route Table has a built-in label called ‘Default’. When you propagate connection routes to the ‘Default’ label, it automatically applies to all the Default Route Tables across every hub in the Virtual WAN.
Creating a label
Associations
Now, we can discuss associations. Each connection is associated with one route table. This means that the connection can send to the destination indicated as routes in the route table it is associated with. The routing configuration of the connection will show the associated route table. This is very important for connected VNETs. Multiple connections can be associated with the same route table. Note that all VPN, ExpressRoute, and User VPN connections are associated with the same (default) route table.
Association
By default, all connections are associated with the Default route table in a virtual hub. Each virtual hub has its own Default route table. You can add one or more static routes to the default Route table. Static routes take precedence over dynamically learned routes for the same prefixes.
Propagations
Last but not least, connections dynamically propagate routes to one or more route table. VPN, ExpressRoute, and User VPN connections propagate routes to the same set of route tables. With connections like a Site-2-Site VPN, Express Route, or Point-2-Site VPN, routes are propagated from the virtual hub to the on-premises router using BGP.
Propagations for BranchesPropagation for a connected VNET
A “None” route table is also available for each virtual hub. Propagating to the None route table implies that no routes are propagated from the connection.
Some need to ask
Finally, some customers need to reach out to support in order to get Azure Virtual WAN Custom route tables to light up.
Contact Microsoft support if and when needed to enable custom route tables for you.
As a result, I suggest you do so to start kicking the tires and then dive in deeper. This is a cornerstone technology for Azure networking going forward.
Automation
I have not found any documentation or guidance in regards to automation with PowerShell, Azure CLI, or ARM templates yet. I expect this to be forthcoming as this is much needed. As a result, I hope we’ll see this by Microsft Ignite 2020.
Conclusion
Azure Virtual WAN with the secured Virtual Hub and custom route tables offers the capabilities we have been waiting for. With these capabilities in place. Azure Virtual WAN is the future of Azure virtual networking. Therefore, I fully expect to hear a lot more about it during Microsoft Ignite in September. I personally will focus on this part of networking in the coming months. It is a stock part of any Azure initiative and project in the near future.
When it comes to Azure Virtual WAN, you might have the impression it is only useful for huge, international entities. Entities like the big Fortune 500 companies, with a significant, distributed global presence.
I can understand why. That is where the attention is going, and it makes for excellent examples to showcase. Also, the emphasis with SD-WAN has too often been about such cases. SD-WAN also enables economically feasible, reliable, and redundant connectivity for smaller locations and companies than ever before. My take is that Azure Virtual WAN is for everyone!
Azure Virtual WAN is for everyone
I would also like to emphasize that Azure Virtual WAN is so much more than just SD-WAN. That does not distract from SD-WAN’s value. SD-WAN is a crucial aspect of it in terms of connectivity to and from your Azure environment. I would even say that the ability to leverage Microsoft’s global network via Azure Virtual WAN is the most significant force multiplier that SD-WAN has gotten in the past year.
Network appliance vendors are signing on to integrate with Azure Virtual WAN for a good reason. It makes sense to leverage one of the biggest, best, and fastest global networks in the world to provide connectivity for your customers.
One extreme use case would be to use Azure Virtual WAN only as an SD-WAN carrier just to connect your sites without using anything in Azure. An example of this would be a business that is still on-prem but wants to move to Azure. That is a good start. It modernizes connectivity between the locations while becoming ready to move workloads to Azure, where the landing zone is integrated into Azure Virtual WAN when it is time to do so.
A Medium Enterprise example
But let’s step back a minute. The benefits of Azure Virtual WAN go beyond SD-WAN deployments for multinational companies spanning the globe. Make no mistake about this. SD-WAN is also very interesting for Small and Medium Enterprises (SME), and the benefits of Azure Virtual WAN go beyond on-premises to Azure connectivity. It extends to connecting any location to any location.
SD-WAN leveraging Azure Virtual WAN and the Microsoft Global Network
On-premises connectivity is more than a data center, a corporate HQ, and branch offices with ExpressRoute and/or Site-to-Site VPN (S2S). It is also a user via a Point-to-Site VPN (P2S). All of these can be anywhere in the world but also distributed across your city, country, or continent. Think about what that means for “remote work by default” shops. Every individual, whether working with you as an employee, partner, customer, consultant or contractor, can be connected to your Azure virtual WAN and your on-premises locations thanks to the any-to-any connectivity.
Some people might have an NGFW at home, depending on their role and needs. Many others will be fine with a point-to-site VPN, which serves both work-from-home profiles as well as road warriors.
People, if this Coronavirus global pandemic has not awakened you to this importance and possibility of remote work, I do not know what to tell you. Drink a lot more coffee?
For example, a national retailer, a school, a medical provider with lots of small local presences can all benefit from Azure Virtual WAN. When they merge with others, within or across the borders, Azure Virtual WAN with SD-WAN puts them in a great position to extend and integrate their network.
There is more to Azure Virtual WAN than SD-WAN
We have not touched on the other benefits Azure Virtual WAN brings. These benefits are there, even if you have no on-premises locations to connect. That would be another extreme, Azure Virtual WAN without any SD-WAN deployment. While the on-premises deployment of apps goes down over time, it will not go ways 100% for everyone. Also, even in a 100% cloud-native environment, having other connectivity options than over the internet and public services can help with security, speed, and cost reduction.
The Any-to-Any capabilities, the ease of use, leading to operational cost saving, are game-changing. Combined with the integration with Azure Firewall manager to create a Secure Virtual HUB and custom routing, it makes for a very flexible way of securing and managing network access and security.
Hybrid scenarios
Don’t think that SMEs will only have 2 to 5 subscriptions, or even less if they are just consumers of cloud services outsourced to a service provider, with one or a couple of vNETs.
If you do not have many subscriptions, you can still have a lot of vNETs. You create vNETs per application, business unit, etc. On top of that, in many cases, you will have development, testing, acceptance, and production environments for these applications.
You might very well do what we do, and what we see more of again, lots of subscriptions. You can create subscriptions for every application environment, business unit, etc. The benefits are clear and easy to measure distinction in ownership, responsibilities, costs, and security. That means a company can have dozens to hundreds of subscriptions that way. These can all have multiple vNETs. When an SME wants to protect itself against downtime, two regions come into play. That means that the hub-to-hub transitive nature excels.
Azure Virtual WAN – Hybrid scenario
Now, managing VNET peering, transit vNETs, Network Gateways, Firewalls, and route tables all become a bit of a chore fast when the environment grows. Rolling all that work into a convenient, centralized virtual global service makes sense to reduce complexity, reduce operational costs, and simplify your network architecture and design.
Going cloud first and cloud native
In a later stage, your organization can reduce its on-premises footprint and go for an all cloud-based approach. Be realistic, there might very well be needs for some on-premises solutions but Azure Stack has you covered there. You can leverage Azure Stack HCI, Edge, or even hub or those needs but still integrate deployment, management, operations, and monitoring into Azure.
Azure Virtual WAM – Cloud first scenario
Global Transit Architecture with Azure Virtual WAN
I still need to drive the capabilities and benefits of the Global Transit Architecture with Azure Virtual WAN home for you. For one, it is any-to-any by default. You can control and limit this where needed, but it works automagically for you out of the box. Second, this is true for ExpressRoute, S2S VPN, P2S VPN, VNET peers, and virtual hubs in all directions.
Branch-to-VNet
Branch-to-branch
ExpressRoute Global Reach and Virtual WAN
Remote User-to-VNet
Remote User-to-branch
VNet-to-VNet
Branch-to-hub-hub-to-Branch
Branch-to-hub-hub-to-VNet
VNet-to-hub-hub-to-VNet
This means that a user with a P2S VPN connected to a virtual hub has access to a datacenter that connects to that same hub or another one within the same Virtual WAN. You can go crisscross all over the place. I love it. Remember that we can secure this, control this.
Any-to-Any – crisscross along locations and connection types – Image adapted from MSFT
Think about that for a moment. When I am on the road connected via a P2S VPN to an azure virtual hub, I can reach my datacenter (ExpressRoute), my office, store, factory, and potentially even my home office (S2S VPN). Next to that, I can reach all my vNETs. It is the same deal when I am working from home or in the office, store, or factory. That is impressive. The default is any-to-any, automagically done for you. But you can restrict and secure this to your needs with custom routing and a secure virtual hub (Azure Firewall Manager).
Conclusion
The benefits of Azure Virtual WAN are plenty, for many scenarios in large, medium and small enterprises. So, I invite you to take a better look. I did. As a result, I have been investing time in diving into its possibilities and potential. I will be presenting on this topic to share my insights into what, to me, is the future of Azure networking. Do not think this is only for the biggest corporations or organizations.
Cookie Consent
We use cookies to improve your experience on our site. By using our site, you consent to cookies.
Websites store cookies to enhance functionality and personalise your experience. You can manage your preferences, but blocking some cookies may impact site performance and services.
Essential cookies enable basic functions and are necessary for the proper function of the website.
Name
Description
Duration
Cookie Preferences
This cookie is used to store the user's cookie consent preferences.
30 days
These cookies are needed for adding comments on this website.
Name
Description
Duration
comment_author_email
Used to track the user across multiple sessions.
Session
comment_author_url
Used to track the user across multiple sessions.
Session
comment_author
Used to track the user across multiple sessions.
Session
These cookies are used for managing login functionality on this website.
Name
Description
Duration
wordpress_logged_in
Used to store logged-in users.
Persistent
wordpress_sec
Used to track the user across multiple sessions.
15 days
wordpress_test_cookie
Used to determine if cookies are enabled.
Session
Statistics cookies collect information anonymously. This information helps us understand how visitors use our website.
Google Analytics is a powerful tool that tracks and analyzes website traffic for informed marketing decisions.
Used to monitor number of Google Analytics server requests when using Google Tag Manager
1 minute
__utmx
Used to determine whether a user is included in an A / B or Multivariate test.
18 months
_ga
ID used to identify users
2 years
_gali
Used by Google Analytics to determine which links on a page are being clicked
30 seconds
_ga_
ID used to identify users
2 years
_gid
ID used to identify users for 24 hours after last activity
24 hours
__utma
ID used to identify users and sessions
2 years after last activity
__utmt
Used to monitor number of Google Analytics server requests
10 minutes
__utmb
Used to distinguish new sessions and visits. This cookie is set when the GA.js javascript library is loaded and there is no existing __utmb cookie. The cookie is updated every time data is sent to the Google Analytics server.
30 minutes after last activity
__utmc
Used only with old Urchin versions of Google Analytics and not with GA.js. Was used to distinguish between new sessions and visits at the end of a session.
End of session (browser)
__utmz
Contains information about the traffic source or campaign that directed user to the website. The cookie is set when the GA.js javascript is loaded and updated when data is sent to the Google Anaytics server
6 months after last activity
__utmv
Contains custom information set by the web developer via the _setCustomVar method in Google Analytics. This cookie is updated every time new data is sent to the Google Analytics server.
2 years after last activity
_gac_
Contains information related to marketing campaigns of the user. These are shared with Google AdWords / Google Ads when the Google Ads and Google Analytics accounts are linked together.
