How Time broke AD (and Group Policy saved the day)

I was recently discussing Time Configuration in Active Directory with another engineer and promised to send along details on how I like to configure Domain Controllers. Instead of sending along some links, I decided to add a post to talk a bit about Time Configuration settings on Domain Controllers and automation using Group Policy.

 

Most IT organizations have at least one story of “that one time” when the clocks skewed on all of their DCs or resource outages happened due to Time issues….or perhaps worse: parts of the organization unpredictably had access problems and inconsistent time.

One of the common scenarios looks something like this:

  1. The Domain Controller owning the PDC FSMO role in the root domain was taken offline, replaced, or failed.
  2. Time Configuration was never updated on the new PDC to take over external NTP sync functions.
  3. Even worse, the new PDC is a Virtual Machine and is now only getting time from the free-running system clock (in this case software) and begins skewing very quickly.

The recommendation I, and many of my peers, give is to automate the Time Configuration of Domain Controllers. This way, we can ensure that the NTP configuration follows the owner of the PDC FSMO role and minimizes the manual efforts required.

As a standard configuration, the Domain Controller holding the PDC Emulator FSMO role in the ROOT domain should be configured to receive time via NTP from an external time source (note: not the PDC in any child domains, only the Forest Root). In addition, all other Domain Controllers in all other Domains are configured to use NT5DS. This is normally (at least prior to using GPOs) controlled by manually running the w32time utility to reconfigure new/old PDCs.

However, these settings can be automated and enforced by using a combination of two GPOs and a WMI Filter.

 

WMI Filter

This WMI filter is used to filter a PDC targeted GPO and will only be true on the PDC. Note that it can be used on a GPO in any domain against any domain’s PDC, but we’re specifically talking about the Root PDC in this scenario.

Name: PDC

Namespace: root\CIMv2

Filter: Select DomainRole from Win32_ComputerSystem Where DomainRole=5

To create:

  1. In the root domain, open the Group Policy Management Console.
  2. Right-click the WMI Filters container and select New
  3. At the New WMI Filter window, enter the name PDC and click Add
  4. At the WMI Query window, enter Select DomainRole from Win32_ComputerSystem Where DomainRole=5

 

Group Policies

These GPOs will be configured with settings located here:

Computer Configuration \ Policies \ Administrative Templates \ System \ Windows Time Service \ Time Providers \ Configure Windows NTP Client

 

GPO 1

This GPO sets NTP synchronization with a least one external time source. It is enforced (to keep the policy at highest precedence), but filtered with a WMI query to only apply to the PDCE role holder.

Name: [DC] W32Time – PDC – NTP

Time Server: ntp1.contoso.com,0x1 ntp2.contoso.com,0x2 (Note: Spaces are the delimiter)

Type: NTP

CrossSiteSyncFlags: 2 (default)

ResolvePeerBackoffMinutes: 15 (default)

ResolvePeerBackoffMaxTimes: 7 (default)

SpecialPollInterval: 900 (15 minutes; default in GPMC is 3600)

EventLogFlags: 0/1/2/3 (I prefer 3 for full logging)

WMI Filter: PDC

Security Filter: Domain Controllers

GPLinks: Domain Controllers OU – Enforced link

 

GPO 2

GPO configures DCs to use NT5DS with default settings. This will force DCs to synchronize up the AD hierarchy, eventually reaching the Forest Root PDC.

Name: [DC] W32Time – OtherDCs – NT5DS

Time Server: time.windows.com,0x9 (default)

Type: NT5DS

CrossSiteSyncFlags: 2 (default)

ResolvePeerBackoffMinutes: 15 (default)

ResolvePeerBackoffMaxTimes: 7 (default)

SpecialPollInterval: 900 (15 minutes; default in GPMC is 3600)

EventLogFlags: 0/1/2/3 (I prefer 3 for full logging)

WMI Filter: NONE

Security Filter: Domain Controllers

GPLinks: Domain Controllers OU

 

Implementation

The actual process to implement should look something like the following:

  1. Create the WMI filter
  2. Create GPO1 (PDC) in the Root Domain.
    1. Ensure the WMI filter is set
  3. Create GPO2 (Other DCs)
  4. Link GPO1 to the Domain Controllers OU and Enforce the Link
  5. Log on to the current PDC:
    • Run gpupdate /force,
    • Run gpresult /R
      • Verify that GPO1 is applied
    • Run w32tm /query /configuration
      • Verify that NTP configurations settings from the GPO are present and show (Policy)
  6. Log on to another DC
    • Run gpupdate /force,
    • Run gpresult /R
      • Verify that GPO1 is filtered out due to WMI (PDC filter)
  7. Link GPO2 to the Domain Controllers OU
  8. Redo steps 5 & 7
    • Ensure that GPO1 and GPO2 are both applied to the PDC
    • Ensure that w32tm command output shows the GPO1 settings as winning
    • Ensure that GPO2 is applied to the non-PDC Domain Controller and GPO1 is filtered by WMI

 

Some final notes on configuration

NTP Servers in the PDC GPO:

My preference is two have at least two (primary and backup) NTP appliances available internally. One in your primary data center and one in a secondary location. These will likely have better accuracy and ability to identify time skew issues from particular sources.

In addition to defining the specific time servers, you also can set the Flags (the ,0x# part of the server.domain.com,0x1). These flags change various NTP service behaviors. For the first NTP server in the list, my preferences is to use the 0x1 (use SpecialPollInterval setting) flag when querying time from an external time source and no flag when using an internal appliance / server. For the second, I generally prefer the 0x2 flag (use as fallback only). If you have the primary server configured with 0x1, it especially useful to have a secondary that does not also use the 0x1. This will enable time synchronization to happen immediately after the Windows Time service is started rather than having to wait until the SpecialPollInterval has passed. Note that this is even more important when the default 3600 (60m) is used as it can mean that your time GPO doesn’t appear to work when time just won’t synchronize after configuration…or at least not until 60 minutes is up. Which, as we all “know” as AD admins, if you try a fix and it doesn’t work in 30 minutes, it’s still broke ;-). To avoid that bit of panic and rolling back settings, make sure you identify an alternate server that doesn’t use the SpecialPollInterval and/or reduce the SpecialPollInterval to something more reasonable like 15 minutes.

Note: This is all discussed very well in the article Configure Your PDCE with Alternate Time Sources (added to references at the end).

 

NTP Type:

While the PDC should definitely be NTP, there are some other options for other Domain Controllers. In the end, the most important piece for is ensuring that all Domain Controllers have the same time…even if its the same Bad time. And it’s important to note that the PDC role shoudl be seized if the PDC is down for an extended period of time. Ideally, transfer the role if you know there will be an extended outage. If, however, you have something unexpected like a hardware failure…metadata cleanup. With this assumption, the classic AD time configuration usually works just fine. However, here’s a couple of alternates that I’ve seen implemented:

Alternate 1: Other DCs are configured to use AllSync instead of NT5DS. In this configuration, you would specify the same NTP servers in the Other DCs GPO as in the PDC GPO. In this configuration, all DCs would fall over to using the direct NTP servers for time when NT5DS does not work (ex: PDC is down). This has the benefit of all Domain Controllers able to get good time in the event of an unexpected PDC outage. Since they would all be getting time from the same sources the PDC would be using anyways, time should stay in sync until the PDC is brought back online.

Alternate 2: All DCs are configured to use NTP. In this scenario, there would be know following the PDC role as all DCs all receive valid external time.When done, the environment is generally one with Domain Controllers in a small number of locations. For example: an environment with all DCs in one of two datacenters with an NTP appliance in both and all DCs configured to point at both.

Importantly, however, note that it is more important in Active Directory to have consistent time than accurate time. In both alternate approaches, if the NTP server specified is an NTP pool (like pool.ntp.org), you run the risk that all DCs will pull time from a different NTP sources. This could be very bad if one or more of the external NTP servers in a pool starts advertising bad time. This could leave one or more Domain Controllers with skewed time and cause some really exciting and erratic authentication problems, service outages, and weird behavior.

 

MaxPosPhaseCorrection and MaxNegPhaseCorrection

Not as much of an issue anymore, but if you still have Windows 2003 Domain Controllers, particularly as your PDC, these should be changed from defaults. This can easily be done in these GPOs. Just look one folder higher in the Group Policy Editor and you’ll find Global Configuration Settings. You can configured a number of settings here, just make sure you have the defaults for a Windows 2008 R2 or higher Domain Controller…including 172,800 (48 hours) for the two settings mentioned.

 

Member Servers and Clients?

In some cases, you may also want to enforce NT5DS settings on members server and/or clients. This is doable and you can definitely create a GPO meant for clients/servers with settings to enforce NT5DS. However, be very cautious of manipulating the Global Configuration Settings when applying to clients as Windows may have different default values in newer version of the Operating System. This was true from Windows 2003 to 2008 / 2008 R2, and will certainly continue to change as better values are identified.

 

Finally, here’s some extra reading:

Configure time for PDC via Group Policy

Configure Your PDCE with Alternate Time Sources

Configuring the Time Service-Enabling the Debug Log

Preventing Large Time Offset Problems

Group Policy Settings Explained (Windows Time Service)

Restore Windows Time service on local computer to default settings

How to disable IPv6 or its components in Windows

 

Tagged with: , , ,
Posted in Active Directory, Group Policy, Windows Time
4 comments on “How Time broke AD (and Group Policy saved the day)
  1. Dak says:

    Should the Type setting in GPO2 above be NT5DS instead of NTP?

  2. […] a script that should only run from a single Domain Controller, I prefer using the method mention in this post to create a WMI filter for the PDC role holder and WMI filter the GPO to that […]

  3. […] approach to filter this to only be done by the PDC using the WMI filter mentioned in my other post here. You can also run this by hand or from a tools system as a user with delegated privileges to modify […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: