Head: Meet Wall.

I spent way more time than I’m comfortable disclosing, troubleshooting an issue with an AD-attached Oracle Linux server that wouldn’t accept ssh logons by domain users.

We use the recommended sssd and realmd to ensure AD membership. Everything looked good, and I could log on using an account that’s a member of the Domain Admins group, and so I released the machine to our developers for further work.

Only they couldn’t log on.

After spending most of the morning looking through my logs and config files, and detaching and re-attaching the server to the domain after tweaking various settings, I suddenly saw the light.

Note to my future self:

Windows runs NetBIOS under the hood! Any machine name over 14 characters of length in a domain joined computer will cause trouble!

Naturally, after setting a more Windows-like hostname and re-joining the domain, everything worked as I expected.

Restoring a really old domain controller from backups

I had an interesting experience this week, where I was faced with the need to restore an entire Active Directory environment from backups that were more than a year old.

The company whose servers I was restoring had been using an older version of Veeam Backup and Recovery, which always simplifies matters a lot: The entire thing was delivered to me over sneaker net, on a 2.5″ USB drive containing several restore points for each machine.

The restore was uneventful, as expected, and most machines simply started up in their new home. Unfortunately, one of the Active Directory controllers would bluescreen on boot, with a C00002E2 error message.

After some reading up on things, I realized the machine had passed the Active Directory tombstone period: as I wrote, the backups were taken over a year ago. Since I had one good domain controller, I figured I would simply cheat with the local time on the failing DC. It would boot successfully into Directory Services Recovery Mode, so I could set the local clock, but anybody who has a bit of experience with the VMware line of virtualization products knows that by default, VMware ESXi synchronizes the guest system clock in a few situations; amongst them on reboot.

Fortunately VMware has a knowledgebase article covering how to disable all synchronization of time between guests and hosts. A total of eight advanced settings must be set to False, with the guest turned off:


The procedure is documented in KB1189.

After setting these properties on the machine, I started it back up, with the system time set well back into the range before the tombstone cutoff date, let it start up and rest for a while for all services to realize everything was alright, and then I set the time forward to the current date, waited a bit longer, and restarted the VM. After this, the system started working as intended.