Performance Series | EP1, The Basics: How IT Pros Troubleshoot Slow PC’s and Servers (or, how they SHOULD)

Episode 1, The Basics. Performance Series.

Why is your computer/PC/server slow? Well… I feel like computer and server performance troubleshooting still remains a mystery to not only the average Joe who is good at computers, but also IT professionals with years of field experience.

There are a ton of misunderstood concepts. There are also a ton of unknown or underutilized free tools available. Many of which are built into the OS. And sadly when it comes to learning how to use them, you’ll find misinformation and bad advice at every corner of the internet. (Oh, and a slew of shady companies promising to “fix” your issues for you magically using only your credit card.)

In this series I will cover the basics of determining why your system is slow. It all starts with understanding what part of the computer is giving you trouble (processor, memory, disk, network) and then moving on to pinpointing what application or service is the culprit.

We start with some really basic tools like Task Manager and Resource Monitor. But then we get into the meat of it with Perfmon (Performance Monitor). Perfmon is a terrific tool and anybody that’s “good at computers” will have no issue using… The tool is very powerful and intimidating, but easily conquerable.

Finally, this series will tell you about what counters to use, when to use them, what their thresholds are, and why they are important. The troubleshooting flow is easy. We first cover the key indicators of a problem that will point you to disk, memory, processor, or network. Once you know that, we’ll cover what counters to use to figure out which process is causing it.

This is a multi-part series I have been wanting to do for a long time. I hope you enjoy it!

No Dll or exported function was found to verify revocation

I’m writing this as much to myself as to everyone else, because this is the second time I’ve run across it at a customer site and then (because my memory doesn’t seem to keep this one in cache) have to go hunt it down again.

Anyway, you might get the error “No Dll or exported function was found to verify revocation” along with an error at the GINA (logon screen) saying, “The system could not log you on. The revocation status of the domain controller certificate used for smart card authentication could not be determined. There is additional information in the system event log. Please contact your system administrator.”

Most likely if you saw the first error you’ve already enabled CAPI2 logging. Also, it is likely that the server you’re on used to have a third party CAPI2 provider (such as Desktop Validator or Tumbleweed). If this is the case, the uninstall may have damaged Cryptographic Services. Have no fear, it’s fixable.

First, make sure all the DLL’s are still registered in Windows. This can be done by running (or copy/pasting) the following at an administrative command prompt:

regsvr32 CERTCLI.DLL
regsvr32 CRYPTDLG.DLL
regsvr32 CRYPTEXT.DLL
regsvr32 CRYPTNET.DLL
regsvr32 CRYPTUI.DLL
regsvr32 SOFTPUB.DLL
regsvr32 WINTRUST.DLL
regsvr32 INITPKI.DLL
regsvr32 DSSENH.DLL
regsvr32 RSAENH.DLL
regsvr32 GPKCSP.DLL
regsvr32 SCCBASE.DLL
regsvr32 SLBCSP.DLL
regsvr32 CRYPTDLG.DLL

There is also a registry key that may not have been set back after the 3rd party software uninstall. If so, it may still be holding on to the old DLL (which is tmwdcapiclient.dll for instance with Tumbleweed).
HKEY_LOCAL_MACHINE\Software\Microsoft\Cryptography\OID\Encoding Type 1\CertDllVerifyRevocation\Default
And:
HKEY_LOCAL_MACHINE\Software\Wow6432Node\Microsoft\Cryptography\OID\Encoding Type 1\CertDllVerifyRevocation\Default

Which should be:
REG_MULTI_SZ:  Dll
Value data:  Cryptnet.dll

It is possible that you’ll see both the cryptnet.dll AND the 3rd party dll together or just the 3rd party one. In either case, remove them. If all you see is cryptnet you’re fine.

A reboot seems to be required. Restarting cryptographic services alone had no effect for me.

***NOTE*** If you still intend to use a 3rd party CAPI provider, you may want to avoid these directions as it will likely disable the program in favor of Windows’ built-in mechanisms. Try re-installing that software to fix your issue.

Let’s Tech: Hyper-V Replica & Azure, Recent Updates


In this episode of Let’s Tech I cover some of the recent changes to Azure and Hyper-V Replica integration. For instance, no more SCVMM requirements. This video covers mostly just the changes so if you want more details:
This is the initial Azure video:

And this was the initial Replica introduction I did in my “What’s new in Windows Server 2012” video series:

Let’s Tech: How to Set Up RemoteApp on Windows Azure

http://youtu.be/N7lNglzSbHc

In this episode of Let’s Tech, I create a new VM for use in RemoteApp for Windows Azure in the cloud, configure it, show you the gotcha’s to watch out for, give you some extra tips, then show you how to import it into RemoteApp, configure it, and finally publish your apps. This also covers how to customize the image after the fact without having to re-upload your gold image.

Let’s Tech: Windows Azure with Hyper-V Replica (Step-By-Step) Creating a Virtual Machine Cloud DR

In this episode of “Let’s Tech” I take a couple of Hyper-V Virtual Machines and enable them for Replica, but this time to the Cloud using Microsoft Azure. I cover the setup, planned and unplanned failovers, DR testing, and a host of other options. This is an incredibly easy BC DR solution that can help you get your feet wet with IaaS. Admittedly… it is a pretty long video but this is a huge topic. Worth the time if you’re wanting to extend your VM’s beyond your on-premises (on-prem) datacenter. We frequently call this “extending your private cloud”.

http://youtu.be/URY8CA2QOVQ

Populate “Terminals” Favorites XML Using A.D. – and keep your OU’s!

So a quick little bit of background on this one. If you’re already using RDCMan (a.k.a. Remote Desktop Connection Manager) you’ve probably noticed it hasn’t been updated in a while. It isn’t an “official” Microsoft project, it isn’t open source, and the last version of it was released quite a while ago.

I won’t go into why, let’s just say that it wasn’t *ever* an official product and they guy who wrote it didn’t really ever expect it to get as large as it got.

There is a similar product that is still being actively updated. It is called Terminals, it lives at codeplex, and it is open source. Which means that if there are problems with it in the future, anybody can help resolve them.

One major issue I ran into was organization. You can import computers from AD or you can import computers from an existing RDG file (RDCMan’s database) but you cannot get the groups to import this way. I manage thousands of servers so just having a big flat file wasn’t going to work.

I wrote a PowerShell script to help you to populate a Terminals XML Favorites file using your Active Directory OU structure. You can download it here (right-click and then save as). Hope it helps!

Let’s Tech: What’s New In Windows Server Technical Preview

Video Link:

http://youtu.be/yZqad04CQxQ

In this episode of Let's Tech, will it be called Windows Server 10? Who knows, but I downloaded a copy and fired it up. The commentary is about the new features and in the video we take a look at the new Desktop and the new Start Menu (not a start screen in this version). This is a pretty quick video and it is at a technical level of less than 100, just a quick look at the new interface.

Performance Series: Why Is My DC Slow?

I’m starting a new grouping of videos called the “Performance Series” which will be a break-out of the “Let’s Tech” videos. I’m releasing the first two videos in this series today, and I’m really excited about them. Hope you enjoy!

In this two part Performance Series on slow Domain Controllers I cover how to identify causes of slow DC's, how to spot and fix bad LDAP queries and searches against the directory, run diagnostics on Active Directory, what tools to use and when. Then I cover how to fix various problems by optimizing queries, indexing attributes, and when different solutions are more appropriate.

Part 1:
http://youtu.be/28pWAxR1zv0
Part 2:
http://youtu.be/s1aHH2E733I

Let’s Tech: Import Lots of Users into AD For Testing

V-Log Post Available Here:

http://youtu.be/cj1OsUqJaTM

Welcome to another Let's Tech… I don't particularly like having user1, user2, user3, user4 as a bed of test users. So in this episode I'll show you a more interesting way to populate your pre-prod environment. This I am doing ahead of another Let's Tech episode that I am working on about AD Performance, LDAP query troubleshooting, and how to brow beat lazy programmers. Enjoy!

Active Directory ACL’s Randomly Revert

I ran into a strange one recently. I have been trying to delegate some permissions to people to manage objects in AD. For *some* but not all, after I’d make the change, the ACL’s were reverting back. Sometimes in a few minutes and sometimes nearly an hour later. I thought I had a replication issue but… no other issues were present and nothing else was reverting back. I was pounding my head against the wall (as usual, because it is so helpful) and there was an alarm ringing in the back of my head, “Chris, you’ve heard of this before” but it finally just clicked yesterday.

Turns out that a whole handful of users in my org belonged to the “Server Operators” built-in security group. This is a protected group, as are:

  • Account Operators
  • Administrator
  • Administrators
  • Backup Operators
  • Domain Admins
  • Domain Controllers
  • Enterprise Admins
  • Krbtgt
  • Print Operators
  • Read-only Domain
  • Controllers
  • Replicator
  • Schema Admins
  • Server Operators

I wasn’t aware that Server Operators was now protected (initially they weren’t), but I guess that happened in 2000 SP4. Oops, missed that on an MCSE test exam question somewhere along the line I’m sure.

Anyway, here’s what is happening. A protected group gets its ACL’s reset back automatically (if they’ve changed) once per hour. The template for this reset is called AdminSDHolder and is located at CN=AdminSDHolder,CN=System,DC=mydomain,DC=com.

So if, for instance, you change the ACL’s on somebody’s user account and give some other group the ability to reset the password for that object… one hour later that permission will disappear and the group won’t have access to reset the password. But only if that user account you tried to change is a member of one of the protected groups above.

The PDC Emulator runs this job once every 60 minutes. So after making the change, you won’t even notice that the problem for up to an hour. If fact, you’ll think you finished the task.

To make things worse, the box for inheriting permissions will uncheck as well – meaning if you tried to delegate a whole OU it will work for some accounts and not for others.

So… how to fix it? For me I just yanked them out of the Server Operators group and that fixed it. But if you need to keep the security group mentioned above, you have two options. Modify AdminSDHolder with new permissions or take the group in question out from under the protection of AdminSDHolder. Ned (as usual) has a good article about these options here so no need to go into them and re-invent the wheel.