Performance Series | EP7, Windows Memory Architecture. How IT Pros (SHOULD) Troubleshoot Slow PC’s and Servers

Episode 7, Windows Memory Architecture. Performance Series.

In episode 7 we’ll start talking about “Fake Memory” as I’ve been eluding to for several episodes. We cover the microkernel memory architecture of 32 and 64 bit operating systems, the Virtual Memory Manager, and why user mode processes sometimes crash when they exceed it. We cover this in a whiteboarding session and then move into a demo of an actual application crash, and show what a typical crash looks like in Perfmon.

Performance Series | EP6, Dude… Where’s My RAM? How IT Pros (SHOULD) Troubleshoot Slow PC’s and Servers

In episode 6 we’ll finish off the RAM and Pagefile discussion, then move on to leaks of just RAM. We cover the differences between working set and private bytes counters in the process object. Then we crash our test box again by setting a process lose to go suck up all the Available Mbytes counter and then investigate what that looks like both live and in a capture.

Performance Series | EP5, Tracking RAM and Page File Exhaustion: How IT Pros (SHOULD) Troubleshoot Slow PC’s and Servers

Episode 5, Tracking RAM and Page File Exhaustion. Performance Series.

In episode 5 we start the discussion of memory leaks. This is a topic that will span several issues. Memory can mean the virtual address space of a process or the kernel itself, or it could mean you ran out or RAM. Or, you ran out of RAM and Page File. How do you tell which one? How do you figure out which process stole all your precious memory. Specifically in this episode we’ll be talking about what it looks like when you run out of both RAM and Page File (the most common type of process memory leak). We also cover the difference between committed bytes of a process verses private bytes. Finally, we cover concepts like the Virtual Memory Manager of Windows. Trimming of the working set. What the page file really is, etc. The next episode will continue with a discussion more related to the working set.

Performance Series | Episode 4, Troubleshooting Disk Bottlenecks. Performance Series.

In episode 4 we cover the first of the actual counter sets used to identify problems, and the methodology to do so. Identifying a disk bottleneck involves more than just latency counters, it also includes ruling out whether your server or PC was actually the culprit. So what we will do in this episode is load up a computer with a lot of disk activity, then additionally load up the hypervisor that runs that computer and compare the differences. Finally, we will do some process analysis and determine who the bad guy was.

Performance Series | EP3, Loading and Interpreting Counters: How IT Pros (SHOULD) Troubleshoot Slow PC’s and Servers

Episode 3, Loading and Interpreting Counters. Performance Series.

In episode 3 we finish up with the intro to perfmon, such as how to actually load the data collector sets we captured. We then cover some of the basics of interpreting the counters (scaling graphs, scaling counters, looking for patterns, and zooming into the problem).

Performance Series | EP2, Perfmon Unleashed: How IT Pros Troubleshoot Slow PC’s and Servers (or, how they SHOULD)

In episode 2 we continue working with some of the more advanced features of Perfmon, such as setting up data collector sets, managing Perfmon through the command line with logman, and integration with task scheduler.

Performance Series | EP1, The Basics: How IT Pros Troubleshoot Slow PC’s and Servers (or, how they SHOULD)

Episode 1, The Basics. Performance Series.

Why is your computer/PC/server slow? Well… I feel like computer and server performance troubleshooting still remains a mystery to not only the average Joe who is good at computers, but also IT professionals with years of field experience.

There are a ton of misunderstood concepts. There are also a ton of unknown or underutilized free tools available. Many of which are built into the OS. And sadly when it comes to learning how to use them, you’ll find misinformation and bad advice at every corner of the internet. (Oh, and a slew of shady companies promising to “fix” your issues for you magically using only your credit card.)

In this series I will cover the basics of determining why your system is slow. It all starts with understanding what part of the computer is giving you trouble (processor, memory, disk, network) and then moving on to pinpointing what application or service is the culprit.

We start with some really basic tools like Task Manager and Resource Monitor. But then we get into the meat of it with Perfmon (Performance Monitor). Perfmon is a terrific tool and anybody that’s “good at computers” will have no issue using… The tool is very powerful and intimidating, but easily conquerable.

Finally, this series will tell you about what counters to use, when to use them, what their thresholds are, and why they are important. The troubleshooting flow is easy. We first cover the key indicators of a problem that will point you to disk, memory, processor, or network. Once you know that, we’ll cover what counters to use to figure out which process is causing it.

This is a multi-part series I have been wanting to do for a long time. I hope you enjoy it!

No Dll or exported function was found to verify revocation

I’m writing this as much to myself as to everyone else, because this is the second time I’ve run across it at a customer site and then (because my memory doesn’t seem to keep this one in cache) have to go hunt it down again.

Anyway, you might get the error “No Dll or exported function was found to verify revocation” along with an error at the GINA (logon screen) saying, “The system could not log you on. The revocation status of the domain controller certificate used for smart card authentication could not be determined. There is additional information in the system event log. Please contact your system administrator.”

Most likely if you saw the first error you’ve already enabled CAPI2 logging. Also, it is likely that the server you’re on used to have a third party CAPI2 provider (such as Desktop Validator or Tumbleweed). If this is the case, the uninstall may have damaged Cryptographic Services. Have no fear, it’s fixable.

First, make sure all the DLL’s are still registered in Windows. This can be done by running (or copy/pasting) the following at an administrative command prompt:

regsvr32 CERTCLI.DLL
regsvr32 CRYPTDLG.DLL
regsvr32 CRYPTEXT.DLL
regsvr32 CRYPTNET.DLL
regsvr32 CRYPTUI.DLL
regsvr32 SOFTPUB.DLL
regsvr32 WINTRUST.DLL
regsvr32 INITPKI.DLL
regsvr32 DSSENH.DLL
regsvr32 RSAENH.DLL
regsvr32 GPKCSP.DLL
regsvr32 SCCBASE.DLL
regsvr32 SLBCSP.DLL
regsvr32 CRYPTDLG.DLL

There is also a registry key that may not have been set back after the 3rd party software uninstall. If so, it may still be holding on to the old DLL (which is tmwdcapiclient.dll for instance with Tumbleweed).
HKEY_LOCAL_MACHINE\Software\Microsoft\Cryptography\OID\Encoding Type 1\CertDllVerifyRevocation\Default
And:
HKEY_LOCAL_MACHINE\Software\Wow6432Node\Microsoft\Cryptography\OID\Encoding Type 1\CertDllVerifyRevocation\Default

Which should be:
REG_MULTI_SZ:  Dll
Value data:  Cryptnet.dll

It is possible that you’ll see both the cryptnet.dll AND the 3rd party dll together or just the 3rd party one. In either case, remove them. If all you see is cryptnet you’re fine.

A reboot seems to be required. Restarting cryptographic services alone had no effect for me.

***NOTE*** If you still intend to use a 3rd party CAPI provider, you may want to avoid these directions as it will likely disable the program in favor of Windows’ built-in mechanisms. Try re-installing that software to fix your issue.

Let’s Tech: Hyper-V Replica & Azure, Recent Updates


In this episode of Let’s Tech I cover some of the recent changes to Azure and Hyper-V Replica integration. For instance, no more SCVMM requirements. This video covers mostly just the changes so if you want more details:
This is the initial Azure video:

And this was the initial Replica introduction I did in my “What’s new in Windows Server 2012” video series: