Fix Edge Crash After Update
Two days ago I was browsing with Edge, suddenly it started crashing out of blue. There was no Windows update, no system settings change, Edge just started showing the site crash page. Few minutes later, I could not open Edge anymore, not even got to see the site error page.
Since I didn’t make any system change, this might be caused by an update to Edge itself; looking at program list, it does appear Edge updated itself to 109.0.1518.78 right before the crash.
I didn’t give it too much thought, maybe it is a bad update, Microsoft will release a new version/patch soon. Two days later, there is no update, still cannot open Edge, searching internet didn’t reveal other people having similar issue from 109.0.1518.78, so it must be something unique to my machine. Time to dig in.
Solutions I tried (in the order) and failed:
- Rename Edge profiles folder: maybe the crash was caused by something in bookmarks, browsing history, or even extensions; so I tried rename Edge profile folders, but crash remains;
- Repair Edge: from Apps and Features, there is an option of Repair Edge; unfortunately it failed with Installer error 106; reboot and try repair again didn’t help;
- Force Uninstall: then I tried to uninstall Edge; unfortunately it also failed due to same crash;
Now I decided to take a deeper look, to investigate the activities from Edge right before it crashed. The best tool for that job would be Windows Performance Recorder, but since it would require me to install Windows SDK, I decided to use Process Monitor first, simpler to use, besides it doesn’t require install.
Process Monitor helped me find the culprit quicker than expected, the trace shows right before msedge.exe crashed, it was trying to load msedge.dll, many of read operations failed with error 0xC0000242:
Decode 0xC0000242 with certutil:
C:>certutil -error 0xc0000242
0xc0000242 (NT: 0xc0000242 STATUS_BAD_COMPRESSION_BUFFER) -- 3221226050 (-1073741246)
Error message text: The specified buffer contains ill-formed data.
CertUtil: -error command completed successfully.
I took a look at C:\Program Files (x86)\Microsoft\Edge\Application\109.0.1518.78\msedge.dll, although it has embedded digital signature, viewing the signature in File Explorer shows Windows couldn’t verify it:
So the theory is that msedge.dll on hard disk is corrupted, causing crash from msedge.exe. I need to find a good copy to replace it.
Fortunately Edge keeps a copy of the installer along with the version installed, I found C:\Program Files (x86)\Microsoft\EdgeUpdate\Download{56EB18F8-B008-4CBD-B6D2-8C97FE7E9062}\109.0.1518.78\MicrosoftEdge_X64_109.0.1518.78.exe, which apparently is the full installer of Edge version 109.0.1518.78. It also looks like to be a self-extract 7-Zip executable, as it can be opened with 7-zip, I was able to find a copy of msedge.dll inside it:
I extracted msedge.dll from the installer, looked at its digital signature in File Explorer, Windows was able to verify it:
I replaced C:\Program Files (x86)\Microsoft\Edge\Application\109.0.1518.78\msedge.dll (which causes 0xc0000242 in ReadFile as well as digital signature verification failure) with the one from C:\Program Files (x86)\Microsoft\EdgeUpdate\Download{56EB18F8-B008-4CBD-B6D2-8C97FE7E9062}\109.0.1518.78\MicrosoftEdge_X64_109.0.1518.78.exe (can be opened with 7-Zip), the crash goes away, Edge runs normally with all existing data intact.
How did original msedge.dll got corrupted in the first place? Was it a software bug caused by Edge update process, or it is a hardware failure, like from the 3-years old SSD?
Before replacing msedge.dll, I renamed it to msedge_bad.dll so I could look into it further. Trying to access the file confirmed it is not accessible, although this time certutil reported a NT status code STATUS_IN_PAGE_ERROR:
C:\Program Files (x86)\Microsoft\Edge\Application\109.0.1518.78>certutil -hashfile .\msedge_bad.dll sha256
CertUtil: -hashfile command FAILED: 0xc0000006 (NT: 0xc0000006 STATUS_IN_PAGE_ERROR)
CertUtil: The instruction at 0x%p referenced memory at 0x%p. The required data was not placed into memory because of an I/O error status of 0x%x.
It is interesting that compact.exe shows it is actually compressed:
C:\Program Files (x86)\Microsoft\Edge\Application\109.0.1518.78>compact .\msedge_bad.dll
…
240089032 : 137281536 = 1.7 to 1 X msedge_bad.dll
Of 1 files within 1 directories
1 are compressed and 0 are not compressed.
240,089,032 total bytes of data are stored in 137,281,536 bytes.
The compression ratio is 1.7 to 1.
Still no conclusion yet, but I am happy Edge no longer crashes and will monitor the SSD status going forward.
The Case of CoCreateInstance Failed Randomly with E_CLASS_NOT_REGISTERED
Last week I investigated an issue in the product where the calls made to CoCreateInstance failed randomly. There were some twists in the investigation; the result surprised almost everybody, and the process worth mentioning.
Since beginning the issue has been found for weeks, developers noticed that CoCreateInstance may randomly return error E_CLASS_NOT_REGISTERED, the “random” here means that it can happen on different code path, it can happen on arbitrary in-proc COM objects, it can happen on any thread, and it can recover automatically.
First, we ruled out the possibility of the COM objects were not registered, despite of what the error code says. We register these COM objects during product installation; they stay registered until product uninstall, so the issue must be somewhere else.
Next, we used Process Monitor captured trace log when we reproduced the issue. Process Monitor log shows that COM library was able locate the CLSID in registry, read out the value of InProcServer32 under it, and was able to open the DLL to map it into memory. However, it seems to be where COM library stopped, as there was no evidence from log that tells us the DllMain function from the DLL was ever called.
Here I started suspecting that somehow the NT loader actually failed to load the DLL. The best option of looking into NT loader issue is to enable show loader snaps with gflags. It is simple as run following command line and re-run the program within debugger:
gflags.exe -i program.exe +sls
After enabled show loader snaps and run the product again in WinDbg this time, I found following output from WinDbg:
1fac:133c @ 01643484 - LdrpInitializeNode - ERROR: Init routine 000000006CC81844 for DLL "C:\Program Files\Company\abc.dll" failed during DLL_PROCESS_ATTACH 1fac:133c @ 01643484 - LdrpProcessDetachNode - INFO: Uninitializing DLL "C:\Program Files\ Company\abc.dll" (Init routine: 000000006CC81844) 1fac:133c @ 01643484 - LdrpUnloadNode - INFO: Unmapping DLL "C:\Program Files\ Company\abc.dll" 1fac:133c @ 01643484 - LdrpLoadDllInternal - RETURN: Status: 0xc0000142 1fac:133c @ 01643484 - LdrLoadDll - RETURN: Status: 0xc0000142
Therefore, NT loader was able to map abc.dll into memory, but its initialize routine returned FALSE to loader, and loader in turned failed LoadLibrary. Normally in this scenario LoadLibrary will return error 1114, “A dynamic link library (DLL) initialization routine failed”.
That abc.dll was created with Visual Studio with default settings; it statically links to CRT with entry point of CRT function _DllMainCRTStartup. The next logic step is to use WinDbg to step through the function to find out where it failed.
Fortunately _DllMainCRTStartup is not complicated and I quickly narrowed the failure down to following function:
extern "C" bool __cdecl __vcrt_initialize_ptd()
{
__vcrt_flsindex = __vcrt_FlsAlloc(&__vcrt_freefls);
if (__vcrt_flsindex == FLS_OUT_OF_INDEXES)
{
return false;
}
if (!store_and_initialize_ptd(&__vcrt_startup_thread_ptd))
{
__vcrt_uninitialize_ptd();
return false;
}
return true;
}
FlsAlloc returned FLS_OUT_OF_INDEXES, which in turned failed the CRT initialization. A little research shows that Fiber Local Storage has 128 slots per process; each DLL statically linked to CRT will take a slot during CRT initialization. There is possibility that if process has loaded too many DLLs and exhausted FLS slots, then new DLL statically linked to CRT will fail to load, and that is exactly what we ran into.
Since we knew what is happening, a solution is working in progress.
Convert number to literal string
Here is a little function that translates numbers like 123 into literal strings like “one hundred twenty three”, the result is in English only:
#include <string>
#include <map>
#include <stack>
#include <iostream>
#include <stdint.h>
using namespace std;
string num2str(uint64_t num) {
using mapping_t = map<uint64_t, string>;
static mapping_t mapping = { {0, "zero"},{1, "one"}, {2, "two"}, {3, "three"},{4, "four"},
{5, "five"},{6, "six"},{7, "seven"},{8, "eight"},{9, "nine"},
{10, "ten"}, {11, "eleven"}, {12, "twelve"}, {13, "thirteen"},
{14, "fourteen"}, {15, "fifteen"}, {16, "sixteen"}, {17, "seventeen"},
{18, "eighteen"}, {19, "nineteen"}, {20, "twenty"}, {30, "thirty"},
{40, "fourty"}, {50, "fifty"}, {60, "sixty"}, {70, "seventy"},
{80, "eighty"}, {90, "ninety"}, {100, "hundred"}, {1000, "thousand"},
{1000000, "million"}, {1000000000, "billion"}, { 1000000000000, "trillion"} };
string str;
if (num == 0) {
str = mapping[0];
}
else {
enum DATA_FLAGS {
FLAG_REMAINING = 1,
FLAG_UNIT
};
using stack_t = stack<uint64_t>;
stack_t s;
s.push(num);
s.push(FLAG_REMAINING);
while (!s.empty()) {
uint64_t flag = s.top();
s.pop();
uint64_t data = s.top();
s.pop();
if (flag == FLAG_UNIT) {
str += " ";
str += mapping[data];
} else {
if (data <= 20) {
if (!str.empty())
str += " ";
str += mapping[data];
} else if (data < 100) {
uint64_t tens = (data / 10) * 10;
uint64_t digits = data % 10;
if (!str.empty())
str += " ";
str += mapping[tens];
if (digits > 0) {
if (!str.empty())
str += "-";
str += mapping[digits];
}
}
else {
for (mapping_t::reverse_iterator it_last = mapping.rbegin(); it_last != mapping.rend(); ++it_last) {
uint64_t base = it_last->first;
if (data >= base) {
uint64_t units = data / base;
uint64_t remaining = data % base;
s.push(remaining);
s.push(FLAG_REMAINING);
s.push(base);
s.push(FLAG_UNIT);
s.push(units);
s.push(FLAG_REMAINING);
break;
}
}
}
}
}
}
return str;
}
Test code with a large number:
int main() {
string str = num2str(1234567890123);
cout << str.c_str() << endl;
return 0;
}
Output:
one trillion two hundred thirty-four billion five hundred sixty-seven million eight hundred ninety thousand one hundred twenty-three
Read digital signatures from multi-signed files
Here is code example of how to use WinTrust to retrieve digital signatures and certificates from a file. It supports multi-signed and catalog-signed files.
What it does:
- Use WinTrust API to enumerate all embedded digital signatures (including nested ones and timestamps) from the file and display the properties of each signature;
- For each digital signature, establish the trust chain of the signing certiifcate, display propertie of each certificate in the trust chain;
- Check if the file is catalog signed, and if yes, do the same to the catalog file: display properties of catalog file’s signatures and their certificates;
What it doesn’t do:
- Verify the signature
- Verify the certificate
- Verify the file against the signature
signature.cpp
#include "signature.h"
#include <vector>
#include <memory>
#include <map>
#include <sstream>
#include <functional>
#include <algorithm>
#include <ctime>
#include <iomanip>
#include <locale>
#include <codecvt>
#include <windows.h>
#include <wincrypt.h>
#include <softpub.h>
#include <mscat.h>
#pragma comment (lib, "crypt32.lib")
#pragma comment (lib, "wintrust.lib")
template<typename T, typename T::pointer invalid_value = nullptr> struct pointer_wrapper {
pointer_wrapper(T& pointer) : _pointer(pointer), _raw_pointer(invalid_value) {}
~pointer_wrapper() {
if (_raw_pointer != invalid_value)
_pointer.reset(_raw_pointer);
}
pointer_wrapper(pointer_wrapper&& _other) : _pointer(_other._pointer) {
std::swap(_raw_pointer, _other._raw_pointer);
}
operator typename T::pointer*() && { return &_raw_pointer; }
operator typename T::pointer&() && { return _raw_pointer; }
private:
T& _pointer;
typename T::pointer _raw_pointer;
pointer_wrapper(pointer_wrapper&) = delete;
pointer_wrapper operator = (const pointer_wrapper&) = delete;
pointer_wrapper operator = (pointer_wrapper&&) = delete;
static void* operator new (size_t) = delete;
static void* operator new (size_t, void*) = delete;
static void* operator new[](size_t) = delete;
static void* operator new[](size_t, void*) = delete;
};
template<typename T, typename T::pointer invalid_value = nullptr> pointer_wrapper<T, invalid_value> get_pointer_wrapper(T& _pointer) {
return pointer_wrapper<T, invalid_value>(_pointer);
}
template <typename T, BOOL(__stdcall *f)(T), T null_t = 0> struct deleter_t {
using pointer = T;
void operator() (pointer value) {
if (value != null_t)
f(value);
}
};
using HCRYPTMSG_t = std::unique_ptr<HCRYPTMSG, deleter_t<HCRYPTMSG, CryptMsgClose>>;
using CERT_CONTEXT_t = std::unique_ptr<CERT_CONTEXT, deleter_t<PCCERT_CONTEXT, CertFreeCertificateContext>>;
using HANDLE_t = std::unique_ptr<HANDLE, deleter_t<HANDLE, CloseHandle, INVALID_HANDLE_VALUE>>;
BOOL __stdcall CertCloseStore0(HCERTSTORE pcrypt_cert_store) {
return CertCloseStore(pcrypt_cert_store, 0);
}
using HCERTSTORE_t = std::unique_ptr<HCERTSTORE, deleter_t<HCERTSTORE, CertCloseStore0>>;
BOOL __stdcall CertFreeCertificateChain0(PCCERT_CHAIN_CONTEXT pcrypt_cert_chain) {
CertFreeCertificateChain(pcrypt_cert_chain);
return true;
}
using CERT_CHAIN_CONTEXT_t = std::unique_ptr<const CERT_CHAIN_CONTEXT, deleter_t<PCCERT_CHAIN_CONTEXT, CertFreeCertificateChain0>>;
BOOL __stdcall CryptCATAdminReleaseContext0(HCATADMIN hcatadmin) {
return CryptCATAdminReleaseContext(hcatadmin, 0);
}
using HCATADMIN_t = std::unique_ptr<HCATADMIN, deleter_t<HCATADMIN, CryptCATAdminReleaseContext0>>;
struct deleter_HCATINFO {
using pointer = HCATINFO;
deleter_HCATINFO(HCATADMIN catadmin) : _catadmin(catadmin) {}
void operator() (pointer hcatinfo) {
if (hcatinfo != 0)
CryptCATAdminReleaseCatalogContext(_catadmin, hcatinfo, 0);
}
HCATADMIN _catadmin;
};
using HCATINFO_t = std::unique_ptr<HCATADMIN, deleter_HCATINFO>;
struct crypt_allocator : std::allocator<BYTE> {
BYTE* allocate(std::size_t n) {
return reinterpret_cast<BYTE*>(::CryptMemAlloc(static_cast<ULONG>(n)));
}
void deallocate(BYTE* p, std::size_t n) {
if (p)
::CryptMemFree(p);
}
};
std::unique_ptr<tm> filetime_to_tm(const FILETIME* pft) {
FILETIME ft_local = {};
FileTimeToLocalFileTime(pft, &ft_local);
SYSTEMTIME st = {};
FileTimeToSystemTime(&ft_local, &st);
std::unique_ptr<tm> t(std::make_unique<tm>());
t->tm_year = st.wYear - 1900;
t->tm_mon = st.wMonth - 1;
t->tm_mday = st.wDay;
t->tm_hour = st.wHour;
t->tm_min = st.wMinute;
t->tm_sec = st.wSecond;
t->tm_isdst = -1;
return std::move(t);
}
std::wstring string_from_filetime(const FILETIME* pft) {
wchar_t str[128] = {};
wcsftime(str, _countof(str), L"%c", filetime_to_tm(pft).get());
return str;
}
std::wstring get_cert_field(PCCERT_CONTEXT pcrypt_cert, DWORD type, DWORD flags) {
DWORD size = CertGetNameString(pcrypt_cert, type, flags, 0, 0, 0);
if (size > 0) {
std::vector<WCHAR, crypt_allocator> value(size + 1);
size = CertGetNameString(pcrypt_cert, type, flags, 0, value.data(), static_cast<DWORD>(value.size()));
return std::wstring(value.data());
}
return L"";
}
std::wstring get_cert_name(CERT_NAME_BLOB* pcrypt_blob, DWORD type) {
DWORD size = CertNameToStr(X509_ASN_ENCODING, pcrypt_blob, type, 0, 0);
if (size > 0) {
std::vector<WCHAR, crypt_allocator> value(size + 1);
size = CertNameToStr(X509_ASN_ENCODING, pcrypt_blob, type, value.data(), static_cast<DWORD>(value.size()));
return std::wstring(value.data());
}
return L"";
}
std::wstring get_oid_info(DWORD type, void* p, DWORD groupid) {
const PCCRYPT_OID_INFO pcrypt_oid_info = CryptFindOIDInfo(type, p, groupid);
if (pcrypt_oid_info && pcrypt_oid_info->pwszName)
return pcrypt_oid_info->pwszName;
return L"";
}
std::wstring format_crypt_blob(const CRYPT_INTEGER_BLOB* pcrypt_blob, bool reverse = false, DWORD limit = 36) {
std::wstringstream ss;
ss << std::hex << std::setfill(L'0');
DWORD size = min(limit - 3, pcrypt_blob->cbData);
if (reverse) {
for (DWORD n = size; n > 0; --n) {
ss << std::setw(2) << pcrypt_blob->pbData[n - 1];
}
}
else {
for (DWORD n = 0; n < size; ++n) {
ss << std::setw(2) << pcrypt_blob->pbData[n];
}
}
if (size < pcrypt_blob->cbData)
ss << L"...";
return ss.str();
}
DWORD get_cert_property(const PCCERT_CONTEXT pcrypt_cert, DWORD property_id, std::vector<BYTE, crypt_allocator>& _buffer) {
DWORD size = 0;
if (!CertGetCertificateContextProperty(pcrypt_cert, property_id, 0, &size))
return GetLastError();
std::vector<BYTE, crypt_allocator> buffer(size);
if (!CertGetCertificateContextProperty(pcrypt_cert, property_id, buffer.data(), &size))
return GetLastError();
std::swap(buffer, _buffer);
return 0;
}
DWORD crypt_decode(LPCSTR decode_type, const std::vector<BYTE, crypt_allocator>& encoded, std::vector<BYTE, crypt_allocator>& decoded) {
DWORD size = 0;
if (!CryptDecodeObject(X509_ASN_ENCODING | PKCS_7_ASN_ENCODING, decode_type, encoded.data(), static_cast<DWORD>(encoded.size()), 0, 0, &size))
return GetLastError();
std::vector<BYTE, crypt_allocator> _decoded(size);
if (!CryptDecodeObject(X509_ASN_ENCODING | PKCS_7_ASN_ENCODING, decode_type, encoded.data(), static_cast<DWORD>(encoded.size()), 0, _decoded.data(), &size))
return GetLastError();
std::swap(_decoded, decoded);
return 0;
}
void decode_usage(CTL_USAGE* pcert_usage, pairs& usages) {
std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>> utf8_to_utf16;
for (DWORD n = 0; n < pcert_usage->cUsageIdentifier; ++n) {
const CCRYPT_OID_INFO* pcrypt_oidinfo = CryptFindOIDInfo(CRYPT_OID_INFO_OID_KEY, pcert_usage->rgpszUsageIdentifier[n], 0);
std::wstring name, usage = utf8_to_utf16.from_bytes(pcert_usage->rgpszUsageIdentifier[n]);
name = (pcrypt_oidinfo && pcrypt_oidinfo->pwszName) ? pcrypt_oidinfo->pwszName : usage;
usages.push_back({ name, usage });
}
}
DWORD get_cert_from_CERT_CONTEXT(const PCCERT_CONTEXT pcrypt_cert, CERT* pcert) {
pcert->version = pcrypt_cert->pCertInfo->dwVersion;
pcert->serial_number = format_crypt_blob(&pcrypt_cert->pCertInfo->SerialNumber, true);
pcert->issuer = get_cert_field(pcrypt_cert, CERT_NAME_SIMPLE_DISPLAY_TYPE, CERT_NAME_ISSUER_FLAG);
pcert->name = get_cert_field(pcrypt_cert, CERT_NAME_FRIENDLY_DISPLAY_TYPE, 0);
pcert->valid_from = string_from_filetime(&pcrypt_cert->pCertInfo->NotBefore);
pcert->valid_to = string_from_filetime(&pcrypt_cert->pCertInfo->NotAfter);
pcert->subject = get_cert_name(&pcrypt_cert->pCertInfo->Subject, CERT_X500_NAME_STR | CERT_NAME_STR_REVERSE_FLAG);
pcert->hash_algorithm = get_oid_info(CRYPT_OID_INFO_OID_KEY, pcrypt_cert->pCertInfo->SignatureAlgorithm.pszObjId, 0);
pcert->algorithm = get_oid_info(CRYPT_OID_INFO_OID_KEY, pcrypt_cert->pCertInfo->SubjectPublicKeyInfo.Algorithm.pszObjId, 0);
std::vector<BYTE, crypt_allocator> buffer;
if (0 == get_cert_property(pcrypt_cert, CERT_HASH_PROP_ID, buffer)) {
CRYPT_INTEGER_BLOB blob = { static_cast<DWORD>(buffer.size()), buffer.data() };
pcert->thumbprint = format_crypt_blob(&blob);
}
if (0 == get_cert_property(pcrypt_cert, CERT_SIGNATURE_HASH_PROP_ID, buffer)) {
CRYPT_INTEGER_BLOB blob = { static_cast<DWORD>(buffer.size()), buffer.data() };
pcert->signature_hash = format_crypt_blob(&blob);
}
if (0 == get_cert_property(pcrypt_cert, CERT_SUBJECT_PUB_KEY_BIT_LENGTH_PROP_ID, buffer)) {
pcert->publickey_length = *reinterpret_cast<DWORD*>(buffer.data());
}
if (0 == get_cert_property(pcrypt_cert, CERT_CTL_USAGE_PROP_ID, buffer)) {
std::vector<BYTE, crypt_allocator> cert_usage;
if (0 == crypt_decode(X509_ENHANCED_KEY_USAGE, buffer, cert_usage)) {
decode_usage(reinterpret_cast<CTL_USAGE*>(cert_usage.data()), pcert->usages);
}
}
DWORD size = 0;
if (CertGetEnhancedKeyUsage(pcrypt_cert, 0, 0, &size)) {
buffer.resize(size);
if (CertGetEnhancedKeyUsage(pcrypt_cert, 0, reinterpret_cast<PCERT_ENHKEY_USAGE>(buffer.data()), &size)) {
decode_usage(reinterpret_cast<CERT_ENHKEY_USAGE*>(buffer.data()), pcert->enhanced_key_usages);
}
}
pcert->publickey = format_crypt_blob(reinterpret_cast<CRYPT_INTEGER_BLOB*>(&pcrypt_cert->pCertInfo->SubjectPublicKeyInfo.PublicKey));
return 0;
}
void oid_publisher(CRYPT_ATTRIBUTE& crypt_attr, ATTR& attr, SIGNATURE* psignature) {
DWORD size = 0;
if (!CryptDecodeObject(X509_ASN_ENCODING | PKCS_7_ASN_ENCODING, SPC_SP_OPUS_INFO_OBJID,
crypt_attr.rgValue[0].pbData, crypt_attr.rgValue[0].cbData, 0, 0, &size))
return;
std::vector<BYTE, crypt_allocator> opusinfo(size);
if (!CryptDecodeObject(X509_ASN_ENCODING | PKCS_7_ASN_ENCODING, SPC_SP_OPUS_INFO_OBJID,
crypt_attr.rgValue[0].pbData, crypt_attr.rgValue[0].cbData, 0, opusinfo.data(), &size))
return;
const PSPC_SP_OPUS_INFO popus = reinterpret_cast<PSPC_SP_OPUS_INFO>(opusinfo.data());
if (popus->pwszProgramName) {
attr.value += L"program:";
attr.value += popus->pwszProgramName;
attr.value += L"; ";
}
if (popus->pPublisherInfo) {
switch (popus->pPublisherInfo->dwLinkChoice) {
case SPC_URL_LINK_CHOICE:
attr.value += L"publisher:";
attr.value += popus->pPublisherInfo->pwszUrl;
attr.value += L"; ";
break;
case SPC_FILE_LINK_CHOICE:
attr.value += L"publisher:";
attr.value += popus->pPublisherInfo->pwszFile;
attr.value += L"; ";
break;
}
}
if (popus->pMoreInfo) {
switch (popus->pMoreInfo->dwLinkChoice) {
case SPC_URL_LINK_CHOICE:
attr.value += L"moreinfo:";
attr.value += popus->pMoreInfo->pwszUrl;
attr.value += L"; ";
break;
case SPC_FILE_LINK_CHOICE:
attr.value += L"moreinfo:";
attr.value += popus->pMoreInfo->pwszFile;
attr.value += L"; ";
break;
}
}
}
void oid_signingtime(CRYPT_ATTRIBUTE& crypt_attr, ATTR& attr, SIGNATURE* psignature) {
FILETIME ft = {};
DWORD size = sizeof(ft);
if (CryptDecodeObject(X509_ASN_ENCODING | PKCS_7_ASN_ENCODING, szOID_RSA_signingTime,
crypt_attr.rgValue[0].pbData, crypt_attr.rgValue[0].cbData, 0, &ft, &size)) {
attr.value += L"timestamp:";
attr.value += string_from_filetime(&ft);
attr.value += L"; ";
}
}
void oid_general(CRYPT_ATTRIBUTE& crypt_attr, ATTR& attr, SIGNATURE* psignature) {
attr.value = format_crypt_blob(&crypt_attr.rgValue[0]);
}
std::map<std::string, void(*)(CRYPT_ATTRIBUTE&, ATTR&, SIGNATURE*)> oidfuncs = {
{ SPC_SP_OPUS_INFO_OBJID, oid_publisher },
{ szOID_RSA_signingTime, oid_signingtime }
};
void process_attrs(CRYPT_ATTRIBUTES& crypt_attrs, attrs_t& attrs, SIGNATURE* psignature) {
for (DWORD index = 0; index < crypt_attrs.cAttr; ++index) {
std::unique_ptr<ATTR> attr(std::make_unique<ATTR>());
std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>> conv;
attr->oid = conv.from_bytes(crypt_attrs.rgAttr[index].pszObjId);
attr->size = crypt_attrs.rgAttr[index].rgValue->cbData;
PCCRYPT_OID_INFO pcrypt_oidinfo = CryptFindOIDInfo(CRYPT_OID_INFO_OID_KEY, crypt_attrs.rgAttr[index].pszObjId, 0);
if (pcrypt_oidinfo && pcrypt_oidinfo->pwszName)
attr->name = pcrypt_oidinfo->pwszName;
else
attr->name = attr->oid;
auto& it = oidfuncs.find(crypt_attrs.rgAttr[index].pszObjId);
if (it != oidfuncs.end())
it->second(crypt_attrs.rgAttr[index], *attr.get(), psignature);
else
oid_general(crypt_attrs.rgAttr[index], *attr.get(), psignature);
attrs.insert(std::make_pair(attr->oid, std::move(attr)));
}
}
DWORD get_signature_from_CMSG_SIGNER_INFO(CMSG_SIGNER_INFO* pcrypt_signature, HCERTSTORE hcrypt_certstore, SIGNATURE* psignature) {
psignature->issuer = get_cert_name(&pcrypt_signature->Issuer, CERT_X500_NAME_STR | CERT_NAME_STR_REVERSE_FLAG);
psignature->version = pcrypt_signature->dwVersion;
psignature->serial_number = format_crypt_blob(&pcrypt_signature->SerialNumber, true);
psignature->encryption_algorithm = get_oid_info(CRYPT_OID_INFO_OID_KEY, pcrypt_signature->HashEncryptionAlgorithm.pszObjId, 0);
psignature->hash_algorithm = get_oid_info(CRYPT_OID_INFO_OID_KEY, pcrypt_signature->HashAlgorithm.pszObjId, 0);
CERT_INFO certinfo = {};
certinfo.Issuer = pcrypt_signature->Issuer;
certinfo.SerialNumber = pcrypt_signature->SerialNumber;
CERT_CONTEXT_t cert_ctx(CertFindCertificateInStore(hcrypt_certstore, X509_ASN_ENCODING | PKCS_7_ASN_ENCODING, 0, CERT_FIND_SUBJECT_CERT, &certinfo, 0));
if (!cert_ctx)
return GetLastError();
std::unique_ptr<CERT> cert(std::make_unique<CERT>());
DWORD last_error = get_cert_from_CERT_CONTEXT(cert_ctx.get(), cert.get());
if (last_error != 0)
return last_error;
CERT_CHAIN_PARA crypt_chain_para = { sizeof(CERT_CHAIN_PARA) };
crypt_chain_para.RequestedUsage.dwType = USAGE_MATCH_TYPE_AND;
CERT_CHAIN_CONTEXT_t cert_chain_ctx;
if (!CertGetCertificateChain(0, cert_ctx.get(), 0, 0, &crypt_chain_para, 0, 0, get_pointer_wrapper(cert_chain_ctx)))
return GetLastError();
if (cert_chain_ctx->cChain > 0 && cert_chain_ctx->rgpChain[0]->cElement > 1) {
CERT* pcert_current = cert.get();
for (DWORD index = 1; index < cert_chain_ctx->rgpChain[0]->cElement; ++index) {
pcert_current->chained_cert = std::make_unique<CERT>();
if ((last_error = get_cert_from_CERT_CONTEXT(cert_chain_ctx->rgpChain[0]->rgpElement[index]->pCertContext, pcert_current->chained_cert.get())) != 0)
return last_error;
pcert_current = pcert_current->chained_cert.get();
}
}
process_attrs(pcrypt_signature->AuthAttrs, psignature->attrs_auth, psignature);
process_attrs(pcrypt_signature->UnauthAttrs, psignature->attrs_unauth, psignature);
psignature->cert = std::move(cert);
return 0;
}
DWORD get_signature_from_wtstatedata(HANDLE state_data, SIGNATURE* psignature) {
CRYPT_PROVIDER_DATA* pcrypt_prov_data = WTHelperProvDataFromStateData(state_data);
std::unique_ptr<SIGNATURE> countersign(std::make_unique<SIGNATURE>());
BOOL is_countersign[2] = { FALSE, TRUE };
SIGNATURE* psignatures[2] = { psignature, countersign.get() };
for (size_t n = 0; n < _countof(is_countersign); ++n) {
CRYPT_PROVIDER_SGNR *pcrypt_signature = WTHelperGetProvSignerFromChain(pcrypt_prov_data, 0, is_countersign[n], 0);
if (!pcrypt_signature)
continue;
CRYPT_PROVIDER_CERT *pcrypt_cert = WTHelperGetProvCertFromChain(pcrypt_signature, 0);
if (!pcrypt_cert)
continue;
HCERTSTORE_t cert_store(CertOpenStore(CERT_STORE_PROV_COLLECTION, X509_ASN_ENCODING | PKCS_7_ASN_ENCODING, 0, 0, 0));
if (!cert_store)
return GetLastError();
for (DWORD index = 0; index < pcrypt_prov_data->chStores; ++index) {
if (!CertAddStoreToCollection(cert_store.get(), pcrypt_prov_data->pahStores[index], 0, 0))
return GetLastError();
}
DWORD last_error = get_signature_from_CMSG_SIGNER_INFO(pcrypt_signature->psSigner, cert_store.get(), psignatures[n]);
if (last_error != 0)
return last_error;
}
psignature->countersign = std::move(countersign);
return 0;
}
unsigned int get_file_signatures(SIGNATURE_INFO& siginfo) {
WINTRUST_DATA wt_data = { sizeof(WINTRUST_DATA) };
wt_data.dwUIChoice = WTD_UI_NONE;
wt_data.fdwRevocationChecks = WTD_REVOKE_NONE;
wt_data.dwUnionChoice = WTD_CHOICE_FILE;
WINTRUST_FILE_INFO wt_fileinfo = { sizeof(WINTRUST_FILE_INFO) };
wt_fileinfo.pcwszFilePath = siginfo.target.c_str();
wt_data.pFile = &wt_fileinfo;
WINTRUST_SIGNATURE_SETTINGS wt_sig_settings = { sizeof(WINTRUST_SIGNATURE_SETTINGS) };
wt_sig_settings.dwFlags = WSS_GET_SECONDARY_SIG_COUNT | WSS_VERIFY_SPECIFIC;
wt_data.pSignatureSettings = &wt_sig_settings;
GUID wt_guid = WINTRUST_ACTION_GENERIC_VERIFY_V2;
DWORD last_error = 0;
while (true) {
wt_data.dwStateAction = WTD_STATEACTION_VERIFY;
if ((last_error = WinVerifyTrust(NULL, &wt_guid, &wt_data)) != 0)
return last_error;
std::unique_ptr<SIGNATURE> signature = std::make_unique<SIGNATURE>();
if ((last_error = get_signature_from_wtstatedata(wt_data.hWVTStateData, signature.get())) != 0)
return last_error;
siginfo.signatures.push_back(std::move(signature));
wt_data.dwStateAction = WTD_STATEACTION_CLOSE;
if ((last_error = WinVerifyTrust(NULL, &wt_guid, &wt_data)) != 0)
return last_error;
wt_data.hWVTStateData = NULL;
if (wt_data.pSignatureSettings == nullptr || ++wt_data.pSignatureSettings->dwIndex > wt_data.pSignatureSettings->cSecondarySigs)
break;
}
return 0;
}
unsigned int get_file_signatures(const wchar_t* file, SIGNATURE_INFO& catalog, SIGNATURE_INFO& embedded) {
embedded.target = file;
embedded.result = get_file_signatures(embedded);
HCATADMIN_t catadmin;
const GUID guid = DRIVER_ACTION_VERIFY;
if (!CryptCATAdminAcquireContext(get_pointer_wrapper(catadmin), &guid, 0))
return (catalog.result = GetLastError());
HANDLE_t h(CreateFile(file, GENERIC_READ, FILE_SHARE_READ, 0, OPEN_EXISTING, 0, 0));
if (!h)
return (catalog.result = GetLastError());
DWORD size = 0;
if (!CryptCATAdminCalcHashFromFileHandle(h.get(), &size, 0, 0))
return (catalog.result = GetLastError());
std::vector<BYTE, crypt_allocator> buffer(size);
if (!CryptCATAdminCalcHashFromFileHandle(h.get(), &size, buffer.data(), 0))
return (catalog.result = GetLastError());
HCATINFO_t hcatinfo(CryptCATAdminEnumCatalogFromHash(catadmin.get(), buffer.data(), static_cast<DWORD>(buffer.size()), 0, 0), deleter_HCATINFO(catadmin.get()));
if (!hcatinfo)
return (catalog.result = GetLastError());
CATALOG_INFO catinfo = {};
if (!CryptCATCatalogInfoFromContext(hcatinfo.get(), &catinfo, 0))
return (catalog.result = GetLastError());
catalog.target = catinfo.wszCatalogFile;
catalog.result = get_file_signatures(catalog);
return 0;
}
signature.h
#pragma once
#include <string>
#include <list>
#include <map>
#include <vector>
#include <memory>
using pairs = std::vector<std::pair<std::wstring, std::wstring>>;
struct CERT {
unsigned int version;
std::wstring serial_number;
std::wstring name;
std::wstring issuer;
std::wstring subject;
std::wstring algorithm;
std::wstring hash_algorithm;
std::wstring thumbprint;
std::wstring signature_hash;
std::wstring publickey;
unsigned long publickey_length;
pairs usages;
pairs enhanced_key_usages;
std::wstring valid_from;
std::wstring valid_to;
std::unique_ptr<CERT> chained_cert;
};
struct ATTR {
unsigned int size;
std::wstring oid;
std::wstring name;
std::wstring value;
};
using attrs_t = std::map<std::wstring, std::unique_ptr<ATTR>>;
struct SIGNATURE {
unsigned int version;
std::wstring issuer;
std::wstring serial_number;
std::wstring encryption_algorithm;
std::wstring hash_algorithm;
attrs_t attrs_auth;
attrs_t attrs_unauth;
std::unique_ptr<CERT> cert;
std::unique_ptr<SIGNATURE> countersign;
};
using signatures_t = std::list<std::unique_ptr<SIGNATURE>>;
struct SIGNATURE_INFO {
std::wstring target;
signatures_t signatures;
unsigned int result;
};
unsigned int get_file_signatures(const wchar_t* file, SIGNATURE_INFO& catalog, SIGNATURE_INFO& embedded);
test.cpp
#include "signature.h"
#include <string>
#include <list>
#include <vector>
#include <iterator>
#include <stdarg.h>
using strings_t = std::vector<std::wstring>;
std::wstring formatstring(const wchar_t* format, ...) {
va_list arguments;
va_start(arguments, format);
int size = _vscwprintf(format, arguments) + 1;
std::unique_ptr<wchar_t[]> buffer(new wchar_t[size]);
vswprintf_s(buffer.get(), size, format, arguments);
va_end(arguments);
return std::wstring(buffer.get());
}
strings_t format_cert(const CERT* pcert) {
strings_t strings;
strings.push_back(formatstring(L"version: %lu", pcert->version));
strings.push_back(formatstring(L"serial number: %s", pcert->serial_number.c_str()));
strings.push_back(formatstring(L"name: %s", pcert->name.c_str()));
strings.push_back(formatstring(L"issuer: %s", pcert->issuer.c_str()));
strings.push_back(formatstring(L"subject: %s", pcert->subject.c_str()));
strings.push_back(formatstring(L"algorithm: %s", pcert->algorithm.c_str()));
strings.push_back(formatstring(L"hashing: %s", pcert->hash_algorithm.c_str()));
strings.push_back(formatstring(L"thumbprint: %s", pcert->thumbprint.c_str()));
strings.push_back(formatstring(L"signature hash: %s", pcert->signature_hash.c_str()));
strings.push_back(formatstring(L"public key (%lu bits): %s", pcert->publickey_length, pcert->publickey.c_str()));
strings.push_back(formatstring(L"valid from: %s", pcert->valid_from.c_str()));
strings.push_back(formatstring(L"valid to: %s", pcert->valid_to.c_str()));
if (!pcert->usages.empty()) {
strings.push_back(formatstring(L"%zu cert usage(s) ->", pcert->usages.size()));
for (const auto& it : pcert->usages) {
strings.push_back(formatstring(L"%zs : %zs", it.first.c_str(), it.second.c_str()));
}
}
if (!pcert->enhanced_key_usages.empty()) {
strings.push_back(formatstring(L"%zu extended key usage(s) ->", pcert->enhanced_key_usages.size()));
for (const auto& it : pcert->enhanced_key_usages) {
strings.push_back(formatstring(L"%zs : %zs", it.first.c_str(), it.second.c_str()));
}
}
return strings;
}
strings_t format_signature(const SIGNATURE* psignature) {
strings_t strings;
strings.push_back(formatstring(L"version: %lu", psignature->version));
strings.push_back(formatstring(L"issuer: %s", psignature->issuer.c_str()));
strings.push_back(formatstring(L"serial number: %s", psignature->serial_number.c_str()));
strings.push_back(formatstring(L"algorithm: %s", psignature->encryption_algorithm.c_str()));
strings.push_back(formatstring(L"hashing: %s", psignature->hash_algorithm.c_str()));
strings.push_back(L"authenticated attributes ->");
for (const auto& it : psignature->attrs_auth) {
const std::wstring name = it.second->oid == it.second->name ? it.second->oid : formatstring(L"%s(%s)", it.second->oid.c_str(), it.second->name.c_str());
strings.push_back(formatstring(L"%s(%lu bytes): %s", name.c_str(), it.second->size, it.second->value.c_str()));
}
strings.push_back(L"unauthenticated attributes ->");
for (const auto& it : psignature->attrs_unauth) {
const std::wstring name = it.second->oid == it.second->name ? it.second->oid : formatstring(L"%s(%s)", it.second->oid.c_str(), it.second->name.c_str());
strings.push_back(formatstring(L"%s(%lu bytes): %s", name.c_str(), it.second->size, it.second->value.c_str()));
}
return strings;
}
void print_with_indent(const std::wstring& indent, const strings_t& strings) {
for (const auto& it : strings) {
wprintf(L"%s%s\n", indent.c_str(), it.c_str());
}
}
void print_certs(std::wstring indent, const CERT* pcert) {
while (true) {
indent += L'\t';
print_with_indent(indent, format_cert(pcert));
pcert = pcert->chained_cert.get();
if (pcert == nullptr)
break;
wprintf(L"\n");
print_with_indent(indent, { L"chained certificate ->" });
}
}
void print_signature(std::wstring indent, const SIGNATURE* psignature) {
print_with_indent(indent, format_signature(psignature));
if (psignature->cert) {
wprintf(L"\n");
print_with_indent(indent, { L"signing certificate ->" });
indent += L'\t';
print_certs(indent, psignature->cert.get());
}
}
void print_signatures(std::wstring indent, const signatures_t& signatures) {
size_t index = 0;
for (auto& it : signatures) {
wprintf(L"\n");
print_with_indent(indent, { formatstring(L"signature %zu of %zu ->", ++index, signatures.size()) });
indent += L'\t';
print_signature(indent, it.get());
if (it->countersign) {
wprintf(L"\n");
print_with_indent(indent, { L"counter-signing signature ->" });
wprintf(L"\n");
indent += L'\t';
print_signature(indent, it->countersign.get());
}
}
}
int wmain(int argc, wchar_t* argv[])
{
if (argc < 2) {
wprintf(L"error: no file specified\n");
return -1;
}
SIGNATURE_INFO catalog, embedded;
get_file_signatures(argv[1], catalog, embedded);
if (catalog.result == 0 && !catalog.signatures.empty()) {
wprintf(L"%s is catalog signed by %s ->", argv[1], catalog.target.c_str());
wprintf(L"\n\n");
print_signatures(L"\t", catalog.signatures);
wprintf(L"\n");
}
if (embedded.result == 0 && !embedded.signatures.empty()) {
wprintf(L"%s has %zu embedded signature(s) ->", embedded.target.c_str(), embedded.signatures.size());
wprintf(L"\n\n");
print_signatures(L"\t", embedded.signatures);
wprintf(L"\n");
}
return 0;
}
Test output for C:\Windows\system32\msvcp140.dll:
C:\WINDOWS\system32\msvcp140.dll has 2 embedded signature(s) -> signature 1 of 2 -> version: 1 issuer: CN=Microsoft Code Signing PCA, O=Microsoft Corporation, L=Redmond, S=Washington, C=US serial number: 330000014096a9ee7056fecc07000100000140 algorithm: RSA hashing: sha1 authenticated attributes -> 1.2.840.113549.1.9.3(Content Type)(12 bytes): 060a2b060104018237020104 1.2.840.113549.1.9.4(Message Digest)(22 bytes): 04144ea7d086546229b36cfb647ded03417b134ae818 1.3.6.1.4.1.311.2.1.11(14 bytes): 300c060a2b060104018237020115 1.3.6.1.4.1.311.2.1.12(54 bytes): program:msvcp140.dll; moreinfo:http://microsoft.com; unauthenticated attributes -> 1.2.840.113549.1.9.6(Counter Sign)(533 bytes): 3082021102010130818e3077310b30090603550406130255533113301106035504... 1.3.6.1.4.1.311.2.4.1(9212 bytes): 308223f806092a864886f70d010702a08223e9308223e5020101310f300d060960... signing certificate -> version: 2 serial number: 330000014096a9ee7056fecc07000100000140 name: Microsoft Corporation issuer: Microsoft Code Signing PCA subject: CN=Microsoft Corporation, OU=MOPR, O=Microsoft Corporation, L=Redmond, S=Washington, C=US algorithm: RSA hashing: sha1RSA thumbprint: 98ed99a67886d020c564923b7df25e9ac019df26 signature hash: 2ccee2e34061e1c24c1dd6aa3147b6c0dfb4eafc public key (2048 bits): 3082010a0282010100db4b8be9036a5fc81b5d0a0539e4d50e92d0c4e18be8a40d... valid from: Sun Aug 18 16:17:17 2016 valid to: Sun Nov 2 16:17:17 2017 1 extended key usage(s) -> Code Signing : 1.3.6.1.5.5.7.3.3 chained certificate -> version: 2 serial number: 6133261a000000000031 name: Microsoft Code Signing PCA issuer: Microsoft Root Certificate Authority subject: CN=Microsoft Code Signing PCA, O=Microsoft Corporation, L=Redmond, S=Washington, C=US algorithm: RSA hashing: sha1RSA thumbprint: 3caf9ba2db5570caf76942ff99101b993888e257 signature hash: 27543a3f7612de2261c7228321722402f63a07de public key (2048 bits): 3082010a0282010100b272595c193064bf1d9a602020429976536c3e1bd66fcccb... valid from: Sun Aug 31 18:19:32 2010 valid to: Sun Aug 31 18:29:32 2020 chained certificate -> version: 2 serial number: 79ad16a14aa0a5ad4c7358f407132e65 name: Microsoft Root Certificate Authority issuer: Microsoft Root Certificate Authority subject: CN=Microsoft Root Certificate Authority, DC=microsoft, DC=com algorithm: RSA hashing: sha1RSA thumbprint: cdd4eeae6000ac7f40c3802c171e30148030c072 signature hash: 391be92883d52509155bfeae27b9bd340170b76b public key (4096 bits): 3082020a0282020100f35dfa8067d45aa7a90c2c9020d035083c7584cdb707899c... valid from: Sun May 9 19:19:22 2001 valid to: Sun May 9 19:28:13 2021 counter-signing signature -> version: 1 issuer: CN=Microsoft Time-Stamp PCA, O=Microsoft Corporation, L=Redmond, S=Washington, C=US serial number: 33000000c59640604bf4deae2e0000000000c5 algorithm: sha1RSA hashing: sha1 authenticated attributes -> 1.2.840.113549.1.9.3(Content Type)(11 bytes): 06092a864886f70d010701 1.2.840.113549.1.9.4(Message Digest)(22 bytes): 04145d47a122156c313c94853b1cf0297d649d7e7dd5 1.2.840.113549.1.9.5(Signing Time)(15 bytes): timestamp:Sun Feb 8 03:09:39 2017; unauthenticated attributes -> signing certificate -> version: 2 serial number: 33000000c59640604bf4deae2e0000000000c5 name: Microsoft Time-Stamp Service issuer: Microsoft Time-Stamp PCA subject: CN=Microsoft Time-Stamp Service, OU=nCipher DSE ESN:C0F4-3086-DEF8, OU=MOPR, O=Microsoft Corporation, L=Redmond, S=Washington, C=US algorithm: RSA hashing: sha1RSA thumbprint: 7ba57715b0f79ca2cf921e5f2a72be11c2fadc67 signature hash: 82a0b845f75deb250c3ef6a214fa5895edc4e693 public key (2048 bits): 3082010a0282010100b6bc33e0258ea6f9f010154e7b572b28496bb3709097fcae... valid from: Sun Sep 7 13:58:52 2016 valid to: Sun Sep 7 13:58:52 2018 1 extended key usage(s) -> Time Stamping : 1.3.6.1.5.5.7.3.8 chained certificate -> version: 2 serial number: 6116683400000000001c name: Microsoft Time-Stamp PCA issuer: Microsoft Root Certificate Authority subject: CN=Microsoft Time-Stamp PCA, O=Microsoft Corporation, L=Redmond, S=Washington, C=US algorithm: RSA hashing: sha1RSA thumbprint: 375fcb825c3dc3752a02e34eb70993b4997191ef signature hash: 023cf1c5e99dc2f24133dae6937145bb481306e6 public key (2048 bits): 3082010a02820101009fa16cb1dfdb48922a7c6b2e19e1bde2e3c599512350adce... valid from: Sun Apr 3 08:53:09 2007 valid to: Sun Apr 3 09:03:09 2021 1 extended key usage(s) -> Time Stamping : 1.3.6.1.5.5.7.3.8 chained certificate -> version: 2 serial number: 79ad16a14aa0a5ad4c7358f407132e65 name: Microsoft Root Certificate Authority issuer: Microsoft Root Certificate Authority subject: CN=Microsoft Root Certificate Authority, DC=microsoft, DC=com algorithm: RSA hashing: sha1RSA thumbprint: cdd4eeae6000ac7f40c3802c171e30148030c072 signature hash: 391be92883d52509155bfeae27b9bd340170b76b public key (4096 bits): 3082020a0282020100f35dfa8067d45aa7a90c2c9020d035083c7584cdb707899c... valid from: Sun May 9 19:19:22 2001 valid to: Sun May 9 19:28:13 2021 signature 2 of 2 -> version: 1 issuer: CN=Microsoft Code Signing PCA 2011, O=Microsoft Corporation, L=Redmond, S=Washington, C=US serial number: 330000008e8791a4571a5fca3e00000000008e algorithm: RSA hashing: sha256 authenticated attributes -> 1.2.840.113549.1.9.25.4(3 bytes): 020101 1.2.840.113549.1.9.3(Content Type)(12 bytes): 060a2b060104018237020104 1.2.840.113549.1.9.4(Message Digest)(34 bytes): 0420efbf66f54719eaa363ada4fa9cbe0dc4d722ec7a1563bb406bf90bb7f25272... 1.3.6.1.4.1.311.2.1.11(14 bytes): 300c060a2b060104018237020115 1.3.6.1.4.1.311.2.1.12(54 bytes): program:msvcp140.dll; moreinfo:http://microsoft.com; unauthenticated attributes -> 1.3.6.1.4.1.311.3.3.1(4921 bytes): 3082133506092a864886f70d010702a082132630821322020103310f300d060960... signing certificate -> version: 2 serial number: 330000008e8791a4571a5fca3e00000000008e name: Microsoft Corporation issuer: Microsoft Code Signing PCA 2011 subject: CN=Microsoft Corporation, OU=MOPR, O=Microsoft Corporation, L=Redmond, S=Washington, C=US algorithm: RSA hashing: sha256RSA thumbprint: b9eaa034c821c159b05d3521bcf7feb796ebd6ff signature hash: 84d8717a416c8c9e214c6e0dbd091860d8133f413bcff35673998f27bba084ca public key (2048 bits): 3082010a0282010100d087d4422b7e9dd9c67ad4a2c3e31592d2539d9517c95236... valid from: Sun Nov 17 18:09:21 2016 valid to: Sun Feb 17 18:09:21 2018 2 extended key usage(s) -> Microsoft Publisher : 1.3.6.1.4.1.311.76.8.1 Code Signing : 1.3.6.1.5.5.7.3.3 chained certificate -> version: 2 serial number: 610e90d2000000000003 name: Microsoft Code Signing PCA 2011 issuer: Microsoft Root Certificate Authority 2011 subject: CN=Microsoft Code Signing PCA 2011, O=Microsoft Corporation, L=Redmond, S=Washington, C=US algorithm: RSA hashing: sha256RSA thumbprint: f252e794fe438e35ace6e53762c0a234a2c52135 signature hash: f6f717a43ad9abddc8cefdde1c505462535e7d1307e630f9544a2d14fe8bf26e public key (4096 bits): 3082020a0282020100abf0fa72101c2eadd86eaa82104d34baf2b658219f421b2a... valid from: Sun Jul 8 16:59:09 2011 valid to: Sun Jul 8 17:09:09 2026 chained certificate -> version: 2 serial number: 3f8bc8b5fc9fb29643b569d66c42e144 name: Microsoft Root Certificate Authority 2011 issuer: Microsoft Root Certificate Authority 2011 subject: CN=Microsoft Root Certificate Authority 2011, O=Microsoft Corporation, L=Redmond, S=Washington, C=US algorithm: RSA hashing: sha256RSA thumbprint: 8f43288ad272f3103b6fb1428485ea3014c0bcfe signature hash: 279cd652c4e252bfbe5217ac722205d7729ba409148cfa9e6d9e5b1cb94eaff1 public key (4096 bits): 3082020a0282020100b28041aa35384d13723268224db8b2f1ffd552bc6cc7f5d2... valid from: Sun Mar 22 18:05:28 2011 valid to: Sun Mar 22 18:13:04 2036 counter-signing signature -> version: 1 issuer: CN=Microsoft Time-Stamp PCA 2010, O=Microsoft Corporation, L=Redmond, S=Washington, C=US serial number: 33000000a1a5fdb9eea7fc9e9d0000000000a1 algorithm: sha256RSA hashing: sha256 authenticated attributes -> 1.2.840.113549.1.9.16.2.12(210 bytes): 3081cf3081cc3081b1041482ae89f5b4e63f165c20bc17e4c1c340cd39eb683081... 1.2.840.113549.1.9.3(Content Type)(13 bytes): 060b2a864886f70d0109100104 1.2.840.113549.1.9.4(Message Digest)(34 bytes): 04201fb274bf8d2388a9f67a01264c6be27f3c51e852bccd15a91287092ddd4799... unauthenticated attributes -> signing certificate -> version: 2 serial number: 33000000a1a5fdb9eea7fc9e9d0000000000a1 name: Microsoft Time-Stamp Service issuer: Microsoft Time-Stamp PCA 2010 subject: CN=Microsoft Time-Stamp Service, OU=nCipher DSE ESN:BBEC-30CA-2DBE, OU=MOPR, O=Microsoft Corporation, L=Redmond, S=Washington, C=US algorithm: RSA hashing: sha256RSA thumbprint: 82ae89f5b4e63f165c20bc17e4c1c340cd39eb68 signature hash: fd68d99c338b8ebfc6990fccd15e2ed939f78bea8a2fd528bb197c0a4256af0b public key (2048 bits): 3082010a02820101009bd00179e622af7a7c1b751ef3b767d61ddea7de0034d812... valid from: Sun Sep 7 13:56:48 2016 valid to: Sun Sep 7 13:56:48 2018 1 extended key usage(s) -> Time Stamping : 1.3.6.1.5.5.7.3.8 chained certificate -> version: 2 serial number: 6109812a000000000002 name: Microsoft Time-Stamp PCA 2010 issuer: Microsoft Root Certificate Authority 2010 subject: CN=Microsoft Time-Stamp PCA 2010, O=Microsoft Corporation, L=Redmond, S=Washington, C=US algorithm: RSA hashing: sha256RSA thumbprint: 2aa752fe64c49abe82913c463529cf10ff2f04ee signature hash: 85975b97560b1c3698f6ea90e4423691bfcc76ef2b95971293c079363f4decd7 public key (2048 bits): 3082010a0282010100a91d0dbc77118a3a20ecfc1397f5fa7f69946b745410d5a5... valid from: Sun Jul 1 17:36:55 2010 valid to: Sun Jul 1 17:46:55 2025 chained certificate -> version: 2 serial number: 28cc3a25bfba44ac449a9b586b4339aa name: Microsoft Root Certificate Authority 2010 issuer: Microsoft Root Certificate Authority 2010 subject: CN=Microsoft Root Certificate Authority 2010, O=Microsoft Corporation, L=Redmond, S=Washington, C=US algorithm: RSA hashing: sha256RSA thumbprint: 3b1efd3a66ea28b16697394703a72ca340a05bd5 signature hash: 08fba831c08544208f5208686b991ca1b2cfc510e7301784ddf1eb5bf0393239 public key (4096 bits): 3082020a0282020100b9089e28e4e4ec064e5068b341c57bebaeb68eaf81ba2244... valid from: Sun Jun 23 17:57:24 2010 valid to: Sun Jun 23 18:04:01 2035
Test output for C:\Windows\system32\drivers\wdboot.sys:
C:\WINDOWS\system32\drivers\wdboot.sys is catalog signed by C:\WINDOWS\system32\CatRoot\{F750E6C3-38EE-11D1-85E5-00C04FC295EE}\Windows-Defender-Core-Group-onecore-Package~31bf3856ad364e35~amd64~~10.0.15063.0.cat ->
signature 1 of 1 ->
version: 1
issuer: CN=Microsoft Windows Production PCA 2011, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
serial number: 33000001066ec325c431c9180e000000000106
algorithm: RSA
hashing: sha256
authenticated attributes ->
1.2.840.113549.1.9.3(Content Type)(11 bytes): 06092b0601040182370a01
1.2.840.113549.1.9.4(Message Digest)(34 bytes): 042067c948513cff6cf1449df945fd6366c98e8d15912dc7a50d881df4e2fa4eb0...
1.3.6.1.4.1.311.2.1.11(14 bytes): 300c060a2b060104018237020115
1.3.6.1.4.1.311.2.1.12(76 bytes): program:Microsoft Windows; moreinfo:http://www.microsoft.com/windows;
unauthenticated attributes ->
1.3.6.1.4.1.311.3.3.1(4919 bytes): 3082133306092a864886f70d010702a082132430821320020103310f300d060960...
signing certificate ->
version: 2
serial number: 33000001066ec325c431c9180e000000000106
name: Microsoft Windows
issuer: Microsoft Windows Production PCA 2011
subject: CN=Microsoft Windows, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
algorithm: RSA
hashing: sha256RSA
thumbprint: afdd80c4ebf2f61d3943f18bb566d6aa6f6e5033
signature hash: 0ae3a29cfb54cd16c853b2246cc428219bb87f7e4ea299b0374b2ac43f2a61d8
public key (2048 bits): 3082010a0282010100c9671a0988213489e4dde729caedae67b2c257d412ec6598...
valid from: Sun Oct 11 16:39:31 2016
valid to: Sun Jan 11 16:39:31 2018
2 extended key usage(s) ->
Windows System Component Verification : 1.3.6.1.4.1.311.10.3.6
Code Signing : 1.3.6.1.5.5.7.3.3
chained certificate ->
version: 2
serial number: 61077656000000000008
name: Microsoft Windows Production PCA 2011
issuer: Microsoft Root Certificate Authority 2010
subject: CN=Microsoft Windows Production PCA 2011, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
algorithm: RSA
hashing: sha256RSA
thumbprint: 580a6f4cc4e4b669b9ebdc1b2b3e087b80d0678d
signature hash: 4e80be107c860de896384b3eff50504dc2d76ac7151df3102a4450637a032146
public key (2048 bits): 3082010a0282010100dd0cbba2e42e09e3e7c5f79669bc0021bd693333efad04cb...
valid from: Sun Oct 19 14:41:42 2011
valid to: Sun Oct 19 14:51:42 2026
chained certificate ->
version: 2
serial number: 28cc3a25bfba44ac449a9b586b4339aa
name: Microsoft Root Certificate Authority 2010
issuer: Microsoft Root Certificate Authority 2010
subject: CN=Microsoft Root Certificate Authority 2010, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
algorithm: RSA
hashing: sha256RSA
thumbprint: 3b1efd3a66ea28b16697394703a72ca340a05bd5
signature hash: 08fba831c08544208f5208686b991ca1b2cfc510e7301784ddf1eb5bf0393239
public key (4096 bits): 3082020a0282020100b9089e28e4e4ec064e5068b341c57bebaeb68eaf81ba2244...
valid from: Sun Jun 23 17:57:24 2010
valid to: Sun Jun 23 18:04:01 2035
counter-signing signature ->
version: 1
issuer: CN=Microsoft Time-Stamp PCA 2010, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
serial number: 33000000b235056837221c0da70000000000b2
algorithm: sha256RSA
hashing: sha256
authenticated attributes ->
1.2.840.113549.1.9.16.2.12(210 bytes): 3081cf3081cc3081b10414bdffc5956390f1139c60d619c0afe9b89e1e201d3081...
1.2.840.113549.1.9.3(Content Type)(13 bytes): 060b2a864886f70d0109100104
1.2.840.113549.1.9.4(Message Digest)(34 bytes): 0420282e8f34290e60ad90ff3d2e6a4769d1085b7cb498601afb4aed9836ab5caf...
unauthenticated attributes ->
signing certificate ->
version: 2
serial number: 33000000b235056837221c0da70000000000b2
name: Microsoft Time-Stamp Service
issuer: Microsoft Time-Stamp PCA 2010
subject: CN=Microsoft Time-Stamp Service, OU=nCipher DSE ESN:728D-C45F-F9EB, OU=MOPR, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
algorithm: RSA
hashing: sha256RSA
thumbprint: bdffc5956390f1139c60d619c0afe9b89e1e201d
signature hash: 07448ef8afa3dcda224722d295b2929d226982c4bbe4960143c07bf3a4bd143f
public key (2048 bits): 3082010a0282010100984a01bbd158f57f749142e9a12e535df22c206f122e9bd3...
valid from: Sun Sep 7 13:56:57 2016
valid to: Sun Sep 7 13:56:57 2018
1 extended key usage(s) ->
Time Stamping : 1.3.6.1.5.5.7.3.8
chained certificate ->
version: 2
serial number: 6109812a000000000002
name: Microsoft Time-Stamp PCA 2010
issuer: Microsoft Root Certificate Authority 2010
subject: CN=Microsoft Time-Stamp PCA 2010, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
algorithm: RSA
hashing: sha256RSA
thumbprint: 2aa752fe64c49abe82913c463529cf10ff2f04ee
signature hash: 85975b97560b1c3698f6ea90e4423691bfcc76ef2b95971293c079363f4decd7
public key (2048 bits): 3082010a0282010100a91d0dbc77118a3a20ecfc1397f5fa7f69946b745410d5a5...
valid from: Sun Jul 1 17:36:55 2010
valid to: Sun Jul 1 17:46:55 2025
chained certificate ->
version: 2
serial number: 28cc3a25bfba44ac449a9b586b4339aa
name: Microsoft Root Certificate Authority 2010
issuer: Microsoft Root Certificate Authority 2010
subject: CN=Microsoft Root Certificate Authority 2010, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
algorithm: RSA
hashing: sha256RSA
thumbprint: 3b1efd3a66ea28b16697394703a72ca340a05bd5
signature hash: 08fba831c08544208f5208686b991ca1b2cfc510e7301784ddf1eb5bf0393239
public key (4096 bits): 3082020a0282020100b9089e28e4e4ec064e5068b341c57bebaeb68eaf81ba2244...
valid from: Sun Jun 23 17:57:24 2010
valid to: Sun Jun 23 18:04:01 2035
C:\WINDOWS\system32\drivers\wdboot.sys has 1 embedded signature(s) ->
signature 1 of 1 ->
version: 1
issuer: CN=Microsoft Code Signing PCA 2010, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
serial number: 330000014b617d5ba87a68e2db00000000014b
algorithm: RSA
hashing: sha256
authenticated attributes ->
1.2.840.113549.1.9.3(Content Type)(12 bytes): 060a2b060104018237020104
1.2.840.113549.1.9.4(Message Digest)(34 bytes): 042062954297f4e99bf2dd8d3caa05995f44dd488f4d5a3dfa81dfeb4e6ca8440a...
1.3.6.1.4.1.311.10.3.28(Platform Manifest Binary ID)(46 bytes): 0c2c7134377a473137304f7a4f724737375770554a6845773039515339722f6a49...
1.3.6.1.4.1.311.2.1.11(14 bytes): 300c060a2b060104018237020115
1.3.6.1.4.1.311.2.1.12(76 bytes): program:Microsoft Windows; moreinfo:http://www.microsoft.com/windows;
unauthenticated attributes ->
1.3.6.1.4.1.311.3.3.1(4915 bytes): 3082132f06092a864886f70d010702a08213203082131c020103310f300d060960...
signing certificate ->
version: 2
serial number: 330000014b617d5ba87a68e2db00000000014b
name: Microsoft Windows Early Launch Anti-malware Publisher
issuer: Microsoft Code Signing PCA 2010
subject: CN=Microsoft Windows Early Launch Anti-malware Publisher, OU=MOPR, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
algorithm: RSA
hashing: sha256RSA
thumbprint: 1412f186eeadbfbc37982d36a45e4bef823ee2ca
signature hash: b4042b1d4310481b79ba8bc0f14d7ad09af03a5a35fa875d158e903996b0996c
public key (2048 bits): 3082010a0282010100a0ee2f90342edf3f43f05d1c574654cd06c702cab0adf3bf...
valid from: Sun Nov 17 17:59:06 2016
valid to: Sun Feb 17 17:59:06 2018
2 extended key usage(s) ->
Early Launch Antimalware Driver : 1.3.6.1.4.1.311.61.4.1
Code Signing : 1.3.6.1.5.5.7.3.3
chained certificate ->
version: 2
serial number: 610c524c000000000003
name: Microsoft Code Signing PCA 2010
issuer: Microsoft Root Certificate Authority 2010
subject: CN=Microsoft Code Signing PCA 2010, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
algorithm: RSA
hashing: sha256RSA
thumbprint: 8bfe3107712b3c886b1c96aaec89984914dc9b6b
signature hash: 121af4b922a74247ea49df50de37609cc1451a1fe06b2cb7e1e079b492bd8195
public key (2048 bits): 3082010a0282010100e90e64507967b5c4e3fd09004c9e94acf75668ea44d8cfc5...
valid from: Sun Jul 6 16:40:17 2010
valid to: Sun Jul 6 16:50:17 2025
chained certificate ->
version: 2
serial number: 28cc3a25bfba44ac449a9b586b4339aa
name: Microsoft Root Certificate Authority 2010
issuer: Microsoft Root Certificate Authority 2010
subject: CN=Microsoft Root Certificate Authority 2010, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
algorithm: RSA
hashing: sha256RSA
thumbprint: 3b1efd3a66ea28b16697394703a72ca340a05bd5
signature hash: 08fba831c08544208f5208686b991ca1b2cfc510e7301784ddf1eb5bf0393239
public key (4096 bits): 3082020a0282020100b9089e28e4e4ec064e5068b341c57bebaeb68eaf81ba2244...
valid from: Sun Jun 23 17:57:24 2010
valid to: Sun Jun 23 18:04:01 2035
counter-signing signature ->
version: 1
issuer: CN=Microsoft Time-Stamp PCA 2010, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
serial number: 33000000b4433a1cfeb6ba52890000000000b4
algorithm: sha256RSA
hashing: sha256
authenticated attributes ->
1.2.840.113549.1.9.16.2.12(210 bytes): 3081cf3081cc3081b1041407c09597ae2e4e6ec9d375f6d98951988b206d3a3081...
1.2.840.113549.1.9.3(Content Type)(13 bytes): 060b2a864886f70d0109100104
1.2.840.113549.1.9.4(Message Digest)(34 bytes): 042015571b68ee3a60fc0f568bc76ab99943ea4ab997b59fb7d2f822098d89a84b...
unauthenticated attributes ->
signing certificate ->
version: 2
serial number: 33000000b4433a1cfeb6ba52890000000000b4
name: Microsoft Time-Stamp Service
issuer: Microsoft Time-Stamp PCA 2010
subject: CN=Microsoft Time-Stamp Service, OU=nCipher DSE ESN:148C-C4B9-2066, OU=MOPR, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
algorithm: RSA
hashing: sha256RSA
thumbprint: 07c09597ae2e4e6ec9d375f6d98951988b206d3a
signature hash: b54f26fad46985a2e2ef7b93e5a4a223149273ce39b052dc38b350bad09280ec
public key (2048 bits): 3082010a0282010100e0814fbbb5d1303a3f8300b7cc569ff79bacbab35d64111e...
valid from: Sun Sep 7 13:56:58 2016
valid to: Sun Sep 7 13:56:58 2018
1 extended key usage(s) ->
Time Stamping : 1.3.6.1.5.5.7.3.8
chained certificate ->
version: 2
serial number: 6109812a000000000002
name: Microsoft Time-Stamp PCA 2010
issuer: Microsoft Root Certificate Authority 2010
subject: CN=Microsoft Time-Stamp PCA 2010, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
algorithm: RSA
hashing: sha256RSA
thumbprint: 2aa752fe64c49abe82913c463529cf10ff2f04ee
signature hash: 85975b97560b1c3698f6ea90e4423691bfcc76ef2b95971293c079363f4decd7
public key (2048 bits): 3082010a0282010100a91d0dbc77118a3a20ecfc1397f5fa7f69946b745410d5a5...
valid from: Sun Jul 1 17:36:55 2010
valid to: Sun Jul 1 17:46:55 2025
chained certificate ->
version: 2
serial number: 28cc3a25bfba44ac449a9b586b4339aa
name: Microsoft Root Certificate Authority 2010
issuer: Microsoft Root Certificate Authority 2010
subject: CN=Microsoft Root Certificate Authority 2010, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
algorithm: RSA
hashing: sha256RSA
thumbprint: 3b1efd3a66ea28b16697394703a72ca340a05bd5
signature hash: 08fba831c08544208f5208686b991ca1b2cfc510e7301784ddf1eb5bf0393239
public key (4096 bits): 3082020a0282020100b9089e28e4e4ec064e5068b341c57bebaeb68eaf81ba2244...
valid from: Sun Jun 23 17:57:24 2010
valid to: Sun Jun 23 18:04:01 2035
Code can be downloaded from https://1drv.ms/u/s!Am6jBwy5CzMPhmfMRhTZzaExRHAP.
Error from Process Monitor: “Unable to write PROCMON23.SYS”
Today I was trying to start a boot logging with Process Monitor on Windows 10 and received following error:
Unable to write PROCMON23.SYS. Make sure that you have permission to write to the %%SystemRoot%%\System32\Drivers directory.
Sharing violation was observed when removed procmon.exe and procmon64.exe from filters:
The workaround is go into C:\Windows\System32\drivers and rename procmon23.sys to something else, then try to enable boot logging from options menu again, it would succeed this time.
Get process command line
There is one officially supported way of getting command line from other processes, that is through WMI Win32_Process class. Here is a command that will display command line and pid of running processes:
WMIC PROCESS get Commandline,Processid
It is also possible to get command line of other processes by reading their memory and track down the information through path PEB -> ProcessParameters -> CommandLine. It involves using unsupported API or structures, and on 64-bit machines extra care needs to be taken for pointer size and structure alignment. Here is the code that can be built and run from 32-bit or 64-bit processes:
#include "stdafx.h"
#include <stddef.h>
#include <Windows.h>
#include <Winternl.h>
typedef NTSTATUS (NTAPI *PFN_NtQueryInformationProcess)(
HANDLE ProcessHandle,
DWORD ProcessInformationClass,
PVOID ProcessInformation,
DWORD ProcessInformationLength,
PDWORD ReturnLength
);
#ifdef _WIN64
typedef struct _UNICODE_STRING_32 {
USHORT Length;
USHORT MaximumLength;
UINT Buffer;
} UNICODE_STRING_32;
typedef struct _RTL_USER_PROCESS_PARAMETERS_32 {
BYTE Reserved1[16];
UINT32 Reserved2[10];
UNICODE_STRING_32 ImagePathName;
UNICODE_STRING_32 CommandLine;
} RTL_USER_PROCESS_PARAMETERS_32;
#else
typedef struct _UNICODE_STRING_64 {
USHORT Length;
USHORT MaximumLength;
UINT32 pad;
UINT64 Buffer;
} UNICODE_STRING_64;
typedef struct _PROCESS_BASIC_INFORMATION_64 {
UINT64 Reserved1;
UINT64 PebBaseAddress;
UINT64 Reserved2[2];
UINT64 UniqueProcessId;
UINT64 Reserved3;
} PROCESS_BASIC_INFORMATION_64;
typedef struct _RTL_USER_PROCESS_PARAMETERS_64 {
BYTE Reserved1[16];
UINT64 Reserved2[10];
UNICODE_STRING_64 ImagePathName;
UNICODE_STRING_64 CommandLine;
} RTL_USER_PROCESS_PARAMETERS_64;
typedef NTSTATUS(NTAPI *PFN_NtWow64ReadVirtualMemory64)(
HANDLE ProcessHandle,
UINT64 BaseAddress,
PVOID Buffer,
ULONG64 Size,
PULONG64 NumberOfBytesRead);
PFN_NtQueryInformationProcess g_pfnNtQueryInformationProcess64 = NULL;
PFN_NtWow64ReadVirtualMemory64 g_pfnWow64ReadVirtualMemory64 = NULL;
#endif
PFN_NtQueryInformationProcess g_pfnNtQueryInformationProcess = NULL;
SYSTEM_INFO g_sysinfo = { 0 };
void get_process_info_initialize()
{
g_pfnNtQueryInformationProcess = (PFN_NtQueryInformationProcess)GetProcAddress(GetModuleHandle(L"ntdll.dll"), "NtQueryInformationProcess");
#ifndef _WIN64
g_pfnNtQueryInformationProcess64 = (PFN_NtQueryInformationProcess)GetProcAddress(GetModuleHandle(L"ntdll.dll"), "NtWow64QueryInformationProcess64");
g_pfnWow64ReadVirtualMemory64 = (PFN_NtWow64ReadVirtualMemory64)GetProcAddress(GetModuleHandle(L"ntdll.dll"), "NtWow64ReadVirtualMemory64");
#endif
GetNativeSystemInfo(&g_sysinfo);
}
LONG get_process_info_by_ntqueryinformationprocess(HANDLE hprocess, PWSTR *info, DWORD pi_class, DWORD info_max_size)
{
LONG error = 0;
PCHAR buffer = 0;
do
{
DWORD info_size = 0;
error = g_pfnNtQueryInformationProcess(hprocess, pi_class, 0, 0, &info_size);
if (info_size == 0 || info_size > info_max_size)
break;
buffer = (PCHAR)malloc(info_size);
if (buffer == 0)
{
error = ERROR_NOT_ENOUGH_MEMORY;
break;
}
error = g_pfnNtQueryInformationProcess(hprocess, pi_class, buffer, info_size, &info_size);
if(!NT_SUCCESS(error))
break;
PUNICODE_STRING str = (PUNICODE_STRING)buffer;
size_t size = str->Length + sizeof(WCHAR);
PWSTR temp_str = (PWSTR)malloc(size);
if (temp_str == 0)
{
error = ERROR_NOT_ENOUGH_MEMORY;
break;
}
memset(temp_str, 0, size);
memcpy(temp_str, str->Buffer, str->Length);
*info = temp_str;
error = 0;
}while(0);
if (buffer != 0)
free(buffer);
return error;
}
template <class unicode_string_type>
LONG read_process_memory_unicode_string(HANDLE hprocess, LPCVOID remote_address, PWSTR *output)
{
LONG error = 0;
do
{
unicode_string_type str = { 0 };
ReadProcessMemory(hprocess, remote_address, &str, sizeof(str), 0);
error = GetLastError();
if (str.Length == 0 || str.Buffer == 0)
break;
size_t size = str.Length + sizeof(WCHAR);
PWSTR temp_str = (PWSTR)malloc(size);
if (temp_str == 0)
{
error = ERROR_NOT_ENOUGH_MEMORY;
break;
}
memset(temp_str, 0, size);
ReadProcessMemory(hprocess, (PCHAR)str.Buffer, temp_str, str.Length, 0);
*output = temp_str;
error = 0;
} while (0);
return error;
}
LONG get_process_info_by_peb_native(HANDLE hprocess, PWSTR *image, PWSTR *command_line)
{
LONG error = 0;
do
{
PROCESS_BASIC_INFORMATION pbi = {0};
error = g_pfnNtQueryInformationProcess(hprocess, ProcessBasicInformation, &pbi, sizeof(pbi), NULL);
if(pbi.PebBaseAddress == 0)
break;
SIZE_T process_parameter = 0;
ReadProcessMemory(hprocess, (PCHAR)pbi.PebBaseAddress + offsetof(PEB, ProcessParameters), &process_parameter, sizeof(process_parameter), 0);
error = GetLastError();
if (process_parameter == 0)
break;
if (image != 0)
error = read_process_memory_unicode_string<UNICODE_STRING>(hprocess, (PCHAR)process_parameter + offsetof(RTL_USER_PROCESS_PARAMETERS, ImagePathName), image);
if (command_line != 0)
error = read_process_memory_unicode_string<UNICODE_STRING>(hprocess, (PCHAR)process_parameter + offsetof(RTL_USER_PROCESS_PARAMETERS, CommandLine), command_line);
}while(0);
return error;
}
#ifdef _WIN64
#if 1
LONG get_process_info_by_peb_to_wow64(HANDLE hprocess, PWSTR *image, PWSTR *command_line)
{
LONG error = 0;
do
{
UINT64 peb32 = 0;
error = g_pfnNtQueryInformationProcess(hprocess, 26 /*ProcessWow64Information*/, &peb32, sizeof(peb32), NULL);
if(peb32 == 0)
break;
UINT64 process_parameter = 0;
ReadProcessMemory(hprocess, (PCHAR)peb32 + 0x10, &process_parameter, sizeof(process_parameter), 0);
error = GetLastError();
if (process_parameter == 0)
break;
if (image != 0)
error = read_process_memory_unicode_string<UNICODE_STRING_32>(hprocess, (PCHAR)process_parameter + offsetof(RTL_USER_PROCESS_PARAMETERS_32, ImagePathName), image);
if (command_line != 0)
error = read_process_memory_unicode_string<UNICODE_STRING_32>(hprocess, (PCHAR)process_parameter + offsetof(RTL_USER_PROCESS_PARAMETERS_32, CommandLine), command_line);
}while(0);
return error;
}
#endif
#endif
#ifndef _WIN64
template <typename unicode_string_type>
LONG read_virtual_memory_from_wow64_unicode_string(HANDLE hprocess, UINT64 remote_address, PWSTR *output)
{
LONG error = 0;
do
{
unicode_string_type str = { 0 };
g_pfnWow64ReadVirtualMemory64(hprocess, remote_address, &str, sizeof(str), 0);
error = GetLastError();
if (str.Length == 0 || str.Buffer == 0)
break;
size_t size = str.Length + sizeof(WCHAR);
PWSTR temp_str = (PWSTR)malloc(size);
if (temp_str == 0)
{
error = ERROR_NOT_ENOUGH_MEMORY;
break;
}
memset(temp_str, 0, size);
g_pfnWow64ReadVirtualMemory64(hprocess, str.Buffer, temp_str, str.Length, 0);
*output = temp_str;
error = 0;
} while (0);
return error;
}
LONG get_process_info_by_peb_from_wow64(HANDLE hprocess, PWSTR *image, PWSTR *command_line)
{
LONG error = 0;
do
{
PROCESS_BASIC_INFORMATION_64 pbi = {0};
error = g_pfnNtQueryInformationProcess64(hprocess, ProcessBasicInformation, &pbi, sizeof(pbi), NULL);
if(pbi.PebBaseAddress == 0)
break;
UINT64 process_parameter = 0;
g_pfnWow64ReadVirtualMemory64(hprocess, (UINT64)(pbi.PebBaseAddress + 0x20), &process_parameter, sizeof(process_parameter), 0);
error = GetLastError();
if(process_parameter == 0)
break;
if (image != 0)
error = read_virtual_memory_from_wow64_unicode_string<UNICODE_STRING_64>(hprocess, process_parameter + offsetof(RTL_USER_PROCESS_PARAMETERS_64, ImagePathName), image);
if(command_line != 0)
error = read_virtual_memory_from_wow64_unicode_string<UNICODE_STRING_64>(hprocess, process_parameter + offsetof(RTL_USER_PROCESS_PARAMETERS_64, CommandLine), command_line);
}while(0);
return error;
}
#endif
LONG get_process_commandline(DWORD pid, BOOL *is32bit, LPWSTR *image, LPWSTR *command_line)
{
LONG error = 0;
HANDLE hprocess = NULL;
do
{
DWORD access[] = { PROCESS_QUERY_INFORMATION | PROCESS_QUERY_LIMITED_INFORMATION | PROCESS_VM_READ,
PROCESS_QUERY_INFORMATION | PROCESS_QUERY_LIMITED_INFORMATION,
PROCESS_QUERY_LIMITED_INFORMATION };
for (size_t n = 0; n < _countof(access); ++n)
{
hprocess = OpenProcess(access[n], FALSE, pid);
error = GetLastError();
if (hprocess != NULL)
break;
}
if (NULL == hprocess)
break;
#ifdef _WIN64
*is32bit = FALSE;
#else
*is32bit = TRUE;
#endif
IsWow64Process(hprocess, is32bit);
#ifdef _WIN64
{
if (*is32bit)
error = get_process_info_by_peb_to_wow64(hprocess, image, command_line);
else
error = get_process_info_by_peb_native(hprocess, image, command_line);
}
#else
{
if (*is32bit || g_sysinfo.wProcessorArchitecture == PROCESSOR_ARCHITECTURE_INTEL)
error = get_process_info_by_peb_native(hprocess, image, command_line);
else
error = get_process_info_by_peb_from_wow64(hprocess, image, command_line);
}
#endif
if (image == 0)
error = get_process_info_by_ntqueryinformationprocess(hprocess, image, 43 /* ProcessImageFileNameWin32 */, 2048);
if (command_line == 0)
error = get_process_info_by_ntqueryinformationprocess(hprocess, command_line, 60 /* ProcessCommandLineInformation */, 2048);
} while (0);
if (hprocess != NULL)
CloseHandle(hprocess);
return error;
}
Here is the testing code:
// cmdline.cpp : Defines the entry point for the console application.
//
#include "stdafx.h"
#include <iostream>
#include <memory>
#include <map>
#include <stack>
#include <algorithm>
#include <Windows.h>
#include <Psapi.h>
#include <tlhelp32.h>
#pragma comment (lib, "Psapi.lib")
void get_process_info_initialize();
LONG get_process_commandline(DWORD pid, BOOL *is32bit, LPWSTR *image, LPWSTR *command_line);
struct _PROCESS_INFO;
typedef std::map<DWORD, _PROCESS_INFO*> processlist;
typedef std::stack<_PROCESS_INFO*> processtack;
typedef struct _PROCESS_INFO
{
DWORD pid;
BOOL is32bit;
PWSTR image;
PWSTR command_line;
processlist child_processes;
}PROCESS_INFO, *PPROCESS_INFO;
PPROCESS_INFO process_alloc();
void process_free(PPROCESS_INFO process);
PPROCESS_INFO process_alloc()
{
PPROCESS_INFO process = new(std::nothrow)PROCESS_INFO;
if (process != 0)
{
process->pid = 0;
process->is32bit = 0;
process->image = 0;
process->command_line = 0;
}
return process;
}
void process_free(PPROCESS_INFO process)
{
processtack st;
st.push(process);
while (!st.empty())
{
process = st.top();
st.pop();
std::for_each(process->child_processes.begin(), process->child_processes.end(), [&st](processlist::value_type& val){ st.push(val.second); });
if (process->image != 0)
free(process->image);
if (process->command_line != 0)
free(process->command_line);
delete process;
}
}
#ifndef _WIN64
BOOL is_running_as_wow64()
{
BOOL is_wow64 = FALSE;
typedef BOOL(WINAPI *PFN_IsWow64Process) (HANDLE, PBOOL);
PFN_IsWow64Process pfnIsWow64Process = (PFN_IsWow64Process)GetProcAddress(GetModuleHandle(L"kernel32"), "IsWow64Process");
if (pfnIsWow64Process != NULL)
pfnIsWow64Process(GetCurrentProcess(), &is_wow64);
return is_wow64;
}
#endif
PPROCESS_INFO get_process_list()
{
PPROCESS_INFO root = process_alloc();
do
{
if (root == 0)
break;
HANDLE hProcessSnap = CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0);
if (INVALID_HANDLE_VALUE == hProcessSnap)
break;
processlist processes;
PROCESSENTRY32 pe32 = { 0 };
pe32.dwSize = sizeof(PROCESSENTRY32);
Process32First(hProcessSnap, &pe32);
do
{
if (pe32.th32ProcessID == 0)
continue;
PPROCESS_INFO process = process_alloc();
if (process == NULL)
break;
process->pid = pe32.th32ProcessID;
PWSTR path = pe32.szExeFile;
MODULEENTRY32 me32 = { 0 };
me32.dwSize = sizeof(MODULEENTRY32);
HANDLE hModuleSnap = CreateToolhelp32Snapshot(TH32CS_SNAPMODULE, pe32.th32ProcessID);
if (INVALID_HANDLE_VALUE != hModuleSnap && Module32First(hModuleSnap, &me32))
path = me32.szExePath;
get_process_commandline(process->pid, &process->is32bit, &process->image, &process->command_line);
if (process->image == 0)
process->image = _wcsdup(path);
if (INVALID_HANDLE_VALUE != hModuleSnap)
CloseHandle(hModuleSnap);
processlist::iterator itor = processes.find(pe32.th32ParentProcessID);
if (itor == processes.end())
{
root->child_processes.insert(std::make_pair(pe32.th32ProcessID, process));
}
else
{
itor->second->child_processes.insert(std::make_pair(pe32.th32ProcessID, process));
}
processes.insert(std::make_pair(pe32.th32ProcessID, process));
} while (Process32Next(hProcessSnap, &pe32));
CloseHandle(hProcessSnap);
} while (0);
return root;
}
void dump_process_list(const processlist& processes)
{
std::wstring indent;
processtack st;
std::for_each(processes.rbegin(), processes.rend(), [&st](const processlist::value_type& val){ st.push(val.second); });
while (!st.empty())
{
const PPROCESS_INFO process = st.top();
st.pop();
if (process == 0)
{
indent.resize(indent.size() - 1);
continue;
}
wprintf(indent.c_str());
wprintf(L"pid:%d image=%s, cmdline=%s\n\n", process->pid, process->image, process->command_line);
indent += L'\t';
st.push(0);
std::for_each(process->child_processes.rbegin(), process->child_processes.rend(), [&st](const processlist::value_type& val){ st.push(val.second); });
}
}
void enable_debug_privilege(LPCWSTR priviledge)
{
HANDLE token = 0;
OpenProcessToken(GetCurrentProcess(), TOKEN_ADJUST_PRIVILEGES | TOKEN_QUERY, &token);
if (token != 0)
{
TOKEN_PRIVILEGES privileges = { 0 };
privileges.PrivilegeCount = 1;
privileges.Privileges[0].Attributes = SE_PRIVILEGE_ENABLED;
LookupPrivilegeValue(0, priviledge, &privileges.Privileges[0].Luid);
AdjustTokenPrivileges(token, FALSE, &privileges, 0, 0, 0);
CloseHandle(token);
}
}
int _tmain(int argc, _TCHAR* argv[])
{
enable_debug_privilege(SE_DEBUG_NAME);
get_process_info_initialize();
PPROCESS_INFO root = get_process_list();
if (root != 0)
{
dump_process_list(root->child_processes);
process_free(root);
}
return 0;
}
Looking for the terminated thread leaving loader lock orphaned
Sometimes a process hang is caused by doing something dangerous while holding the loader lock, if you can get a dump file, it is pretty easy to run !locks command to find out the locks causing the trouble. It can be a little bit complicated if one of the locking mechanism is not a critical section(for example, event or mutex), but it is still manageable by looking for similar function calls among the stacks. However today I ran into a dump file from a hung process which shows the loader lock was locked by a thread that has been terminated and being left orphaned:
0:000> !locks
CritSec ntdll!LdrpLoaderLock+0 at 00007ffa0aa3d8a8
WaiterWoken No
LockCount 9
RecursionCount 1
OwningThread 710
EntryCount 0
ContentionCount 39
*** Locked
Scanned 76 critical sections
0:000> ~~[710]k
^ Illegal thread error in '~~[710]k'
Most likely, thread 710 was terminated while it was holding the loader lock. Because of this, there were many other threads in same process were being blocked and waiting for it to be unlocked, which unfortunately will never happen. Since the thread 710 no longer exists, there was no TEB, we don’t know what it was running when being terminated, and most importantly, we don’t know who terminated it. If you have worked on certain amount of crash dumps, eventually you will run into some internal NT structures, one of which is _CLIENT_ID:
0:000> dt _CLIENT_ID combase!_CLIENT_ID +0x000 UniqueProcess : Ptr64 Void +0x008 UniqueThread : Ptr64 Void
It is simply a structure of process id and thread id, and it is being used by many other structures like _TEB. It has been a technique to search them in memory in order to identify the parent structure that contains them. Let’s try this here to see what we can get.
0:000> | . 0 id: 608 examine name: C:\Program Files\ABC\xyz.exe 0:000> s 0 L?ffffffffffffffff 00 00 00 00 08 06 00 00 00 00 00 00 10 07 00 00 00007ffa`07eb6214 00 00 00 00 08 06 00 00-00 00 00 00 10 07 00 00 ................
First we use “|” command to get process id 608, then use the “s” command to search for memory that can be interpreted as a _CLIENT_ID structure that has 608 as process id and 710 as thread id. “s” command did return an address that contains matching data, let’s find out what it is:
0:000> !address 00007ffa`07eb6214 ... Usage: Image Base Address: 00007ffa`07eb3000 End Address: 00007ffa`07eb7000 Region Size: 00000000`00004000 State: 00001000 MEM_COMMIT Protect: 00000004 PAGE_READWRITE Type: 01000000 MEM_IMAGE Allocation Base: 00007ffa`07dc0000 Allocation Protect: 00000080 PAGE_EXECUTE_WRITECOPY Image Path: C:\Windows\System32\KERNELBASE.dll Module Name: KERNELBASE Loaded Image Name: KERNELBASE.dll Mapped Image Name: More info: lmv m KERNELBASE More info: !lmi KERNELBASE More info: ln 0x7ffa07eb6214 More info: !dh 0x7ffa07dc0000 0:000> ln 0x7ffa07eb6214 (00007ffa`07eb6200) KERNELBASE!BaseTerminatedLoaderLockOwner+0x14 | (00007ffa`07eb6268) KERNELBASE!BaseDataFileHandleTableElementCount
It is very surprising, the memory found belongs to a global data structure of KERNELBASE.dll, and based on the name given by symbol, looks like Windows is tracking the threads if they were terminated when holding the loader lock. Could it save the stack as well?
0:000> dps 00007ffa`07eb6200 00007ffa`07eb6200 00000000`00000001 00007ffa`07eb6208 00000000`00000608 00007ffa`07eb6210 00000000`00000c64 00007ffa`07eb6218 00000000`00000608 00007ffa`07eb6220 00000000`00000710 00007ffa`07eb6228 00007ffa`07e64263 KERNELBASE!TerminateThread+0xaf 00007ffa`07eb6230 00007ff9`fa494995 somedll+0x4995 00007ffa`07eb6238 00007ff9`fa492421 somedll+0x2421 00007ffa`07eb6240 00007ff9`fa4924e9 somedll+0x24e9 00007ffa`07eb6248 00007ffa`0a1113d2 kernel32!BaseThreadInitThunk+0x22 00007ffa`07eb6250 00007ffa`0a9803c4 ntdll!RtlUserThreadStart+0x34 00007ffa`07eb6258 00000000`00000000 00007ffa`07eb6260 00000000`00000000 00007ffa`07eb6268 00000000`00000000 00007ffa`07eb6270 00000000`00000000 00007ffa`07eb6278 00000000`00000000
It did! After fixing symbols for somedll and re-run dps command, I received the full stack of the thread when it is calling TerminateThread when the thread being terminated was holding loader lock, it’s time to notify the team responsible for somedll.
Bonus read: Raymond has another two excellent articles about finding ghost threads causing crashing from kernel or user dumps:
How to view the stack of threads that were terminated as part of process teardown from the kernel debugger
How to view the stack of threads that were terminated as part of process teardown from user mode
Crash from HeapLock
One of my routine work is investigating crash dumps collected by Windows Error Reporting service (also known as Winqual). This morning I found cabs are available for some of high volume buckets, I downloaded the first one, opened it in WinDbg, corrected symbols, and here is the stack after .ecxr and kb command:
0:019> kb *** Stack trace for last set context - .thread/.cxr resets it ChildEBP RetAddr Args to Child 01a1ac80 74b76dee 01c70000 01a1ed04 62d54ab3 ntdll!RtlLockHeap+0x16 01a1ac8c 62d54ab3 01c70000 bfb3fcb0 01a1f050 KERNELBASE!HeapLock+0xe 01a1ed04 62d55239 01b42b71 00000200 01a1f064 MyModule!IsAddressOnHeap+0x83 ...
Here is the pseudo code for MyModule!IsAddressOnHeap, it takes a parameter of void*, then walk through heaps in current process and returns whether the address might be coming from a heap.
BOOL IsAddressOnHeap(void* p) {
HANDLE handles[4096];
int heaps = GetProcessHeaps(sizeof(handles)/sizeof(HANDLE), handles);
if(heaps == 0)
return FALSE;
for(int i=0 ; i < heaps ; i++) {
if (HeapLock(handles[i])) {
__try {
// use HeapWalk to determine whether the address is on heap
}
__finally {
HeapUnlock(handles[i]);
}
}
}
return FALSE;
}
I already had a good guess about what went wrong, but let’s confirm it. First, let’s see how the execution was transferred from MyModule!IsAddressOnHeap to KERNELBASE!HeapLock:
0:019> ub 62d54ab3 MyModule!IsAddressOnHeap+0x62 62d54a92 0f8447010000 je MyModule!IsAddressOnHeap+0x1af (62d54bdf) 62d54a98 33db xor ebx,ebx 62d54a9a 895de4 mov dword ptr [ebp-1Ch],ebx 62d54a9d 3bdf cmp ebx,edi 62d54a9f 0f8d34010000 jge MyModule!IsAddressOnHeap+0x1a9 (62d54bd9) 62d54aa5 8b8c9da4bfffff mov ecx,dword ptr [ebp+ebx*4-405Ch] 62d54aac 51 push ecx 62d54aad ff155ca0d662 call dword ptr [MyModule!_imp__HeapLock (62d6a05c)]
The parameter passed to KERNELBASE!HeapLock was pushed to stack, which came from ecx, which in turn came from address at ebp+ebx*4-405Ch. Also we know that ebx is used as the index (variable i) and edi is used as the counter (variable heaps). We can get the value of registers from context record saved at the time of crash, but they might not be the same as they were in MyModule!IsAddressOnHeap considering there are two more functions (KERNELBASE!HeapLock+0xe and ntdll!RtlLockHeap+0x16) on the stack above MyModule!IsAddressOnHeap. Things gets simple in this case as these two functions didn’t run much (offsets are 0xe and 0x16), which means we can disassemble them from start to the offset to make sure ebx and edi were not changed (I did it and they were not changed). In case they were changed and since they are non-volatile registers, caller is responsible in preserving the old values (by pushing/pop them to/from stack) so they will still be retrievable. Since they are not changed in this case, let’s use the registers in crash context record to see what were in handles array:
0:019> .ecxr eax=3b94b284 ebx=00000004 ecx=01c70000 edx=00000000 esi=01c70000 edi=00000005 eip=77481641 esp=01a1ac54 ebp=01a1ac80 iopl=0 nv up ei pl nz na pe nc cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00010206 ntdll!RtlLockHeap+0x16: 77481641 f7464400000001 test dword ptr [esi+44h],1000000h ds:0023:01c70044=???????? * 01a1ed04 is ChildEBP of MyModule!IsAddressOnHeap from kb command, the output is in sync with first arg from kb 0:019> dd 01a1ed04+4*4-405c l1 01a1acb8 01c70000 * edi has value of 5, dump first 5 elements in array heaps 0:019> dd 01a1ed04-405c l5 01a1aca8 006d0000 004b0000 00690000 01a80000 01a1acb8 01c70000
Now we know what were in array heaps. Let’s take a look at the heaps in PEB:
0:019> dt _PEB @$peb MyModule!_PEB ... +0x088 NumberOfHeaps : 4 +0x08c MaximumNumberOfHeaps : 0x10 +0x090 ProcessHeaps : 0x7754d3a0 -> 0x006d0000 Void ... 0:019> dd 0x7754d3a0 l4 7754d3a0 006d0000 004b0000 00690000 01a80000
So there were 4 heaps reported from PEB, while our array had 5, the first 4 were same as in PEB, the fifth one was not in PEB heaps and wasn’t in a committed page:
0:019> !address 01c70000 Usage: Free Base Address: 01b48000 End Address: 5ffd0000 Region Size: 5e488000 State: 00010000 MEM_FREE Protect: 00000001 PAGE_NOACCESS Type: <info not present at the target>
So it looks like by the time function MyModule!IsAddressOnHeap walked to the last heap, it had been destroyed and memory had been freed. We need to put HeapLock into another SEH block to catch the access violation, existing block cannot be reused as it sole purpose is to use the __finally block to unlock the heap (if it was locked successfully).
I wrote a simple test application and validated the theory: after calling HeapDestroy on a heap, calling HeapLock with same heap handle will cause access violation. HeapValidate can be used to check if the heap was valid (even with a handle that has been used with HeapDestroy), I though of calling it before HeapLock, but ditched the idea quickly due to the potential racing condition that HeapDestroy could called in-between.
Get this Pointer from 64-bit crash dump
64-bit calling convention means that under most scenarios, the first four parameters are passed using rcx, rdx, r8 and r9 registers rather than using stack. It’s difficult to find out actual parameters passed to functions in the calling stack from a 64-bit crash dump. CodeMachine has a great article X64 Deep Dive on this topic, it also talks about x64 compiler optimization, exception handling, parameter passing as well as parameter retrieval. It is a must read if crash dump analysis is part of your job.
There are lots of times we want to find out “this” pointer from the crash dump. “this” pointer is passed using rcx, and if the function accesses class members, very likely rcx will be saved into rsi or rdi after function’s prolog. I created a MFC application using VS2012 project wizard, inserted DebugBreak() at the end of CMFCApplication1App::InitInstance, built it using x64|Release configuration, ran it, save the crash dump and open it with WinDbg, this is what the initial stack looks like:
0:000> kL Child-SP RetAddr Call Site 000000cc`aa65e468 000007fb`01df12d2 ntdll!NtWaitForMultipleObjects+0xa 000000cc`aa65e470 000007fb`0211d20e KERNELBASE!WaitForMultipleObjectsEx+0xe5 000000cc`aa65e750 000007fb`0211cfd2 kernel32!WerpReportFaultInternal+0x1fa 000000cc`aa65e7f0 000007fb`01e6fc87 kernel32!WerpReportFault+0x42 000000cc`aa65e820 000007fb`04db9183 KERNELBASE!UnhandledExceptionFilter+0x1d7 000000cc`aa65e920 000007fb`04ce4fea ntdll! ?? ::FNODOBFM::`string'+0x9e3 000000cc`aa65e950 000007fb`04ce464d ntdll!_C_specific_handler+0x8e 000000cc`aa65e9c0 000007fb`04ce567c ntdll!RtlpExecuteHandlerForException+0xd 000000cc`aa65e9f0 000007fb`04cc4bba ntdll!RtlDispatchException+0x392 000000cc`aa65f100 000007fb`01e9478a ntdll!KiUserExceptionDispatch+0x2e 000000cc`aa65f828 000007f7`5be74c8b KERNELBASE!DebugBreak+0x2 000000cc`aa65f830 000007f7`5c059a09 MFCApplication1!CMFCApplication1App::InitInstance+0x2cb 000000cc`aa65f920 000007f7`5c0417fc MFCApplication1!AfxWinMain+0x75 000000cc`aa65f960 000007fb`020d167e MFCApplication1!__tmainCRTStartup+0x148 000000cc`aa65f9a0 000007fb`04ce3501 kernel32!BaseThreadInitThunk+0x1a 000000cc`aa65f9d0 00000000`00000000 ntdll!RtlUserThreadStart+0x1d
Let’s say I want to know what does MFCApplication1!CMFCApplication1App look like at this time. First let us disassemble first few instructions in CMFCApplication1App::InitInstance:
0:000> u MFCApplication1!CMFCApplication1App::InitInstance l10 MFCApplication1!CMFCApplication1App::InitInstance [d:\temp\mfcapplication1\mfcapplication1\mfcapplication1.cpp @ 73]: 000007f7`5be749c0 488bc4 mov rax,rsp 000007f7`5be749c3 55 push rbp 000007f7`5be749c4 488d68a1 lea rbp,[rax-5Fh] 000007f7`5be749c8 4881ece0000000 sub rsp,0E0h 000007f7`5be749cf 48c745affeffffff mov qword ptr [rbp-51h],0FFFFFFFFFFFFFFFEh 000007f7`5be749d7 48895810 mov qword ptr [rax+10h],rbx 000007f7`5be749db 48897018 mov qword ptr [rax+18h],rsi 000007f7`5be749df 48897820 mov qword ptr [rax+20h],rdi 000007f7`5be749e3 488b0536f72b00 mov rax,qword ptr [MFCApplication1!__security_cookie (000007f7`5c134120)] 000007f7`5be749ea 4833c4 xor rax,rsp 000007f7`5be749ed 48894547 mov qword ptr [rbp+47h],rax 000007f7`5be749f1 488bf1 mov rsi,rcx <-- rcx was saved to rsi here 000007f7`5be749f4 c745b708000000 mov dword ptr [rbp-49h],8 000007f7`5be749fb c745bbff000000 mov dword ptr [rbp-45h],0FFh 000007f7`5be74a02 488d4db7 lea rcx,[rbp-49h] 000007f7`5be74a06 ff157c961f00 call qword ptr [MFCApplication1!_imp_InitCommonControlsEx (000007f7`5c06e088)]
We know that “this” pointer of MFCApplication1!CMFCApplication1App object was passed using rcx, rcx was saved into rsi not far from the prolog, and since the crash was happening inside this function, let’s restore the crash context record and check whether we can get MFCApplication1!CMFCApplication1App pointer from rsi:
0:000> .ecxr
rax=0000000000000001 rbx=0000000000000000 rcx=000007fb04b04aaa
rdx=0000000000000000 rsi=000007f75c1418b0 rdi=000000ccaa860ba0
rip=000007fb01e9478a rsp=000000ccaa65f828 rbp=000000ccaa65f8b9
r8=000000ccaa65f828 r9=000000ccaa65f8b9 r10=0000000000000000
r11=0000000000000202 r12=000007f75c1418b0 r13=0000000000000000
r14=000007f75c1418b0 r15=00000000ffffffff
iopl=0 nv up ei pl nz na pe nc
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202
KERNELBASE!DebugBreak+0x2:
000007fb`01e9478a cc int 3
0:000> dt /r MFCApplication1!CMFCApplication1App @rsi
+0x000 __VFN_table : 0x000007f7`5c0df898
=000007f7`5c074d10 classCObject : CRuntimeClass
+0x000 m_lpszClassName : 0x000007f7`5c074d40 "CObject"
+0x008 m_nObjectSize : 0n8
+0x00c m_wSchema : 0xffff
+0x010 m_pfnCreateObject : (null)
+0x018 m_pBaseClass : (null)
+0x020 m_pNextClass : (null)
+0x028 m_pClassInit : (null)
=000007f7`5c071698 classCCmdTarget : CRuntimeClass
+0x000 m_lpszClassName : 0x000007f7`5c071808 "CCmdTarget"
+0x008 m_nObjectSize : 0n64
+0x00c m_wSchema : 0xffff
+0x010 m_pfnCreateObject : (null)
+0x018 m_pBaseClass : 0x000007f7`5c074d10 CRuntimeClass
+0x000 m_lpszClassName : 0x000007f7`5c074d40 "CObject"
+0x008 m_nObjectSize : 0n8
+0x00c m_wSchema : 0xffff
+0x010 m_pfnCreateObject : (null)
+0x018 m_pBaseClass : (null)
+0x020 m_pNextClass : (null)
+0x028 m_pClassInit : (null)
+0x020 m_pNextClass : (null)
+0x028 m_pClassInit : (null)
...
+0x180 m_strRegSection : ATL::CStringT<wchar_t,StrTraitMFC<wchar_t,ATL::ChTraitsCRT<wchar_t> > >
+0x000 m_pszData : 0x000000cc`aa851548 "Workspace"
...
+0x1cc m_bHiColorIcons : 0n1
It looks good. Of course it is completely possible that after the copying “this” pointer from rcx, rsi was changed again before the crash. The only way be sure about it is to disassemble the function from start to the crash point and read the assembly code. I made a bet here that rsi wan’t changed, and I will dump the complete structure to make sure it looks intact.
Another tricky part of 64-bit crash dump analysis is that executable images are as important as symbols. Many times WinDbg needs access to the binaries to be able to fully restore the call stack, there are lots more function information stored in 64-bit binaries than 32-bit binaries. If images are not available, it is still possible to manually construct the stack by walking through output from dps , but it is going to be very painful since rbp is no longer used as frame base pointer.
Use Windows Performance Toolkit to Trackdown Slow Service Start Problem
Terminology
- ETW: Event Tracing for Windows
- WPT: Windows Performance Toolkit
- ADK: Windows Assessment and Deployment Kit
Recommended Readings
Bruce Dawson has an excellent article “Xperf Wait Analysis–Finding Idle Time” which explains how to use Windows Performance Toolkit (WPT) to analyze CPU idle issue. He actually has a series of articles about WPT and I would recommend read all of them if you are new to WPT.
The Issue
Our product has several Windows services, and to reduce boot time impact to the computer, we have an internal guideline that the each service should be started in 0.5 second during boot. One day I received report that a service took more than 1 second to start, they supplied the boot tracing etl file collected by XbootMgr (part of WPT and now replaced by Windows Performance Recorder in Windows Assessment and Deployment Kit). I opened the etl file, loaded symbols, opened services table and filtered to the service in question:
- Container Init Time: this starts when the service process being created by services.exe, ends when the service process calls StartServiceCtrlDispatcher; if there are multiple services sharing same host process, Container Init Time will only be counted for the first service.
- Service Init Time: this starts when ServiceMain was called, ends when service called SetServiceStatus with status SERVICE_RUNNING.
- Duration: this is how long service took to start, the value came from “Ended At” minus “Started At”, or “Container Init Time” plus “Service Init Time”.
From the table, we can see that MyService did take 1.079s to start, most of the time was spent in Container Init Time, this means we should make our focus on looking at what the process did at its start up. Fortunately the etl file has ReadyThead (please see this article for details) information and stack-walking was enabled on it, let’s try to find out what is its first thread and what it did.
Zoom Into The Range
Before jumping to ReadyThread, it is recommended to zoom into the time range we are interested in, it not only reduces noises but also improves the data loading and ordering performance. Since I already filtered the data to MyService only, there will be a vertical blue bar in the graph, you can right click on it and select “Zoom” command; alternatively you can right click anywhere on the graph, select “Select Time Range…” and fill in Start and End manually. For example, I set the start and end to same value as MyService start and end time since I will only be interested in data occurred during this time frame:
First Thread in MyService.exe
First, I need to find out which is the first thread in MyService.exe. Let’s open ReadThread table, filter selection to MyService.exe, make “SwitchInTime(s)” and “NewThreadId” the second and third column, order data by “SwitchInTime(s)”, and here is what I saw:
Again, please see this article for information about the meaning of these columns.
Here not only we know thread 1592 was first thread ran in MyService.exe, but also we noticed something was wrong: thread 1592 was first switched into CPU to be run at 13.207940543s, the next time it was switched in was at 14.080725766s, more than 0.8s after the first switch in. It looks like it was blocked and waiting for some other threads. Let’s re-order the columns to find out which thread awoke (also known as “readied”) thread 1592:
From the graph above, thread 672 in services.exe is the readying thread of thread 1592. Also from the stack, we can tell that services.exe created thread 1592 (in suspended status) at around 13.207940543s, but didn’t call ResumeThread to let it run until at about 14.080671757s (column ReadyTime(s) in the graph), was it blocked too? Let’s take a look at what’s happening in thread 672 of services.exe.
We saw similar thing was happening in thread 672 as well: after being switched into CPU at 13.208368780s, the next switch in happened at 14.080603959s, again 0.8s after last switch in. Its readying thread was thread 800, in same process. It’s time to take a look at that thread, and it started feeling like falling into a rabbit hole…
Dead End?
I expanded thread 800 in services.exe and following is a snippet of switches from it:
It looks actually pretty normal, the switch in time here was distributed evenly without any noticeable gap from 13.20s all the way down to 14.08s. It looks like a dead end. Or is it?
Switch To CPU Busy Diagnostic
So far we are using ReadyThread to diagnostic CPU idle issue, which are caused by thread was being blocked for too long. If the search reached to a thread without visible delay or blocking, it is time to switch to CPU busy diagnostic model. Most often people are using CPU sampling graph (also known as profiling) for CPU busy issues, here we can also use ReadyThread to investigate which function took most of CPU time, and it is actually more accurate than CPU sampling. Still in ReadyThread table, let’s re-order the columns: remove everything, leave only NewProcess, NewThreadId, NewThreadStack and %CPU Usage, order data by %CPU Usage:
This is cool. The butterfly stack graph shows that during 13.20s to 14.08s, in thread 800, most of CPU time was spent in function services.exe!RSetServiceObjectSecurity and children functions called by it. Based on stack, we also know that thread 800 was responding to an ALPC message and was modifying a service’s security descriptor. As the result of the modification, it resulted a registry flush which took about 0.8s to finish. This whole process was inside a semaphore object (recall that thread 800 readied thread 672 by calling ntkrnlmp.exe!NtReleaseSemaphore from services.exe!RSetServiceObjectSecurity), which ultimately delayed start of MyService.
Who Did It?
Now we know that the root cause is somebody was modifying service security descriptor during boot, caused registry flush, and delayed other service’s start. The next step is to find out who did it. Based on the knowledge I know how Windows service works, it is sechost.dll that creates worker threads and calls ServiceMain function implemented by the service, and there is a good chance sechost.dll!SetServiceObjectSecurity was called from this thread. Let’s re-order the column in ReadyThread table and see whether we can find somebody called sechost.dll!SetServiceObjectSecurity. It only took a minute for me to find it:
AnotherSvc.exe’s thread 1584 made a call to sechost.dll!SetServiceObjectSecurity, it transformed into an ALPC message, handled by thread 800 in services.exe, and once it is done, thread 800 readied thread 1584 as well as another thread 672 in services.exe almost at same time due to the releasing semaphore. Now all the pieces are put together and we got a clear picture of what was going on.
I contacted the team responsible for AnotherSvc.exe and they fixed the issue by moving the call to sechost.dll!SetServiceObjectSecurity at a later stage.
The End
Here we used ReadyThread table (and different columns and orderings) to track down the root cause of the service slow start, across 4 threads in 3 different processes, this is so far the most complicated slow performance issue I have met, and I hope won’t run into another in future.
Radec 