Andre's Blog
Perfection is when there is nothing left to take away
Printing from a 32-bit IIS process on 64-bit Windows

Last week I came across a very strange problem with Windows printing - an attempt to create a printer device context through CreateDC within a 32-bit IIS worker process would fail and return NULL, but GetLastError would indicate that there was no error and there was nothing in the system or application event log to help me identify the problem.

Naturally, my first thought was that the IIS anonymous user is missing some access rights and I spent some time double-checking various permissions and privileges, but found nothing that would be relevant in this case. Suspecting that the problem lies elsewhere, I added the anonymous user to the Administrators group, which actually made things worse. Now not only CreateDC would still return NULL, but it would actually take about a minute for this call to fail!

The only way I could make CreateDC work was to call it from the security context of the Network Service user, which was undesirable in the context of the application I was working with. This started to get interesting.

What's my session ID?

I found the first bug quite quickly. It turned out that IIS sets up the worker process (w3wp.exe) such that the anonymous user cannot query the process. CreateDC, on the other hand, calls ProcessidToSessionId in order to figure out its Terminal Services session ID, which requires the user to have process query rights. So, ProcessIdToSessionId failed with the error Access Denied, which was thrown away by the caller, so upon return CreateDC returned no error.

In order to test this theory, I granted the Users group, which includes all authenticated users, the right to query the worker process using code similar to this simplified fragment:

CDacl dacl;
CSecurityDesc secdesc;
SECURITY_DESCRIPTOR *_secdesc = NULL;
ACL *_dacl = NULL;

if(GetSecurityInfo(GetCurrentProcess(), 
          SE_KERNEL_OBJECT, DACL_SECURITY_INFORMATION, 
          NULL, NULL, &_dacl, NULL, 
          (PSECURITY_DESCRIPTOR*) &_secdesc) == ERROR_SUCCESS) {
	secdesc = *_secdesc;
	dacl = *_dacl;
		
	dacl.AddAllowedAce(Sids::Users(), PROCESS_QUERY_INFORMATION);

	SetSecurityInfo(GetCurrentProcess(), SE_KERNEL_OBJECT,
          DACL_SECURITY_INFORMATION, NULL, NULL, 
          (PACL) dacl.GetPACL(), NULL);
}

, and then restored the original access rights after calling CreateDC. This allowed me to go past the first problem, but now CreateDC started to pause for about one minute before failing the same was it did when the anonymous user was a member of the Administrators group.

What RPC server?!

I attached a debugger to the worker process and after jumping through a few call stacks figured out that the process was busy calling __rpc_mgmt_is_server_listening@8, which was consistently failing to locate some RPC server, then calling __imp__Sleep@4 and repeating the entire sequence for about a minute.

The call stack indicated that IIS was trying to create a proxy object to allow the 32-bit IIS worker process call 64-bit printer driver:

rpcrt4.dll!_RpcMgmtIsServerListening@4()  + 0x28 bytes	
winspool.drv!_ConnectToLd64In32ServerWorker@8()  + 0x19b bytes	
winspool.drv!_ExternalConnectToLd64In32Server@4()  + 0x1a bytes	
gdi32.dll!PROXYPORT::PROXYPORT()  + 0x189 bytes	
gdi32.dll!_LoadUserModePrinterDriverEx@20()  + 0xfda bytes	
gdi32.dll!_LoadUserModePrinterDriver@16()  + 0x40 bytes	
gdi32.dll!_hdcCreateDCW@20()  + 0x51e3 bytes	
gdi32.dll!_bCreateDCW@20()  + 0x91 bytes	
gdi32.dll!_CreateDCW@16()  + 0x18 bytes	

After spending some time going through all of the RPC endpoints on my machine, I realized that the one IIS was trying to connect to was not available, so I spent some time looking for a reason this RPC server wasn't running.

Where did it go?!

After more digging through numerous screens of assembly instructions, I found out that the RPC server is supposed to be started by the worker process itself and, interestingly enough, the CreateProcess call:

winspool.drv!_ConnectToLd64In32ServerWorker@8()  + 0x20f bytes	
winspool.drv!_ExternalConnectToLd64In32Server@4()  + 0x1a bytes	
gdi32.dll!PROXYPORT::PROXYPORT()  + 0x189 bytes	
gdi32.dll!_LoadUserModePrinterDriverEx@20()  + 0xfda bytes	
gdi32.dll!_LoadUserModePrinterDriver@16()  + 0x40 bytes	
gdi32.dll!_hdcCreateDCW@20()  + 0x51e3 bytes	
gdi32.dll!_bCreateDCW@20()  + 0x91 bytes	
gdi32.dll!_CreateDCW@16()  + 0x18 bytes	

was actually successful. The name of the executable that was being launched was splwow64.exe, which was the missing RPC server, whose purpose is to host 64-bit printer drivers and communicate with 32-bit processes that need to print.

I started Process Monitor and set the filters to record all activity generated by splwow64.exe. Much to my surprise, none of what this process did triggered any errors (aside from some of the debug entries it did not find in the registry). splwow64.exe simply started, read a bunch of registry entries, loaded a few DLLs and then exited, leaving IIS puzzled about what just happened.

Hacky x86/x64 printing

Seeing some of the code and reading about splwow64.exe, I couldn't shake the feeling that this whole x86/x64 printing bridge was a hack somebody put together at the last minute before the release and then just left it there as is. For example, one knowledge base article indicates that only one user can use splwow64.exe at a time:

http://support.microsoft.com/kb/923357

It seems that printing from 32-bit processes definitely wasn't too high on the list of Windows deliverables.

Dear Microsoft

At this point I was running out of time and decided to contact Microsoft, to let people with source code take a stab at the problem. I created a sample project demonstrating the problem and accompanied it with a detailed description of what was going on.

Unlike with many other companies, I got through to Microsoft quite quickly. May be having to pay $300 for a premium support incident was helping a bit. A support engineer connected to my machine and I walked him through the code and demonstrated all the problems I identified. A few hours later I had to repeat everything to another guy who specialized in IIS/.Net and then again the next day to somebody from the printing team.

September 16th, 2009

Microsoft finally called me to inform that they have completed the investigation and created a bug. They also told me that this bug is low priority and will not be fixed in the foreseeable future, if ever due to small number of complaints. All customers who encountered this bug are advised to move to Windows 7 and Windows Server 2008 RC2, where all listed issues have been fixed. I hope they are right.

Comments:
Posted Wed Oct 6 11:26:37 EDT 2010 by Pierre
Under what version of Windows did you experience this bug? I have a few customers reporting problems with Amyuni PDF Converter. My app is 32-bit, not IIS, running under Win 7/64-bit. The PDF driver is 64-bit. Pretty impressive problem analysis BTW.
Posted Wed Oct 6 15:26:44 EDT 2010 by Andre

Thanks! I was running x64 Windows 2003 Server when I run into this issue.

Posted Wed Oct 6 15:40:34 EDT 2010 by Andre

Oh, one more thing. I recall that I was able to work around the problem by calling CreateDC from the application's startup code and hanging onto the returned DC, which caused this DC to be cached somewhere in the printing code.

Posted Fri May 13 06:01:53 EDT 2011 by Christoph

Just as a note:

The 32-64 bit printer thunking i still FUBAR in Windows 2008 R2

Posted Tue May 24 11:27:29 EDT 2011 by yogeshddd

Hi Andre,

I am ble to reproduce a similar issue on Windows 7. Creating a prin ter device context using CreateDC returns null and GetLastError() indicates that there is no error. I disabled the UAC feature on Windows 7 but to no avail. The process runs as under User-Account that has Administrator privileges. This issue is reproducible while printing from Adobe 32 bit Engine using a 64 bit driver but while printing from other 32 bit apps, i don't see this problem. I am a little clueless at this point. Your inputs will be appreciated. Thanks in Advance!

Posted Thu Apr 26 12:31:57 EDT 2012 by ChrisG

Hi there,

do you, by any chance, know if MS has been doing something on that matter?

We have the same issue with Windows7 + IIS + 32bit web app which tries to create PDF documents by printing to PS printer and using Ghostscript to convert to PDF.

When trying to query DocumentProperties on the PS printer we still fail with "RPC server can't be located".

Cheers,

Christoph

Posted Mon May 7 21:38:58 EDT 2012 by Andre

MS said they won't fix it and that moving to a 64-bit app is the only way. It was a while ago and I don't remember all details, but they confirmed that holding onto the device context would work with some limitations (I think one printing user was one of them).

Posted Wed Jan 9 17:50:54 EST 2013 by Bird Programmer

Thanks Andre! I have waisted several months of development time pinpointing this problem on Windows 2008 64 bit server. Honestly, Windows 2008 64 bit sucks.

Name:

Comment: