为应用程序池“XX”提供服务的进程在与 Windows Process Activation Service 通信时出现严重错误

场景

WCF应用程序部署在IIS7中，使用net.tcp协议对外给几百台客户端提供服务，应用程序池不断崩溃重启。

分析过程

在事件查看器中看到的错误信息类似于

为应用程序池“XX”提供服务的进程在与 Windows Process Activation Service 通信时出现严重错误

使用windbg在崩溃时进行分析，找到引发崩溃的堆栈的最顶部信息进行搜索，找到两个补丁包安装上

用于搜索的顶部异常信息为：IOCompletionThunk.UnhandledExceptionFrame BeginReadCore

0:084> !analyze -v
*******************************************************************************
* *
* Exception Analysis *
* *
*******************************************************************************

***** OS symbols are WRONG. Please fix symbols to do analysis.

*************************************************************************
*** ***
*** ***
*** Either you specified an unqualified symbol, or your debugger ***
*** doesn‘t have full symbol information. Unqualified symbol ***
*** resolution is turned off by default. Please either specify a ***
*** fully qualified symbol module!symbolname, or enable resolution ***
*** of unqualified symbols by typing ".symopt- 100". Note that ***
*** enabling unqualified symbol resolution with network symbol ***
*** server shares in the symbol path may cause the debugger to ***
*** appear to hang for long periods of time when an incorrect ***
*** symbol name is typed or the network symbol server is down. ***
*** ***
*** For some commands to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: ntdll!_PEB ***
*** ***
*************************************************************************
*************************************************************************
*** ***
*** ***
*** Either you specified an unqualified symbol, or your debugger ***
*** doesn‘t have full symbol information. Unqualified symbol ***
*** resolution is turned off by default. Please either specify a ***
*** fully qualified symbol module!symbolname, or enable resolution ***
*** of unqualified symbols by typing ".symopt- 100". Note that ***
*** enabling unqualified symbol resolution with network symbol ***
*** server shares in the symbol path may cause the debugger to ***
*** appear to hang for long periods of time when an incorrect ***
*** symbol name is typed or the network symbol server is down. ***
*** ***
*** For some commands to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: nt!IMAGE_NT_HEADERS32 ***
*** ***
*************************************************************************

FAULTING_IP:
KERNELBASE!RaiseException+3d
000007fe`fdbfac3d 4881c4c8000000 add rsp,0C8h

EXCEPTION_RECORD: 0000000000000001 -- (.exr 0x1)
Cannot read Exception record @ 0000000000000001

DEFAULT_BUCKET_ID: WRONG_SYMBOLS

PROCESS_NAME: w3wp.exe

ADDITIONAL_DEBUG_TEXT:
You can run ‘.symfix; .reload‘ to try to fix the symbol path and load symbols.

FAULTING_MODULE: 0000000077930000 ntdll

DEBUG_FLR_IMAGE_TIMESTAMP: 0

ERROR_CODE: (NTSTATUS) 0xe0434352 - <Unable to get error code text>

EXCEPTION_CODE: (NTSTATUS) 0xe0434352 - <Unable to get error code text>

EXCEPTION_PARAMETER1: ffffffff80131501

EXCEPTION_PARAMETER2: 0000000000000000

EXCEPTION_PARAMETER3: 0000000000000000

EXCEPTION_PARAMETER4: 0

APP: w3wp.exe

MANAGED_STACK:
SP IP Function
0000000009DC9CE0 000007FEE1BBF617 System_ServiceModel_ni!System.ServiceModel.Channels.StreamConnection.BeginRead(Int32, Int32, System.TimeSpan, System.Threading.WaitCallback, System.Object)+0xfa7b67
0000000009DCE2B0 000007FEE0C3AE5E System_ServiceModel_ni!System.ServiceModel.Channels.SessionConnectionReader.BeginReceive(System.TimeSpan, System.Threading.WaitCallback, System.Object)+0x1ee
0000000009DCE340 000007FEE0C3AC56 System_ServiceModel_ni!System.ServiceModel.Channels.SynchronizedMessageSource+ReceiveAsyncResult.PerformOperation(System.TimeSpan)+0x46
0000000009DCE380 000007FEE0C3AB1B System_ServiceModel_ni!System.ServiceModel.Channels.SynchronizedMessageSource+SynchronizedAsyncResult`1[[System.__Canon, mscorlib]]..ctor(System.ServiceModel.Channels.SynchronizedMessageSource, System.TimeSpan, System.AsyncCallback, System.Object)+0x12b
0000000009DCE410 000007FEE0C3A997 System_ServiceModel_ni!System.ServiceModel.Channels.FramingDuplexSessionChannel.BeginReceive(System.TimeSpan, System.AsyncCallback, System.Object)+0xe7
0000000009DCE4E0 000007FEE0C3A846 System_ServiceModel_ni!System.ServiceModel.Channels.FramingDuplexSessionChannel+TryReceiveAsyncResult..ctor(System.ServiceModel.Channels.FramingDuplexSessionChannel, System.TimeSpan, System.AsyncCallback, System.Object)+0x76
0000000009DCE550 000007FEE0C3A7BB System_ServiceModel_ni!System.ServiceModel.Channels.FramingDuplexSessionChannel.BeginTryReceive(System.TimeSpan, System.AsyncCallback, System.Object)+0x3b
0000000009DCE5B0 000007FEE132B224 System_ServiceModel_ni!System.ServiceModel.Channels.InputChannelWrapper.BeginTryReceive(System.TimeSpan, System.AsyncCallback, System.Object)+0x114
0000000009DCE650 000007FEE13386E8 System_ServiceModel_ni!System.ServiceModel.Channels.ReplyOverDuplexChannelBase`1[[System.__Canon, mscorlib]].BeginTryReceiveRequest(System.TimeSpan, System.AsyncCallback, System.Object)+0x18
0000000009DCE680 000007FEE162EE18 System_ServiceModel_ni!System.ServiceModel.Channels.SecurityChannelListener`1+ReceiveItemAndVerifySecurityAsyncResult`2[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].StartInnerReceive()+0x88
0000000009DCE6D0 000007FEE162ED30 System_ServiceModel_ni!System.ServiceModel.Channels.SecurityChannelListener`1+ReceiveItemAndVerifySecurityAsyncResult`2[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].Start()+0x10
0000
EXCEPTION_OBJECT: !pe 1bffc9ab8
Exception object: 00000001bffc9ab8
Exception type: System.ServiceModel.CommunicationException
Message: 套接字连接已中止。这可能是由于处理消息时出错或远程主机超过接收超时或者潜在的网络资源问题导致的。本地套接字超时是“10675199.02:48:05.4775807”。
InnerException: System.IO.IOException, Use !PrintException 00000001bffc9878 to see more.
StackTrace (generated):
SP IP Function
0000000009DC9CE0 000007FEE1BBF617 System_ServiceModel_ni!System.ServiceModel.Channels.StreamConnection.BeginRead(Int32, Int32, System.TimeSpan, System.Threading.WaitCallback, System.Object)+0xfa7b67
0000000009DCE2B0 000007FEE0C3AE5E System_ServiceModel_ni!System.ServiceModel.Channels.SessionConnectionReader.BeginReceive(System.TimeSpan, System.Threading.WaitCallback, System.Object)+0x1ee
0000000009DCE340 000007FEE0C3AC56 System_ServiceModel_ni!System.ServiceModel.Channels.SynchronizedMessageSource+ReceiveAsyncResult.PerformOperation(System.TimeSpan)+0x46
0000000009DCE380 000007FEE0C3AB1B System_ServiceModel_ni!System.ServiceModel.Channels.SynchronizedMessageSource+SynchronizedAsyncResult`1[[System.__Canon, mscorlib]]..ctor(System.ServiceModel.Channels.SynchronizedMessageSource, System.TimeSpan, System.AsyncCallback, System.Object)+0x12b
0000000009DCE410 000007FEE0C3A997 System_ServiceModel_ni!System.ServiceModel.Channels.FramingDuplexSessionChannel.BeginReceive(System.TimeSpan, System.AsyncCallback, System.Object)+0xe7
0000000009DCE4E0 000007FEE0C3A846 System_ServiceModel_ni!System.ServiceModel.Channels.FramingDuplexSessionChannel+TryReceiveAsyncResult..ctor(System.ServiceModel.Channels.FramingDuplexSessionChannel, System.TimeSpan, System.AsyncCallback, System.Object)+0x76
0000000009DCE550 000007FEE0C3A7BB System_ServiceModel_ni!System.ServiceModel.Channels.FramingDuplexSessionChannel.BeginTryReceive(System.TimeSpan, System.AsyncCallback, System.Object)+0x3b
0000000009DCE5B0 000007FEE132B224 System_ServiceModel_ni!System.ServiceModel.Channels.InputChannelWrapper.BeginTryReceive(System.TimeSpan, System.AsyncCallback, System.Object)+0x114
0000000009DCE650 000007FEE13386E8 System_ServiceModel_ni!System.ServiceModel.Channels.ReplyOverDuplexChannelBase`1[[System.__Canon, mscorlib]].BeginTryReceiveRequest(System.TimeSpan, System.AsyncCallback, System.Object)+0x18
0000000009DCE680 000007FEE162EE18 System_ServiceModel_ni!System.ServiceModel.Channels.SecurityChannelListener`1+ReceiveItemAndVerifySecurityAsyncResult`2[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].StartInnerReceive()+0x88
0000000009DCE6D0 000007FEE162ED30 System_ServiceModel_ni!System.ServiceModel.Channels.SecurityChannelListener`1+ReceiveItemAndVerifySecurityAsyncResult`2[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].Start()+0x10
0000000009DCE700 000007FEE615EDB1 System_Runtime_DurableInstancing_ni!System.Runtime.IOThreadScheduler+ScheduledOverlapped.IOCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)+0x71
0000000009DCE760 000007FEE6236A10 System_Runtime_DurableInstancing_ni!System.Runtime.Fx+IOCompletionThunk.UnhandledExceptionFrame(UInt32, UInt32, System.Threading.NativeOverlapped*)+0xdbc70
0000000009DCE7C0 000007FEF7F95DA6 mscorlib_ni!System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)+0x96

StackTraceString: <none>
HResult: 80131501
There are nested exceptions on this thread. Run with -nested for details

EXCEPTION_OBJECT: !pe 1bffc9878
Exception object: 00000001bffc9878
Exception type: System.IO.IOException
Message: 读取操作失败，请参见内部异常。
InnerException: System.ServiceModel.CommunicationException, Use !PrintException 0000000120009910 to see more.
StackTrace (generated):
SP IP Function
0000000009DCBD10 000007FEF697945C System_ni!System.Net.Security._SslStream.ProcessRead(Byte[], Int32, Int32, System.Net.AsyncProtocolRequest)+0x28c
0000000009DCE1A0 000007FEF697AD4D System_ni!System.Net.Security._SslStream.BeginRead(Byte[], Int32, Int32, System.AsyncCallback, System.Object)+0xbd
0000000009DCE210 000007FEE0C17BC4 System_ServiceModel_ni!System.ServiceModel.Channels.StreamConnection.BeginRead(Int32, Int32, System.TimeSpan, System.Threading.WaitCallback, System.Object)+0x114

StackTraceString: <none>
HResult: 80131620
There are nested exceptions on this thread. Run with -nested for details

EXCEPTION_OBJECT: !pe 120009910
Exception object: 0000000120009910
Exception type: System.ServiceModel.CommunicationException
Message: 套接字连接已中止。这可能是由于处理消息时出错或远程主机超过接收超时或者潜在的网络资源问题导致的。本地套接字超时是“10675199.02:48:05.4775807”。
InnerException: System.Net.Sockets.SocketException, Use !PrintException 0000000120008e50 to see more.
StackTrace (generated):
SP IP Function
0000000009DCDD40 000007FF001874CC System_ServiceModel_ni!System.ServiceModel.Channels.SocketConnection.BeginReadCore(Int32, Int32, System.TimeSpan, System.Threading.WaitCallback, System.Object)+0x4bc
0000000009DCDE40 000007FEE15142A4 System_ServiceModel_ni!System.ServiceModel.Channels.TracingConnection.BeginRead(Int32, Int32, System.TimeSpan, System.Threading.WaitCallback, System.Object)+0x144
0000000009DCDEE0 000007FEE0C17D6E System_ServiceModel_ni!System.ServiceModel.Channels.ConnectionStream+ReadAsyncResult..ctor(System.ServiceModel.Channels.IConnection, Byte[], Int32, Int32, System.TimeSpan, System.AsyncCallback, System.Object)+0xbe
0000000009DCDF40 000007FEE0C17C98 System_ServiceModel_ni!System.ServiceModel.Channels.ConnectionStream.BeginRead(Byte[], Int32, Int32, System.AsyncCallback, System.Object)+0x78
0000000009DCDFC0 000007FEF697AE1C System_ni!System.Net.FixedSizeReader.StartReading()+0x5c
0000000009DCE010 000007FEF69798DF System_ni!System.Net.Security._SslStream.StartFrameHeader(Byte[], Int32, Int32, System.Net.AsyncProtocolRequest)+0x10f
0000000009DCE090 000007FEF6979528 System_ni!System.Net.Security._SslStream.StartReading(Byte[], Int32, Int32, System.Net.AsyncProtocolRequest)+0x78
0000000009DCE120 000007FEF6979331 System_ni!System.Net.Security._SslStream.ProcessRead(Byte[], Int32, Int32, System.Net.AsyncProtocolRequest)+0x161

StackTraceString: <none>
HResult: 80131501
There are nested exceptions on this thread. Run with -nested for details

EXCEPTION_OBJECT: !pe 120008e50
Exception object: 0000000120008e50
Exception type: System.Net.Sockets.SocketException
Message: 远程主机强迫关闭了一个现有的连接。
InnerException: <none>
StackTrace (generated):
<none>
StackTraceString: <none>
HResult: 80004005
There are nested exceptions on this thread. Run with -nested for details

MANAGED_OBJECT: !dumpobj 120009358
Name: System.String
MethodTable: 000007fef79b6738
EEClass: 000007fef753ed68
Size: 60(0x3c) bytes
File: C:\Windows\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll
String: 远程主机强迫关闭了一个现有的连接。
Fields:
MT Field Offset Type VT Attr Value Name
000007fef79bc620 4000103 8 System.Int32 1 instance 17 m_stringLength
000007fef79bb160 4000104 c System.Char 1 instance 8fdc m_firstChar
000007fef79b6738 4000105 10 System.String 0 shared static Empty
>> Domain:Value 00000000015cee70:000000015fc50488 00000000029b99b0:000000015fc50488 <<

EXCEPTION_MESSAGE: ??3ì?÷?ú???è1?±?á?ò?????óDμ?á??ó?￡

MANAGED_OBJECT_NAME: SYSTEM.NET.SOCKETS.SOCKETEXCEPTION

MANAGED_STACK_COMMAND: !do 0000000120008E50

CONTEXT: 0000000000000001 -- (.cxr 0x1)
Unable to read context, HRESULT 0x80004002

LAST_CONTROL_TRANSFER: from 000007fef8b86d1d to 000007fefdbfac3d

PRIMARY_PROBLEM_CLASS: WRONG_SYMBOLS

BUGCHECK_STR: APPLICATION_FAULT_WRONG_SYMBOLS_CLR_EXCEPTION

STACK_TEXT:
00000000`09dc9ce0 00000000`ffffffff system_servicemodel_ni!System.ServiceModel.Channels.StreamConnection.BeginRead+0xfa7b67
00000000`09dce2b0 00000000`ffffffff system_servicemodel_ni!System.ServiceModel.Channels.SessionConnectionReader.BeginReceive+0x1ee
00000000`09dce340 00000000`ffffffff system_servicemodel_ni!System.ServiceModel.Channels.SynchronizedMessageSource+ReceiveAsyncResult.PerformOperation+0x46
00000000`09dce380 00000000`ffffffff system_servicemodel_ni!System.ServiceModel.Channels.SynchronizedMessageSource+SynchronizedAsyncResult_1[[System.__Canon,_mscorlib]]..ctor+0x12b
00000000`09dce410 00000000`ffffffff system_servicemodel_ni!System.ServiceModel.Channels.FramingDuplexSessionChannel.BeginReceive+0xe7
00000000`09dce4e0 00000000`ffffffff system_servicemodel_ni!System.ServiceModel.Channels.FramingDuplexSessionChannel+TryReceiveAsyncResult..ctor+0x76
00000000`09dce550 00000000`ffffffff system_servicemodel_ni!System.ServiceModel.Channels.FramingDuplexSessionChannel.BeginTryReceive+0x3b
00000000`09dce5b0 00000000`ffffffff system_servicemodel_ni!System.ServiceModel.Channels.InputChannelWrapper.BeginTryReceive+0x114
00000000`09dce650 00000000`ffffffff system_servicemodel_ni!System.ServiceModel.Channels.ReplyOverDuplexChannelBase_1[[System.__Canon,_mscorlib]].BeginTryReceiveRequest+0x18
00000000`09dce680 00000000`ffffffff system_servicemodel_ni!System.ServiceModel.Channels.SecurityChannelListener_1+ReceiveItemAndVerifySecurityAsyncResult_2[[System.__Canon,_mscorlib],[System.__Canon,_mscorlib],[System.__Canon,_mscorlib]].StartInnerReceive+0x88
00000000`09dce6d0 00000000`ffffffff system_servicemodel_ni!System.ServiceModel.Channels.SecurityChannelListener_1+ReceiveItemAndVerifySecurityAsyncResult_2[[System.__Canon,_mscorlib],[System.__Canon,_mscorlib],[System.__Canon,_mscorlib]].Start+0x10
00000000`09dce700 00000000`ffffffff system_runtime_durableinstancing_ni!System.Runtime.IOThreadScheduler+ScheduledOverlapped.IOCallback+0x71
00000000`09dce760 00000000`ffffffff system_runtime_durableinstancing_ni!System.Runtime.Fx+IOCompletionThunk.UnhandledExceptionFrame+0xdbc70
00000000`09dce7c0 00000000`ffffffff mscorlib_ni!System.Threading._IOCompletionCallback.PerformIOCompletionCallback+0x96

FOLLOWUP_IP:
unknown!fillpattern+0
00000000`ffffffff 006d00 add byte ptr [rbp],ch

SYMBOL_STACK_INDEX: b

SYMBOL_NAME: system_runtime_durableinstancing_ni!System.Runtime.IOThreadScheduler+ScheduledOverlapped.IOCallback+71

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: system_runtime_durableinstancing_ni

STACK_COMMAND: !do 0000000120008E50
; ** Pseudo Context ** ; kb

BUCKET_ID: WRONG_SYMBOLS

IMAGE_NAME: System.Runtime.DurableInstancing.ni.dll

FAILURE_BUCKET_ID: WRONG_SYMBOLS_e0434352_System.Runtime.DurableInstancing.ni.dll!System.Runtime.IOThreadScheduler+ScheduledOverlapped.IOCallback

WATSON_STAGEONE_URL: http://watson.microsoft.com/StageOne/w3wp_exe/7_5_7600_16385/4a5bd0eb/KERNELBASE_dll/6_1_7600_17206/50e669a2/e0434352/0000ac3d.htm?Retriage=1

Followup: MachineOwner
---------

0:084> !clrstack
OS Thread Id: 0x854 (84)
Child SP IP Call Site
0000000009dc7ba8 000007fefdbfac3d [HelperMethodFrame: 0000000009dc7ba8]
0000000009dc7cc0 000007fee6236a10 System.Runtime.Fx+IOCompletionThunk.UnhandledExceptionFrame(UInt32, UInt32, System.Threading.NativeOverlapped*)
0000000009dcdbf8 000007fef8b865ed [HelperMethodFrame: 0000000009dcdbf8]
0000000009dcdd40 000007ff001874cc System.ServiceModel.Channels.SocketConnection.BeginReadCore(Int32, Int32, System.TimeSpan, System.Threading.WaitCallback, System.Object)
0000000009dcde40 000007fee15142a4 System.ServiceModel.Channels.TracingConnection.BeginRead(Int32, Int32, System.TimeSpan, System.Threading.WaitCallback, System.Object)
0000000009dcdee0 000007fee0c17d6e System.ServiceModel.Channels.ConnectionStream+ReadAsyncResult..ctor(System.ServiceModel.Channels.IConnection, Byte[], Int32, Int32, System.TimeSpan, System.AsyncCallback, System.Object)
0000000009dcdf40 000007fee0c17c98 System.ServiceModel.Channels.ConnectionStream.BeginRead(Byte[], Int32, Int32, System.AsyncCallback, System.Object)
0000000009dcdfc0 000007fef697ae1c System.Net.FixedSizeReader.StartReading()
0000000009dce010 000007fef69798df System.Net.Security._SslStream.StartFrameHeader(Byte[], Int32, Int32, System.Net.AsyncProtocolRequest)
0000000009dce090 000007fef6979528 System.Net.Security._SslStream.StartReading(Byte[], Int32, Int32, System.Net.AsyncProtocolRequest)
0000000009dce120 000007fef6979331 System.Net.Security._SslStream.ProcessRead(Byte[], Int32, Int32, System.Net.AsyncProtocolRequest)
0000000009dce1a0 000007fef697ad4d System.Net.Security._SslStream.BeginRead(Byte[], Int32, Int32, System.AsyncCallback, System.Object)
0000000009dce210 000007fee0c17bc4 System.ServiceModel.Channels.StreamConnection.BeginRead(Int32, Int32, System.TimeSpan, System.Threading.WaitCallback, System.Object)
0000000009dce2b0 000007fee0c3ae5e System.ServiceModel.Channels.SessionConnectionReader.BeginReceive(System.TimeSpan, System.Threading.WaitCallback, System.Object)
0000000009dce340 000007fee0c3ac56 System.ServiceModel.Channels.SynchronizedMessageSource+ReceiveAsyncResult.PerformOperation(System.TimeSpan)
0000000009dce380 000007fee0c3ab1b System.ServiceModel.Channels.SynchronizedMessageSource+SynchronizedAsyncResult`1[[System.__Canon, mscorlib]]..ctor(System.ServiceModel.Channels.SynchronizedMessageSource, System.TimeSpan, System.AsyncCallback, System.Object)
0000000009dce410 000007fee0c3a997 System.ServiceModel.Channels.FramingDuplexSessionChannel.BeginReceive(System.TimeSpan, System.AsyncCallback, System.Object)
0000000009dce4e0 000007fee0c3a846 System.ServiceModel.Channels.FramingDuplexSessionChannel+TryReceiveAsyncResult..ctor(System.ServiceModel.Channels.FramingDuplexSessionChannel, System.TimeSpan, System.AsyncCallback, System.Object)
0000000009dce550 000007fee0c3a7bb System.ServiceModel.Channels.FramingDuplexSessionChannel.BeginTryReceive(System.TimeSpan, System.AsyncCallback, System.Object)
0000000009dce5b0 000007fee132b224 System.ServiceModel.Channels.InputChannelWrapper.BeginTryReceive(System.TimeSpan, System.AsyncCallback, System.Object)
0000000009dce650 000007fee13386e8 System.ServiceModel.Channels.ReplyOverDuplexChannelBase`1[[System.__Canon, mscorlib]].BeginTryReceiveRequest(System.TimeSpan, System.AsyncCallback, System.Object)
0000000009dce680 000007fee162ee18 System.ServiceModel.Channels.SecurityChannelListener`1+ReceiveItemAndVerifySecurityAsyncResult`2[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].StartInnerReceive()
0000000009dce6d0 000007fee162ed30 System.ServiceModel.Channels.SecurityChannelListener`1+ReceiveItemAndVerifySecurityAsyncResult`2[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].Start()
0000000009dce700 000007fee615edb1 System.Runtime.IOThreadScheduler+ScheduledOverlapped.IOCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)
0000000009dce760 000007fee615ade0 System.Runtime.Fx+IOCompletionThunk.UnhandledExceptionFrame(UInt32, UInt32, System.Threading.NativeOverlapped*)
0000000009dce7c0 000007fef7f95da6 System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)
0000000009dcea60 000007fef8aaee74 [GCFrame: 0000000009dcea60]
0000000009dcec30 000007fef8aaee74 [DebuggerU2MCatchHandlerFrame: 0000000009dcec30]
0000000009dcee08 000007fef8aaee74 [ContextTransitionFrame: 0000000009dcee08]
0000000009dceff0 000007fef8aaee74 [DebuggerU2MCatchHandlerFrame: 0000000009dceff0]
0:084> !threads
ThreadCount: 63
UnstartedThread: 0
BackgroundThread: 62
PendingThread: 0
DeadThread: 0
Hosted Runtime: no
PreEmptive Lock
ID OSID ThreadOBJ State GC GC Alloc Context Domain Count APT Exception
4 1 1c5c 00000000028e2250 8220 Enabled 0000000000000000:0000000000000000 00000000015cee70 0 Ukn
31 2 1bf8 00000000028f8bc0 b220 Enabled 000000011fff0be8:000000011fff0cb0 00000000015cee70 0 MTA (Finalizer)
34 3 12c8 0000000002958eb0 100a220 Enabled 0000000000000000:0000000000000000 00000000015cee70 0 MTA (Threadpool Worker)
35 4 1580 00000000029ba610 1220 Enabled 0000000000000000:0000000000000000 00000000015cee70 0 Ukn
36 5 13e4 0000000002a4dd70 8009220 Disabled 000000014029cfe0:000000014029cff8 00000000029b99b0 0 MTA (Threadpool Completion Port) System.ServiceModel.CommunicationException (00000001401cca08)
21 6 19e0 0000000002a4e480 1008220 Enabled 000000010046df58:000000010046e608 00000000015cee70 0 MTA (Threadpool Worker)
37 7 243c 0000000002a4f070 a009220 Enabled 0000000000000000:0000000000000000 00000000029b99b0 0 MTA (Threadpool Completion Port)
38 8 15d0 0000000002a80c40 200b220 Enabled 0000000000000000:0000000000000000 00000000029b99b0 0 MTA
39 9 1644 0000000002a8a520 200b020 Enabled 0000000120034a90:0000000120036458 00000000029b99b0 0 MTA
41 a 1054 0000000005cdb0c0 3009220 Enabled 0000000000000000:0000000000000000 00000000029b99b0 0 MTA (Threadpool Worker)
42 b d48 0000000005cf9610 1009220 Enabled 0000000000000000:0000000000000000 00000000015cee70 0 MTA (Threadpool Worker)
43 c 189c 0000000005cebf50 8009220 Disabled 0000000160aa8e70:0000000160aaa5b0 00000000029b99b0 0 MTA (Threadpool Completion Port)
44 d 193c 0000000005d0df50 8009220 Enabled 00000001802849e0:0000000180286900 00000000029b99b0 0 MTA (Threadpool Completion Port)
45 e f9c 0000000005d17f70 8009220 Enabled 00000001dffaa410:00000001dffac330 00000000029b99b0 0 MTA (Threadpool Completion Port)
46 f 224 0000000005d15b00 8009220 Enabled 000000011fff6d90:000000011fff8cb0 00000000029b99b0 0 MTA (Threadpool Completion Port)
47 10 1038 0000000005d22410 a009220 Enabled 0000000000000000:0000000000000000 00000000029b99b0 0 MTA (Threadpool Completion Port)
48 11 1698 0000000005dac730 8009220 Enabled 00000001802a49b8:00000001802a6900 00000000029b99b0 0 MTA (Threadpool Completion Port)
49 12 1228 0000000005dc78f0 1009220 Enabled 0000000000000000:0000000000000000 00000000015cee70 0 MTA (Threadpool Worker)
50 13 1f08 0000000005dc8000 8009220 Enabled 00000001a0393b00:00000001a0394158 00000000015cee70 0 MTA (Threadpool Completion Port)
51 14 16a0 0000000005dc8710 8009220 Enabled 00000001e03aaac8:00000001e03ab968 00000000015cee70 0 MTA (Threadpool Completion Port)
52 15 1498 0000000005dc8e20 1009220 Enabled 0000000000000000:0000000000000000 00000000015cee70 0 MTA (Threadpool Worker)
53 16 22b0 0000000005dc9530 1009220 Enabled 0000000000000000:0000000000000000 00000000015cee70 0 MTA (Threadpool Worker)
54 17 e30 0000000005dc9c40 1009220 Enabled 0000000000000000:0000000000000000 00000000015cee70 0 MTA (Threadpool Worker)
55 18 1650 0000000005dca350 1009220 Enabled 0000000000000000:0000000000000000 00000000015cee70 0 MTA (Threadpool Worker)
56 19 1514 0000000005dcaa60 1009220 Enabled 0000000000000000:0000000000000000 00000000015cee70 0 MTA (Threadpool Worker)
57 1a a14 0000000005dcb170 1009220 Enabled 0000000000000000:0000000000000000 00000000015cee70 0 MTA (Threadpool Worker)
58 1b 4a8 0000000005dcb880 8009220 Disabled 00000001c0369568:00000001c03697d0 00000000029b99b0 0 MTA (Threadpool Completion Port) System.UnauthorizedAccessException (00000001c03471b0) (nested exceptions)
59 1c a80 0000000005dcbf90 a009220 Enabled 0000000000000000:0000000000000000 00000000029b99b0 0 MTA (Threadpool Completion Port)
61 1d 1f14 0000000005dcc6a0 200b220 Enabled 0000000000000000:0000000000000000 00000000029b99b0 1 MTA
62 1e 1250 0000000005dccdb0 8009220 Enabled 000000011fffad40:000000011fffccb0 00000000029b99b0 0 MTA (Threadpool Completion Port)
63 1f 22c4 0000000005dcd4c0 8009220 Enabled 00000001401a2ae8:00000001401a4a08 00000000029b99b0 0 MTA (Threadpool Completion Port)
64 20 1690 0000000005dcdbd0 a009220 Enabled 0000000000000000:0000000000000000 00000000029b99b0 0 MTA (Threadpool Completion Port)
65 21 2648 0000000005dce2e0 8009220 Enabled 00000001401a4ac0:00000001401a6a08 00000000029b99b0 0 MTA (Threadpool Completion Port)
67 22 1134 0000000005dce9f0 a009220 Enabled 0000000000000000:0000000000000000 00000000029b99b0 0 MTA (Threadpool Completion Port)
68 23 1b20 0000000005dcf100 8009220 Disabled 00000001e03b7b40:00000001e03b9968 00000000029b99b0 0 MTA (Threadpool Completion Port)
69 24 1574 0000000005d4dc00 8009220 Enabled 00000001802869e0:0000000180288900 00000000029b99b0 0 MTA (Threadpool Completion Port)
70 25 18a0 0000000005d4e310 8009220 Enabled 00000001a01ab2d0:00000001a01ad240 00000000029b99b0 0 MTA (Threadpool Completion Port)
71 26 1954 0000000005d4ea20 8009220 Enabled 00000001bff2eb28:00000001bff30a48 00000000029b99b0 0 MTA (Threadpool Completion Port)
72 27 db8 0000000005d4f130 8009220 Disabled 0000000160aaafa0:0000000160aac5b0 00000000029b99b0 1 MTA (Threadpool Completion Port)
73 28 1604 0000000005d4f840 8009220 Enabled 0000000160974b98:00000001609760b8 00000000015cee70 0 MTA (Threadpool Completion Port)
74 29 11fc 0000000005d4ff50 8009220 Enabled 00000001401aa1b8:00000001401aaa08 00000000015cee70 0 MTA (Threadpool Completion Port)
75 2a 137c 0000000005d50660 1009220 Enabled 0000000000000000:0000000000000000 00000000015cee70 0 MTA (Threadpool Worker)
76 2b 25b4 0000000005d50d70 1009220 Enabled 0000000000000000:0000000000000000 00000000015cee70 0 MTA (Threadpool Worker)
77 2c fec 0000000005d51480 1009220 Enabled 0000000000000000:0000000000000000 00000000015cee70 0 MTA (Threadpool Worker)
78 2d 22b8 0000000007f86b00 1009220 Enabled 0000000000000000:0000000000000000 00000000015cee70 0 MTA (Threadpool Worker)
80 2e 1c48 0000000007f87210 1009220 Enabled 0000000000000000:0000000000000000 00000000015cee70 0 MTA (Threadpool Worker)
79 2f 2168 0000000007f87920 1009220 Enabled 0000000000000000:0000000000000000 00000000015cee70 0 MTA (Threadpool Worker)
81 30 1660 0000000007f88030 1009220 Enabled 0000000000000000:0000000000000000 00000000015cee70 0 MTA (Threadpool Worker)
82 31 96c 0000000007f88740 8009220 Enabled 000000011fff4d40:000000011fff6cb0 00000000029b99b0 0 MTA (Threadpool Completion Port)
83 32 1418 0000000007f88e50 8009220 Enabled 00000001001b8378:00000001001ba298 00000000029b99b0 0 MTA (Threadpool Completion Port)
84 33 854 0000000007f89560 8009220 Enabled 0000000100596dd0:0000000100598690 00000000029b99b0 0 MTA (Threadpool Completion Port) System.ServiceModel.CommunicationException (00000001bffc9ab8) (nested exceptions)
85 34 2798 0000000007f89c70 1009220 Enabled 0000000000000000:0000000000000000 00000000015cee70 0 MTA (Threadpool Worker)
86 35 2438 0000000007f8a380 1009220 Enabled 0000000000000000:0000000000000000 00000000015cee70 0 MTA (Threadpool Worker)
87 36 ee4 000000000a973760 1009220 Enabled 0000000000000000:0000000000000000 00000000015cee70 0 MTA (Threadpool Worker)
88 37 2764 000000000a973e70 1009220 Enabled 0000000000000000:0000000000000000 00000000015cee70 0 MTA (Threadpool Worker)
89 38 25c8 000000000a974580 1009220 Enabled 0000000000000000:0000000000000000 00000000015cee70 0 MTA (Threadpool Worker)
90 39 1b38 000000000a974c90 1009220 Enabled 0000000000000000:0000000000000000 00000000015cee70 0 MTA (Threadpool Worker)
91 3a 448 000000000a9753a0 8009220 Disabled 00000001c036f378:00000001c036f7d0 00000000029b99b0 0 MTA (Threadpool Completion Port)
92 3b 184c 000000000a975ab0 8009220 Enabled 00000001002bef30:00000001002c0710 00000000015cee70 0 MTA (Threadpool Completion Port)
93 3c d3c 000000000a9761c0 8009220 Enabled 00000001a01747d0:00000001a01766f0 00000000029b99b0 0 MTA (Threadpool Completion Port)
94 3d d18 000000000a9768d0 8009220 Enabled 000000011fff2d90:000000011fff4cb0 00000000029b99b0 0 MTA (Threadpool Completion Port)
95 3e 13e8 000000000a976fe0 8009220 Enabled 00000001802829b8:0000000180284900 00000000029b99b0 0 MTA (Threadpool Completion Port)
96 3f 26e0 000000000ada4850 8009220 Enabled 00000001a01727d0:00000001a01746f0 00000000029b99b0 0 MTA (Threadpool Completion Port)
0:084> ~58s
clr!CoUninitializeEE+0x31cc3:
000007fe`f8aa5a03 48895010 mov qword ptr [rax+10h],rdx ds:00000000`07e75a38=f054b3f8fe070000
0:058> !clrstack
OS Thread Id: 0x4a8 (58)
Child SP IP Call Site
0000000007e76560 000007fef8aa5a03 [GCFrame: 0000000007e76560]
0000000007e765f8 000007fef8aa5a03 [GCFrame: 0000000007e765f8]
0000000007e76730 000007fef8aa5a03 [GCFrame: 0000000007e76730]
0000000007e767b8 000007fef8aa5a03 [GCFrame: 0000000007e767b8]
0000000007e77948 000007fef8aa5a03 [HelperMethodFrame: 0000000007e77948]
0000000007e77a60 000007fee6236a10 System.Runtime.Fx+IOCompletionThunk.UnhandledExceptionFrame(UInt32, UInt32, System.Threading.NativeOverlapped*)
0000000007e7d998 000007fef8b865ed [HelperMethodFrame: 0000000007e7d998]
0000000007e7dae0 000007ff001874cc System.ServiceModel.Channels.SocketConnection.BeginReadCore(Int32, Int32, System.TimeSpan, System.Threading.WaitCallback, System.Object)
0000000007e7dbe0 000007fee15142a4 System.ServiceModel.Channels.TracingConnection.BeginRead(Int32, Int32, System.TimeSpan, System.Threading.WaitCallback, System.Object)
0000000007e7dc80 000007fee0c17d6e System.ServiceModel.Channels.ConnectionStream+ReadAsyncResult..ctor(System.ServiceModel.Channels.IConnection, Byte[], Int32, Int32, System.TimeSpan, System.AsyncCallback, System.Object)
0000000007e7dce0 000007fee0c17c98 System.ServiceModel.Channels.ConnectionStream.BeginRead(Byte[], Int32, Int32, System.AsyncCallback, System.Object)
0000000007e7dd60 000007fef697ae1c System.Net.FixedSizeReader.StartReading()
0000000007e7ddb0 000007fef69798df System.Net.Security._SslStream.StartFrameHeader(Byte[], Int32, Int32, System.Net.AsyncProtocolRequest)
0000000007e7de30 000007fef6979528 System.Net.Security._SslStream.StartReading(Byte[], Int32, Int32, System.Net.AsyncProtocolRequest)
0000000007e7dec0 000007fef6979331 System.Net.Security._SslStream.ProcessRead(Byte[], Int32, Int32, System.Net.AsyncProtocolRequest)
0000000007e7df40 000007fef697ad4d System.Net.Security._SslStream.BeginRead(Byte[], Int32, Int32, System.AsyncCallback, System.Object)
0000000007e7dfb0 000007fee0c17bc4 System.ServiceModel.Channels.StreamConnection.BeginRead(Int32, Int32, System.TimeSpan, System.Threading.WaitCallback, System.Object)
0000000007e7e050 000007fee0c3ae5e System.ServiceModel.Channels.SessionConnectionReader.BeginReceive(System.TimeSpan, System.Threading.WaitCallback, System.Object)
0000000007e7e0e0 000007fee0c3ac56 System.ServiceModel.Channels.SynchronizedMessageSource+ReceiveAsyncResult.PerformOperation(System.TimeSpan)
0000000007e7e120 000007fee0c3ab1b System.ServiceModel.Channels.SynchronizedMessageSource+SynchronizedAsyncResult`1[[System.__Canon, mscorlib]]..ctor(System.ServiceModel.Channels.SynchronizedMessageSource, System.TimeSpan, System.AsyncCallback, System.Object)
0000000007e7e1b0 000007fee0c3a997 System.ServiceModel.Channels.FramingDuplexSessionChannel.BeginReceive(System.TimeSpan, System.AsyncCallback, System.Object)
0000000007e7e280 000007fee0c3a846 System.ServiceModel.Channels.FramingDuplexSessionChannel+TryReceiveAsyncResult..ctor(System.ServiceModel.Channels.FramingDuplexSessionChannel, System.TimeSpan, System.AsyncCallback, System.Object)
0000000007e7e2f0 000007fee0c3a7bb System.ServiceModel.Channels.FramingDuplexSessionChannel.BeginTryReceive(System.TimeSpan, System.AsyncCallback, System.Object)
0000000007e7e350 000007fee132b224 System.ServiceModel.Channels.InputChannelWrapper.BeginTryReceive(System.TimeSpan, System.AsyncCallback, System.Object)
0000000007e7e3f0 000007fee13386e8 System.ServiceModel.Channels.ReplyOverDuplexChannelBase`1[[System.__Canon, mscorlib]].BeginTryReceiveRequest(System.TimeSpan, System.AsyncCallback, System.Object)
0000000007e7e420 000007fee162ee18 System.ServiceModel.Channels.SecurityChannelListener`1+ReceiveItemAndVerifySecurityAsyncResult`2[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].StartInnerReceive()
0000000007e7e470 000007fee162ed30 System.ServiceModel.Channels.SecurityChannelListener`1+ReceiveItemAndVerifySecurityAsyncResult`2[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].Start()
0000000007e7e4a0 000007fee615edb1 System.Runtime.IOThreadScheduler+ScheduledOverlapped.IOCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)
0000000007e7e500 000007fee615ade0 System.Runtime.Fx+IOCompletionThunk.UnhandledExceptionFrame(UInt32, UInt32, System.Threading.NativeOverlapped*)
0000000007e7e560 000007fef7f95da6 System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)
0000000007e7e800 000007fef8aaee74 [GCFrame: 0000000007e7e800]
0000000007e7e9d0 000007fef8aaee74 [DebuggerU2MCatchHandlerFrame: 0000000007e7e9d0]
0000000007e7eba8 000007fef8aaee74 [ContextTransitionFrame: 0000000007e7eba8]
0000000007e7ed90 000007fef8aaee74 [DebuggerU2MCatchHandlerFrame: 0000000007e7ed90]
0:058> ~36s
clr!CoUninitializeEE+0xe90:
000007fe`f8a74bd0 488361f800 and qword ptr [rcx-8],0 ds:00000001`4029f008=0000000000000000
0:036> !clrstack
OS Thread Id: 0x13e4 (36)
Child SP IP Call Site
00000000053cbfd8 000007fef8a74bd0 [HelperMethodFrame_2OBJ: 00000000053cbfd8] System.Diagnostics.StackTrace.GetStackFramesInternal(System.Diagnostics.StackFrameHelper, Int32, System.Exception)
00000000053cc340 000007fef78e18db System.Diagnostics.StackTrace.CaptureStackTrace(Int32, Boolean, System.Threading.Thread, System.Exception)
00000000053cc3a0 000007fef7ece60e System.Diagnostics.StackTrace..ctor(Int32, Boolean)
00000000053cc3f0 000007fee61c961c System.Runtime.Diagnostics.DiagnosticTrace.StackTraceString(System.Exception)
00000000053cc460 000007fee61c915c System.Runtime.Diagnostics.DiagnosticTrace.ExceptionToTraceString(System.Exception)
00000000053cc4f0 000007fee61c941a System.Runtime.Diagnostics.DiagnosticTrace.ExceptionToTraceString(System.Exception)
00000000053cc580 000007fee61c941a System.Runtime.Diagnostics.DiagnosticTrace.ExceptionToTraceString(System.Exception)
00000000053cc610 000007fee61c941a System.Runtime.Diagnostics.DiagnosticTrace.ExceptionToTraceString(System.Exception)
00000000053cc6a0 000007fee623645d System.Runtime.Diagnostics.DiagnosticTrace.GetSerializedPayload(System.Object, System.Runtime.Diagnostics.TraceRecord, System.Exception, Boolean)
00000000053cc760 000007fee61c6f07 System.Runtime.TraceCore.ThrowingException(System.Runtime.Diagnostics.DiagnosticTrace, System.String, System.Exception)
00000000053cc840 000007fee6236878 System.Runtime.AsyncResult.End[[System.__Canon, mscorlib]](System.IAsyncResult)
00000000053cc880 000007fee1465e39 System.ServiceModel.Channels.FramingDuplexSessionChannel+TryReceiveAsyncResult.End(System.IAsyncResult, System.ServiceModel.Channels.Message ByRef)
00000000053cc8c0 000007fee133850e System.ServiceModel.Channels.ReplyOverDuplexChannelBase`1[[System.__Canon, mscorlib]].EndTryReceiveRequest(System.IAsyncResult, System.ServiceModel.Channels.RequestContext ByRef)
00000000053cc910 000007fee162ef16 System.ServiceModel.Channels.SecurityChannelListener`1+ReceiveItemAndVerifySecurityAsyncResult`2[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].InnerTryReceiveCompletedCallback(System.IAsyncResult)
00000000053cc980 000007fee61605e2 System.Runtime.Fx+AsyncThunk.UnhandledExceptionFrame(System.IAsyncResult)
00000000053cc9d0 000007fee615c6eb System.Runtime.AsyncResult.Complete(Boolean)
00000000053cca90 000007fee0c3b9da System.ServiceModel.Channels.FramingDuplexSessionChannel+TryReceiveAsyncResult.OnReceive(System.IAsyncResult)
00000000053ccb00 000007fee61605e2 System.Runtime.Fx+AsyncThunk.UnhandledExceptionFrame(System.IAsyncResult)
00000000053ccb50 000007fee615c6eb System.Runtime.AsyncResult.Complete(Boolean)
00000000053ccc10 000007fee0c3b903 System.ServiceModel.Channels.SynchronizedMessageSource+ReceiveAsyncResult.OnReceiveComplete(System.Object)
00000000053ccc70 000007fee0c3b070 System.ServiceModel.Channels.SessionConnectionReader.OnAsyncReadComplete(System.Object)
00000000053cccd0 000007fee61605e2 System.Runtime.Fx+AsyncThunk.UnhandledExceptionFrame(System.IAsyncResult)
00000000053ccd20 000007fef69545c5 System.Net.LazyAsyncResult.Complete(IntPtr)
00000000053ccd80 000007fef7106a29 System.Net.FixedSizeReader.ReadCallback(System.IAsyncResult)
00000000053cec48 000007fef8b865ed [HelperMethodFrame: 00000000053cec48]
00000000053ced90 000007fee6236880 System.Runtime.AsyncResult.End[[System.__Canon, mscorlib]](System.IAsyncResult)
00000000053cedd0 000007fee145e504 System.ServiceModel.Channels.ConnectionStream+ReadAsyncResult.End(System.IAsyncResult)
00000000053cee00 000007fef697af12 System.Net.FixedSizeReader.ReadCallback(System.IAsyncResult)
00000000053cee80 000007fee615c6eb System.Runtime.AsyncResult.Complete(Boolean)
00000000053cef40 000007fee0c17e73 System.ServiceModel.Channels.ConnectionStream+ReadAsyncResult.OnAsyncReadComplete(System.Object)
00000000053cefb0 000007fee15144b6 System.ServiceModel.Channels.TracingConnection+TracingConnectionState.ExecuteCallback()
00000000053cf000 000007ff00187a8b System.ServiceModel.Channels.SocketConnection.AsyncReadCallback(Boolean, Int32, Int32)
00000000053cf080 000007fee615ade0 System.Runtime.Fx+IOCompletionThunk.UnhandledExceptionFrame(UInt32, UInt32, System.Threading.NativeOverlapped*)
00000000053cf0e0 000007fef7f95da6 System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)
00000000053cf380 000007fef8aaee74 [GCFrame: 00000000053cf380]
00000000053cf550 000007fef8aaee74 [DebuggerU2MCatchHandlerFrame: 00000000053cf550]
00000000053cf728 000007fef8aaee74 [ContextTransitionFrame: 00000000053cf728]
00000000053cf910 000007fef8aaee74 [DebuggerU2MCatchHandlerFrame: 00000000053cf910]