c# - HttpWebResponse.GetResponseStream() (seems to be) failing with no exception -

i've been puzzling on while. on 1 of our client (windows 7, .net 2.0) machines, following code seems failing (causing application shut down) seems no exceptions thrown.

in logs 'sending rest request' printed nothing. expectation 1 of following log messages printed (either 1 of exceptions or 1 of info messages).

going through event log shows windows error report service starting stopping time there no dump files created.

any insight happening or how more information debug appreciated. thanks

   try    {        system.net.servicepointmanager.expect100continue = false;        system.net.servicepointmanager.usenaglealgorithm = false;        httpwebrequest webrequest = proxiedhttpwebrequest.createwithproxy(web_uri);        webrequest.method = request_method;        if (content_type != null)        {            webrequest.contenttype = content_type;        }        webrequest.method = request_method;        if (data != null)        {            webrequest.contentlength = data.length;   //count bytes send        }        else        {            webrequest.contentlength = 0;        }        webrequest.keepalive = true;        if (webrequest.method == "post")        {            //write post data            using (stream os = webrequest.getrequeststream())            {                 os.write(data, 0, data.length);             }        }        if (headers != null)        {            foreach (keyvaluepair<string, string> pair in headers)            {               webrequest.headers.add(pair.key, pair.value);            }        }        logger.info("sending rest request");        using (httpwebresponse webresponse = (httpwebresponse)webrequest.getresponse())        {             if (webresponse != null)             {                 logger.info("received response");                 responsecode = webresponse.statuscode;                 using (streamreader sr = new streamreader(webresponse.getresponsestream(), encoding.utf8))                 {                     string response = sr.readtoend().trim();                     return response;                 }             }             else             {                 logger.info("a null response received web");             }         }         return null;     }     catch (webexception we)     {         logger.warn(tracelogger.generateerrorstringfromexception(we));         throw we;     }     catch (exception e)     {         logger.warn(tracelogger.generateerrorstringfromexception(e));         throw e;     } 

edit: should add transient error (it fails once every couple of days , on machine). code running on 200+ machines

edit2: start code following catch unhanled exceptions:

           // add event handler handling non-ui thread exceptions event.              appdomain.currentdomain.unhandledexception += currentdomain_unhandledexception;              // add event handler handling ui thread exceptions event.             application.threadexception += application_threadexception;             // set unhandled exception mode force windows forms errors go through             // our handler.             application.setunhandledexceptionmode(unhandledexceptionmode.catchexception); 

edit3: logging. it's possible logging code cause of crash. i'm using log4net apache pretty stable , rollingfileappender defaults flush after each log request. error happens when log level such warnings wouldn't getting logged.

edit4: after running procdump able capture dump of program when crashes. opened in windbg , ran !analyze -v

faulting_ip:  +ae31be0 00000000`00000000 ??              ??? exception_record:  ffffffffffffffff -- (.exr 0xffffffffffffffff) exceptionaddress: 0000000000000000    exceptioncode: 80000003 (break instruction exception)   exceptionflags: 00000000 numberparameters: 0 faulting_thread:  0000000000001404  process_name:  app_2_0_0_122.exe error_code: (ntstatus) 0x80000003 - {exception}  breakpoint  breakpoint has been reached. exception_code: (hresult) 0x80000003 (2147483651) - 1 or more arguments invalid mod_list: <analysis/> ntglobalflag:  0 application_verifier_flags:  0 managed_stack: !dumpstack -ee no export dumpstack found additional_debug_text:  followup set based on attribute [is_chosencrashfollowupthread] frame:[0] on thread:[pseudo_thread] last_control_transfer:  000007fef6ebce72 0000000076c0933a default_bucket_id:  stackimmune primary_problem_class:  stackimmune bugcheck_str:  application_fault_stackimmune_nosos_wrong_symbols stack_text:   00000000`00000000 00000000`00000000 app_2_0_0+0x0  symbol_name:  app_2_0_0 followup_name:  machineowner module_name: app_2_0_0 image_name:  app_2_0_0 debug_flr_image_timestamp:  0 stack_command:  ** pseudo context ** ; kb failure_bucket_id:  stackimmune_80000003_app_2_0_0!unloaded bucket_id:  x64_application_fault_stackimmune_nosos_wrong_symbols_app_2_0_0 

edit: - output of winddbg

0:000> !threads - .. output removed , correct output below

0:000> !pe no export pe found .. correct output below

0:000> !eestack -ee no export eestack found - correct output below

after loading sos , clr , running !analyze -v winddbg produced following stack

stack_text:   00000000`0024e6d8 000007fe`f6ebce72 : 00000000`0024f000 00000000`0024e5d1 00000000`00000000 00000000`00000000 : user32!zwuserwaitmessage+0xa 00000000`0024e6e0 000007fe`f0a1ec14 : 00000000`027f3260 00000000`0024e7c0 000007fe`f095f548 00000000`00000001 : mscorwks!dondirectcallworker+0x62 00000000`0024e770 000007fe`f0a1e278 : 00000000`02857a00 00000000`00000001 00000000`ffffffff 00000000`00000000 : system_windows_forms_ni+0x35ec14 00000000`0024e9c0 000007fe`f0a1dce5 : 00000000`027f3260 00000000`ffffffff 00000000`02857998 000007fe`f104cd48 : system_windows_forms_ni+0x35e278 00000000`0024eb10 000007ff`001804a5 : 00000000`00000000 00000000`00000000 00000000`0024e888 00000000`0000000c : system_windows_forms_ni+0x35dce5 00000000`0024eb70 000007fe`f6ebd432 : 00000000`027f3668 00000000`00000000 000007ff`00033fa8 000007fe`f6d62fa9 : 0x7ff`001804a5 00000000`0024ebf0 000007fe`f6dab7a3 : 00000000`00000002 00000000`00000027 000007fe`f6c77340 00000000`00000000 : mscorwks!calldescrworker+0x82 00000000`0024ec40 000007fe`f729a981 : 00000000`0024ed78 00000000`00000000 00000000`0024ef80 00000000`00000008 : mscorwks!calldescrworkerwithhandler+0xd3 00000000`0024ece0 000007fe`f6e086db : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`0024f1e0 : mscorwks!methoddesc::calldescr+0x2b1 00000000`0024ef20 000007fe`f6e2bd54 : 00000000`00000000 00000000`00000000 0000000d`00d6000a 00000000`00000000 : mscorwks!classloader::runmain+0x22b 00000000`0024f180 000007fe`f7386c9d : 00000000`0024f7d0 00000000`00000000 00000000`0076bbc8 00000000`00000200 : mscorwks!assembly::executemainmethod+0xbc 00000000`0024f470 000007fe`f6e3929f : 00000000`00000000 00000000`00000000 00000000`00000000 000007fe`f6e55c12 : mscorwks!systemdomain::executemainmethod+0x47d 00000000`0024fa40 000007fe`f6e1bfdc : ffffffff`fffffffe 00000000`00000000 00009fee`00000000 00000000`00000000 : mscorwks!executeexe+0x47 00000000`0024fa90 000007fe`f7923309 : ffffffff`ffffffff 00000000`0075dbd0 00000000`00000000 00000000`0024fa98 : mscorwks!corexemain+0xac 00000000`0024faf0 000007fe`f7a15b21 : 000007fe`f6e1bf30 000007fe`f79232c0 00000000`00000000 00000000`00000000 : mscoreei!corexemain+0x41 00000000`0024fb20 00000000`7677652d : 000007fe`f7920000 00000000`00000000 00000000`00000000 00000000`00000000 : mscoree!corexemain_exported+0x57 00000000`0024fb50 00000000`76e6c541 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : kernel32!basethreadinitthunk+0xd 00000000`0024fb80 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!rtluserthreadstart+0x1d 

edit: results of !pe below. interestingly dll trying load exists. auto generated serialization of utilities dll used both service , application components.

0:000> !pe exception object: 0000000002831e48 exception type: system.badimageformatexception message: not load file or assembly 'utilities_2_0_0_122.xmlserializers, version=2.0.0.122, culture=neutral, publickeytoken=0bd8b79f92cc7463' or 1 of dependencies. assembly built runtime newer loaded runtime , cannot loaded. innerexception: system.badimageformatexception, use !printexception 00000000028325c0 see more stacktrace (generated):     sp               ip               function     000000000024e740 0000000000000001 system.reflection.assembly._nload(system.reflection.assemblyname, system.string, system.security.policy.evidence, system.reflection.assembly, system.threading.stackcrawlmark byref, boolean, boolean)     000000000024e740 000007fef5debf61 system.reflection.assembly.internalload(system.reflection.assemblyname, system.security.policy.evidence, system.threading.stackcrawlmark byref, boolean)     000000000024e7d0 000007fef5e249e4 system.reflection.assembly.load(system.reflection.assemblyname)     000000000024e810 000007fef42a5c0a system.xml.serialization.tempassembly.loadgeneratedassembly(system.type, system.string, system.xml.serialization.xmlserializerimplementation byref)  stacktracestring: <none> hresult: 8013101b 0:000> !printexception 00000000028325c0 exception object: 00000000028325c0 exception type: system.badimageformatexception message: not load file or assembly 'utilities_2_0_0_122.xmlserializers, version=2.0.0.122, culture=neutral, publickeytoken=0bd8b79f92cc7463' or 1 of dependencies. assembly built runtime newer loaded runtime , cannot loaded. innerexception: <none> stacktrace (generated): <none> stacktracestring: <none> hresult: 8013101b 

edit: requested source generateerrorstringfromexception

    public static string generateerrorstringfromexception(exception e)     {         try         {             string s = "";             s += "outer exception." + "\n";             s += "exceptiontype: " + e.gettype().name + "\n";             s += "message: " + e.message + "\n";             s += "source: " + e.source + "\n";             s += "stacktrace: " + e.stacktrace + "\n";             s += "targetsite: " + e.targetsite + "\n";             string indent = "\t";             exception ie = e;             while (ie.innerexception != null)             {                 ie = ie.innerexception;                  s += "inner exception." + "\n";                 s += indent + "exceptiontype: " +                      ie.gettype().name + "\n";                 s += indent + "message: " + ie.message + "\n";                 s += indent + "source: " + ie.source + "\n";                 s += indent + "stacktrace: " + ie.stacktrace + "\n";                 s += indent + "targetsite: " + ie.targetsite + "\n";                  indent += "\t";             }             return s;         }         catch (exception)         {             return                 "fatal problem, error generated while printing exception - no message given on orginal issue ";         }     } 

edit: should have included earlier. client machine x64, target build any_cpu. client has .net 4.5 installed, target platform .net 2.0.

edit: ouput of !threads , !eestack -ee

0:000> !threads threadcount: 8 unstartedthread: 0 backgroundthread: 6 pendingthread: 0 deadthread: 1 hosted runtime: no                                               preemptive                                                lock        id osid        threadobj     state   gc     gc alloc context                  domain           count apt exception    0    1 1404 000000000075dbd0      6020 enabled  0000000000000000:0000000000000000 0000000000754c60     0 ukn system.badimageformatexception (0000000002831e48)    2    2 1540 0000000000763bd0      b220 enabled  0000000000000000:0000000000000000 0000000000754c60     0 ukn (finalizer)    5    3 1ac0 000000001aedff20   880b220 enabled  0000000002b11a50:0000000002b135c0 0000000000754c60     0 ukn (threadpool completion port)    6    4 1530 000000001aee0da0    80a220 enabled  0000000000000000:0000000000000000 0000000000754c60     0 ukn (threadpool completion port)    8    6  41c 000000001c5d4960   180b220 enabled  0000000000000000:0000000000000000 0000000000754c60     0 ukn (threadpool worker) system.argumentexception (0000000002a676d8)   10    5  dfc 000000001ba94600   180b220 enabled  0000000002b19680:0000000002b1b5c0 0000000000754c60     2 ukn (threadpool worker) system.argumentexception (0000000002add658)   14    8 1e6c 000000001c9c2bf0   200b220 enabled  0000000000000000:0000000000000000 0000000000754c60     1 ukn xxxx    7    0 000000001c9b69a0      9820 enabled  0000000000000000:0000000000000000 0000000000754c60     0 ukn 0:000> !eeestack -ee no export eeestack found 0:000> !eestack -ee --------------------------------------------- thread   0 unable load image c:\windows\assembly\nativeimages_v2.0.50727_64\system.windows.forms\95674cb72317e3a5380ea450b913786f\system.windows.forms.ni.dll, win32 error 0n2 *** warning: unable verify checksum system.windows.forms.ni.dll child-sp         retaddr          call site 000000000024e770 000007fef0a1e278 system_windows_forms_ni!system.windows.forms.application+componentmanager.system.windows.forms.unsafenativemethods.imsocomponentmanager.fpushmessageloop(int32, int32, int32)+0x7d4 000000000024e9c0 000007fef0a1dce5 system_windows_forms_ni!system.windows.forms.application+threadcontext.runmessageloopinner(int32, system.windows.forms.applicationcontext)+0x578 000000000024eb10 000007ff001804a5 system_windows_forms_ni!system.windows.forms.application+threadcontext.runmessageloop(int32, system.windows.forms.applicationcontext)+0x65 000000000024eb70 000007fef6ebd432 app_2_0_0_122!app_2_0_0_122.exe!unknown+0x245 --------------------------------------------- thread   2 child-sp         retaddr          call site --------------------------------------------- thread   5 child-sp         retaddr          call site --------------------------------------------- thread   6 child-sp         retaddr          call site --------------------------------------------- thread   8 child-sp         retaddr          call site --------------------------------------------- thread  10 *** warning: unable verify checksum mscorlib.ni.dll child-sp         retaddr          call site 000000001d82e220 000007fef0a1e278 system_windows_forms_ni!system.windows.forms.application+componentmanager.system.windows.forms.unsafenativemethods.imsocomponentmanager.fpushmessageloop(int32, int32, int32)+0x7d4 000000001d82e470 000007fef0a1dce5 system_windows_forms_ni!system.windows.forms.application+threadcontext.runmessageloopinner(int32, system.windows.forms.applicationcontext)+0x578 000000001d82e5c0 000007fef109f253 system_windows_forms_ni!system.windows.forms.application+threadcontext.runmessageloop(int32, system.windows.forms.applicationcontext)+0x65 000000001d82e620 000007ff001ac37d system_windows_forms_ni!system.windows.forms.form.showdialog(system.windows.forms.iwin32window)+0x763 000000001d82e910 000007ff001abce4 app_2_0_0_122!app_2_0_0_122.exe!unknown+0x4dd 000000001d82ea00 000007ff001ab803 app_2_0_0_122!app_2_0_0_122.exe!unknown+0x34 000000001d82ea50 000007ff001aaf9c app_2_0_0_122!app_2_0_0_122.exe!unknown+0x113 000000001d82ead0 000007ff001a8706 app_2_0_0_122!app_2_0_0_122.exe!unknown+0x23c 000000001d82eb70 000007ff001a85f8 app_2_0_0_122!app_2_0_0_122.exe!unknown+0x96 000000001d82ebf0 000007fef5dedd38 utilities_2_0_0_122!utilities_2_0_0_122.dll!unknown+0x58 000000001d82ec40 000007fef6ebd432 mscorlib_ni!system.threading.executioncontext.runtrycode(system.object)+0x178 000000001d82f4e0 000007fef5e2edd6 mscorlib_ni!system.threading.executioncontext.run(system.threading.executioncontext, system.threading.contextcallback, system.object)+0x62 000000001d82f530 000007fef6ebd432 mscorlib_ni!system.threading._timercallback.performtimercallback(system.object)+0x86 --------------------------------------------- thread  14 unable load image c:\windows\assembly\nativeimages_v2.0.50727_64\system\af0a0b96a02f9925eb84392ee65a5cfa\system.ni.dll, win32 error 0n2 *** warning: unable verify checksum system.ni.dll child-sp         retaddr          call site 000000002202f090 000007fef5104487 mscorlib_ni!system.threading.waithandle.waitany(system.threading.waithandle[], int32, boolean)+0x64 000000002202f0f0 000007fef5dd2bbb system_ni!system.net.timerthread.threadproc()+0x327 000000002202f1c0 000007fef5e6aadd mscorlib_ni!system.threading.executioncontext.run(system.threading.executioncontext, system.threading.contextcallback, system.object)+0x9b 000000002202f210 000007fef6ebd432 mscorlib_ni!system.threading.threadhelper.threadstart()+0x4d 

edit: procdump arguments procdump.exe -h -e -t app_2_0_0_122.exe c:\temp\dump.dump

and worth including first line of windbg output, assume it's trigger dump *** hung window detected: 180a5c

this line strange:

error_code: (ntstatus) 0x80000003 - {exception}  breakpoint  breakpoint has been reached. 

there debbugger.break or debug.assert in code of logger?