Explorer.exe taking all memory until it crashes while doing intensive I/O - Windows Server 2019 Standard Edition
Hi,
I have a fresh Windows Server 2019 Installation and when using my compression tools explorer.exe gets bigger and bigger until it eats whole 64 gb ram and crashes system, system logs don't say anything only that there was a critical shutdown due to no response with a failed memory dump. Swap file is set to auto, tested with static as well with over 70 GB.
When I kill explorer.exe everything works fine but when I execute it, it's starting to eat memory in fast linear fashion.
I have tried using TotalCommander instead of explorer.exe as my file explorer but with the same result, total commander was behaving the same way.
I have since reinstalled the whole system from scratch, installed latest updates and tested again with my portable tools - same result.
It also allocated lot of memory when I installed and run VM from Oracle, it didn't release the memory after closing the VM.
When looked at the process of explorer.exe it shows a lot of I/O and shcore.dll being the source of problem in explorer.exe, when I unload this module it will calm down for a little while and come again to eat memory.
I did not had this problem on Windows Server 2012.
Disk Configuration: 2x 1 TB NVMe SSD, CPU i9-9900K, 64 GB DDR4 RAM
Before installing system from scratch I have tested file integrity and originality - everything was ok.
If this is some new "feature" of Windows caching, please tell me how to turn it off. Back in the days in 2008 server edition it could be done by using DynCache.
Everything is up to date, no possibility of malware, compression tools are not the problem as they work fine on previous 2012 edition.
I'm asking it here because you are more knowledgeable people than that on Microsoft forum which usually just copy/paste some bs about updating your drivers and so on, I have posted there too but I don't believe in help there ;)
I can post tools too, they are based on FreeArc but I doubt it's the source of problem, I have used them for many years without any issues on many systems.
Help much appreciated fellow programmers.
I have a fresh Windows Server 2019 Installation and when using my compression tools explorer.exe gets bigger and bigger until it eats whole 64 gb ram and crashes system, system logs don't say anything only that there was a critical shutdown due to no response with a failed memory dump. Swap file is set to auto, tested with static as well with over 70 GB.
When I kill explorer.exe everything works fine but when I execute it, it's starting to eat memory in fast linear fashion.
I have tried using TotalCommander instead of explorer.exe as my file explorer but with the same result, total commander was behaving the same way.
I have since reinstalled the whole system from scratch, installed latest updates and tested again with my portable tools - same result.
It also allocated lot of memory when I installed and run VM from Oracle, it didn't release the memory after closing the VM.
When looked at the process of explorer.exe it shows a lot of I/O and shcore.dll being the source of problem in explorer.exe, when I unload this module it will calm down for a little while and come again to eat memory.
I did not had this problem on Windows Server 2012.
Disk Configuration: 2x 1 TB NVMe SSD, CPU i9-9900K, 64 GB DDR4 RAM
Before installing system from scratch I have tested file integrity and originality - everything was ok.
If this is some new "feature" of Windows caching, please tell me how to turn it off. Back in the days in 2008 server edition it could be done by using DynCache.
Everything is up to date, no possibility of malware, compression tools are not the problem as they work fine on previous 2012 edition.
|
|
I'm asking it here because you are more knowledgeable people than that on Microsoft forum which usually just copy/paste some bs about updating your drivers and so on, I have posted there too but I don't believe in help there ;)
https://answers.microsoft.com/en-us/windows/forum/windows_10-performance/explorerexe-taking-all-memory-until-it-crashes/fb80f2c8-bc49-477b-80a7-4b023ca749e0?tm=1588626129701/ I can post tools too, they are based on FreeArc but I doubt it's the source of problem, I have used them for many years without any issues on many systems.
Help much appreciated fellow programmers.
Last edited on
< removed hyperlink >
So... this memory leak only starts happening once you use your compression tool?
i.e. the memory is steady-state until you start to use your tool, and then it starts to build up?
If not, what exact event triggers the memory usage to start growing?
So... this memory leak only starts happening once you use your compression tool?
i.e. the memory is steady-state until you start to use your tool, and then it starts to build up?
If not, what exact event triggers the memory usage to start growing?
Last edited on
The site you posted copied my thread without my knowledge, I have posted only here and on Microsoft which I linked.
Yes the problem starts when I run my compression that uses lot of I/O calls.
When I close my tool the memory consumption by explorer.exe stops as well but it never releases the memory.
Yes the problem starts when I run my compression that uses lot of I/O calls.
When I close my tool the memory consumption by explorer.exe stops as well but it never releases the memory.
Last edited on
That's interesting. I removed the link in my post.
My first guess is that the tool is at fault, and is hooking into explorer.exe in some way, perhaps making incorrect assumptions about the Windows API calls that it's making, i.e. behavior that has inadvertently been broken by an update.
The tool is open-source, so theoretically someone could try to pin down exactly what is happening. Not sure if I have time to dig that deep, though. If you tell me the steps to reproduce the issue, I could at least running in my Windows Server 2019 to see if I get the same result.
My first guess is that the tool is at fault, and is hooking into explorer.exe in some way, perhaps making incorrect assumptions about the Windows API calls that it's making, i.e. behavior that has inadvertently been broken by an update.
The tool is open-source, so theoretically someone could try to pin down exactly what is happening. Not sure if I have time to dig that deep, though. If you tell me the steps to reproduce the issue, I could at least running in my Windows Server 2019 to see if I get the same result.
Last edited on
Tomorrow I will try to further analyze the source of problem but I think it's just very bad microsoft policy on caching ? will post all my findings, maybe someone here on forum knows something about this behavior or seen it somewhere else... anyway thank you for your time. See you later.
Last edited on
It's based around arc, with precomp and srep advanced compression tools.
didn't have time yet to check if bare freearc does the same but I bet it will do the same on big files.
Will send you my compression tools, you just need to edit compress.bat with path to directory you want to compress, use big collective of files (60+gb) to reproduce the same env.
I would think the tools are the problem too in this new system but I had issue with Oracle VM as well and I only run winrar extraction for a test on a Win7 and the host machine did allocate memory in explorer the same way and didn't release it.
Maybe it's due to disks being in Raid0 but I can't afford another server down time just to check it out.
Will check this forum tomorrow as I need to sleep, been fighting this problem whole day...
didn't have time yet to check if bare freearc does the same but I bet it will do the same on big files.
Will send you my compression tools, you just need to edit compress.bat with path to directory you want to compress, use big collective of files (60+gb) to reproduce the same env.
|
|
I would think the tools are the problem too in this new system but I had issue with Oracle VM as well and I only run winrar extraction for a test on a Win7 and the host machine did allocate memory in explorer the same way and didn't release it.
Maybe it's due to disks being in Raid0 but I can't afford another server down time just to check it out.
Will check this forum tomorrow as I need to sleep, been fighting this problem whole day...
Last edited on
I've never heard of anything like this. Can you give more details about what you were running?
What were you trying to compress? Was it very large? What sort of data was it?
Where did the compression tools come from? Did you make them?
The memory consumption suggests that Explorer is trying to do something with a directory that's being constantly modified. Explorer should not even be aware that you're trying to compress anything unless you have an Explorer window open on the directory where the archive will be saved. Have you tried running the compressor from a command line, without opening any Explorer windows?
Another possibility is that the compression utility is poorly written and is causing Explorer to think the original files are constantly being modified. If the files in question are, say, large video files, Explorer might be fruitlessly trying over and over again to generate previews for them.
What were you trying to compress? Was it very large? What sort of data was it?
Where did the compression tools come from? Did you make them?
The memory consumption suggests that Explorer is trying to do something with a directory that's being constantly modified. Explorer should not even be aware that you're trying to compress anything unless you have an Explorer window open on the directory where the archive will be saved. Have you tried running the compressor from a command line, without opening any Explorer windows?
Another possibility is that the compression utility is poorly written and is causing Explorer to think the original files are constantly being modified. If the files in question are, say, large video files, Explorer might be fruitlessly trying over and over again to generate previews for them.
I got my tools from trusted source, here's one:
I am compressing games so a lot of variety in files.
Explorer window is not even open.
The funny thing is that explorer.exe is never releasing the memory when I kill the compression, also if I start explorer in the middle of compression it starts to allocate heap memory as well.
It must be some function that is also used by norton commander as it behaves the same when I kill explorer and use it instead.
I will try to trace it down with x64dbg, I think you're right about explorer.exe trying to do some indexing or previews for the files that are constantly modified but weird that it doesn't release the memory.
Will post when I find something.
edit1: After viewing memory map for the allocated heap I can see it's not doing anything with it, all zeroes.
edit2: I can see on explorer stack there's path to temporary file that my compression tool is working at and before that some appresolver return addresses.
edit3: Interesting line on stack little before path to my working file:
&"pcshell\\shell\\appresolver\\lib\\appresolvercache.cpp"
edit4: found some relations with windows font cache service, will diable it and see if it helps.
edit5: didn't help. I think some windows service called by explorer.exe got problem with names that include "$" and "~", unfortunately I must wait for my current compression to end to modify configs to use normal names and test it.
edit6: disabled all the visual effects in the system didn't help, set manually reg:
And waiting, so far so good.
edit7: didn't help..
edit8: trying newer version of compression tools, so far so good.
Problem solved, it seems old version of precomp.exe was causing the trouble... Thank You for your time my friends, I didn't suspect my tools as I am using them for so long but this is new system and I should have checked that first...
Cheers!
@edit: problem returned, i will just kill explorer.exe for the time of compression. It eats memory but at a much lower rate.
http://schnaader.info/precomp.php I am compressing games so a lot of variety in files.
Explorer window is not even open.
The funny thing is that explorer.exe is never releasing the memory when I kill the compression, also if I start explorer in the middle of compression it starts to allocate heap memory as well.
It must be some function that is also used by norton commander as it behaves the same when I kill explorer and use it instead.
I will try to trace it down with x64dbg, I think you're right about explorer.exe trying to do some indexing or previews for the files that are constantly modified but weird that it doesn't release the memory.
Will post when I find something.
edit1: After viewing memory map for the allocated heap I can see it's not doing anything with it, all zeroes.
edit2: I can see on explorer stack there's path to temporary file that my compression tool is working at and before that some appresolver return addresses.
edit3: Interesting line on stack little before path to my working file:
&"pcshell\\shell\\appresolver\\lib\\appresolvercache.cpp"
edit4: found some relations with windows font cache service, will diable it and see if it helps.
edit5: didn't help. I think some windows service called by explorer.exe got problem with names that include "$" and "~", unfortunately I must wait for my current compression to end to modify configs to use normal names and test it.
edit6: disabled all the visual effects in the system didn't help, set manually reg:
|
|
And waiting, so far so good.
edit7: didn't help..
edit8: trying newer version of compression tools, so far so good.
Problem solved, it seems old version of precomp.exe was causing the trouble... Thank You for your time my friends, I didn't suspect my tools as I am using them for so long but this is new system and I should have checked that first...
Cheers!
@edit: problem returned, i will just kill explorer.exe for the time of compression. It eats memory but at a much lower rate.
Last edited on
I don't know what causes it but you are not alone. I have had any number of programs (commercial, not my stuff) cause explorer.exe to go kind of nuts on both memory and CPU. I used to kill it for increased performance but winx has improved this problem a lot; it still happens but its 'better' in other words. Killing it is fine: it does not really do anything important, it just visualizes stuff (taskbar, folders, etc).
maybe try 7zip or something else? Just to check if it really is the tool?
maybe try 7zip or something else? Just to check if it really is the tool?
Last edited on
My system crashes after a while when explorer.exe is killed and running my compression tools... I first killed explorer.exe then started my compression from cmd so it had no reason to crash...
I am going back to Windows Server 2012 and hope they didn't update it with this junk, also hope it's not because of Raid0.
I think I finally found the issue, Windows Server 2019 is a piece of $#!^
^_^
Winrar didn't act the same but it's not so I/O intensive as my more advanced tools, the only time when explorer reacted the same was with Oracle VM by allocating amount of RAM set in the VM and not releasing it after closing VM and then crashing whole system after killing explorer.exe. Weirdo...
And now they are saying at Hetzner I can only go back as far as of 2016 and can't turn off Raid...
Will test on VM again, hope it will work with out too much performance hit.
@edit: Performance on a VM is not bad, could get better with raw disk access but I'm afraid explorer will start acting crazy again, will do my business on a virtual drive and then copy to host machine.
I suggest anyone with the same problem doing the same or downgrade to 2012 system, if the tools I use would be available for linux... would need to check them on wine. *actually precomp is available for linux, need to check others ;)
@edit: the performance of VM is too low for my tasks, I think I'll try 2016 edition....
Now they say they don't offer anything except 2019... will need to try find another provider of dedicated servers, any suggestions ? I need it to be in Germany and with 2012 Windows...
Just from curiosity I will download 2019 version to locally test the problem on a VM, will report here back if I find what's exactly this explorer is doing.
I am going back to Windows Server 2012 and hope they didn't update it with this junk, also hope it's not because of Raid0.
I think I finally found the issue, Windows Server 2019 is a piece of $#!^
^_^
Winrar didn't act the same but it's not so I/O intensive as my more advanced tools, the only time when explorer reacted the same was with Oracle VM by allocating amount of RAM set in the VM and not releasing it after closing VM and then crashing whole system after killing explorer.exe. Weirdo...
And now they are saying at Hetzner I can only go back as far as of 2016 and can't turn off Raid...
Will test on VM again, hope it will work with out too much performance hit.
@edit: Performance on a VM is not bad, could get better with raw disk access but I'm afraid explorer will start acting crazy again, will do my business on a virtual drive and then copy to host machine.
I suggest anyone with the same problem doing the same or downgrade to 2012 system, if the tools I use would be available for linux... would need to check them on wine. *actually precomp is available for linux, need to check others ;)
@edit: the performance of VM is too low for my tasks, I think I'll try 2016 edition....
Now they say they don't offer anything except 2019... will need to try find another provider of dedicated servers, any suggestions ? I need it to be in Germany and with 2012 Windows...
Just from curiosity I will download 2019 version to locally test the problem on a VM, will report here back if I find what's exactly this explorer is doing.
Last edited on
After realizing how unstable 2019 edition is, I have ditched the pursuit of the explorer problem.
It did crash for the second time while using simple proxy server without any reason in system logs.
Before this system I have used 2012 R2 edition which was very stable, never had a crash for over 7 years I've been using it.
Stay away from 2019 edition, I have switched now to ubuntu distro as my server provider don't sell 2012 anymore.
Cheers.
It did crash for the second time while using simple proxy server without any reason in system logs.
Before this system I have used 2012 R2 edition which was very stable, never had a crash for over 7 years I've been using it.
Stay away from 2019 edition, I have switched now to ubuntu distro as my server provider don't sell 2012 anymore.
Cheers.
Topic archived. No new replies allowed.