Node.js High Memory Usage?

For the past 2-3 weeks I've noticed A-parser performance will slowly get worse until rebooting server and then after reboot performance would improve 40-50%

Today, I noticed Node.js was using over 8GB memory for each A-parser instance with 4 instances of A-parser running on the server using around 32GB of memory.

Instead of rebooting, I just restarted each A-parser instance and performance improved 40-50% again and Node.js memory usage dropped considerably.

I'm using NET::HTTP and Google Modern parsers with A-parser versions 1.2.1222 and 1.2.1226 . The instances using NET::HTTP seem to cause Node.js to consume more memory but Google Parser also causes Node.js to use very high memory.

Not sure if this is a bug/memory leak or just normal behavior but I didn't always have this problem when running the same number of instances/threads/parsers/etc...




After restarting A-parser:

 

1) Windows Server 2012 R2. I am using multiple server providers and they all have this issue so I don't think it is server related. Also tried disabling malware/virus protection but same result.

2) Tasks run fine for 4-6 hours after restarting A-parser before memory usage gets so high performance begins to slow down. Memory usage slowly increases as soon as A-parser restarts and tasks start running again. Also, pausing or stopping tasks has no effect on memory usage.

3) There are 4-5 presets I use and this happens with all of them. But they all use either Google Modern or NET::HTTP. I have included 2 samples below.

4) Attached is a portion of a log file from when I pause the currently running task to successfully restarting A-parser. Maybe it will help.

Spoiler: NET:HTTP

eJyNVm1z2jgQ/isaTRuSG2qbJuFyZJiblAvXXhPIBTL9ABlGtRfQIUtGkkNylP/e
lTDG6VyT8zCMtdpnXx7trrymlpmFudFgwBraGq1p5t9pi/Yuh62Pw+ENrdOMaQPa
7Y9oD2xrJ09gynJhaX1N7VMGCFIPoDVPADd5guuZUklH+fUDEzmqjN5H0f3m54gc
Pb1jM5B2j6HX6l8uBAtPg4gcfuEyUStDekPSiILonKCgeXJOHpsnR+QiywR8ga+f
uQ1Pj38Njpvk8PPH4fVVnQi+APInxAt1RDpzrVIIz86CKDg5fn8SNM6aZMCmTPMC
Rl+IMdPq8UmD1RxMJcpGFEWvwr4yKSGJBTCZZxXwi0jLU1B5lZHmi/pIone2B1id
wwsADcscjE1AsKf/G5T30JkjoaArmF+eYTLLlTQ7SOy0O0ra58c7t6kg3EJqYpVB
e0zH1C+dEVzNrc1aYWgQnbJA6Vk4AKbj+S0YrD5zg9WCAMHkDJVBjimaZknCnWsm
aGv9zO81s/F86MNrbDb393VM3tvpKp0yV/mjt+RTl2SNgMupCkwex2AMabdJ43ws
xxY3EmYZmRAMFAUUpZe9P/D/bnBJ0PwcXxOuSdsv0Io9RHb1UyDzlITktyg6Im/v
ael4wB5gqNDxlAvYi7u46rEUXo8I97YOxINw0gilBJ+xrUREiueVyArNbUJjW8N2
zCaarcLcvKthzg4+IbXQvW+xi5UTBO4Qa+eI2+a2P4Atr9vpYbHIJlPGRXg3mLxB
GsHlHEy9yuFRYB9dXfw03cjHNKZvdp4r/G/dul/lTO8kX/oSkwp3HApbtou9v2fb
m9qd/cj5XiI/ZWrfSK45evgvjmuuMg2W5mq1CnDUzTCVWAX5IjS+QH9ftj1Njp+D
bGXa0cFMtHNzgHS3T6PaPnCMOPeh/r0NcduwdWqwNLpIFyTlxpQJgzvYH5pZpftF
h2GVK3khxBU8gCjxPuQPORcJzu+LKWI+FbjS0DOV/o8mNiWVVUc4CVYaIyiN+NWH
/nUhQFSirtRsR7zgKbe4Nh2VS+sHS50uALLyfHpOLVUaSjeF5cI7XlEZSDdD9g2C
s74UPcvCX2RG5Tp2pv0WqhQjyeDYlrNBJrgrNRyUrOCDnjsl5YpgsXK3G9f0frOv
pYp1FMZKTvmsX8zEQqhzOcQLtS87KsV7yBEkcyHqbiDf7rv6whSn6Rb7TH8Ed7wL
DKy8ZzE8Jcxfg23OmeYY7akLMMUjqXotTMZMiLvbq+oOrfSB9z7aLUNkwnIEhgt4
WimdmAk26WXvOPJt6XLGfp0pJNMnu9xVJKWOpeIjofyWWFc+FVrrDZbEPziqvY5z
7DRQhvwZPG83i78DIGrZAg==

Spoiler: Google::Modern

eJx9VW1v2zYQ/isC0QLb4Mpyhw2oin5I3XhtkcRZHH+yA4GVTjZrimRIyo5h+L/v
SOotXVrEQMzjcy+8e+7xiVhqduZWgwFrSLo6EeW/k5T8I+WGQ3YtC9CCjIii2oB2
oBVZXKZpuE/TDmDB2MzkmirQZHQi9qgA48g9aM0KQAQr8Ky0fDpOt5DvHIzsKa8d
7A9y/rlPISvKxAB9OBzijS8gzmVc737lvOEDx+XiV9DtEAqCnB8eRgQbUnNrZlJX
1DXmlZrEim7AxElcUEtJB1nQPdxLhJSMQ2+e4emGVi4mdlBlmh7Gq9fRl1mEkZgo
ZWzqPAdjog8fokn0+gEvl4vL6Jra7ftoTWrzZk0yf0S4/e2xBn2MRV1F4+hdkvwe
ZWsydgh/4U7x1lZ8TUKoy5tP+AXLoUXBLJOC8vCWMEtfUkmxxmIc/sX2ySL8pyUm
GO6Vz7Vei2EC9xl0bCnYo++kkHjjHBiYmZZV3yEfpe3sioSoBEPU3vff4EPSknID
I2KwwTNf4483zIKmVuq5ci9E+4lIccH5FeyB9zAf/2PNOHLWXJTo9KVxfBky/1+M
c/e8YSrk0UFjDV0Uf/o4v+69CnklN20zOKuYxbOZylo4UiVo3AGormc3DlZJDV2a
JnKTHXdWgXCk7Yl2oXrTs2dgb/uxPDfmUpRsM28WoUXW4h6FYS6mslIc3LtEzTmO
xcBdT+oL04zBHfoCf3Se+hRujaGk6Ou0Qkpuvi5CqUozXJq/XIEVdnKYtQmZU86X
d1fDGzKglM++ao9jVCrL0HG8g+NB6sJky0V2efP2nSe2ezO1sJFINP/Yx5ZKBPe9
07lOE08vqR1an8ldegq6psE2sSZJ4maKDfMXJLW6btlVdlIS+D4iwRIG1EhMIy3e
+xsVAoqcAxW1athiWQWydui/E08BjGVsAZweG8QzpXUKiySEPcvBj8LsrFRe1zeQ
BxZOHDHFzuCLnBVnkoy8PoR2485aFym8xNldDXiH02l/IIS0JQYrWpQoVb9ajYy/
IN+YWIckqNZepUdOjL0I453EieESBMTSMp7dwZQqm2/p26z7xerppdvbu24efw7N
t9TggP1YmtJwTvhnwqaF0gsp7D3dwSLYOyjK6GegYYOI7+m3epMJKeCJGWuy8Mp+
YeHJahoG7Sh2Ri34bm4Dyxx1HcfQhhsY0kzO/wGIoJWb

 

250-300 threads. Sending sample now. You can use this query file for either task preset.

 

Problem appears to be fixed! Running 6 hours with no increase in memory usage. Thanks for the help. Performance is much better!