For the past 2-3 weeks I've noticed A-parser performance will slowly get worse until rebooting server and then after reboot performance would improve 40-50% Today, I noticed Node.js was using over 8GB memory for each A-parser instance with 4 instances of A-parser running on the server using around 32GB of memory. Instead of rebooting, I just restarted each A-parser instance and performance improved 40-50% again and Node.js memory usage dropped considerably. I'm using NET::HTTP and Google Modern parsers with A-parser versions 1.2.1222 and 1.2.1226 . The instances using NET::HTTP seem to cause Node.js to consume more memory but Google Parser also causes Node.js to use very high memory. Not sure if this is a bug/memory leak or just normal behavior but I didn't always have this problem when running the same number of instances/threads/parsers/etc... After restarting A-parser:
Please send us the presets on which this problem occurs. And also tell the OS version and after what time of work of these presets there is an increase in the amount of consumed memory.
1) Windows Server 2012 R2. I am using multiple server providers and they all have this issue so I don't think it is server related. Also tried disabling malware/virus protection but same result. 2) Tasks run fine for 4-6 hours after restarting A-parser before memory usage gets so high performance begins to slow down. Memory usage slowly increases as soon as A-parser restarts and tasks start running again. Also, pausing or stopping tasks has no effect on memory usage. 3) There are 4-5 presets I use and this happens with all of them. But they all use either Google Modern or NET::HTTP. I have included 2 samples below. 4) Attached is a portion of a log file from when I pause the currently running task to successfully restarting A-parser. Maybe it will help. Spoiler: NET:HTTP eJyNVm1z2jgQ/isaTRuSG2qbJuFyZJiblAvXXhPIBTL9ABlGtRfQIUtGkkNylP/e lTDG6VyT8zCMtdpnXx7trrymlpmFudFgwBraGq1p5t9pi/Yuh62Pw+ENrdOMaQPa 7Y9oD2xrJ09gynJhaX1N7VMGCFIPoDVPADd5guuZUklH+fUDEzmqjN5H0f3m54gc Pb1jM5B2j6HX6l8uBAtPg4gcfuEyUStDekPSiILonKCgeXJOHpsnR+QiywR8ga+f uQ1Pj38Njpvk8PPH4fVVnQi+APInxAt1RDpzrVIIz86CKDg5fn8SNM6aZMCmTPMC Rl+IMdPq8UmD1RxMJcpGFEWvwr4yKSGJBTCZZxXwi0jLU1B5lZHmi/pIone2B1id wwsADcscjE1AsKf/G5T30JkjoaArmF+eYTLLlTQ7SOy0O0ra58c7t6kg3EJqYpVB e0zH1C+dEVzNrc1aYWgQnbJA6Vk4AKbj+S0YrD5zg9WCAMHkDJVBjimaZknCnWsm aGv9zO81s/F86MNrbDb393VM3tvpKp0yV/mjt+RTl2SNgMupCkwex2AMabdJ43ws xxY3EmYZmRAMFAUUpZe9P/D/bnBJ0PwcXxOuSdsv0Io9RHb1UyDzlITktyg6Im/v ael4wB5gqNDxlAvYi7u46rEUXo8I97YOxINw0gilBJ+xrUREiueVyArNbUJjW8N2 zCaarcLcvKthzg4+IbXQvW+xi5UTBO4Qa+eI2+a2P4Atr9vpYbHIJlPGRXg3mLxB GsHlHEy9yuFRYB9dXfw03cjHNKZvdp4r/G/dul/lTO8kX/oSkwp3HApbtou9v2fb m9qd/cj5XiI/ZWrfSK45evgvjmuuMg2W5mq1CnDUzTCVWAX5IjS+QH9ftj1Njp+D bGXa0cFMtHNzgHS3T6PaPnCMOPeh/r0NcduwdWqwNLpIFyTlxpQJgzvYH5pZpftF h2GVK3khxBU8gCjxPuQPORcJzu+LKWI+FbjS0DOV/o8mNiWVVUc4CVYaIyiN+NWH /nUhQFSirtRsR7zgKbe4Nh2VS+sHS50uALLyfHpOLVUaSjeF5cI7XlEZSDdD9g2C s74UPcvCX2RG5Tp2pv0WqhQjyeDYlrNBJrgrNRyUrOCDnjsl5YpgsXK3G9f0frOv pYp1FMZKTvmsX8zEQqhzOcQLtS87KsV7yBEkcyHqbiDf7rv6whSn6Rb7TH8Ed7wL DKy8ZzE8Jcxfg23OmeYY7akLMMUjqXotTMZMiLvbq+oOrfSB9z7aLUNkwnIEhgt4 WimdmAk26WXvOPJt6XLGfp0pJNMnu9xVJKWOpeIjofyWWFc+FVrrDZbEPziqvY5z 7DRQhvwZPG83i78DIGrZAg== Spoiler: Google::Modern eJx9VW1v2zYQ/isC0QLb4Mpyhw2oin5I3XhtkcRZHH+yA4GVTjZrimRIyo5h+L/v SOotXVrEQMzjcy+8e+7xiVhqduZWgwFrSLo6EeW/k5T8I+WGQ3YtC9CCjIii2oB2 oBVZXKZpuE/TDmDB2MzkmirQZHQi9qgA48g9aM0KQAQr8Ky0fDpOt5DvHIzsKa8d 7A9y/rlPISvKxAB9OBzijS8gzmVc737lvOEDx+XiV9DtEAqCnB8eRgQbUnNrZlJX 1DXmlZrEim7AxElcUEtJB1nQPdxLhJSMQ2+e4emGVi4mdlBlmh7Gq9fRl1mEkZgo ZWzqPAdjog8fokn0+gEvl4vL6Jra7ftoTWrzZk0yf0S4/e2xBn2MRV1F4+hdkvwe ZWsydgh/4U7x1lZ8TUKoy5tP+AXLoUXBLJOC8vCWMEtfUkmxxmIc/sX2ySL8pyUm GO6Vz7Vei2EC9xl0bCnYo++kkHjjHBiYmZZV3yEfpe3sioSoBEPU3vff4EPSknID I2KwwTNf4483zIKmVuq5ci9E+4lIccH5FeyB9zAf/2PNOHLWXJTo9KVxfBky/1+M c/e8YSrk0UFjDV0Uf/o4v+69CnklN20zOKuYxbOZylo4UiVo3AGormc3DlZJDV2a JnKTHXdWgXCk7Yl2oXrTs2dgb/uxPDfmUpRsM28WoUXW4h6FYS6mslIc3LtEzTmO xcBdT+oL04zBHfoCf3Se+hRujaGk6Ou0Qkpuvi5CqUozXJq/XIEVdnKYtQmZU86X d1fDGzKglM++ao9jVCrL0HG8g+NB6sJky0V2efP2nSe2ezO1sJFINP/Yx5ZKBPe9 07lOE08vqR1an8ldegq6psE2sSZJ4maKDfMXJLW6btlVdlIS+D4iwRIG1EhMIy3e +xsVAoqcAxW1athiWQWydui/E08BjGVsAZweG8QzpXUKiySEPcvBj8LsrFRe1zeQ BxZOHDHFzuCLnBVnkoy8PoR2485aFym8xNldDXiH02l/IIS0JQYrWpQoVb9ajYy/ IN+YWIckqNZepUdOjL0I453EieESBMTSMp7dwZQqm2/p26z7xerppdvbu24efw7N t9TggP1YmtJwTvhnwqaF0gsp7D3dwSLYOyjK6GegYYOI7+m3epMJKeCJGWuy8Mp+ YeHJahoG7Sh2Ri34bm4Dyxx1HcfQhhsY0kzO/wGIoJWb
Also specify in how many threads you run these tasks and send files with queries for each of these tasks (you can send it to PM).
Problem appears to be fixed! Running 6 hours with no increase in memory usage. Thanks for the help. Performance is much better!
