Installation and Updates

Before Installation

Before installing, you need to perform the following actions:

1. In your Personal Account, check if your IP address is correctly entered (for users with a dynamic IP address, it is necessary to change the IP address when it changes to a new one)

   

2. Go to the A-Parser -> Downloads tab

3. Before downloading the necessary version, you should first click Update to update to the latest version of A-Parser

   

A-Parser Release - the stable version of A-Parser, you can read more about the release of new versions in the News section

A-Parser Beta - an intermediate version of A-Parser, which is constantly supplemented, you can read more about the changes on the forum in the Next Release section

A-Parser Installation

⏩ A-Parser Video: Installation, Launch, and Update

TODO: (next) update the video

Installation on Windows

After completing the Before Installation step, you need to download the program archive and unzip it to the folder of your choice (you can unzip it to any folder, even on the desktop)

Enter the aparser folder and run aparser.exe

The first launch may take a long time, from 30 seconds to several minutes. After the server window appears, open http://127.0.0.1:9091/ in your browser

The default password is empty. Just press the Login button, after which the A-Parser web interface will appear

Troubleshooting

Incompatible Programs

There are known compatibility issues with the following antivirus software and programs:

• Norton Internet Security - the scraper may work unstably
• Emsisoft Anti-Malware - the scraper may work unstably
• Guard Mail.ru - needs to be completely removed from the system
• HTTPDebugger - interrupts the operation of A-Parser two hours after launch

Disabling the Windows Indexing Service

The indexing service may block access to files during the operation of A-Parser. If the scraper does not start or crashes during operation, you need to check the A-Parser log for the following entries:

IO error: ... Append: cannot write
или
sysopen: Permission denied

To disable the indexing service, perform the following actions:

1. Open Services Management: Start - Run or the key combination Win + R, enter services.msc
2. In the window that opens, select Windows Search, right-click and choose Properties
3. In the properties window on the General tab, change Startup type to Disabled and click Apply
4. If after that the Stop button is active - press it and stop the service

Error While Updating Distribution

In some cases, Windows may block access to the A-Parser distribution files, in which case the following entry will be in aparser.log:

remove_tree failed for dist\nodejs\node_modules\...

To solve the problem:

• make sure there are no stuck processes in the task manager aparser.exe or aparser-node.exe
• delete the dist folder in the A-Parser directory, if Windows reports a deletion error - rename the dist folder to dist_

Installation on MacOS

Currently, installation on MacOS is possible using Docker

Default password is empty. Just press the Login button, after which the A-Parser web interface will appear

caution

On MacOS with Apple processors, it is necessary to enable the following option in Docker Desktop

Installation on Linux

We recommend using docker or docker-compose to install A-Parser on Linux, for a classic installation follow these instructions

note

A-Parser implements its own web server, so choose a directory without web access for installation

For convenience of downloading from the server, temporary links are implemented. In the Personal Account you need to click One-time link (in the English version Get one-time link) - by the received link you can download the distribution once. In the terminal, go to the installation directory (for example ~/) and execute the following commands using the received link:

wget https://a-parser.com/members/onetime/ce42f308eaa577b5/aparser-linux-x64.tar.gz
tar zxf aparser-linux-x64.tar.gz
rm -f aparser-linux-x64.tar.gz
cd aparser/
chmod +x aparser
./aparser

The first launch may take a long time, from 30 seconds to several minutes. After the server window appears, open http://127.0.0.1:9091/ in your browser, you can also go to A-Parser using the public IP address of the server

Default password is empty. Just press the Login button, after which the A-Parser web interface will appear

caution

Please note that by default A-Parser is available on all interfaces. We recommend setting a strong password and, if necessary, restrict access using iptables

Troubleshooting

Sometimes the server may lack some libraries, for example:

./aparser
./aparser: error while loading shared libraries: libz.so.1: cannot open shared object file: No such file or directory

You need to install the missing libraries:

yum -y install zlib

And restart the scraper:

./aparser

If no messages are issued - this indicates a successful launch of A-Parser. You can check with the command tail -f aparser.log

If aparser.log contains lines like:

./dist/nodejs/bin/aparser-node: /lib64/libc.so.6: version `GLIBC_2.25' not found (required by ./dist/nodejs/bin/aparser-node)
./dist/nodejs/bin/aparser-node: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by ./dist/nodejs/bin/aparser-node)

then this indicates the use of an outdated version of Linux and in this case, it is recommended to update the OS or use docker or docker-compose

Tuning Linux for more threads

By default, Linux limits the number of open files and sockets to 1024 per user, to increase the limit execute the following commands:

echo 'root soft nofile 10240' >> /etc/security/limits.conf
echo 'root hard nofile 10240' >> /etc/security/limits.conf

If you are running the scraper not as root, but as another user, then replace root with the username

It is also necessary to increase the size of the ip_conntrack table:

sysctl -w net.ipv4.netfilter.ip_conntrack_max=262144
echo 'net.ipv4.netfilter.ip_conntrack_max=262144' >> /etc/sysctl.conf

If there is no iptables firewall, this command will result in an error - just ignore it

You need to re-login to the terminal (ssh), after which restart A-Parser. To check the current limit, you need to execute:

ulimit -n

On some systems, it is additionally required to write the following line in the file /etc/pam.d/common-session:

session required pam_limits.so

Installation on FreeBSD

Currently, the Linux emulator for FreeBSD is unable to work stably with A-Parser

File structure of the installed program

The structure of the A-Parser working directory:

File	Description
config	Directory with configuration files, it is recommended to make a backup before updating A-Parser
dist	Directory with the A-Parser distribution, includes NodeJS and other additional modules
files/proxy	Directory with proxy checker settings
files/parsers	Directory contains JavaScript scraper source codes that were created or imported by the user
logs	Execution logs of tasks
queries	Directory with queries for scrapers
results	Parsing results directory
tmp	Temporary directory
.htaccess	Apache web server configuration file to protect the working directory from web access
aparser.exe or aparser	A-Parser executable file
aparser.log	A-Parser operation log, the main way to diagnose the current state and possible errors

Structure of the config directory:

File	Description
tasks	Directory with task files (working and completed)
unique	Directory with uniqueness files
config.db	Main configuration file, which stores settings and presets
queue.db	File with task queue data
scheduler.db	File with scheduled tasks data
config.txt	Additional Configuration File

tip

To display file extensions in the Windows operating system, enable the following setting:

Initial Setup

When starting to work with A-Parser, it is necessary to initially configure it for yourself in the menu Settings -> General settings

• Password - the default password is set to empty, you can create a new password to log into the system.
• Language - the ability to choose the language of the interface, news, and tips - Russian and English are available.
• Check for updates and Updates channel - notifications about the release of new versions of the scraper. The Updates channel allows you to choose between stable, beta, and alpha versions.

The setup of other options is described in the section General settings

Default Password

The default password is empty. Simply press the Login button, after which the A-Parser web interface will appear. To reset the password, see below.

Password Reset

You can reset the scraper access password by running it from the command line with the option -resetpassword

For Windows:

aparser.exe -resetpassword

For Linux:

./aparser -resetpassword

The password will be reset to empty, to log into A-Parser just press Login

Updating A-Parser

Update via Interface

Select the Updates channel in General settings:

In the Tools menu, go to the Update A-Parser tab:

Select the files to update, A-Parser will be restarted:

caution

Attention! Unless otherwise specified, it is sufficient to update only the executable file (aparser.exe or aparser)

Manual Update on Windows

In general, it is sufficient to replace the executable file of the scraper.

• Stop A-Parser - press Stop server
• Download the archive from Personal Account and overwrite aparser.exe
• Launch aparser.exe

Manual update on Linux

• Stop A-Parser - in the console execute killall aparser
• Download the archive from Personal Account and overwrite the file aparser
• Launch aparser - in the console execute ./aparser

Update using a one-time link

wget https://a-parser.com/members/onetime/0d19621928c25a48/aparser.tar.gz
kill $(cat files/pid)
sleep 1
tar xzf aparser.tar.gz -O aparser/aparser > aparser
rm -f aparser.tar.gz
chmod +x aparser
./aparser

Installing one license on multiple computers

Each license can be used simultaneously on only one PC/server. However, having one license, it is not prohibited to install A-Parser on several computers. But in this case, A-Parser can only be launched on the PC/server whose IP is registered in the Personal Account.

An example of such use could be a work PC and a home laptop: the scraper can be installed on both machines, but used either on the work PC or on the home laptop. The number of IP changes in the Personal Account is unlimited, but it is not recommended to do this more than 5 times a day, as otherwise, you may need to confirm the absence of fraudulent actions.

To use A-Parser simultaneously on several computers or servers, perform the following actions:

1. Add Additional licenses
2. In the Personal Account, go to the tab A-Parser -> IP Configuration
3. Enter the IP addresses of additional computers

Installing multiple copies on one PC or server

Each license allows you to install and run an unlimited number of copies of A-Parser on the same machine simultaneously. This allows you to make the most of powerful systems where one copy does not utilize all resources and at the same time, there is a need to increase performance.

The process of installing multiple copies of the scraper on one system:

• you need to download and install each copy in a separate directory according to the standard installation instructions
• in each copy of the scraper, you need to create a file config.txt in the config directory and specify a setting of this kind:

bind: 0.0.0.0:9092

• instead of 9092 specify the port on which this copy will operate

After that, the scraper can be launched and it will be available on the specified port

caution

Any methods of shared access, as well as renting out A-Parser, are prohibited and in case of detection, the license will be revoked without a refund or the possibility of restoration.