Multithreading and network technologies

This information can be interesting only to programmers as reasons for high performance and efficiency of a parser

A-Parser was created when using the following technologies:

• Asynchronous input/output - allows to support at the same time thousands of connections (libev, epoll & select)
• Easy "green" threads (coro, anyevent)
• Own productive HTTP/Socket client with support of HTTP/1.1, HTTPS, HTTP and SOCKS5 proxy with support authorization
• SQLite and LevelDB as productive storages
• Absence of the graphic interface - A-Parser is executed on client-server technology
  • Client represents the web application (sencha) and is responsible for setup and control of a parser
  • Server is engaged only in parsing and information processing
• JavaScript engine V8 for bypass of JS protection, and also for writing own JavaScript parsers
• Template Toolkit as the main template editor
• UTF-8 in requests and results

A-Parser performs parallel all important operations:

• Each task (a set of requests) parsing in the specified number of threads
• When using several parsers in one task each request to different parsers is executed in different threads at the same time
• Parser is able to launch some tasks parallely
• Check and loading of a proxy from sources also takes place in the multithreaded mode