1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
  2. Join our Telegram chat: https://t.me/a_parser_en
    Dismiss Notice

Selective processing of JavaScript: decoding phones numbers

Discussion in 'Share Experience' started by Support, Oct 15, 2015.

  1. Support

    Support Administrator
    Staff Member A-Parser Enterprise

    Joined:
    Mar 16, 2012
    Messages:
    4,371
    Likes Received:
    2,101
    From version 1.1.318 to A-Parser adds support for JS. Therefore, in this article we examine an example of this functionality, as well as will be shown, both directly during the task, you can add and execute JS from the sites.

    Task. Necessary parse all phone numbers from a private ads about selling used cars on the site http://ru.auto.soov.ee/

    Open and analyze site. In the search we set all the necessary parameters and obtain a link to the results. The first step is the parsing of links to ads. In the second step, we will parse the numbers from each advertisement.

    1) Parsing links:
    [​IMG]
    Code:
    eyJwcmVzZXQiOiJKU190ZWxzIDEiLCJ2YWx1ZSI6eyJwcmVzZXQiOiJKU190ZWxz
    IDEiLCJwYXJzZXJzIjpbWyJIVE1MOjpMaW5rRXh0cmFjdG9yIiwiZGVmYXVsdCIs
    eyJ0eXBlIjoiZmlsdGVyIiwicmVzdWx0IjpbImludGxpbmtzIiwibGluayJdLCJm
    aWx0ZXJUeXBlIjoicmVtYXRjaCIsInZhbHVlIjoib2JqYXZsZW5pamEvLis/XFwu
    aHRtbCQiLCJvcHRpb24iOiIifSx7InR5cGUiOiJvdmVycmlkZSIsImlkIjoiZm9y
    bWF0cmVzdWx0IiwidmFsdWUiOiIkaW50bGlua3MuZm9ybWF0KCckbGlua1xcbicp
    In1dXSwicmVzdWx0c0Zvcm1hdCI6IiRwMS5wcmVzZXQiLCJyZXN1bHRzU2F2ZVRv
    IjoiZmlsZSIsInJlc3VsdHNGaWxlTmFtZSI6IiRkYXRlZmlsZS5mb3JtYXQoKS50
    eHQiLCJhZGRpdGlvbmFsRm9ybWF0cyI6W10sInJlc3VsdHNVbmlxdWUiOiJzdHJp
    bmciLCJxdWVyeUZvcm1hdCI6WyIkcXVlcnkiXSwidW5pcXVlUXVlcmllcyI6ZmFs
    c2UsInNhdmVGYWlsZWRRdWVyaWVzIjpmYWxzZSwiaXRlcmF0b3JPcHRpb25zIjp7
    Im9uQWxsTGV2ZWxzIjpmYWxzZSwicXVlcnlCdWlsZGVyc0FmdGVySXRlcmF0b3Ii
    OmZhbHNlLCJxdWVyeUJ1aWxkZXJzT25BbGxMZXZlbHMiOmZhbHNlfSwicmVzdWx0
    c09wdGlvbnMiOnsib3ZlcndyaXRlIjpmYWxzZX0sImRvTG9nIjoibm8iLCJrZWVw
    VW5pcXVlIjoiTm8iLCJtb3JlT3B0aW9ucyI6ZmFsc2UsInJlc3VsdHNQcmVwZW5k
    IjoiIiwicmVzdWx0c0FwcGVuZCI6IiIsInF1ZXJ5QnVpbGRlcnMiOltdLCJyZXN1
    bHRzQnVpbGRlcnMiOltdLCJjb25maWdPdmVycmlkZXMiOltdfX0=
    • In the query uses a macro {num:1:1000} to loop through all the pages.
    2) Given a list of links to ads, we begin to parsing phone numbers. But if you look at the code of any page with the ad, you may notice, that the phone number where encrypted by script:
    [​IMG]
    Thanks tools.js we are able to run this script and get a result. For this we need parse the script and substitute it in tools.js.
    [​IMG]
    Code:
    eyJwcmVzZXQiOiJKU190ZWxzIDIiLCJ2YWx1ZSI6eyJwcmVzZXQiOiJKU190ZWxz
    IDIiLCJwYXJzZXJzIjpbWyJOZXQ6OkhUVFAiLCJkZWZhdWx0Iix7InR5cGUiOiJv
    dmVycmlkZSIsImlkIjoiZ29vZENvZGUiLCJ2YWx1ZSI6MjAwfSx7InR5cGUiOiJj
    dXN0b21SZXN1bHQiLCJyZXN1bHQiOiJkYXRhIiwicmVnZXgiOiI8dGg+XHUwNDFh
    XHUwNDNlXHUwNDNkXHUwNDQyXHUwNDMwXHUwNDNhXHUwNDQyXHUwNDNkXHUwNDRi
    XHUwNDM5IFx1MDQ0Mlx1MDQzNVx1MDQzYlx1MDQzNVx1MDQ0NFx1MDQzZVx1MDQz
    ZDo8XFwvdGg+PHRkPjxzdHJvbmc+PHNjcmlwdCB0eXBlPVwidGV4dFxcL2phdmFz
    Y3JpcHRcIj4oLis/KTxcXC9zY3JpcHQ+IiwicmVnZXhUeXBlIjoicyIsInJlc3Vs
    dFR5cGUiOiJmbGF0IiwiYXJyYXlOYW1lIjoiIiwicmVzdWx0cyI6WyJzdGVsIl19
    LHsidHlwZSI6Im92ZXJyaWRlIiwiaWQiOiJmb3JtYXRyZXN1bHQiLCJ2YWx1ZSI6
    IlslIHRvb2xzLmpzLmNhdGNoV3JpdGUoc3RlbCkgJV0ifV1dLCJyZXN1bHRzRm9y
    bWF0IjoiJHAxLnByZXNldCIsInJlc3VsdHNTYXZlVG8iOiJmaWxlIiwicmVzdWx0
    c0ZpbGVOYW1lIjoiJGRhdGVmaWxlLmZvcm1hdCgpLnR4dCIsImFkZGl0aW9uYWxG
    b3JtYXRzIjpbXSwicmVzdWx0c1VuaXF1ZSI6InN0cmluZyIsInF1ZXJ5Rm9ybWF0
    IjpbIiRxdWVyeSJdLCJ1bmlxdWVRdWVyaWVzIjpmYWxzZSwic2F2ZUZhaWxlZFF1
    ZXJpZXMiOmZhbHNlLCJpdGVyYXRvck9wdGlvbnMiOnsib25BbGxMZXZlbHMiOmZh
    bHNlLCJxdWVyeUJ1aWxkZXJzQWZ0ZXJJdGVyYXRvciI6ZmFsc2UsInF1ZXJ5QnVp
    bGRlcnNPbkFsbExldmVscyI6ZmFsc2V9LCJyZXN1bHRzT3B0aW9ucyI6eyJvdmVy
    d3JpdGUiOmZhbHNlfSwiZG9Mb2ciOiJubyIsImtlZXBVbmlxdWUiOiJObyIsIm1v
    cmVPcHRpb25zIjpmYWxzZSwicmVzdWx0c1ByZXBlbmQiOiIiLCJyZXN1bHRzQXBw
    ZW5kIjoiIiwicXVlcnlCdWlsZGVycyI6W10sInJlc3VsdHNCdWlsZGVycyI6W10s
    ImNvbmZpZ092ZXJyaWRlcyI6W119fQ==
    • To use this preset, you must first register function catchWrite in tools.js. To do this, insert the following code into the JavaScript Editor and save:
    As a result we get a list of almost 13K unique phone numbers. The average speed of parsing on 300 threads was almost 4k queries per minute.
    [​IMG]

    P.S. To use presets in this article is needed A-Parser version not below 1.1.318.
     

Share This Page