Google Parser...Save To Text And Save Full HTML Same Time?

Discussion in 'A-Parser Support Forum' started by scrapefun, Oct 12, 2015.

  1. scrapefun

    scrapefun A-Parser Enterprise License
    A-Parser Enterprise

    Joined:
    Feb 24, 2015
    Messages:
    184
    Likes Received:
    34
    Using the Google Parser I want to save the parsed results to a .txt file as it does by default but also save the full source of each query to individual .html files.

    Is this possible?

    Also,my current settings grab the top 10 results for the first 2 pages of results so for each keyword/query there would need to be two html files saved if that makes sense (ie is the query was "a-parser" it would save two html files for that query...a-parser.html and a-parser2.html)
     
  2. Support

    Support Administrator
    Staff Member A-Parser Enterprise

    Joined:
    Mar 16, 2012
    Messages:
    4,545
    Likes Received:
    2,163
    It's easy to do:
    [​IMG]
    Code:
    eyJwcmVzZXQiOiJkZWZhdWx0IiwidmFsdWUiOnsicHJlc2V0IjoiZGVmYXVsdCIs
    InBhcnNlcnMiOltbIlNFOjpHb29nbGUiLCJkZWZhdWx0Iix7InR5cGUiOiJvdmVy
    cmlkZSIsImlkIjoicGFnZWNvdW50IiwidmFsdWUiOjJ9LHsidHlwZSI6Im92ZXJy
    aWRlIiwiaWQiOiJsaW5rc3BlcnBhZ2UiLCJ2YWx1ZSI6MTB9LHsidHlwZSI6Im92
    ZXJyaWRlIiwiaWQiOiJyYXdkYXRhIiwidmFsdWUiOnRydWV9XV0sInJlc3VsdHNG
    b3JtYXQiOiIkcDEucHJlc2V0IiwicmVzdWx0c1NhdmVUbyI6ImZpbGUiLCJyZXN1
    bHRzRmlsZU5hbWUiOiIkZGF0ZWZpbGUuZm9ybWF0KCkudHh0IiwiYWRkaXRpb25h
    bEZvcm1hdHMiOltbIiR7cXVlcnl9MS5odG1sIiwiJHAxLnBhZ2VzLjAuZGF0YSJd
    LFsiJHtxdWVyeX0yLmh0bWwiLCIkcDEucGFnZXMuMS5kYXRhIl1dLCJyZXN1bHRz
    VW5pcXVlIjoibm8iLCJxdWVyeUZvcm1hdCI6WyIkcXVlcnkiXSwidW5pcXVlUXVl
    cmllcyI6ZmFsc2UsInNhdmVGYWlsZWRRdWVyaWVzIjpmYWxzZSwiaXRlcmF0b3JP
    cHRpb25zIjp7Im9uQWxsTGV2ZWxzIjpmYWxzZSwicXVlcnlCdWlsZGVyc0FmdGVy
    SXRlcmF0b3IiOmZhbHNlLCJxdWVyeUJ1aWxkZXJzT25BbGxMZXZlbHMiOmZhbHNl
    fSwicmVzdWx0c09wdGlvbnMiOnsib3ZlcndyaXRlIjpmYWxzZX0sImRvTG9nIjoi
    bm8iLCJrZWVwVW5pcXVlIjoiTm8iLCJtb3JlT3B0aW9ucyI6ZmFsc2UsInJlc3Vs
    dHNQcmVwZW5kIjoiIiwicmVzdWx0c0FwcGVuZCI6IiIsInF1ZXJ5QnVpbGRlcnMi
    OltdLCJyZXN1bHRzQnVpbGRlcnMiOltdLCJjb25maWdPdmVycmlkZXMiOltdfX0=
    

    Result:
    [​IMG]
     
  3. scrapefun

    scrapefun A-Parser Enterprise License
    A-Parser Enterprise

    Joined:
    Feb 24, 2015
    Messages:
    184
    Likes Received:
    34
    Thanks! The "pages.o" and "pages.1" parameters I didn't know about.
     
  4. Support

    Support Administrator
    Staff Member A-Parser Enterprise

    Joined:
    Mar 16, 2012
    Messages:
    4,545
    Likes Received:
    2,163
    These are elements in the array. You can directly access any element, such as $serp.23.link - it is the 24th link in the array $serp.
     
  5. scrapefun

    scrapefun A-Parser Enterprise License
    A-Parser Enterprise

    Joined:
    Feb 24, 2015
    Messages:
    184
    Likes Received:
    34
    Is there a list or reference in the documentation somewhere that lists all the possible elements of the array or at least the standard/common ones? This would definitely help me out and hopefully I would not have to bother you so much haha
     
  6. Support

    Support Administrator
    Staff Member A-Parser Enterprise

    Joined:
    Mar 16, 2012
    Messages:
    4,545
    Likes Received:
    2,163
  7. scrapefun

    scrapefun A-Parser Enterprise License
    A-Parser Enterprise

    Joined:
    Feb 24, 2015
    Messages:
    184
    Likes Received:
    34
    I'm running into an issue where when saving the .html files sometimes the "page2" file is empty. I assume this is because the IP/proxy was blocked or had a timeout. Is there a way to ensure that if both queries (for page1 and page2) are not successful neither is written to the file?

    I manually checked and there are definitely results for the second page.

    Also, sometimes the results will be appended to the html file multiple times so when you open it it will have the search results listed multiple time.

    Here is what I have in my settings:

    [​IMG]

    eyJwcmVzZXQiOiJnb29nbGUgc2NyYXBlIiwidmFsdWUiOnsicHJlc2V0IjoiZ29v
    Z2xlIHNjcmFwZSIsInBhcnNlcnMiOltbIlNFOjpHb29nbGUiLCJLZXl3b3JkLCBS
    YW5rLCBVUkwiXV0sInJlc3VsdHNGb3JtYXQiOiIkcDEucHJlc2V0IiwicmVzdWx0
    c1NhdmVUbyI6ImZpbGUiLCJyZXN1bHRzRmlsZU5hbWUiOiJbJSBJRiBwMS5pbmZv
    LnN1Y2Nlc3MgPT0gMSAlXVslIGxpbmVzID0gbGluZXMgKyBwMS5zZXJwLnNpemU7
    IFVTRSBNYXRoOyBcInNlcnBzMy9cIl8gTWF0aC5pbnQobGluZXMgLyAxMDAwMDAw
    KSBfXCIudHh0XCIgJV1bJSBFTkQgJV0iLCJhZGRpdGlvbmFsRm9ybWF0cyI6W1si
    c2VycF9yYXcvWyUgSUYgcDEuaW5mby5zdWNjZXNzID09IDEgJV1bJSBVU0UgTWF0
    aDsgXCJ1c19cIl8gTWF0aC5pbnQocXVlcnkubnVtIC8gMjUwMCkgX1wiL1wiXyBx
    dWVyeSBfXCIuaHRtbFwiICVdWyUgRU5EICVdIiwiJHAxLnBhZ2VzLjAuZGF0YSJd
    LFsic2VycF9yYXcvWyUgSUYgcDEuaW5mby5zdWNjZXNzID09IDEgJV1bJSBVU0Ug
    TWF0aDsgXCJ1c19cIl8gTWF0aC5pbnQocXVlcnkubnVtIC8gMjUwMCkgX1wiL1wi
    XyBxdWVyeSBfXCJfcGFnZTIuaHRtbFwiICVdWyUgRU5EICVdIiwiJHAxLnBhZ2Vz
    LjEuZGF0YSJdLFsic2VycF9mYWlsZWQvZmFpbGVkLnR4dCIsIlslIElGIHAxLmlu
    Zm8uc3VjY2VzcyA9PSAwICVdJHF1ZXJ5XFxuWyUgRU5EICVdIl1dLCJyZXN1bHRz
    VW5pcXVlIjoibm8iLCJxdWVyeUZvcm1hdCI6WyIkcXVlcnkiXSwidW5pcXVlUXVl
    cmllcyI6ZmFsc2UsInNhdmVGYWlsZWRRdWVyaWVzIjpmYWxzZSwiaXRlcmF0b3JP
    cHRpb25zIjp7Im9uQWxsTGV2ZWxzIjpmYWxzZSwicXVlcnlCdWlsZGVyc0FmdGVy
    SXRlcmF0b3IiOmZhbHNlLCJxdWVyeUJ1aWxkZXJzT25BbGxMZXZlbHMiOmZhbHNl
    fSwicmVzdWx0c09wdGlvbnMiOnsib3ZlcndyaXRlIjpmYWxzZX0sImRvTG9nIjoi
    bm8iLCJrZWVwVW5pcXVlIjoiTm8iLCJtb3JlT3B0aW9ucyI6ZmFsc2UsInJlc3Vs
    dHNQcmVwZW5kIjoiIiwicmVzdWx0c0FwcGVuZCI6IiIsInF1ZXJ5QnVpbGRlcnMi
    OltdLCJyZXN1bHRzQnVpbGRlcnMiOltdLCJjb25maWdPdmVycmlkZXMiOltdfSwi
    cGFyc2Vyc0NvbmZQcmVzZXRzIjp7IlNFOjpHb29nbGUiOnsiS2V5d29yZCwgUmFu
    aywgVVJMIjp7InByb3h5cmV0cmllcyI6IjMwIiwidXNlcHJveHkiOnRydWUsInF1
    ZXJ5Zm9ybWF0IjoiJHF1ZXJ5IiwiZm9ybWF0cmVzdWx0IjoiWyUgRk9SRUFDSCBz
    ZXJwIC0lXSRxdWVyeTsgJG1pc3NwZWxsOyAkbG9vcC5jb3VudDsgJGxpbmtcXG5b
    JSBFTkQgJV0iLCJtYXhfc2l6ZSI6IjIwNDgwMCIsInByb3h5YmFubmVkY2xlYW51
    cCI6IjAiLCJ0aW1lb3V0IjoiNjAiLCJyZXF1ZXN0ZGVsYXkiOiIwIiwibGlua3Nw
    ZXJwYWdlIjoxMCwicGFnZWNvdW50IjoyLCJkb21haW4iOiJ3d3cuZ29vZ2xlLmNv
    LnVrIiwibHIiOiJsYW5nX2VuIiwiZ2wiOiJVUyIsImxvY2F0aW9uIjoiIiwiZmls
    dGVyIjp0cnVlLCJzZXJwdGltZSI6IiIsInNlcnAiOiIiLCJwYXJzZW5vdGZvdW5k
    Ijp0cnVlLCJ1c2VhbnRpZ2F0ZSI6ZmFsc2UsImFudGlnYXRlcHJlc2V0IjoiZGVm
    YXVsdCIsInVzZXNlc3Npb25zIjp0cnVlLCJyYXdkYXRhIjp0cnVlLCJkb19nemlw
    Ijp0cnVlLCJleHRyYXF1ZXJ5IjoiIn19fX0=
     
    #7 scrapefun, Oct 17, 2015
    Last edited: Oct 17, 2015
  8. scrapefun

    scrapefun A-Parser Enterprise License
    A-Parser Enterprise

    Joined:
    Feb 24, 2015
    Messages:
    184
    Likes Received:
    34
    I think I figured out the empty html file issue but still not sure on the multiple results sets being added sometimes.
     
  9. scrapefun

    scrapefun A-Parser Enterprise License
    A-Parser Enterprise

    Joined:
    Feb 24, 2015
    Messages:
    184
    Likes Received:
    34
    Any idea why the results would be added to the .html files multiple times? I've upgraded to the latest version (1.1.323)
     
  10. Support

    Support Administrator
    Staff Member A-Parser Enterprise

    Joined:
    Mar 16, 2012
    Messages:
    4,545
    Likes Received:
    2,163
    This may be because of the fact that identical requests are maked. Enable Unique queries. Or maybe you do not delete the previous results of parsing and the file is added...
     
  11. scrapefun

    scrapefun A-Parser Enterprise License
    A-Parser Enterprise

    Joined:
    Feb 24, 2015
    Messages:
    184
    Likes Received:
    34
    Enabling unique queries worked! Thanks!
     

Share This Page