1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
  2. Join our Telegram chat: https://t.me/a_parser_en
    Dismiss Notice
Cookies Telegram чат EN

Automatically Control Size of Results Files

Discussion in 'A-Parser Support Forum' started by scrapefun, Mar 18, 2015.

  1. scrapefun

    scrapefun A-Parser Enterprise License
    A-Parser Enterprise

    Joined:
    Feb 24, 2015
    Messages:
    212
    Likes Received:
    38
    It would be great if there was a way to control the size of results files.

    Sometimes they can contain over 10 million lines and get very large and hard to handle. So if there was a way to create a new result file when a certain number of lines is reached or a certain file size is reached that would be great.
     
    #1 scrapefun, Mar 18, 2015
  2. Forbidden

    Forbidden Administrator
    Staff Member A-Parser Enterprise

    Joined:
    Mar 9, 2013
    Messages:
    3,375
    Likes Received:
    1,820
    Easy solution:

    [​IMG]

    results splitted approximately by 1000 lines
     
    #2 Forbidden, Mar 21, 2015
  3. scrapefun

    scrapefun A-Parser Enterprise License
    A-Parser Enterprise

    Joined:
    Feb 24, 2015
    Messages:
    212
    Likes Received:
    38
    Great! This is perfect. Thanks for the help.
     
    #3 scrapefun, Mar 21, 2015
  4. scrapefun

    scrapefun A-Parser Enterprise License
    A-Parser Enterprise

    Joined:
    Feb 24, 2015
    Messages:
    212
    Likes Received:
    38
    How would I add "$datefile.format()" to the filename?

    I've tried adding it to the solution you gave but the files are not split properly?
     
    #4 scrapefun, Mar 22, 2015
  5. Forbidden

    Forbidden Administrator
    Staff Member A-Parser Enterprise

    Joined:
    Mar 9, 2013
    Messages:
    3,375
    Likes Received:
    1,820
    You can't mix together $datefile.format() and other variables because it will create new date each time when save new portion of results
     
    #5 Forbidden, Mar 22, 2015
  6. scrapefun

    scrapefun A-Parser Enterprise License
    A-Parser Enterprise

    Joined:
    Feb 24, 2015
    Messages:
    212
    Likes Received:
    38
    ok, thanks for clarifying and for the help!
     
    #6 scrapefun, Mar 22, 2015
(You must log in or sign up to post here.)

Share This Page