Automatically Control Size of Results Files

Discussion in 'A-Parser Support Forum' started by scrapefun, Mar 18, 2015.

  1. scrapefun

    scrapefun A-Parser Enterprise License
    A-Parser Enterprise

    Joined:
    Feb 24, 2015
    Messages:
    184
    Likes Received:
    34
    It would be great if there was a way to control the size of results files.

    Sometimes they can contain over 10 million lines and get very large and hard to handle. So if there was a way to create a new result file when a certain number of lines is reached or a certain file size is reached that would be great.
     
  2. Forbidden

    Forbidden Administrator
    Staff Member A-Parser Enterprise

    Joined:
    Mar 9, 2013
    Messages:
    3,337
    Likes Received:
    1,793
    Easy solution:

    [​IMG]

    results splitted approximately by 1000 lines
     
  3. scrapefun

    scrapefun A-Parser Enterprise License
    A-Parser Enterprise

    Joined:
    Feb 24, 2015
    Messages:
    184
    Likes Received:
    34
    Great! This is perfect. Thanks for the help.
     
  4. scrapefun

    scrapefun A-Parser Enterprise License
    A-Parser Enterprise

    Joined:
    Feb 24, 2015
    Messages:
    184
    Likes Received:
    34
    How would I add "$datefile.format()" to the filename?

    I've tried adding it to the solution you gave but the files are not split properly?
     
  5. Forbidden

    Forbidden Administrator
    Staff Member A-Parser Enterprise

    Joined:
    Mar 9, 2013
    Messages:
    3,337
    Likes Received:
    1,793
    You can't mix together $datefile.format() and other variables because it will create new date each time when save new portion of results
     
  6. scrapefun

    scrapefun A-Parser Enterprise License
    A-Parser Enterprise

    Joined:
    Feb 24, 2015
    Messages:
    184
    Likes Received:
    34
    ok, thanks for clarifying and for the help!
     

Share This Page