SE::Startpage::Images - Startpage Image Scraper

Scraper overview

Scraper for Startpage search engine images. Using the SE::Startpage::Images scraper, you can obtain image databases ready for further use. You can use queries in the same format as you enter them in the Startpage search bar

A-Parser functionality allows saving scraping settings for the Startpage image scraper for future use (presets), set scraping schedules, and much more. You can use automatic query multiplication, subquery substitution from files, and iteration through alphanumeric combinations and lists to get the maximum possible number of results.

Saving results is possible in any format and structure you need, thanks to the powerful built-in template engine Template Toolkit which allows applying additional logic to results and outputting data in various formats, including JSON, SQL and CSV.

Go to DEMO Buy A-Parser Pro ($299)

Collected data

Links, anchors, and image snippets
Links to source pages
Links to previews
Image width and height
Image types

Capabilities

Scrapes the maximum number of results provided - 50 pages with 10 elements per SERP
Supports selection of search country, results language, and interface language
Ability to specify filters by size, color, type, and usage rights

Use cases

Collecting images for blog content
Building avatar databases

Queries

Search phrases should be specified as queries, for example:

Waterfall  
Speak in english  
Cats and dogs  
cars

Query substitutions

You can use built-in macros for query multiplication; for example, if we want to get a very large database of forums, we can specify several main queries in different languages:

forum
forum
foro
论坛

In the query format, specify character iteration from a to zzzz, this method allows for maximum rotation of search results and obtaining many new unique results:

$query {az:a:zzzz}

This macro will create 475254 additional queries for each original search query, which in total gives 4 x 475254 = 1901016 search queries—an impressive figure, but no problem for A-Parser. At a speed of 2000 queries per minute, such a task will be processed in just 16 hours.

Output results examples

A-Parser supports flexible result formatting thanks to the built-in template engine Template Toolkit, allowing it to output results in arbitrary forms, as well as structured ones like CSV or JSON

Default output

Result format:

$serp.format('$link\n')

Result example:

https://techcrunch.com/wp-content/uploads/2015/04/codecode.jpg
https://specials-images.forbesimg.com/imageserve/5f302109ffad89f9130e07db/960x0.jpg?cropX1=0&cropX2=4800&cropY1=243&cropY2=2943
https://victoria.mediaplanet.com/app/uploads/sites/102/2019/07/mainimage-26.jpg
https://inteng-storage.s3.amazonaws.com/img/iea/9lwjAVnM6E/sizes/ocde_resize_md.jpg
https://miro.medium.com/max/11520/0*Jy3heMl_yP_fQwMO
https://cdn-images.welcometothejungle.com/5DDbrp9_kdlw05Z0hzc7kYEpaaruHhUUWJqs-nW1o8k/rs:auto:980::/q:85/czM6Ly93dHRqLXByb2R1Y3Rpb24vdXBsb2Fkcy9jYXRlZ29yeS9jb3Zlci8yNjYwLzE1NDg4My9jb2xsZWN0aW9uX2NhdGVnb3J5X2JlaGluZF90aGVfY29kZS5qcGc
https://code.org/shared/images/social-media/codeorg2020_social.png
https://miro.medium.com/max/12032/0*ghyQGW_ZCFN_afQ0
https://www.ionos.com/digitalguide/fileadmin/DigitalGuide/Teaser/code-editoren-t.jpg

Output in CSV table

Result format:

[% FOREACH item IN serp;
    tools.CSVline(query, item.link, item.width, item.height,  item.page, item.thumb);
END %]

Result example:

code,https://techcrunch.com/wp-content/uploads/2015/04/codecode.jpg,3888,2592,https://techcrunch.com/2016/05/10/please-dont-learn-to-code/,https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTda13SHf3DRDIZLWnKwu1HLr0JHXzs8QtiQs98Qc5GyEdD9JuHtdk9B8Qko_n-PdDT39k&usqp=CAU,jpg
code,https://specials-images.forbesimg.com/imageserve/5f302109ffad89f9130e07db/960x0.jpg?cropX1=0&cropX2=4800&cropY1=243&cropY2=2943,960,540,https://www.forbes.com/sites/enriquedans/2020/08/09/could-the-no-code-movement-put-programmers-out-of-ajob/,https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSyHw0ZZT5tdDwR4UrT0YNui_R-hKHUCgEYBhNKmTRyKRyZ5XQppBVbYfxa9Tds8Zhx5CI&usqp=CAU,jpg
code,https://victoria.mediaplanet.com/app/uploads/sites/102/2019/07/mainimage-26.jpg,1200,630,https://www.educationandcareernews.com/stem-education/5-reasons-students-should-learn-to-code/,https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQCPak-ygQMNGgIy404t7FticAgqunQe72andfS-QRpaRAno2oJ_qmOshBp85jhLuekQGs&usqp=CAU,jpg
code,https://inteng-storage.s3.amazonaws.com/img/iea/9lwjAVnM6E/sizes/ocde_resize_md.jpg,744,389,https://interestingengineering.com/the-best-ways-to-learn-how-to-code,https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcT90pAbc4h3wggUFQ20JsRJ_XeKAZ0wurtRsOO2rKsUaW4cKx61SntGDg8cpqZrZEcP-Ls&usqp=CAU,jpg
code,https://miro.medium.com/max/11520/0*Jy3heMl_yP_fQwMO,4000,2666,https://medium.com/dealeron-dev/how-to-write-readable-code-8434c58748a1,https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcScySt4wtbwVa_vJIqVVmxkEF943I48xpvWVEbmqOd-PWFgpD1CBCyntWST12MKN7KLrnI&usqp=CAU,
code,https://cdn-images.welcometothejungle.com/5DDbrp9_kdlw05Z0hzc7kYEpaaruHhUUWJqs-nW1o8k/rs:auto:980::/q:85/czM6Ly93dHRqLXByb2R1Y3Rpb24vdXBsb2Fkcy9jYXRlZ29yeS9jb3Zlci8yNjYwLzE1NDg4My9jb2xsZWN0aW9uX2NhdGVnb3J5X2JlaGluZF90aGVfY29kZS5qcGc,980,659,https://www.welcometothejungle.com/en/collections/behind-the-code,https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQOQYtxNtGy1qVMxUO42XWb75nYjE_w6fMlpJNaDxbro0ZorYIcfS0EPhmTEzhSyHBpvvo&usqp=CAU,
code,https://code.org/shared/images/social-media/codeorg2020_social.png,1200,630,https://code.org/,https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcRB6y-6WBM9qNNlVnSnQtFPoHSLh-W0koeQvT_iKLmHasR-h7fPe7Y-U2327bZaRcZV1jk&usqp=CAU,png

Saving in SQL format

Result format:

[% FOREACH serp;
    "INSERT INTO serp VALUES('" _ query _ "', '"; link _ "', '";  page _ "', '";    thumb _ "')\n";
END %]

Result example:

INSERT INTO serp VALUES('code', 'https://techcrunch.com/wp-content/uploads/2015/04/codecode.jpg', '', 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTda13SHf3DRDIZLWnKwu1HLr0JHXzs8QtiQs98Qc5GyEdD9JuHtdk9B8Qko_n-PdDT39k&usqp=CAU')
INSERT INTO serp VALUES('code', 'https://specials-images.forbesimg.com/imageserve/5f302109ffad89f9130e07db/960x0.jpg?cropX1=0&cropX2=4800&cropY1=243&cropY2=2943', '', 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSyHw0ZZT5tdDwR4UrT0YNui_R-hKHUCgEYBhNKmTRyKRyZ5XQppBVbYfxa9Tds8Zhx5CI&usqp=CAU')
INSERT INTO serp VALUES('code', 'https://victoria.mediaplanet.com/app/uploads/sites/102/2019/07/mainimage-26.jpg', '', 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQCPak-ygQMNGgIy404t7FticAgqunQe72andfS-QRpaRAno2oJ_qmOshBp85jhLuekQGs&usqp=CAU')
INSERT INTO serp VALUES('code', 'https://code.org/shared/images/social-media/codeorg2020_social.png', '', 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcRB6y-6WBM9qNNlVnSnQtFPoHSLh-W0koeQvT_iKLmHasR-h7fPe7Y-U2327bZaRcZV1jk&usqp=CAU')
INSERT INTO serp VALUES('code', 'https://cdn-images.welcometothejungle.com/5DDbrp9_kdlw05Z0hzc7kYEpaaruHhUUWJqs-nW1o8k/rs:auto:980::/q:85/czM6Ly93dHRqLXByb2R1Y3Rpb24vdXBsb2Fkcy9jYXRlZ29yeS9jb3Zlci8yNjYwLzE1NDg4My9jb2xsZWN0aW9uX2NhdGVnb3J5X2JlaGluZF90aGVfY29kZS5qcGc', '', 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQOQYtxNtGy1qVMxUO42XWb75nYjE_w6fMlpJNaDxbro0ZorYIcfS0EPhmTEzhSyHBpvvo&usqp=CAU')
INSERT INTO serp VALUES('code', 'https://inteng-storage.s3.amazonaws.com/img/iea/9lwjAVnM6E/sizes/ocde_resize_md.jpg', '', 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcT90pAbc4h3wggUFQ20JsRJ_XeKAZ0wurtRsOO2rKsUaW4cKx61SntGDg8cpqZrZEcP-Ls&usqp=CAU')
INSERT INTO serp VALUES('code', 'https://miro.medium.com/max/11520/0*Jy3heMl_yP_fQwMO', '', 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcScySt4wtbwVa_vJIqVVmxkEF943I48xpvWVEbmqOd-PWFgpD1CBCyntWST12MKN7KLrnI&usqp=CAU')

Dump results to JSON

Общий формат результата:

[% IF notFirst;
  ",\n";
ELSE;
  notFirst = 1;
END;

obj = {};
obj.query = query;
obj.images = [];

FOREACH item IN p1.serp;
    obj.images.push({
        width = item.width
        height = item.height
        link = item.link
        pagelink = item.pagelink
        thumb = item.thumb
    });
END;

obj.json %]

Начальный текст:

Конечный текст:

Result example:

[{
    "images": [
        {
            "link": "https://techcrunch.com/wp-content/uploads/2015/04/codecode.jpg",
            "width": 3888,
            "page": "https://techcrunch.com/2016/05/10/please-dont-learn-to-code/",
            "thumb": "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTda13SHf3DRDIZLWnKwu1HLr0JHXzs8QtiQs98Qc5GyEdD9JuHtdk9B8Qko_n-PdDT39k&usqp=CAU",
            "height": 2592
        },
        {
            "link": "https://specials-images.forbesimg.com/imageserve/5f302109ffad89f9130e07db/960x0.jpg?cropX1=0&cropX2=4800&cropY1=243&cropY2=2943",
            "width": 960,
            "page": "https://www.forbes.com/sites/enriquedans/2020/08/09/could-the-no-code-movement-put-programmers-out-of-ajob/",
            "thumb": "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSyHw0ZZT5tdDwR4UrT0YNui_R-hKHUCgEYBhNKmTRyKRyZ5XQppBVbYfxa9Tds8Zhx5CI&usqp=CAU",
            "height": 540
        },
        {
            "link": "https://victoria.mediaplanet.com/app/uploads/sites/102/2019/07/mainimage-26.jpg",
            "width": 1200,
            "page": "https://www.educationandcareernews.com/stem-education/5-reasons-students-should-learn-to-code/",
            "thumb": "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQCPak-ygQMNGgIy404t7FticAgqunQe72andfS-QRpaRAno2oJ_qmOshBp85jhLuekQGs&usqp=CAU",
            "height": 630
        },
        
    ],
    "query": "code"
}]

tip

To make the "Start text" and "End text" options available in the Task Editor, you need to activate "More options".

Possible settings

note

Common settings for all scrapers

Parameter	Default value	Description
Pages count	`5`	Number of pages to scrape
Family filter	`Filter depending on search`	Safe search filter
Search language	`English`	Selection of results language
Search country	`All`	Selection of country to search from
Page language	`English`	Selection of interface language
Image size	`Any`	Selection of image size
Image color	`Any`	Selection of image color
Image type	`Any`	Selection of image type
License	`Any`	Image usage license

Scraper overview​

Collected data​

Capabilities​

Use cases​

Queries​

Query substitutions​

Output results examples​

Default output​

Output in CSV table​

Saving in SQL format​

Dump results to JSON​

Possible settings​

Scraper overview

Collected data

Capabilities

Use cases

Queries

Query substitutions

Output results examples

Default output

Output in CSV table

Saving in SQL format

Dump results to JSON

Possible settings