SE::Pinterest - Pinterest search results scraper
Overview of the scraper
Scraper of Pinterest search results. Thanks to the SE::Pinterest scraper, you can get databases of image links or images ready for further use. You can also scrape domain databases or texts for further text generation. You can use queries in the same way you enter them in the Pinterest search bar.
Thanks to the multi-threading of A-Parser, the processing speed can reach more than 4000 requests per minute, which on average allows you to get about 100000 results per minute.
The functionality of A-Parser allows you to save the settings of the Pinterest scraper for further use (presets), set a scraping schedule, and much more. You can use automatic query multiplication, substitution of subqueries from files, permutation of alphanumeric combinations and lists to get the maximum possible number of results.
Saving results is possible in the form and structure that you need, thanks to the built-in powerful template engine Template Toolkit which allows you to apply additional logic to the results and output data in various formats, including JSON, SQL, and CSV.
Use cases for the scraper
Downloading images by link
A-Parser allows you to use a chain of tasks, upon completion of the first, the second will start, with the links from the first task being used as requests for the second one
Download example
How to import an example into A-Parser
eJyNVktT2zAQ/iuMJofQhsQcevGFCZRM6VCSQjiFdEbEa1cgS0aSQxiT/96V7PhV
J3Dzrval3W8/OSOG6mc9U6DBaOIvMpK4b+KTGRMGUDBHVzGN4Oi7fBVc0gAUGZCE
Kg3KeizI3aXvl8Z4FkBIU27IcjkgqMJPPZEqpjZoLzkdFhnKwzu6hrnEw5BxqNQT
lG5oDNYroAbs6TB0gfrHQ7OxEWgQMMOkoDzPYAuqst4L9pJaf20UExHao6gY6ImS
MaoNuCBW+barcEF6TiYYJnX+v3Mf4oeUaxgQjeVOKBYTtE8YtoAaqaaJrQn1GZFi
zPk1rIFXZi7+eco4tlKPQ3S6Khy7Tab/xdiWV6ynWoN6VVhDGcVJ59NflVcgr2WE
Nw8e8d6cxcygrC9kKuxwPFQ+AyRl324kamKpoExjVAplcoRNAiJAw2pq46RSNW7R
mExTuZIiZNEU61csgJ1lKuaIzam4kHHCwV6LOCDqJhJTDbcVYMa6GIoVynLboS5c
QtuHAqoDYqTk+uddXniiGOLxmy03xrbWayhau6Kc399eN6qr8OWwpc2DCKU0j2hK
rIeBSCKw7JI55PnkckOtt8Zj2CRUBBDkNW+X20HWWLIbML7/Yz6f1RYMTRREsMFI
2EIDwpyYtwT84ZczZjs16icien9K4Cx6j1h47KaE9nM0Qh9mF0gp+lZsme1CfrJK
tZFx3tZytKj/C9RNbacqAoWc1tbZ7pC9nrvCmnKHJCEFVPFxlY0bnnS4shsKQtcz
5ZuZ2xVZhDTwklJOtnViqUjCXVmP8vUdijQ+6WXFNxpsh5Z68rAd0BIp53s2ogvs
nTzyGZgdRKtXLaiQH7Jbmzb3QPLTuDvd7uFrJF/asc0Z0TJVK5smZ0y7RHb4tt1k
OSixOeov/oyWX48fHob9M78JyV4HJguQ5O62qPJJ6tr/Dt5p8bbXxWrlinfQuVdf
5DY3NOjQ28No7QfIjfOj58E7/DS0jxvPgrftIivvwOPWTbqHXhWv/qLYhG4GO+QU
ZFX+SmSt/wI/Q4+KxVBE+UnPcje7ADq3wYTakcLp9h+3SP7O
Data collected
- Titles and descriptions of pins
- Links to images
- Domains and links to the source
- Pin ID
- Name and username of the author and pinner
Capabilities
- Scrapes up to 100 search result pages
- Supports specifying the required number of pages
Use cases
- Collecting images for filling your blogs
- Collecting textual data for text generation
- Any other variants using Pinterest data
Queries
As queries, you need to specify search phrases, for example:
Waterfall
Speak in english
Cats and dogs
cars
Query substitutions
You can use built-in macros for query multiplication, for example, we want to get a very large database of forums, we will specify several main queries in different languages:
forum
форум
foro
论坛
In the query format, we will specify a permutation of characters from a to zzzz, this method allows you to maximally rotate the search output and get many new unique results:
$query {az:a:zzzz}
This macro will create 475254 additional queries for each original search query, which in total will give 4 x 475254 = 1901016 search queries, an impressive figure, but this is not a problem for A-Parser. At a speed of 2000 requests per minute, such a task will be processed in just 16 hours.
Output results examples
A-Parser supports flexible formatting of results thanks to the built-in template engine Template Toolkit, which allows it to output results in any form, as well as in structured ones, for example CSV or JSON
Default Output
Result format:
$serp.format('$image\n')
Example of result:
https://i.pinimg.com/originals/dd/ba/c6/ddbac6d4813a82c55a658bf3afe8b90e.jpg
https://i.pinimg.com/originals/2e/73/f4/2e73f44e74756c79950772fca4fbea7d.jpg
https://i.pinimg.com/originals/49/4f/72/494f7289e47ac7988611dc7a191e0f89.jpg
https://i.pinimg.com/originals/43/bb/28/43bb28b987d916f543e128c9cdb3464f.jpg
https://i.pinimg.com/originals/0c/d8/6a/0cd86ab71aae1e16fef47fef803998ee.jpg
https://i.pinimg.com/originals/5c/76/d5/5c76d519525457d9807a37c07d140f22.jpg
https://i.pinimg.com/originals/45/64/66/456466acb284572575ce693a35bb75a3.jpg
https://i.pinimg.com/originals/ad/b9/34/adb934d8393b64177c7a0a4195648e4e.jpg
https://i.pinimg.com/originals/b6/ef/70/b6ef705f7f6635bf4e895ee5d114714b.jpg
https://i.pinimg.com/originals/7a/e5/9c/7ae59c58c87f28ffb4568f7b817852eb.jpg
https://i.pinimg.com/originals/57/0d/9a/570d9a47a72e75c9c5419da8ca7b9776.jpg
...
Output to CSV Table
Result format:
[% FOREACH item IN serp;
tools.CSVline(query, item.link, item.pinner_username, item.title, item.id);
END %]
Example of result:
"Speak in english ",https://www.teflcourse.net/english-grammar-corner/great-ways-to-start-a-sentence/?r=Pinterest,,"Great Ways to Start a Sentence",289637819794664055
"Speak in english ",https://www.buzzfeed.com/maximilianzender/deutschlernen?sub=0_10833256#10833256,,"23 Fotos, die dich zur Weißglut bringen, wenn du gerade Deutsch lernst",84653667980063381
"Speak in english ",https://www.teflcourse.net/english-grammar-corner/great-ways-to-start-a-sentence/?r=Pinterest,,"Great Ways to Start a Sentence",307089268347214342
"Speak in english ",https://www.sprachheld.de/category/englisch-lernen/,,"Englisch lernen: W-Fragen!",13792342597188564
"Speak in english ",http://knowyourmeme.com/photos/1176258-pen-pineapple-apple-pen,,"Avengers - I have a pen spoof | ""Pen Pineapple Apple Pen""",609604499561518518
"Speak in english ",https://lehrermarktplatz.de/material/222137/picture-dictionary-word-banks-all-about-english-vocabulary-wachsendes-paket,,"Picture Dictionary: All about English Vocabulary + Word Banks! Englisch Wörterbuch/Vokabeln UK & USA",19210735900566228
"Speak in english ",https://lehrermarktplatz.de/material/180602/conversation-bubbles-english-for-beginners-bearbeitbare-sprechblasen?utm_medium=social&utm_source=pinterest&utm_campaign=tailwind_tribes&utm_content=tribes&utm_term=1144892530_56228916_845126,,"Beginner Conversation Bubbles | Tafelmaterial Englisch Unterricht Grundschule & Klasse 5",314477986488878805
"Speak in english ",https://mobile.twitter.com/AndereDeutsch/status/648719176048222208,,Twitter,1829656081959551
"Speak in english ",https://die-lernlotsen.com/englische-redewendungen/,,"Wow! Englische Redewendungen, die du immer anwenden kannst",82612974406076695
...
Saving in SQL Format
Result format:
[% FOREACH serp;
"INSERT INTO serp VALUES('" _ query _ "', '";
link _ "', '";
image _ "', '";
pinner_username _ "')\n";
END %]
Example of result:
INSERT INTO serp VALUES('Speak in english ', 'https://www.teflcourse.net/english-grammar-corner/great-ways-to-start-a-sentence/?r=Pinterest', 'https://i.pinimg.com/originals/dd/ba/c6/ddbac6d4813a82c55a658bf3afe8b90e.jpg', '')
INSERT INTO serp VALUES('Speak in english ', 'https://www.buzzfeed.com/maximilianzender/deutschlernen?sub=0_10833256#10833256', 'https://i.pinimg.com/originals/2e/73/f4/2e73f44e74756c79950772fca4fbea7d.jpg', '')
INSERT INTO serp VALUES('Speak in english ', 'https://www.teflcourse.net/english-grammar-corner/great-ways-to-start-a-sentence/?r=Pinterest', 'https://i.pinimg.com/originals/49/4f/72/494f7289e47ac7988611dc7a191e0f89.jpg', 'TEFLcourse')
INSERT INTO serp VALUES('Speak in english ', 'https://www.sprachheld.de/category/englisch-lernen/', 'https://i.pinimg.com/originals/43/bb/28/43bb28b987d916f543e128c9cdb3464f.jpg', 'sprachheld')
INSERT INTO serp VALUES('Speak in english ', 'http://knowyourmeme.com/photos/1176258-pen-pineapple-apple-pen', 'https://i.pinimg.com/originals/0c/d8/6a/0cd86ab71aae1e16fef47fef803998ee.jpg', '')
INSERT INTO serp VALUES('Speak in english ', 'https://lehrermarktplatz.de/material/222137/picture-dictionary-word-banks-all-about-english-vocabulary-wachsendes-paket', 'https://i.pinimg.com/originals/5c/76/d5/5c76d519525457d9807a37c07d140f22.jpg', 'unterrichtmitspass')
INSERT INTO serp VALUES('Speak in english ', 'https://lehrermarktplatz.de/material/180602/conversation-bubbles-english-for-beginners-bearbeitbare-sprechblasen?utm_medium=social&utm_source=pinterest&utm_campaign=tailwind_tribes&utm_content=tribes&utm_term=1144892530_56228916_845126', 'https://i.pinimg.com/originals/45/64/66/456466acb284572575ce693a35bb75a3.jpg', '')
INSERT INTO serp VALUES('Speak in english ', 'https://mobile.twitter.com/AndereDeutsch/status/648719176048222208', 'https://i.pinimg.com/originals/ad/b9/34/adb934d8393b64177c7a0a4195648e4e.jpg', '')
INSERT INTO serp VALUES('Speak in english ', 'https://die-lernlotsen.com/englische-redewendungen/', 'https://i.pinimg.com/originals/b6/ef/70/b6ef705f7f6635bf4e895ee5d114714b.jpg', 'lernlotsen')
...
Dump Results to JSON
Общий формат результата:
[% IF notFirst;
",\n";
ELSE;
notFirst = 1;
END;
obj = {};
obj.query = query;
obj.images = [];
FOREACH item IN p1.serp;
obj.images.push({
title = item.title
pinner_username = item.pinner_username
link = item.link
image = item.image
});
END;
obj.json %]
Начальный текст:
[
Конечный текст:
]
Example of result:
[{
"query": "Speak in english",
"images": [
{
"link": "https://www.teflcourse.net/english-grammar-corner/great-ways-to-start-a-sentence/?r=Pinterest",
"author": "",
"title": "Great Ways to Start a Sentence",
"image": "https://i.pinimg.com/originals/dd/ba/c6/ddbac6d4813a82c55a658bf3afe8b90e.jpg"
},
{
"link": "https://www.buzzfeed.com/maximilianzender/deutschlernen?sub=0_10833256#10833256",
"author": "",
"title": "23 Fotos, die dich zur Weißglut bringen, wenn du gerade Deutsch lernst",
"image": "https://i.pinimg.com/originals/2e/73/f4/2e73f44e74756c79950772fca4fbea7d.jpg"
},
{
"link": "https://www.teflcourse.net/english-grammar-corner/great-ways-to-start-a-sentence/?r=Pinterest",
"author": "TEFLcourse",
"title": "Great Ways to Start a Sentence",
"image": "https://i.pinimg.com/originals/49/4f/72/494f7289e47ac7988611dc7a191e0f89.jpg"
},
...
]
}]
To make the "Start text" and "End text" options available in the Task Editor, you need to activate "More options".
Possible Settings
| Parameter | Default Value | Description |
|---|---|---|
| Pages count | 5 | Number of pages to scrape |