Skip to main content

SE::YouTube - full-featured YouTube parser

YouTube

Overview of the parser​

YouTube search results parser. Thanks to the YouTube parser, you can obtain large databases of video links ready for further use. You can use queries in the same form as you enter them into the YouTube search bar. Using the YouTube parser, you can collect basic data on video clips in multi-threaded mode. To obtain full data about each video, you can use SE::YouTube::VideoSE::YouTube::Video

A-Parser functionality allows you to save YouTube parser settings for future use (presets), set parsing schedules, and much more. You can use automatic query multiplication, substitution of subqueries from files, iteration of alphanumeric combinations and lists to obtain the maximum possible number of results.

Saving results is possible in the form and structure you need, thanks to the built-in powerful Template Toolkit template engine which allows you to apply additional logic to the results and output data in various formats, including JSON, SQL, and CSV.

Collected data​

Data is collected from the http://www.youtube.com/ service.

  • Main search results
    • Link to the video
    • Video title
    • Video description
    • Username
    • Link to preview image
    • View count
    • Video length
    • Video upload date
    • Channel subscriber count
    • AI summary, if available
Collected data
  • "Related queries" array
    • Keyword
    • Preview link

Capabilities​

  • Maximum number of pages for parsing in Youtube - 50 pages
  • Country selection from where the search is performed
  • Search by upload date
  • Choice of result type (videos, channels, playlists)
  • Choice of video duration
  • Advanced search parameters (subtitles, 3D, HD, live, Creative Commons license)
  • Sort by relevance, upload date, rating, view count
  • Additionally parses the video preview link
  • Ability to choose the interface language
  • Ability to enable SafeSearch mode

Use cases​

  • Search, collection, and analysis of information on Youtube

Queries​

Search phrases should be specified as queries, for example:

Football   
Niagara Falls   
Speak in english  
Cats and dogs  
Cars

Query substitutions​

You can use built-in macros for query multiplication, for example, if we want to get a very large database of forums, we specify several main queries in different languages:

forum
forum
foro

In the query format, we specify character iteration from a to zzzz, this method allows for maximum rotation of search results and obtaining many new unique results:

$query {az:a:zzzz}

This macro will create 475254 additional queries for each original search query, which in total will give 4 x 475254 = 1901016 search queries, an impressive figure, but it is not a problem at all for A-Parser. At a speed of 2000 queries per minute, such a task will be processed in just 16 hours.

Output results examples​

A-Parser supports flexible result formatting thanks to the built-in Template Toolkit template engine, which allows it to output results in arbitrary form, as well as in structured formats like CSV or JSON

Exporting a list of links​

Same as in SE::Google.

Result format:

[% FOREACH item IN p1.serp;    loop.count _ ' - ' _ item.link _ ' - ' _ item.title _ ' - ' _ item.desc _ "\n"; END %]

Example result:

1 - https://www.youtube.com/watch?v=dm_T7H6J2U8 - HOW SMART ARE YOU? A Simple Test that many adults will fail - In this video you can check how smart you are. A <b>test</b>,  consisting of simple school questions awaits you ...
2 - https://www.youtube.com/watch?v=iDAYNEV9Kxg - Unique Japanese brain aging test! Be sure to check yourself! - Unique Japanese <b>test</b>  for brain aging! Be sure to check yourself! This <b>test</b>  was developed in Japan. How ...
3 - https://www.youtube.com/watch?v=0PEy2_sSy6A - This Simple Test Will Reveal Your Deepest Fear - Our subconscious is a rather dark place, understanding it takes years of psychoanalysis. And this...
4 - https://www.youtube.com/watch?v=j6K9nIugzAY - India vs England 2nd Test Day 4 Highlights 2021| Royal Sports Tv - India vs England 2nd <b>Test</b> Day 4 Highlights 2021 India vs England 2nd <b>Test</b> Day 4 Highlights 2021 | ind vs eng <b>test</b> series India vs ...
5 - https://www.youtube.com/watch?v=ALDqwSMVYKQ - PSYCHOLOGY TEST/ 929 SECONDS OF LAUGHTER/BEST JOKES FOR FEBRUARY 2021 FUNNY/TRY NOT TO LAUGH! BEST COUB - Telegram channel: https://t.me/CrazyHumor129k ON THE CHANNEL YOU WILL FIND 929 SECONDS OF EXCELLENT LAUGHTER TO THE BEST...
6 - https://www.youtube.com/watch?v=6X1puBtvc_s - Pregnancy Test Series Episode 1 - Russian series 2015 HD - Series Premiere - <b>Test</b>  for pregnancy Episode 1 - Russian series 2015 After the death of a patient, gynecologist Natalya ...
7 - https://www.youtube.com/watch?v=hXuhVD7Dwp0 - Test! Optical Illusions That Will Reveal Unexpected Truths About You! - <b>Test</b>! ! Optical Illusions That Will Reveal Unexpected Truths About You! There are many different types ...
8 - https://www.youtube.com/watch?v=BYA8lY4o33A - Test! WHICH ANIMAL IS YOUR TALISMAN? Which predator hides in your soul? Accurate character test - <b>Test</b>! ! WHICH DIVINE ANIMAL ARE YOU? Which predator hides in your soul? Accurate <b>test</b>  for character In order ...
9 - https://www.youtube.com/watch?v=V-kqty2vAm4 - Test! SOMEONE IS SECRETLY IN LOVE WITH YOU! FIND OUT WHO! - <b>Test</b>! ! SOMEONE IS SECRETLY IN LOVE WITH YOU! FIND OUT WHO! You often feel lonely and dream of finding true ...
10 - https://www.youtube.com/watch?v=9HtbSe_oJto - Take this Test and check your Attention - In this video we will check how developed your attention is. Different types of tasks with several levels await you...
...

The built-in tools.CSVLine tool allows you to create correct tabular documents ready for import into Excel or Google Sheets.

General result format:

[%  FOREACH i IN p1.serp;    tools.CSVline(i.link, i.title, i.desc); END  %]

File name:

$datefile.format().csv

Initial text:

Link,Anchor,Snippet

tip

In the General result format, the Template Toolkit template engine is used to output the $serp array in a FOREACH loop.

In the result file name, you just need to change the file extension to csv.

To make the "Initial text" option available in the Task Editor, you need to activate "More options". In "Initial text", write the column names separated by commas and make the second line empty.

Keyword competition​

Same as in SE::Google.

Saving in SQL format​

Result format:

[%  FOREACH serp;   "INSERT INTO serp VALUES('" _ query _ "', '";   link _ "', '";  title _ "')\n"; END  %]

Example result:

INSERT INTO serp VALUES('test', 'https://www.youtube.com/watch?v=dm_T7H6J2U8', 'HOW SMART ARE YOU? A Simple Test that many adults will fail')
INSERT INTO serp VALUES('test', 'https://www.youtube.com/watch?v=iDAYNEV9Kxg', 'Unique Japanese brain aging test! Be sure to check yourself!')
INSERT INTO serp VALUES('test', 'https://www.youtube.com/watch?v=0PEy2_sSy6A', 'This Simple Test Will Reveal Your Deepest Fear')
INSERT INTO serp VALUES('test', 'https://www.youtube.com/watch?v=BYA8lY4o33A', 'Test! WHICH ANIMAL IS YOUR TALISMAN? Which predator hides in your soul? Accurate character test')
INSERT INTO serp VALUES('test', 'https://www.youtube.com/watch?v=5Se6w0lOkyY', 'New Renault Duster. Test drive. Anton Avtoman.')
INSERT INTO serp VALUES('test', 'https://www.youtube.com/watch?v=Ko8cFdoOV6U', 'Test! WHAT KIND OF GIRL ARE YOU? Who is more in you, a LADY or a TOMBOY?')
INSERT INTO serp VALUES('test', 'https://www.youtube.com/watch?v=j6K9nIugzAY', 'India vs England 2nd Test Day 4 Highlights 2021| Royal Sports Tv')
INSERT INTO serp VALUES('test', 'https://www.youtube.com/watch?v=9HtbSe_oJto', 'Take this Test and check your Attention')
INSERT INTO serp VALUES('test', 'https://www.youtube.com/watch?v=V-kqty2vAm4', 'Test! SOMEONE IS SECRETLY IN LOVE WITH YOU! FIND OUT WHO!')
...

Dumping results to JSON​

Same as in SE::Google.

Results processing​

A-Parser allows you to process results directly during parsing; in this section, we have listed the most popular cases for the YouTube parser

Same as in SE::Google.

Same as in SE::Google.

Extracting domains​

Same as in SE::Google.

Removing tags from video titles and descriptions​

Add Results Builder and in the dropdown list select the source: $p1.serp.$i.title - Title. Select type: Remove HTML tags.

Add Results Builder again and in the dropdown list select the source: $p1.serp.$i.desc - Description. Select type: Remove HTML tags.

Download example

How to import an example into A-Parser

eJyVVMtu2zAQ/BWBMJAGUI3k0ItujlsjLZw4tZ1D4fjASCuBDUWyJOXGEPTv3aVo
y07TQ2/kPmb2MWTLPHcv7sGCA+9YtmmZCWeWsdWXLPuhm3XzDMkSar2D5HZ9N088
r1xSWl0na+EluISrIvkMLrfCeKGVYykz3DqwBLg5xUFPASVvpGdpy/zeAPIgsLWi
IKco8G54BbluFMawHZcNxlx3/w4vta25x6ID6iGDjZDfjHvnh4uRp1KTj8mowEKf
ntTFJeu225T1eW4W4ijNXI/jBI7OFd/BWhOVkDCYZ3i753UgK7gH8h4IL8f+lRB4
UQiaCZc9A01kYH1U4lcoVmmMxaMV4GY4WTR5CABk3B+q27BRuDOEaELu9z6HZSWX
DlLmsNQZx0KKtx7hwXKv7SLuKGuZVhMp57ADOYQF/JtGyALXNykx6WtMfD9k8RdG
d2zvlAqX9ttiDUeUcLtZ3A1ZhZ7rCjsvnrFvKWrh8e6mQQkZu0LjC4A5zuyeZlZr
C0eaiBzZUdMGFClkWNnEDKazNs7WcmJsmdONzZFvc5VuGGkKU4OYGMknatKG53Hr
a0k7t5bv0XgIJuX0GaTi9/BIk/8FFxI6TMi1KkW1iE/i0Eaj1viqF2qqayOBhq4a
KVEzDpaDdicuaoQuw/TeJk8DRSCNLxdr0NJ9W/VzNFZgSZ/SWPUpa4TMuZSPy/mp
hw16D1p3BJvjI6o0ypvmHvSfsUrrKjw6eDX4zwAuz9sGui11H3+Z4+/Vnv01Wduh
qn66hz6KuqQYtOG4HEqG/pU/xum9JQ==
tip

Results Builder can be added as many times as you need.

See also: Results Builder

Same as in SE::Google.

Possible settings​

note

General settings for all parsers

Parameter nameDefault valueDescription
DeviceDesktopChoice of output type (Desktop/Mobile)
Pages count10Number of pages to parse (from 1 to 50)
Search from countryAuto (Based on IP)Choice of country from where the search is performed
Interface languageEnglishChoice of interface language
Restricted mode☐Enable/disable "SafeSearch" mode
Uploaded timeAll timeSearch by upload date
Result typeVideoChoice of result type
DurationAllChoice of video duration
FeaturesAllAdvanced search parameters configuration
Sort byRelevancySorting of results
Advanced filters (param sp=)Allows specifying complex filter combinations. To do this, take the value of the sp parameter from the URL in your browser and paste it into this field. This value takes priority over the filters set in the parser settings.