Task: need to download pictures a certain theme with a resolution 1920x1080 and a minimum size from 1 to 10 megabytes. Themse set a keywords: They must also be sorted by topic. Solution: 1) Looking with a given key all links to pictures using a parser SE::Google::Images and the filters. Use the filters on the height, width, size and availability at the end of links file extension (in order to save with him then). The height and width is 1 less than the required in order to grab the standard format of 1920x1080. Every link save with its keyword, then to use it to the folder name. Turn on unique as to avoid repetitions. Spoiler: Code for import Code: eyJwcmVzZXQiOiJEb3dubG9hZCBwaWN0dXJlcyAtIDEiLCJ2YWx1ZSI6eyJwcmVz ZXQiOiJEb3dubG9hZCBwaWN0dXJlcyAtIDEiLCJwYXJzZXJzIjpbWyJTRTo6R29v Z2xlOjpJbWFnZXMiLCJkZWZhdWx0Iix7InR5cGUiOiJmaWx0ZXIiLCJyZXN1bHQi Olsic2VycCIsIndpZHRoIl0sImZpbHRlclR5cGUiOiI+IiwidmFsdWUiOiIxOTE5 Iiwib3B0aW9uIjoic2VucyJ9LHsidHlwZSI6ImZpbHRlciIsInJlc3VsdCI6WyJz ZXJwIiwiaGVpZ2h0Il0sImZpbHRlclR5cGUiOiI+IiwidmFsdWUiOiIxMDc5Iiwi b3B0aW9uIjoic2VucyJ9LHsidHlwZSI6Im92ZXJyaWRlIiwiaWQiOiJmb3JtYXRy ZXN1bHQiLCJ2YWx1ZSI6IiRzZXJwLmZvcm1hdCgnJHF1ZXJ5Lm9yaWc7JGxpbmtc XG4nKSJ9LHsidHlwZSI6InVuaXF1ZSIsInJlc3VsdCI6WyJzZXJwIiwibGluayJd LCJ1bmlxdWVUeXBlIjoic3RyaW5nIiwidW5pcXVlR2xvYmFsIjpmYWxzZX0seyJ0 eXBlIjoiZmlsdGVyIiwicmVzdWx0IjpbInNlcnAiLCJsaW5rIl0sImZpbHRlclR5 cGUiOiJyZW1hdGNoIiwidmFsdWUiOiJcXC4oXFx3ezMsNH0pJCIsIm9wdGlvbiI6 IiJ9LHsidHlwZSI6ImZpbHRlciIsInJlc3VsdCI6WyJzZXJwIiwic2l6ZSJdLCJm aWx0ZXJUeXBlIjoiPiIsInZhbHVlIjoiMTAyNCIsIm9wdGlvbiI6InNlbnMifSx7 InR5cGUiOiJmaWx0ZXIiLCJyZXN1bHQiOlsic2VycCIsInNpemUiXSwiZmlsdGVy VHlwZSI6IjwiLCJ2YWx1ZSI6IjEwMjQwIiwib3B0aW9uIjoic2VucyJ9XV0sInJl c3VsdHNGb3JtYXQiOiIkcDEucHJlc2V0IiwicmVzdWx0c1NhdmVUbyI6ImZpbGUi LCJyZXN1bHRzRmlsZU5hbWUiOiJwaWN0dXJlc1xcbGlua3NfJGRhdGVmaWxlLmZv cm1hdCgpLnR4dCIsImFkZGl0aW9uYWxGb3JtYXRzIjpbXSwicmVzdWx0c1VuaXF1 ZSI6Im5vIiwicXVlcnlGb3JtYXQiOlsiJHF1ZXJ5Il0sInVuaXF1ZVF1ZXJpZXMi OmZhbHNlLCJzYXZlRmFpbGVkUXVlcmllcyI6ZmFsc2UsIml0ZXJhdG9yT3B0aW9u cyI6eyJvbkFsbExldmVscyI6ZmFsc2UsInF1ZXJ5QnVpbGRlcnNBZnRlckl0ZXJh dG9yIjpmYWxzZX0sInJlc3VsdHNPcHRpb25zIjp7Im92ZXJ3cml0ZSI6dHJ1ZX0s ImRvTG9nIjoibm8iLCJrZWVwVW5pcXVlIjoiTm8iLCJtb3JlT3B0aW9ucyI6ZmFs c2UsInJlc3VsdHNQcmVwZW5kIjoiIiwicmVzdWx0c0FwcGVuZCI6IiIsInF1ZXJ5 QnVpbGRlcnMiOltdLCJyZXN1bHRzQnVpbGRlcnMiOltdLCJjb25maWdPdmVycmlk ZXMiOltdfX0= 2) By the received in the 1st step links, using Net::HTTP, download images and saved into the appropriate folders. Query builder extract link, name for the folder and file extension. Proxies do not use (the different sites, and it's faster). Change Max body size, to large images are not cropped. Filters out everything is less than 100 KB (it can be is given on the link page and not a picture). The file name and path are generating from previously received data. ATTENTION! I do not recommend to set a large number of threads, optimally 50-100 threads. Spoiler: Code for import Code: eyJwcmVzZXQiOiJEb3dubG9hZCBwaWN0dXJlcyAtIDIiLCJ2YWx1ZSI6eyJwcmVz ZXQiOiJEb3dubG9hZCBwaWN0dXJlcyAtIDIiLCJwYXJzZXJzIjpbWyJOZXQ6OkhU VFAiLCJkZWZhdWx0Iix7InR5cGUiOiJvdmVycmlkZSIsImlkIjoidXNlcHJveHki LCJ2YWx1ZSI6ZmFsc2V9LHsidHlwZSI6Im92ZXJyaWRlIiwiaWQiOiJnb29kQ29k ZSIsInZhbHVlIjoyMDB9LHsidHlwZSI6Im92ZXJyaWRlIiwiaWQiOiJxdWVyeWZv cm1hdCIsInZhbHVlIjoiJHF1ZXJ5LmxpbmsifSx7InR5cGUiOiJvdmVycmlkZSIs ImlkIjoiZm9ybWF0cmVzdWx0IiwidmFsdWUiOiIkZGF0YSJ9LHsidHlwZSI6Im92 ZXJyaWRlIiwiaWQiOiJtYXhfc2l6ZSIsInZhbHVlIjoiMTA0ODU3NjAifSx7InR5 cGUiOiJmaWx0ZXIiLCJyZXN1bHQiOiJbJSBkYXRhLmxlbmd0aCAlXSIsImZpbHRl clR5cGUiOiI+IiwidmFsdWUiOiIxMDI0MDAiLCJvcHRpb24iOiJzZW5zIn1dXSwi cmVzdWx0c0Zvcm1hdCI6IiRwMS5wcmVzZXQiLCJyZXN1bHRzU2F2ZVRvIjoiZmls ZSIsInJlc3VsdHNGaWxlTmFtZSI6InBpY3R1cmVzLyRxdWVyeS5uYW1lLyRxdWVy eS5udW0tJGRhdGVmaWxlLmZvcm1hdCgpLiRxdWVyeS5leHQiLCJhZGRpdGlvbmFs Rm9ybWF0cyI6W10sInJlc3VsdHNVbmlxdWUiOiJubyIsInF1ZXJ5Rm9ybWF0Ijpb IiRxdWVyeSJdLCJ1bmlxdWVRdWVyaWVzIjpmYWxzZSwic2F2ZUZhaWxlZFF1ZXJp ZXMiOmZhbHNlLCJpdGVyYXRvck9wdGlvbnMiOnsib25BbGxMZXZlbHMiOmZhbHNl LCJxdWVyeUJ1aWxkZXJzQWZ0ZXJJdGVyYXRvciI6ZmFsc2V9LCJyZXN1bHRzT3B0 aW9ucyI6eyJvdmVyd3JpdGUiOmZhbHNlfSwiZG9Mb2ciOiJubyIsImtlZXBVbmlx dWUiOiJObyIsIm1vcmVPcHRpb25zIjpmYWxzZSwicmVzdWx0c1ByZXBlbmQiOiIi LCJyZXN1bHRzQXBwZW5kIjoiIiwicXVlcnlCdWlsZGVycyI6W3sic291cmNlIjoi cXVlcnkiLCJ0eXBlIjoic3RyaW5nU3BsaXQiLCJzZXBhcmF0b3IiOiI7IiwidG8i OlsibmFtZSIsImxpbmsiXX0seyJzb3VyY2UiOiJsaW5rIiwidHlwZSI6InJlZ2V4 IiwicmVnZXgiOiJcXC4oXFx3KykkIiwicmVnZXhUeXBlIjpudWxsLCJ0byI6WyJl eHQiXX1dLCJyZXN1bHRzQnVpbGRlcnMiOltdLCJjb25maWdPdmVycmlkZXMiOltd fX0= Result: As a result, we have 360 pictures (more than 1 GB) sorted by category in 4 minutes on 50 streams. Not a bad update of wallpaper
