Compare commits
No commits in common. "master" and "2020.09.14" have entirely different histories.
master
...
2020.09.14
6
.github/ISSUE_TEMPLATE/1_broken_site.md
vendored
6
.github/ISSUE_TEMPLATE/1_broken_site.md
vendored
@ -18,7 +18,7 @@ title: ''
|
||||
|
||||
<!--
|
||||
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
|
||||
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2020.09.20. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
|
||||
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2020.09.14. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
|
||||
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
|
||||
- Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in http://yt-dl.org/escape.
|
||||
- Search the bugtracker for similar issues: http://yt-dl.org/search-issues. DO NOT post duplicates.
|
||||
@ -26,7 +26,7 @@ Carefully read and work through this check list in order to prevent the most com
|
||||
-->
|
||||
|
||||
- [ ] I'm reporting a broken site support
|
||||
- [ ] I've verified that I'm running youtube-dl version **2020.09.20**
|
||||
- [ ] I've verified that I'm running youtube-dl version **2020.09.14**
|
||||
- [ ] I've checked that all provided URLs are alive and playable in a browser
|
||||
- [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
|
||||
- [ ] I've searched the bugtracker for similar issues including closed ones
|
||||
@ -41,7 +41,7 @@ Add the `-v` flag to your command line you run youtube-dl with (`youtube-dl -v <
|
||||
[debug] User config: []
|
||||
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
|
||||
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
|
||||
[debug] youtube-dl version 2020.09.20
|
||||
[debug] youtube-dl version 2020.09.14
|
||||
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
|
||||
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
|
||||
[debug] Proxy map: {}
|
||||
|
||||
@ -19,7 +19,7 @@ labels: 'site-support-request'
|
||||
|
||||
<!--
|
||||
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
|
||||
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2020.09.20. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
|
||||
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2020.09.14. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
|
||||
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
|
||||
- Make sure that site you are requesting is not dedicated to copyright infringement, see https://yt-dl.org/copyright-infringement. youtube-dl does not support such sites. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
|
||||
- Search the bugtracker for similar site support requests: http://yt-dl.org/search-issues. DO NOT post duplicates.
|
||||
@ -27,7 +27,7 @@ Carefully read and work through this check list in order to prevent the most com
|
||||
-->
|
||||
|
||||
- [ ] I'm reporting a new site support request
|
||||
- [ ] I've verified that I'm running youtube-dl version **2020.09.20**
|
||||
- [ ] I've verified that I'm running youtube-dl version **2020.09.14**
|
||||
- [ ] I've checked that all provided URLs are alive and playable in a browser
|
||||
- [ ] I've checked that none of provided URLs violate any copyrights
|
||||
- [ ] I've searched the bugtracker for similar site support requests including closed ones
|
||||
|
||||
@ -18,13 +18,13 @@ title: ''
|
||||
|
||||
<!--
|
||||
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
|
||||
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2020.09.20. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
|
||||
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2020.09.14. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
|
||||
- Search the bugtracker for similar site feature requests: http://yt-dl.org/search-issues. DO NOT post duplicates.
|
||||
- Finally, put x into all relevant boxes (like this [x])
|
||||
-->
|
||||
|
||||
- [ ] I'm reporting a site feature request
|
||||
- [ ] I've verified that I'm running youtube-dl version **2020.09.20**
|
||||
- [ ] I've verified that I'm running youtube-dl version **2020.09.14**
|
||||
- [ ] I've searched the bugtracker for similar site feature requests including closed ones
|
||||
|
||||
|
||||
|
||||
6
.github/ISSUE_TEMPLATE/4_bug_report.md
vendored
6
.github/ISSUE_TEMPLATE/4_bug_report.md
vendored
@ -18,7 +18,7 @@ title: ''
|
||||
|
||||
<!--
|
||||
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
|
||||
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2020.09.20. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
|
||||
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2020.09.14. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
|
||||
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
|
||||
- Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in http://yt-dl.org/escape.
|
||||
- Search the bugtracker for similar issues: http://yt-dl.org/search-issues. DO NOT post duplicates.
|
||||
@ -27,7 +27,7 @@ Carefully read and work through this check list in order to prevent the most com
|
||||
-->
|
||||
|
||||
- [ ] I'm reporting a broken site support issue
|
||||
- [ ] I've verified that I'm running youtube-dl version **2020.09.20**
|
||||
- [ ] I've verified that I'm running youtube-dl version **2020.09.14**
|
||||
- [ ] I've checked that all provided URLs are alive and playable in a browser
|
||||
- [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
|
||||
- [ ] I've searched the bugtracker for similar bug reports including closed ones
|
||||
@ -43,7 +43,7 @@ Add the `-v` flag to your command line you run youtube-dl with (`youtube-dl -v <
|
||||
[debug] User config: []
|
||||
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
|
||||
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
|
||||
[debug] youtube-dl version 2020.09.20
|
||||
[debug] youtube-dl version 2020.09.14
|
||||
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
|
||||
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
|
||||
[debug] Proxy map: {}
|
||||
|
||||
4
.github/ISSUE_TEMPLATE/5_feature_request.md
vendored
4
.github/ISSUE_TEMPLATE/5_feature_request.md
vendored
@ -19,13 +19,13 @@ labels: 'request'
|
||||
|
||||
<!--
|
||||
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
|
||||
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2020.09.20. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
|
||||
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2020.09.14. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
|
||||
- Search the bugtracker for similar feature requests: http://yt-dl.org/search-issues. DO NOT post duplicates.
|
||||
- Finally, put x into all relevant boxes (like this [x])
|
||||
-->
|
||||
|
||||
- [ ] I'm reporting a feature request
|
||||
- [ ] I've verified that I'm running youtube-dl version **2020.09.20**
|
||||
- [ ] I've verified that I'm running youtube-dl version **2020.09.14**
|
||||
- [ ] I've searched the bugtracker for similar feature requests including closed ones
|
||||
|
||||
|
||||
|
||||
21
ChangeLog
21
ChangeLog
@ -1,24 +1,3 @@
|
||||
version 2020.09.20
|
||||
|
||||
Core
|
||||
* [extractor/common] Relax interaction count extraction in _json_ld
|
||||
+ [extractor/common] Extract author as uploader for VideoObject in _json_ld
|
||||
* [downloader/hls] Fix incorrect end byte in Range HTTP header for
|
||||
media segments with EXT-X-BYTERANGE (#14748, #24512)
|
||||
* [extractor/common] Handle ssl.CertificateError in _request_webpage (#26601)
|
||||
* [downloader/http] Improve timeout detection when reading block of data
|
||||
(#10935)
|
||||
* [downloader/http] Retry download when urlopen times out (#10935, #26603)
|
||||
|
||||
Extractors
|
||||
* [redtube] Extend URL regular expression (#26506)
|
||||
* [twitch] Refactor
|
||||
* [twitch:stream] Switch to GraphQL and fix reruns (#26535)
|
||||
+ [telequebec] Add support for brightcove videos (#25833)
|
||||
* [pornhub] Extract metadata from JSON-LD (#26614)
|
||||
* [pornhub] Fix view count extraction (#26621, #26614)
|
||||
|
||||
|
||||
version 2020.09.14
|
||||
|
||||
Core
|
||||
|
||||
@ -545,7 +545,7 @@ The basic usage is not to set any template arguments when downloading a single f
|
||||
- `extractor` (string): Name of the extractor
|
||||
- `extractor_key` (string): Key name of the extractor
|
||||
- `epoch` (numeric): Unix epoch when creating the file
|
||||
- `autonumber` (numeric): Number that will be increased with each download, starting at `--autonumber-start`
|
||||
- `autonumber` (numeric): Five-digit number that will be increased with each download, starting at zero
|
||||
- `playlist` (string): Name or id of the playlist that contains the video
|
||||
- `playlist_index` (numeric): Index of the video in the playlist padded with leading zeros according to the total length of the playlist
|
||||
- `playlist_id` (string): Playlist identifier
|
||||
|
||||
@ -994,12 +994,6 @@ class TestUtil(unittest.TestCase):
|
||||
on = js_to_json('{42:4.2e1}')
|
||||
self.assertEqual(json.loads(on), {'42': 42.0})
|
||||
|
||||
on = js_to_json('{ "0x40": "0x40" }')
|
||||
self.assertEqual(json.loads(on), {'0x40': '0x40'})
|
||||
|
||||
on = js_to_json('{ "040": "040" }')
|
||||
self.assertEqual(json.loads(on), {'040': '040'})
|
||||
|
||||
def test_js_to_json_malformed(self):
|
||||
self.assertEqual(js_to_json('42a1'), '42"a1"')
|
||||
self.assertEqual(js_to_json('42a-1'), '42"a"-1')
|
||||
|
||||
@ -141,7 +141,7 @@ class HlsFD(FragmentFD):
|
||||
count = 0
|
||||
headers = info_dict.get('http_headers', {})
|
||||
if byte_range:
|
||||
headers['Range'] = 'bytes=%d-%d' % (byte_range['start'], byte_range['end'] - 1)
|
||||
headers['Range'] = 'bytes=%d-%d' % (byte_range['start'], byte_range['end'])
|
||||
while count <= fragment_retries:
|
||||
try:
|
||||
success, frag_content = self._download_fragment(
|
||||
|
||||
@ -106,12 +106,7 @@ class HttpFD(FileDownloader):
|
||||
set_range(request, range_start, range_end)
|
||||
# Establish connection
|
||||
try:
|
||||
try:
|
||||
ctx.data = self.ydl.urlopen(request)
|
||||
except (compat_urllib_error.URLError, ) as err:
|
||||
if isinstance(err.reason, socket.timeout):
|
||||
raise RetryDownload(err)
|
||||
raise err
|
||||
ctx.data = self.ydl.urlopen(request)
|
||||
# When trying to resume, Content-Range HTTP header of response has to be checked
|
||||
# to match the value of requested Range HTTP header. This is due to a webservers
|
||||
# that don't support resuming and serve a whole file with no Content-Range
|
||||
@ -223,10 +218,9 @@ class HttpFD(FileDownloader):
|
||||
|
||||
def retry(e):
|
||||
to_stdout = ctx.tmpfilename == '-'
|
||||
if ctx.stream is not None:
|
||||
if not to_stdout:
|
||||
ctx.stream.close()
|
||||
ctx.stream = None
|
||||
if not to_stdout:
|
||||
ctx.stream.close()
|
||||
ctx.stream = None
|
||||
ctx.resume_len = byte_counter if to_stdout else os.path.getsize(encodeFilename(ctx.tmpfilename))
|
||||
raise RetryDownload(e)
|
||||
|
||||
@ -239,11 +233,9 @@ class HttpFD(FileDownloader):
|
||||
except socket.timeout as e:
|
||||
retry(e)
|
||||
except socket.error as e:
|
||||
# SSLError on python 2 (inherits socket.error) may have
|
||||
# no errno set but this error message
|
||||
if e.errno in (errno.ECONNRESET, errno.ETIMEDOUT) or getattr(e, 'message', None) == 'The read operation timed out':
|
||||
retry(e)
|
||||
raise
|
||||
if e.errno not in (errno.ECONNRESET, errno.ETIMEDOUT):
|
||||
raise
|
||||
retry(e)
|
||||
|
||||
byte_counter += len(data_block)
|
||||
|
||||
|
||||
@ -275,7 +275,7 @@ class AfreecaTVIE(InfoExtractor):
|
||||
video_element = video_xml.findall(compat_xpath('./track/video'))[-1]
|
||||
if video_element is None or video_element.text is None:
|
||||
raise ExtractorError(
|
||||
'Video %s does not exist' % video_id, expected=True)
|
||||
'Video %s video does not exist' % video_id, expected=True)
|
||||
|
||||
video_url = video_element.text.strip()
|
||||
|
||||
|
||||
@ -10,7 +10,6 @@ import os
|
||||
import random
|
||||
import re
|
||||
import socket
|
||||
import ssl
|
||||
import sys
|
||||
import time
|
||||
import math
|
||||
@ -68,7 +67,6 @@ from ..utils import (
|
||||
sanitized_Request,
|
||||
sanitize_filename,
|
||||
str_or_none,
|
||||
str_to_int,
|
||||
strip_or_none,
|
||||
unescapeHTML,
|
||||
unified_strdate,
|
||||
@ -625,12 +623,9 @@ class InfoExtractor(object):
|
||||
url_or_request = update_url_query(url_or_request, query)
|
||||
if data is not None or headers:
|
||||
url_or_request = sanitized_Request(url_or_request, data, headers)
|
||||
exceptions = [compat_urllib_error.URLError, compat_http_client.HTTPException, socket.error]
|
||||
if hasattr(ssl, 'CertificateError'):
|
||||
exceptions.append(ssl.CertificateError)
|
||||
try:
|
||||
return self._downloader.urlopen(url_or_request)
|
||||
except tuple(exceptions) as err:
|
||||
except (compat_urllib_error.URLError, compat_http_client.HTTPException, socket.error) as err:
|
||||
if isinstance(err, compat_urllib_error.HTTPError):
|
||||
if self.__can_accept_status_code(err, expected_status):
|
||||
# Retain reference to error to prevent file object from
|
||||
@ -1249,10 +1244,7 @@ class InfoExtractor(object):
|
||||
interaction_type = is_e.get('interactionType')
|
||||
if not isinstance(interaction_type, compat_str):
|
||||
continue
|
||||
# For interaction count some sites provide string instead of
|
||||
# an integer (as per spec) with non digit characters (e.g. ",")
|
||||
# so extracting count with more relaxed str_to_int
|
||||
interaction_count = str_to_int(is_e.get('userInteractionCount'))
|
||||
interaction_count = int_or_none(is_e.get('userInteractionCount'))
|
||||
if interaction_count is None:
|
||||
continue
|
||||
count_kind = INTERACTION_TYPE_MAP.get(interaction_type.split('/')[-1])
|
||||
@ -1272,7 +1264,6 @@ class InfoExtractor(object):
|
||||
'thumbnail': url_or_none(e.get('thumbnailUrl') or e.get('thumbnailURL')),
|
||||
'duration': parse_duration(e.get('duration')),
|
||||
'timestamp': unified_timestamp(e.get('uploadDate')),
|
||||
'uploader': str_or_none(e.get('author')),
|
||||
'filesize': float_or_none(e.get('contentSize')),
|
||||
'tbr': int_or_none(e.get('bitrate')),
|
||||
'width': int_or_none(e.get('width')),
|
||||
|
||||
@ -15,7 +15,7 @@ from ..utils import (
|
||||
class ExpressenIE(InfoExtractor):
|
||||
_VALID_URL = r'''(?x)
|
||||
https?://
|
||||
(?:www\.)?(?:expressen|di)\.se/
|
||||
(?:www\.)?expressen\.se/
|
||||
(?:(?:tvspelare/video|videoplayer/embed)/)?
|
||||
tv/(?:[^/]+/)*
|
||||
(?P<id>[^/?#&]+)
|
||||
@ -42,16 +42,13 @@ class ExpressenIE(InfoExtractor):
|
||||
}, {
|
||||
'url': 'https://www.expressen.se/videoplayer/embed/tv/ditv/ekonomistudion/experterna-har-ar-fragorna-som-avgor-valet/?embed=true&external=true&autoplay=true&startVolume=0&partnerId=di',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
'url': 'https://www.di.se/videoplayer/embed/tv/ditv/borsmorgon/implantica-rusar-70--under-borspremiaren-hor-styrelsemedlemmen/?embed=true&external=true&autoplay=true&startVolume=0&partnerId=di',
|
||||
'only_matching': True,
|
||||
}]
|
||||
|
||||
@staticmethod
|
||||
def _extract_urls(webpage):
|
||||
return [
|
||||
mobj.group('url') for mobj in re.finditer(
|
||||
r'<iframe[^>]+\bsrc=(["\'])(?P<url>(?:https?:)?//(?:www\.)?(?:expressen|di)\.se/(?:tvspelare/video|videoplayer/embed)/tv/.+?)\1',
|
||||
r'<iframe[^>]+\bsrc=(["\'])(?P<url>(?:https?:)?//(?:www\.)?expressen\.se/(?:tvspelare/video|videoplayer/embed)/tv/.+?)\1',
|
||||
webpage)]
|
||||
|
||||
def _real_extract(self, url):
|
||||
|
||||
@ -86,8 +86,7 @@ class IPrimaIE(InfoExtractor):
|
||||
(r'<iframe[^>]+\bsrc=["\'](?:https?:)?//(?:api\.play-backend\.iprima\.cz/prehravac/embedded|prima\.iprima\.cz/[^/]+/[^/]+)\?.*?\bid=(p\d+)',
|
||||
r'data-product="([^"]+)">',
|
||||
r'id=["\']player-(p\d+)"',
|
||||
r'playerId\s*:\s*["\']player-(p\d+)',
|
||||
r'\bvideos\s*=\s*["\'](p\d+)'),
|
||||
r'playerId\s*:\s*["\']player-(p\d+)'),
|
||||
webpage, 'real id')
|
||||
|
||||
playerpage = self._download_webpage(
|
||||
|
||||
@ -150,7 +150,7 @@ class IqiyiSDKInterpreter(object):
|
||||
elif function in other_functions:
|
||||
other_functions[function]()
|
||||
else:
|
||||
raise ExtractorError('Unknown function %s' % function)
|
||||
raise ExtractorError('Unknown funcion %s' % function)
|
||||
|
||||
return sdk.target
|
||||
|
||||
|
||||
@ -17,7 +17,6 @@ from ..utils import (
|
||||
determine_ext,
|
||||
ExtractorError,
|
||||
int_or_none,
|
||||
merge_dicts,
|
||||
NO_DEFAULT,
|
||||
orderedSet,
|
||||
remove_quotes,
|
||||
@ -60,14 +59,13 @@ class PornHubIE(PornHubBaseIE):
|
||||
'''
|
||||
_TESTS = [{
|
||||
'url': 'http://www.pornhub.com/view_video.php?viewkey=648719015',
|
||||
'md5': 'a6391306d050e4547f62b3f485dd9ba9',
|
||||
'md5': '1e19b41231a02eba417839222ac9d58e',
|
||||
'info_dict': {
|
||||
'id': '648719015',
|
||||
'ext': 'mp4',
|
||||
'title': 'Seductive Indian beauty strips down and fingers her pink pussy',
|
||||
'uploader': 'Babes',
|
||||
'upload_date': '20130628',
|
||||
'timestamp': 1372447216,
|
||||
'duration': 361,
|
||||
'view_count': int,
|
||||
'like_count': int,
|
||||
@ -84,8 +82,8 @@ class PornHubIE(PornHubBaseIE):
|
||||
'id': '1331683002',
|
||||
'ext': 'mp4',
|
||||
'title': '重庆婷婷女王足交',
|
||||
'uploader': 'Unknown',
|
||||
'upload_date': '20150213',
|
||||
'timestamp': 1423804862,
|
||||
'duration': 1753,
|
||||
'view_count': int,
|
||||
'like_count': int,
|
||||
@ -123,7 +121,6 @@ class PornHubIE(PornHubBaseIE):
|
||||
'params': {
|
||||
'skip_download': True,
|
||||
},
|
||||
'skip': 'This video has been disabled',
|
||||
}, {
|
||||
'url': 'http://www.pornhub.com/view_video.php?viewkey=ph557bbb6676d2d',
|
||||
'only_matching': True,
|
||||
@ -341,10 +338,10 @@ class PornHubIE(PornHubBaseIE):
|
||||
|
||||
video_uploader = self._html_search_regex(
|
||||
r'(?s)From: .+?<(?:a\b[^>]+\bhref=["\']/(?:(?:user|channel)s|model|pornstar)/|span\b[^>]+\bclass=["\']username)[^>]+>(.+?)<',
|
||||
webpage, 'uploader', default=None)
|
||||
webpage, 'uploader', fatal=False)
|
||||
|
||||
view_count = self._extract_count(
|
||||
r'<span class="count">([\d,\.]+)</span> [Vv]iews', webpage, 'view')
|
||||
r'<span class="count">([\d,\.]+)</span> views', webpage, 'view')
|
||||
like_count = self._extract_count(
|
||||
r'<span class="votesUp">([\d,\.]+)</span>', webpage, 'like')
|
||||
dislike_count = self._extract_count(
|
||||
@ -359,11 +356,7 @@ class PornHubIE(PornHubBaseIE):
|
||||
if div:
|
||||
return re.findall(r'<a[^>]+\bhref=[^>]+>([^<]+)', div)
|
||||
|
||||
info = self._search_json_ld(webpage, video_id, default={})
|
||||
# description provided in JSON-LD is irrelevant
|
||||
info['description'] = None
|
||||
|
||||
return merge_dicts({
|
||||
return {
|
||||
'id': video_id,
|
||||
'uploader': video_uploader,
|
||||
'upload_date': upload_date,
|
||||
@ -379,7 +372,7 @@ class PornHubIE(PornHubBaseIE):
|
||||
'tags': extract_list('tags'),
|
||||
'categories': extract_list('categories'),
|
||||
'subtitles': subtitles,
|
||||
}, info)
|
||||
}
|
||||
|
||||
|
||||
class PornHubPlaylistBaseIE(PornHubBaseIE):
|
||||
|
||||
@ -15,7 +15,7 @@ from ..utils import (
|
||||
|
||||
|
||||
class RedTubeIE(InfoExtractor):
|
||||
_VALID_URL = r'https?://(?:(?:\w+\.)?redtube\.com/|embed\.redtube\.com/\?.*?\bid=)(?P<id>[0-9]+)'
|
||||
_VALID_URL = r'https?://(?:(?:www\.)?redtube\.com/|embed\.redtube\.com/\?.*?\bid=)(?P<id>[0-9]+)'
|
||||
_TESTS = [{
|
||||
'url': 'http://www.redtube.com/66418',
|
||||
'md5': 'fc08071233725f26b8f014dba9590005',
|
||||
@ -31,9 +31,6 @@ class RedTubeIE(InfoExtractor):
|
||||
}, {
|
||||
'url': 'http://embed.redtube.com/?bgcolor=000000&id=1443286',
|
||||
'only_matching': True,
|
||||
}, {
|
||||
'url': 'http://it.redtube.com/66418',
|
||||
'only_matching': True,
|
||||
}]
|
||||
|
||||
@staticmethod
|
||||
|
||||
@ -13,24 +13,14 @@ from ..utils import (
|
||||
|
||||
class TeleQuebecBaseIE(InfoExtractor):
|
||||
@staticmethod
|
||||
def _result(url, ie_key):
|
||||
def _limelight_result(media_id):
|
||||
return {
|
||||
'_type': 'url_transparent',
|
||||
'url': smuggle_url(url, {'geo_countries': ['CA']}),
|
||||
'ie_key': ie_key,
|
||||
'url': smuggle_url(
|
||||
'limelight:media:' + media_id, {'geo_countries': ['CA']}),
|
||||
'ie_key': 'LimelightMedia',
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def _limelight_result(media_id):
|
||||
return TeleQuebecBaseIE._result(
|
||||
'limelight:media:' + media_id, 'LimelightMedia')
|
||||
|
||||
@staticmethod
|
||||
def _brightcove_result(brightcove_id):
|
||||
return TeleQuebecBaseIE._result(
|
||||
'http://players.brightcove.net/6150020952001/default_default/index.html?videoId=%s'
|
||||
% brightcove_id, 'BrightcoveNew')
|
||||
|
||||
|
||||
class TeleQuebecIE(TeleQuebecBaseIE):
|
||||
_VALID_URL = r'''(?x)
|
||||
@ -47,27 +37,11 @@ class TeleQuebecIE(TeleQuebecBaseIE):
|
||||
'id': '577116881b4b439084e6b1cf4ef8b1b3',
|
||||
'ext': 'mp4',
|
||||
'title': 'Un petit choc et puis repart!',
|
||||
'description': 'md5:067bc84bd6afecad85e69d1000730907',
|
||||
'description': 'md5:b04a7e6b3f74e32d7b294cffe8658374',
|
||||
},
|
||||
'params': {
|
||||
'skip_download': True,
|
||||
},
|
||||
}, {
|
||||
'url': 'https://zonevideo.telequebec.tv/media/55267/le-soleil/passe-partout',
|
||||
'info_dict': {
|
||||
'id': '6167180337001',
|
||||
'ext': 'mp4',
|
||||
'title': 'Le soleil',
|
||||
'description': 'md5:64289c922a8de2abbe99c354daffde02',
|
||||
'uploader_id': '6150020952001',
|
||||
'upload_date': '20200625',
|
||||
'timestamp': 1593090307,
|
||||
},
|
||||
'params': {
|
||||
'format': 'bestvideo',
|
||||
'skip_download': True,
|
||||
},
|
||||
'add_ie': ['BrightcoveNew'],
|
||||
}, {
|
||||
# no description
|
||||
'url': 'http://zonevideo.telequebec.tv/media/30261',
|
||||
@ -84,14 +58,7 @@ class TeleQuebecIE(TeleQuebecBaseIE):
|
||||
'https://mnmedias.api.telequebec.tv/api/v2/media/' + media_id,
|
||||
media_id)['media']
|
||||
|
||||
source_id = media_data['streamInfo']['sourceId']
|
||||
source = (try_get(
|
||||
media_data, lambda x: x['streamInfo']['source'],
|
||||
compat_str) or 'limelight').lower()
|
||||
if source == 'brightcove':
|
||||
info = self._brightcove_result(source_id)
|
||||
else:
|
||||
info = self._limelight_result(source_id)
|
||||
info = self._limelight_result(media_data['streamInfo']['sourceId'])
|
||||
info.update({
|
||||
'title': media_data.get('title'),
|
||||
'description': try_get(
|
||||
|
||||
@ -8,8 +8,8 @@ from ..utils import int_or_none
|
||||
|
||||
class TwentyThreeVideoIE(InfoExtractor):
|
||||
IE_NAME = '23video'
|
||||
_VALID_URL = r'https?://(?P<domain>[^.]+\.(?:twentythree\.net|23video\.com|filmweb\.no))/v\.ihtml/player\.html\?(?P<query>.*?\bphoto(?:_|%5f)id=(?P<id>\d+).*)'
|
||||
_TESTS = [{
|
||||
_VALID_URL = r'https?://video\.(?P<domain>twentythree\.net|23video\.com|filmweb\.no)/v\.ihtml/player\.html\?(?P<query>.*?\bphoto(?:_|%5f)id=(?P<id>\d+).*)'
|
||||
_TEST = {
|
||||
'url': 'https://video.twentythree.net/v.ihtml/player.html?showDescriptions=0&source=site&photo%5fid=20448876&autoPlay=1',
|
||||
'md5': '75fcf216303eb1dae9920d651f85ced4',
|
||||
'info_dict': {
|
||||
@ -21,14 +21,11 @@ class TwentyThreeVideoIE(InfoExtractor):
|
||||
'uploader_id': '12258964',
|
||||
'uploader': 'Rasmus Bysted',
|
||||
}
|
||||
}, {
|
||||
'url': 'https://bonnier-publications-danmark.23video.com/v.ihtml/player.html?token=f0dc46476e06e13afd5a1f84a29e31e8&source=embed&photo%5fid=36137620',
|
||||
'only_matching': True,
|
||||
}]
|
||||
}
|
||||
|
||||
def _real_extract(self, url):
|
||||
domain, query, photo_id = re.match(self._VALID_URL, url).groups()
|
||||
base_url = 'https://%s' % domain
|
||||
base_url = 'https://video.%s' % domain
|
||||
photo_data = self._download_json(
|
||||
base_url + '/api/photo/list?' + query, photo_id, query={
|
||||
'format': 'json',
|
||||
|
||||
@ -24,6 +24,7 @@ from ..utils import (
|
||||
parse_duration,
|
||||
parse_iso8601,
|
||||
qualities,
|
||||
str_or_none,
|
||||
try_get,
|
||||
unified_timestamp,
|
||||
update_url_query,
|
||||
@ -336,27 +337,19 @@ def _make_video_result(node):
|
||||
class TwitchGraphQLBaseIE(TwitchBaseIE):
|
||||
_PAGE_LIMIT = 100
|
||||
|
||||
_OPERATION_HASHES = {
|
||||
'CollectionSideBar': '27111f1b382effad0b6def325caef1909c733fe6a4fbabf54f8d491ef2cf2f14',
|
||||
'FilterableVideoTower_Videos': 'a937f1d22e269e39a03b509f65a7490f9fc247d7f83d6ac1421523e3b68042cb',
|
||||
'ClipsCards__User': 'b73ad2bfaecfd30a9e6c28fada15bd97032c83ec77a0440766a56fe0bd632777',
|
||||
'ChannelCollectionsContent': '07e3691a1bad77a36aba590c351180439a40baefc1c275356f40fc7082419a84',
|
||||
'StreamMetadata': '1c719a40e481453e5c48d9bb585d971b8b372f8ebb105b17076722264dfa5b3e',
|
||||
'ComscoreStreamingQuery': 'e1edae8122517d013405f237ffcc124515dc6ded82480a88daef69c83b53ac01',
|
||||
'VideoPreviewOverlay': '3006e77e51b128d838fa4e835723ca4dc9a05c5efd4466c1085215c6e437e65c',
|
||||
}
|
||||
|
||||
def _download_gql(self, video_id, ops, note, fatal=True):
|
||||
for op in ops:
|
||||
op['extensions'] = {
|
||||
'persistedQuery': {
|
||||
'version': 1,
|
||||
'sha256Hash': self._OPERATION_HASHES[op['operationName']],
|
||||
}
|
||||
}
|
||||
def _download_gql(self, video_id, op, variables, sha256_hash, note, fatal=True):
|
||||
return self._download_json(
|
||||
'https://gql.twitch.tv/gql', video_id, note,
|
||||
data=json.dumps(ops).encode(),
|
||||
data=json.dumps({
|
||||
'operationName': op,
|
||||
'variables': variables,
|
||||
'extensions': {
|
||||
'persistedQuery': {
|
||||
'version': 1,
|
||||
'sha256Hash': sha256_hash,
|
||||
}
|
||||
}
|
||||
}).encode(),
|
||||
headers={
|
||||
'Content-Type': 'text/plain;charset=UTF-8',
|
||||
'Client-ID': self._CLIENT_ID,
|
||||
@ -376,15 +369,14 @@ class TwitchCollectionIE(TwitchGraphQLBaseIE):
|
||||
}]
|
||||
|
||||
_OPERATION_NAME = 'CollectionSideBar'
|
||||
_SHA256_HASH = '27111f1b382effad0b6def325caef1909c733fe6a4fbabf54f8d491ef2cf2f14'
|
||||
|
||||
def _real_extract(self, url):
|
||||
collection_id = self._match_id(url)
|
||||
collection = self._download_gql(
|
||||
collection_id, [{
|
||||
'operationName': self._OPERATION_NAME,
|
||||
'variables': {'collectionID': collection_id},
|
||||
}],
|
||||
'Downloading collection GraphQL')[0]['data']['collection']
|
||||
collection_id, self._OPERATION_NAME,
|
||||
{'collectionID': collection_id}, self._SHA256_HASH,
|
||||
'Downloading collection GraphQL')['data']['collection']
|
||||
title = collection.get('title')
|
||||
entries = []
|
||||
for edge in collection['items']['edges']:
|
||||
@ -411,16 +403,14 @@ class TwitchPlaylistBaseIE(TwitchGraphQLBaseIE):
|
||||
if cursor:
|
||||
variables['cursor'] = cursor
|
||||
page = self._download_gql(
|
||||
channel_name, [{
|
||||
'operationName': self._OPERATION_NAME,
|
||||
'variables': variables,
|
||||
}],
|
||||
channel_name, self._OPERATION_NAME, variables,
|
||||
self._SHA256_HASH,
|
||||
'Downloading %ss GraphQL page %s' % (self._NODE_KIND, page_num),
|
||||
fatal=False)
|
||||
if not page:
|
||||
break
|
||||
edges = try_get(
|
||||
page, lambda x: x[0]['data']['user'][entries_key]['edges'], list)
|
||||
page, lambda x: x['data']['user'][entries_key]['edges'], list)
|
||||
if not edges:
|
||||
break
|
||||
for edge in edges:
|
||||
@ -563,6 +553,7 @@ class TwitchVideosIE(TwitchPlaylistBaseIE):
|
||||
'views': 'Popular',
|
||||
}
|
||||
|
||||
_SHA256_HASH = 'a937f1d22e269e39a03b509f65a7490f9fc247d7f83d6ac1421523e3b68042cb'
|
||||
_OPERATION_NAME = 'FilterableVideoTower_Videos'
|
||||
_ENTRY_KIND = 'video'
|
||||
_EDGE_KIND = 'VideoEdge'
|
||||
@ -631,6 +622,7 @@ class TwitchVideosClipsIE(TwitchPlaylistBaseIE):
|
||||
# NB: values other than 20 result in skipped videos
|
||||
_PAGE_LIMIT = 20
|
||||
|
||||
_SHA256_HASH = 'b73ad2bfaecfd30a9e6c28fada15bd97032c83ec77a0440766a56fe0bd632777'
|
||||
_OPERATION_NAME = 'ClipsCards__User'
|
||||
_ENTRY_KIND = 'clip'
|
||||
_EDGE_KIND = 'ClipEdge'
|
||||
@ -688,6 +680,7 @@ class TwitchVideosCollectionsIE(TwitchPlaylistBaseIE):
|
||||
'playlist_mincount': 3,
|
||||
}]
|
||||
|
||||
_SHA256_HASH = '07e3691a1bad77a36aba590c351180439a40baefc1c275356f40fc7082419a84'
|
||||
_OPERATION_NAME = 'ChannelCollectionsContent'
|
||||
_ENTRY_KIND = 'collection'
|
||||
_EDGE_KIND = 'CollectionsItemEdge'
|
||||
@ -724,7 +717,7 @@ class TwitchVideosCollectionsIE(TwitchPlaylistBaseIE):
|
||||
playlist_title='%s - Collections' % channel_name)
|
||||
|
||||
|
||||
class TwitchStreamIE(TwitchGraphQLBaseIE):
|
||||
class TwitchStreamIE(TwitchBaseIE):
|
||||
IE_NAME = 'twitch:stream'
|
||||
_VALID_URL = r'''(?x)
|
||||
https?://
|
||||
@ -781,43 +774,28 @@ class TwitchStreamIE(TwitchGraphQLBaseIE):
|
||||
else super(TwitchStreamIE, cls).suitable(url))
|
||||
|
||||
def _real_extract(self, url):
|
||||
channel_name = self._match_id(url).lower()
|
||||
|
||||
gql = self._download_gql(
|
||||
channel_name, [{
|
||||
'operationName': 'StreamMetadata',
|
||||
'variables': {'channelLogin': channel_name},
|
||||
}, {
|
||||
'operationName': 'ComscoreStreamingQuery',
|
||||
'variables': {
|
||||
'channel': channel_name,
|
||||
'clipSlug': '',
|
||||
'isClip': False,
|
||||
'isLive': True,
|
||||
'isVodOrCollection': False,
|
||||
'vodID': '',
|
||||
},
|
||||
}, {
|
||||
'operationName': 'VideoPreviewOverlay',
|
||||
'variables': {'login': channel_name},
|
||||
}],
|
||||
'Downloading stream GraphQL')
|
||||
|
||||
user = gql[0]['data']['user']
|
||||
|
||||
if not user:
|
||||
raise ExtractorError(
|
||||
'%s does not exist' % channel_name, expected=True)
|
||||
|
||||
stream = user['stream']
|
||||
|
||||
if not stream:
|
||||
raise ExtractorError('%s is offline' % channel_name, expected=True)
|
||||
channel_name = self._match_id(url)
|
||||
|
||||
access_token = self._download_access_token(channel_name)
|
||||
token = access_token['token']
|
||||
|
||||
stream_id = stream.get('id') or channel_name
|
||||
token = access_token['token']
|
||||
channel_id = self._extract_channel_id(token, channel_name)
|
||||
|
||||
stream = self._call_api(
|
||||
'kraken/streams/%s?stream_type=all' % channel_id,
|
||||
channel_id, 'Downloading stream JSON').get('stream')
|
||||
|
||||
if not stream:
|
||||
raise ExtractorError('%s is offline' % channel_id, expected=True)
|
||||
|
||||
# Channel name may be typed if different case than the original channel name
|
||||
# (e.g. http://www.twitch.tv/TWITCHPLAYSPOKEMON) that will lead to constructing
|
||||
# an invalid m3u8 URL. Working around by use of original channel name from stream
|
||||
# JSON and fallback to lowercase if it's not available.
|
||||
channel_name = try_get(
|
||||
stream, lambda x: x['channel']['name'],
|
||||
compat_str) or channel_name.lower()
|
||||
|
||||
query = {
|
||||
'allow_source': 'true',
|
||||
'allow_audio_only': 'true',
|
||||
@ -830,39 +808,41 @@ class TwitchStreamIE(TwitchGraphQLBaseIE):
|
||||
'token': token.encode('utf-8'),
|
||||
}
|
||||
formats = self._extract_m3u8_formats(
|
||||
'%s/api/channel/hls/%s.m3u8' % (self._USHER_BASE, channel_name),
|
||||
stream_id, 'mp4', query=query)
|
||||
'%s/api/channel/hls/%s.m3u8?%s'
|
||||
% (self._USHER_BASE, channel_name, compat_urllib_parse_urlencode(query)),
|
||||
channel_id, 'mp4')
|
||||
self._prefer_source(formats)
|
||||
|
||||
view_count = stream.get('viewers')
|
||||
timestamp = unified_timestamp(stream.get('createdAt'))
|
||||
timestamp = parse_iso8601(stream.get('created_at'))
|
||||
|
||||
sq_user = try_get(gql, lambda x: x[1]['data']['user'], dict) or {}
|
||||
uploader = sq_user.get('displayName')
|
||||
description = try_get(
|
||||
sq_user, lambda x: x['broadcastSettings']['title'], compat_str)
|
||||
channel = stream['channel']
|
||||
title = self._live_title(channel.get('display_name') or channel.get('name'))
|
||||
description = channel.get('status')
|
||||
|
||||
thumbnail = url_or_none(try_get(
|
||||
gql, lambda x: x[2]['data']['user']['stream']['previewImageURL'],
|
||||
compat_str))
|
||||
|
||||
title = uploader or channel_name
|
||||
stream_type = stream.get('type')
|
||||
if stream_type in ['rerun', 'live']:
|
||||
title += ' (%s)' % stream_type
|
||||
thumbnails = []
|
||||
for thumbnail_key, thumbnail_url in stream['preview'].items():
|
||||
m = re.search(r'(?P<width>\d+)x(?P<height>\d+)\.jpg$', thumbnail_key)
|
||||
if not m:
|
||||
continue
|
||||
thumbnails.append({
|
||||
'url': thumbnail_url,
|
||||
'width': int(m.group('width')),
|
||||
'height': int(m.group('height')),
|
||||
})
|
||||
|
||||
return {
|
||||
'id': stream_id,
|
||||
'id': str_or_none(stream.get('_id')) or channel_id,
|
||||
'display_id': channel_name,
|
||||
'title': self._live_title(title),
|
||||
'title': title,
|
||||
'description': description,
|
||||
'thumbnail': thumbnail,
|
||||
'uploader': uploader,
|
||||
'uploader_id': channel_name,
|
||||
'thumbnails': thumbnails,
|
||||
'uploader': channel.get('display_name'),
|
||||
'uploader_id': channel.get('name'),
|
||||
'timestamp': timestamp,
|
||||
'view_count': view_count,
|
||||
'formats': formats,
|
||||
'is_live': stream_type == 'live',
|
||||
'is_live': True,
|
||||
}
|
||||
|
||||
|
||||
|
||||
@ -19,7 +19,7 @@ from ..utils import (
|
||||
|
||||
|
||||
class UstreamIE(InfoExtractor):
|
||||
_VALID_URL = r'https?://(?:www\.)?(?:ustream\.tv|video\.ibm\.com)/(?P<type>recorded|embed|embed/recorded)/(?P<id>\d+)'
|
||||
_VALID_URL = r'https?://(?:www\.)?ustream\.tv/(?P<type>recorded|embed|embed/recorded)/(?P<id>\d+)'
|
||||
IE_NAME = 'ustream'
|
||||
_TESTS = [{
|
||||
'url': 'http://www.ustream.tv/recorded/20274954',
|
||||
@ -67,15 +67,12 @@ class UstreamIE(InfoExtractor):
|
||||
'params': {
|
||||
'skip_download': True, # m3u8 download
|
||||
},
|
||||
}, {
|
||||
'url': 'https://video.ibm.com/embed/recorded/128240221?&autoplay=true&controls=true&volume=100',
|
||||
'only_matching': True,
|
||||
}]
|
||||
|
||||
@staticmethod
|
||||
def _extract_url(webpage):
|
||||
mobj = re.search(
|
||||
r'<iframe[^>]+?src=(["\'])(?P<url>http://(?:www\.)?(?:ustream\.tv|video\.ibm\.com)/embed/.+?)\1', webpage)
|
||||
r'<iframe[^>]+?src=(["\'])(?P<url>http://www\.ustream\.tv/embed/.+?)\1', webpage)
|
||||
if mobj is not None:
|
||||
return mobj.group('url')
|
||||
|
||||
|
||||
@ -3181,94 +3181,54 @@ class YoutubeSearchIE(SearchInfoExtractor, YoutubeSearchBaseInfoExtractor):
|
||||
_MAX_RESULTS = float('inf')
|
||||
IE_NAME = 'youtube:search'
|
||||
_SEARCH_KEY = 'ytsearch'
|
||||
_SEARCH_PARAMS = None
|
||||
_EXTRA_QUERY_ARGS = {}
|
||||
_TESTS = []
|
||||
|
||||
def _entries(self, query, n):
|
||||
data = {
|
||||
'context': {
|
||||
'client': {
|
||||
'clientName': 'WEB',
|
||||
'clientVersion': '2.20201021.03.00',
|
||||
}
|
||||
},
|
||||
'query': query,
|
||||
}
|
||||
if self._SEARCH_PARAMS:
|
||||
data['params'] = self._SEARCH_PARAMS
|
||||
total = 0
|
||||
for page_num in itertools.count(1):
|
||||
search = self._download_json(
|
||||
'https://www.youtube.com/youtubei/v1/search?key=AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8',
|
||||
video_id='query "%s"' % query,
|
||||
note='Downloading page %s' % page_num,
|
||||
errnote='Unable to download API page', fatal=False,
|
||||
data=json.dumps(data).encode('utf8'),
|
||||
headers={'content-type': 'application/json'})
|
||||
if not search:
|
||||
break
|
||||
slr_contents = try_get(
|
||||
search,
|
||||
(lambda x: x['contents']['twoColumnSearchResultsRenderer']['primaryContents']['sectionListRenderer']['contents'],
|
||||
lambda x: x['onResponseReceivedCommands'][0]['appendContinuationItemsAction']['continuationItems']),
|
||||
list)
|
||||
if not slr_contents:
|
||||
break
|
||||
isr_contents = try_get(
|
||||
slr_contents,
|
||||
lambda x: x[0]['itemSectionRenderer']['contents'],
|
||||
list)
|
||||
if not isr_contents:
|
||||
break
|
||||
for content in isr_contents:
|
||||
if not isinstance(content, dict):
|
||||
continue
|
||||
video = content.get('videoRenderer')
|
||||
if not isinstance(video, dict):
|
||||
continue
|
||||
video_id = video.get('videoId')
|
||||
if not video_id:
|
||||
continue
|
||||
title = try_get(video, lambda x: x['title']['runs'][0]['text'], compat_str)
|
||||
description = try_get(video, lambda x: x['descriptionSnippet']['runs'][0]['text'], compat_str)
|
||||
duration = parse_duration(try_get(video, lambda x: x['lengthText']['simpleText'], compat_str))
|
||||
view_count_text = try_get(video, lambda x: x['viewCountText']['simpleText'], compat_str) or ''
|
||||
view_count = int_or_none(self._search_regex(
|
||||
r'^(\d+)', re.sub(r'\s', '', view_count_text),
|
||||
'view count', default=None))
|
||||
uploader = try_get(video, lambda x: x['ownerText']['runs'][0]['text'], compat_str)
|
||||
total += 1
|
||||
yield {
|
||||
'_type': 'url_transparent',
|
||||
'ie_key': YoutubeIE.ie_key(),
|
||||
'id': video_id,
|
||||
'url': video_id,
|
||||
'title': title,
|
||||
'description': description,
|
||||
'duration': duration,
|
||||
'view_count': view_count,
|
||||
'uploader': uploader,
|
||||
}
|
||||
if total == n:
|
||||
return
|
||||
token = try_get(
|
||||
slr_contents,
|
||||
lambda x: x[1]['continuationItemRenderer']['continuationEndpoint']['continuationCommand']['token'],
|
||||
compat_str)
|
||||
if not token:
|
||||
break
|
||||
data['continuation'] = token
|
||||
|
||||
def _get_n_results(self, query, n):
|
||||
"""Get a specified number of results for a query"""
|
||||
return self.playlist_result(self._entries(query, n), query)
|
||||
|
||||
videos = []
|
||||
limit = n
|
||||
|
||||
url_query = {
|
||||
'search_query': query.encode('utf-8'),
|
||||
}
|
||||
url_query.update(self._EXTRA_QUERY_ARGS)
|
||||
result_url = 'https://www.youtube.com/results?' + compat_urllib_parse_urlencode(url_query)
|
||||
|
||||
for pagenum in itertools.count(1):
|
||||
data = self._download_json(
|
||||
result_url, video_id='query "%s"' % query,
|
||||
note='Downloading page %s' % pagenum,
|
||||
errnote='Unable to download API page',
|
||||
query={'spf': 'navigate'})
|
||||
html_content = data[1]['body']['content']
|
||||
|
||||
if 'class="search-message' in html_content:
|
||||
raise ExtractorError(
|
||||
'[youtube] No video results', expected=True)
|
||||
|
||||
new_videos = list(self._process_page(html_content))
|
||||
videos += new_videos
|
||||
if not new_videos or len(videos) > limit:
|
||||
break
|
||||
next_link = self._html_search_regex(
|
||||
r'href="(/results\?[^"]*\bsp=[^"]+)"[^>]*>\s*<span[^>]+class="[^"]*\byt-uix-button-content\b[^"]*"[^>]*>Next',
|
||||
html_content, 'next link', default=None)
|
||||
if next_link is None:
|
||||
break
|
||||
result_url = compat_urlparse.urljoin('https://www.youtube.com/', next_link)
|
||||
|
||||
if len(videos) > n:
|
||||
videos = videos[:n]
|
||||
return self.playlist_result(videos, query)
|
||||
|
||||
|
||||
class YoutubeSearchDateIE(YoutubeSearchIE):
|
||||
IE_NAME = YoutubeSearchIE.IE_NAME + ':date'
|
||||
_SEARCH_KEY = 'ytsearchdate'
|
||||
IE_DESC = 'YouTube.com searches, newest videos first'
|
||||
_SEARCH_PARAMS = 'CAI%3D'
|
||||
_EXTRA_QUERY_ARGS = {'search_sort': 'video_date_uploaded'}
|
||||
|
||||
|
||||
class YoutubeSearchURLIE(YoutubeSearchBaseInfoExtractor):
|
||||
|
||||
@ -4088,12 +4088,12 @@ def js_to_json(code):
|
||||
'\\\n': '',
|
||||
'\\x': '\\u00',
|
||||
}.get(m.group(0), m.group(0)), v[1:-1])
|
||||
else:
|
||||
for regex, base in INTEGER_TABLE:
|
||||
im = re.match(regex, v)
|
||||
if im:
|
||||
i = int(im.group(1), base)
|
||||
return '"%d":' % i if v.endswith(':') else '%d' % i
|
||||
|
||||
for regex, base in INTEGER_TABLE:
|
||||
im = re.match(regex, v)
|
||||
if im:
|
||||
i = int(im.group(1), base)
|
||||
return '"%d":' % i if v.endswith(':') else '%d' % i
|
||||
|
||||
return '"%s"' % v
|
||||
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
from __future__ import unicode_literals
|
||||
|
||||
__version__ = '2020.09.20'
|
||||
__version__ = '2020.09.14'
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user