release 2016.08.31

[ChangeLog] Actualize
[pyvideo] Remove debugging code
2016-08-31 02:39:39 +07:00 · 2016-08-31 02:37:40 +07:00 · 2016-08-31 01:56:58 +07:00 · 2016-08-31 01:56:15 +07:00 · 2016-08-31 00:29:49 +07:00 · 2016-08-30 23:51:18 +07:00
44 changed files with 1190 additions and 602 deletions
--- a/.github/ISSUE_TEMPLATE.md
+++ b/.github/ISSUE_TEMPLATE.md
@@ -6,8 +6,8 @@

 ---

-### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *2016.08.24.1*. If it's not read [this FAQ entry](https://github.com/rg3/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
- [ ] I've **verified** and **I assure** that I'm running youtube-dl **2016.08.24.1**
+### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *2016.08.31*. If it's not read [this FAQ entry](https://github.com/rg3/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
+- [ ] I've **verified** and **I assure** that I'm running youtube-dl **2016.08.31**

 ### Before submitting an *issue* make sure you have:
 - [ ] At least skimmed through [README](https://github.com/rg3/youtube-dl/blob/master/README.md) and **most notably** [FAQ](https://github.com/rg3/youtube-dl#faq) and [BUGS](https://github.com/rg3/youtube-dl#bugs) sections
@@ -35,7 +35,7 @@ $ youtube-dl -v <your command line>
 [debug] User config: []
 [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
 [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
-[debug] youtube-dl version 2016.08.24.1
+[debug] youtube-dl version 2016.08.31
 [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
 [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
 [debug] Proxy map: {}
--- a/1
+++ b/1
@@ -181,3 +181,4 @@ Nehal Patel
 Rob van Bekkum
 Petr Zvoníček
 Pratyush Singh
+Aleksander Nitecki
--- a/45
+++ b/45
@@ -1,3 +1,48 @@
+version 2016.08.31
+
+Extractors
+* [soundcloud] Fix URL regular expression to avoid clashes with sets (#10505)
+* [bandcamp:album] Fix title extraction (#10455)
+* [pyvideo] Fix extraction (#10468)
+ [ctv] Add support for tsn.ca, bnn.ca and thecomedynetwork.ca (#10016)
+* [9c9media] Extract more metadata
+* [9c9media] Fix multiple stacks extraction (#10016)
+* [adultswim] Improve video info extraction (#10492)
+* [vodplatform] Improve embed regular expression
+- [played] Remove extractor (#10470)
+ [tbs] Add extractor for tbs.com and tntdrama.com (#10222)
+ [cartoonnetwork] Add extractor for cartoonnetwork.com (#10110)
+* [adultswim] Rework in terms of turner extractor
+* [cnn] Rework in terms of turner extractor
+* [nba] Rework in terms of turner extractor
+ [turner] Add base extractor for Turner Broadcasting System based sites
+* [bilibili] Fix extraction (#10375)
+* [openload] Fix extraction (#10408)
+
+
+version 2016.08.28
+
+Core
+ Add warning message that ffmpeg doesn't support SOCKS
+* Improve thumbnail sorting
+ Extract formats from #EXT-X-MEDIA tags in _extract_m3u8_formats
+* Fill IV with leading zeros for IVs shorter than 16 octets in hlsnative
+ Add ac-3 to the list of audio codecs in parse_codecs
+
+Extractors
+* [periscope:user] Fix extraction (#10453)
+* [douyutv] Fix extraction (#10153, #10318, #10444)
+ [nhk:vod] Add extractor for www3.nhk.or.jp on demand (#4437, #10424)
+- [trutube] Remove extractor (#10438)
+ [usanetwork] Add extractor for usanetwork.com
+* [crackle] Fix extraction (#10333)
+* [spankbang] Fix description and uploader extraction (#10339)
+* [discoverygo] Detect cable provider restricted videos (#10425)
+ [cbc] Add support for watch.cbc.ca
+* [kickstarter] Silent the warning for og:description (#10415)
+* [mtvservices:embedded] Fix extraction for the new 'edge' player (#10363)
+
+
 version 2016.08.24.1

 Extractors
--- a/README.md
+++ b/README.md
@@ -412,11 +412,19 @@ You can configure youtube-dl by placing any supported command line option to a c

 For example, with the following configuration file youtube-dl will always extract the audio, not copy the mtime, use a proxy and save all videos under `Movies` directory in your home directory:
 ```
-x
--no-mtime
--proxy 127.0.0.1:3128
-o ~/Movies/%(title)s.%(ext)s
 # Lines starting with # are comments
+
+# Always extract audio
+-x
+
+# Do not copy the mtime
+--no-mtime
+
+# Use this proxy
+--proxy 127.0.0.1:3128
+
+# Save all videos under Movies directory in your home directory
+-o ~/Movies/%(title)s.%(ext)s
 ```

 Note that options in configuration file are just the same options aka switches used in regular command line calls thus there **must be no whitespace** after `-` or `--`, e.g. `-o` or `--proxy` but not `- o` or `-- proxy`.
@@ -730,7 +738,7 @@ Videos or video formats streamed via RTMP protocol can only be downloaded when [

 ### I have downloaded a video but how can I play it?

-Once the video is fully downloaded, use any video player, such as [mpv](https://mpv.io/), [vlc](http://www.videolan.org) or [mplayer](http://www.mplayerhq.hu/).
+Once the video is fully downloaded, use any video player, such as [mpv](https://mpv.io/), [vlc](http://www.videolan.org/) or [mplayer](http://www.mplayerhq.hu/).

 ### I extracted a video URL with `-g`, but it does not play on another machine / in my webbrowser.

@@ -812,10 +820,32 @@ Either prepend `http://www.youtube.com/watch?v=` or separate the ID from the opt

 ### How do I pass cookies to youtube-dl?

-Use the `--cookies` option, for example `--cookies /path/to/cookies/file.txt`. Note that the cookies file must be in Mozilla/Netscape format and the first line of the cookies file must be either `# HTTP Cookie File` or `# Netscape HTTP Cookie File`. Make sure you have correct [newline format](https://en.wikipedia.org/wiki/Newline) in the cookies file and convert newlines if necessary to correspond with your OS, namely `CRLF` (`\r\n`) for Windows, `LF` (`\n`) for Linux and `CR` (`\r`) for Mac OS. `HTTP Error 400: Bad Request` when using `--cookies` is a good sign of invalid newline format.
+Use the `--cookies` option, for example `--cookies /path/to/cookies/file.txt`.
+
+In order to extract cookies from browser use any conforming browser extension for exporting cookies. For example, [cookies.txt](https://chrome.google.com/webstore/detail/cookiestxt/njabckikapfpffapmjgojcnbfjonfjfg) (for Chrome) or [Export Cookies](https://addons.mozilla.org/en-US/firefox/addon/export-cookies/) (for Firefox).
+
+Note that the cookies file must be in Mozilla/Netscape format and the first line of the cookies file must be either `# HTTP Cookie File` or `# Netscape HTTP Cookie File`. Make sure you have correct [newline format](https://en.wikipedia.org/wiki/Newline) in the cookies file and convert newlines if necessary to correspond with your OS, namely `CRLF` (`\r\n`) for Windows, `LF` (`\n`) for Linux and `CR` (`\r`) for Mac OS. `HTTP Error 400: Bad Request` when using `--cookies` is a good sign of invalid newline format.

 Passing cookies to youtube-dl is a good way to workaround login when a particular extractor does not implement it explicitly. Another use case is working around [CAPTCHA](https://en.wikipedia.org/wiki/CAPTCHA) some websites require you to solve in particular cases in order to get access (e.g. YouTube, CloudFlare).

+### How do I stream directly to media player?
+
+You will first need to tell youtube-dl to stream media to stdout with `-o -`, and also tell your media player to read from stdin (it must be capable of this for streaming) and then pipe former to latter. For example, streaming to [vlc](http://www.videolan.org/) can be achieved with:
+
+    youtube-dl -o - "http://www.youtube.com/watch?v=BaW_jenozKcj" | vlc -
+
+### How do I download only new videos from a playlist?
+
+Use download-archive feature. With this feature you should initially download the complete playlist with `--download-archive /path/to/download/archive/file.txt` that will record identifiers of all the videos in a special file. Each subsequent run with the same `--download-archive` will download only new videos and skip all videos that have been downloaded before. Note that only successful downloads are recorded in the file.
+
+For example, at first,
+
+    youtube-dl --download-archive archive.txt "https://www.youtube.com/playlist?list=PLwiyx1dc3P2JR9N8gQaQN_BCvlSlap7re"
+
+will download the complete `PLwiyx1dc3P2JR9N8gQaQN_BCvlSlap7re` playlist and create a file `archive.txt`. Each subsequent run will only download new videos if any:
+
+    youtube-dl --download-archive archive.txt "https://www.youtube.com/playlist?list=PLwiyx1dc3P2JR9N8gQaQN_BCvlSlap7re"
+
 ### Can you add support for this anime video site, or site which shows current movies for free?

 As a matter of policy (as well as legality), youtube-dl does not include support for services that specialize in infringing copyright. As a rule of thumb, if you cannot easily find a video that the service is quite obviously allowed to distribute (i.e. that has been uploaded by the creator, the creator's distributor, or is published under a free license), the service is probably unfit for inclusion to youtube-dl.
--- a/docs/supportedsites.md
+++ b/docs/supportedsites.md
@@ -13,6 +13,8 @@
 - **5min**
 - **8tracks**
 - **91porn**
+ - **9c9media**
+ - **9c9media:stack**
 - **9gag**
 - **9now.com.au**
 - **abc.net.au**
@@ -115,8 +117,11 @@
 - **Canvas**
 - **CarambaTV**
 - **CarambaTVPage**
- - **CBC**
- - **CBCPlayer**
+ - **CartoonNetwork**
+ - **cbc.ca**
+ - **cbc.ca:player**
+ - **cbc.ca:watch**
+ - **cbc.ca:watch:video**
 - **CBS**
 - **CBSInteractive**
 - **CBSLocal**
@@ -448,6 +453,7 @@
 - **NextMediaActionNews**: 蘋果日報 - 動新聞
 - **nfb**: National Film Board of Canada
 - **nfl.com**
+ - **NhkVod**
 - **nhl.com**
 - **nhl.com:news**: NHL news
 - **nhl.com:videocenter**
@@ -456,7 +462,6 @@
 - **nick.de**
 - **niconico**: ニコニコ動画
 - **NiconicoPlaylist**
- - **NineCNineMedia**
 - **Nintendo**
 - **njoy**: N-JOY
 - **njoy:embed**
@@ -514,7 +519,6 @@
 - **Pinkbike**
 - **Pladform**
 - **play.fm**
- - **played.to**
 - **PlaysTV**
 - **Playtvak**: Playtvak.cz, iDNES.cz and Lidovky.cz
 - **Playvid**
@@ -672,6 +676,7 @@
 - **Tagesschau**
 - **tagesschau:player**
 - **Tass**
+ - **TBS**
 - **TDSLifeway**
 - **teachertube**: teachertube.com videos
 - **teachertube:user:collection**: teachertube.com user and collection videos
@@ -713,7 +718,6 @@
 - **TrailerAddict** (Currently broken)
 - **Trilulilu**
 - **trollvids**
- - **TruTube**
 - **Tube8**
 - **TubiTv**
 - **tudou**
@@ -758,6 +762,7 @@
 - **uplynk:preplay**
 - **Urort**: NRK P3 Urørt
 - **URPlay**
+ - **USANetwork**
 - **USAToday**
 - **ustream**
 - **ustream:channel**
--- a/youtube_dl/YoutubeDL.py
+++ b/youtube_dl/YoutubeDL.py
@@ -1256,8 +1256,10 @@ class YoutubeDL(object):
                info_dict['thumbnails'] = thumbnails = [{'url': thumbnail}]
        if thumbnails:
            thumbnails.sort(key=lambda t: (
-                t.get('preference'), t.get('width'), t.get('height'),
-                t.get('id'), t.get('url')))
+                t.get('preference') if t.get('preference') is not None else -1,
+                t.get('width') if t.get('width') is not None else -1,
+                t.get('height') if t.get('height') is not None else -1,
+                t.get('id') if t.get('id') is not None else '', t.get('url')))
            for i, t in enumerate(thumbnails):
                t['url'] = sanitize_url(t['url'])
                if t.get('width') and t.get('height'):
--- a/youtube_dl/downloader/external.py
+++ b/youtube_dl/downloader/external.py
@@ -220,6 +220,12 @@ class FFmpegFD(ExternalFD):
        if proxy:
            if not re.match(r'^[\da-zA-Z]+://', proxy):
                proxy = 'http://%s' % proxy
+
+            if proxy.startswith('socks'):
+                self.report_warning(
+                    '%s does not support SOCKS proxies. Downloading is likely to fail. '
+                    'Consider adding --hls-prefer-native to your command.' % self.get_basename())
+
            # Since December 2015 ffmpeg supports -http_proxy option (see
            # http://git.videolan.org/?p=ffmpeg.git;a=commit;h=b4eb1f29ebddd60c41a2eb39f5af701e38e0d3fd)
            # We could switch to the following code if we are able to detect version properly
--- a/youtube_dl/downloader/hls.py
+++ b/youtube_dl/downloader/hls.py
@@ -83,7 +83,10 @@ class HlsFD(FragmentFD):

        self._prepare_and_start_frag_download(ctx)

+        extra_query = None
        extra_param_to_segment_url = info_dict.get('extra_param_to_segment_url')
+        if extra_param_to_segment_url:
+            extra_query = compat_urlparse.parse_qs(extra_param_to_segment_url)
        i = 0
        media_sequence = 0
        decrypt_info = {'METHOD': 'NONE'}
@@ -97,8 +100,8 @@ class HlsFD(FragmentFD):
                        if re.match(r'^https?://', line)
                        else compat_urlparse.urljoin(man_url, line))
                    frag_filename = '%s-Frag%d' % (ctx['tmpfilename'], i)
-                    if extra_param_to_segment_url:
-                        frag_url = update_url_query(frag_url, extra_param_to_segment_url)
+                    if extra_query:
+                        frag_url = update_url_query(frag_url, extra_query)
                    success = ctx['dl'].download(frag_filename, {'url': frag_url})
                    if not success:
                        return False
@@ -120,12 +123,12 @@ class HlsFD(FragmentFD):
                    decrypt_info = parse_m3u8_attributes(line[11:])
                    if decrypt_info['METHOD'] == 'AES-128':
                        if 'IV' in decrypt_info:
-                            decrypt_info['IV'] = binascii.unhexlify(decrypt_info['IV'][2:])
+                            decrypt_info['IV'] = binascii.unhexlify(decrypt_info['IV'][2:].zfill(32))
                        if not re.match(r'^https?://', decrypt_info['URI']):
                            decrypt_info['URI'] = compat_urlparse.urljoin(
                                man_url, decrypt_info['URI'])
-                        if extra_param_to_segment_url:
-                            decrypt_info['URI'] = update_url_query(decrypt_info['URI'], extra_param_to_segment_url)
+                        if extra_query:
+                            decrypt_info['URI'] = update_url_query(decrypt_info['URI'], extra_query)
                        decrypt_info['KEY'] = self.ydl.urlopen(decrypt_info['URI']).read()
                elif line.startswith('#EXT-X-MEDIA-SEQUENCE'):
                    media_sequence = int(line[22:])
--- a/youtube_dl/extractor/adultswim.py
+++ b/youtube_dl/extractor/adultswim.py
@@ -3,16 +3,14 @@ from __future__ import unicode_literals

 import re

-from .common import InfoExtractor
+from .turner import TurnerBaseIE
 from ..utils import (
-    determine_ext,
    ExtractorError,
-    float_or_none,
-    xpath_text,
+    int_or_none,
 )


-class AdultSwimIE(InfoExtractor):
+class AdultSwimIE(TurnerBaseIE):
    _VALID_URL = r'https?://(?:www\.)?adultswim\.com/videos/(?P<is_playlist>playlists/)?(?P<show_path>[^/]+)/(?P<episode_path>[^/?#]+)/?'

    _TESTS = [{
@@ -96,7 +94,8 @@ class AdultSwimIE(InfoExtractor):
        'params': {
            # m3u8 download
            'skip_download': True,
-        }
+        },
+        'expected_warnings': ['Unable to download f4m manifest'],
    }]

    @staticmethod
@@ -148,7 +147,10 @@ class AdultSwimIE(InfoExtractor):
                if bootstrapped_data.get('slugged_video', {}).get('slug') == episode_path:
                    video_info = bootstrapped_data['slugged_video']
            if not video_info:
-                video_info = bootstrapped_data.get('heroMetadata', {}).get('trailer').get('video')
+                video_info = bootstrapped_data.get(
+                    'heroMetadata', {}).get('trailer', {}).get('video')
+            if not video_info:
+                video_info = bootstrapped_data.get('onlineOriginals', [None])[0]
            if not video_info:
                raise ExtractorError('Unable to find video info')

@@ -162,70 +164,38 @@ class AdultSwimIE(InfoExtractor):
            elif video_info.get('videoPlaybackID'):
                segment_ids = [video_info['videoPlaybackID']]
            else:
-                raise ExtractorError(
-                    'This video is only available via cable service provider subscription that'
-                    ' is not currently supported. You may want to use --cookies.'
-                    if video_info.get('auth') is True else 'Unable to find stream or clips',
-                    expected=True)
+                if video_info.get('auth') is True:
+                    raise ExtractorError(
+                        'This video is only available via cable service provider subscription that'
+                        ' is not currently supported. You may want to use --cookies.', expected=True)
+                else:
+                    raise ExtractorError('Unable to find stream or clips')

        episode_id = video_info['id']
        episode_title = video_info['title']
-        episode_description = video_info['description']
-        episode_duration = video_info.get('duration')
+        episode_description = video_info.get('description')
+        episode_duration = int_or_none(video_info.get('duration'))
+        view_count = int_or_none(video_info.get('views'))

        entries = []
        for part_num, segment_id in enumerate(segment_ids):
-            segment_url = 'http://www.adultswim.com/videos/api/v0/assets?id=%s&platform=desktop' % segment_id
-
+            segement_info = self._extract_cvp_info(
+                'http://www.adultswim.com/videos/api/v0/assets?id=%s&platform=desktop' % segment_id,
+                segment_id, {
+                    'secure': {
+                        'media_src': 'http://androidhls-secure.cdn.turner.com/adultswim/big',
+                        'tokenizer_src': 'http://www.adultswim.com/astv/mvpd/processors/services/token_ipadAdobe.do',
+                    },
+                })
            segment_title = '%s - %s' % (show_title, episode_title)
            if len(segment_ids) > 1:
                segment_title += ' Part %d' % (part_num + 1)
-
-            idoc = self._download_xml(
-                segment_url, segment_title,
-                'Downloading segment information', 'Unable to download segment information')
-
-            segment_duration = float_or_none(
-                xpath_text(idoc, './/trt', 'segment duration').strip())
-
-            formats = []
-            file_els = idoc.findall('.//files/file') or idoc.findall('./files/file')
-
-            unique_urls = []
-            unique_file_els = []
-            for file_el in file_els:
-                media_url = file_el.text
-                if not media_url or determine_ext(media_url) == 'f4m':
-                    continue
-                if file_el.text not in unique_urls:
-                    unique_urls.append(file_el.text)
-                    unique_file_els.append(file_el)
-
-            for file_el in unique_file_els:
-                bitrate = file_el.attrib.get('bitrate')
-                ftype = file_el.attrib.get('type')
-                media_url = file_el.text
-                if determine_ext(media_url) == 'm3u8':
-                    formats.extend(self._extract_m3u8_formats(
-                        media_url, segment_title, 'mp4', preference=0,
-                        m3u8_id='hls', fatal=False))
-                else:
-                    formats.append({
-                        'format_id': '%s_%s' % (bitrate, ftype),
-                        'url': file_el.text.strip(),
-                        # The bitrate may not be a number (for example: 'iphone')
-                        'tbr': int(bitrate) if bitrate.isdigit() else None,
-                    })
-
-            self._sort_formats(formats)
-
-            entries.append({
+            segement_info.update({
                'id': segment_id,
                'title': segment_title,
-                'formats': formats,
-                'duration': segment_duration,
-                'description': episode_description
+                'description': episode_description,
            })
+            entries.append(segement_info)

        return {
            '_type': 'playlist',
@@ -234,5 +204,6 @@ class AdultSwimIE(InfoExtractor):
            'entries': entries,
            'title': '%s - %s' % (show_title, episode_title),
            'description': episode_description,
-            'duration': episode_duration
+            'duration': episode_duration,
+            'view_count': view_count,
        }
--- a/youtube_dl/extractor/bandcamp.py
+++ b/youtube_dl/extractor/bandcamp.py
@@ -162,6 +162,15 @@ class BandcampAlbumIE(InfoExtractor):
            'uploader_id': 'dotscale',
        },
        'playlist_mincount': 7,
+    }, {
+        # with escaped quote in title
+        'url': 'https://jstrecords.bandcamp.com/album/entropy-ep',
+        'info_dict': {
+            'title': '"Entropy" EP',
+            'uploader_id': 'jstrecords',
+            'id': 'entropy-ep',
+        },
+        'playlist_mincount': 3,
    }]

    def _real_extract(self, url):
@@ -176,8 +185,11 @@ class BandcampAlbumIE(InfoExtractor):
        entries = [
            self.url_result(compat_urlparse.urljoin(url, t_path), ie=BandcampIE.ie_key())
            for t_path in tracks_paths]
-        title = self._search_regex(
-            r'album_title\s*:\s*"(.*?)"', webpage, 'title', fatal=False)
+        title = self._html_search_regex(
+            r'album_title\s*:\s*"((?:\\.|[^"\\])+?)"',
+            webpage, 'title', fatal=False)
+        if title:
+            title = title.replace(r'\"', '"')
        return {
            '_type': 'playlist',
            'uploader_id': uploader_id,
--- a/youtube_dl/extractor/bet.py
+++ b/youtube_dl/extractor/bet.py
@@ -2,7 +2,6 @@ from __future__ import unicode_literals

 from .mtv import MTVServicesInfoExtractor
 from ..utils import unified_strdate
-from ..compat import compat_urllib_parse_urlencode


 class BetIE(MTVServicesInfoExtractor):
@@ -53,9 +52,9 @@ class BetIE(MTVServicesInfoExtractor):
    _FEED_URL = "http://feeds.mtvnservices.com/od/feed/bet-mrss-player"

    def _get_feed_query(self, uri):
-        return compat_urllib_parse_urlencode({
+        return {
            'uuid': uri,
-        })
+        }

    def _extract_mgid(self, webpage):
        return self._search_regex(r'data-uri="([^"]+)', webpage, 'mgid')
--- a/youtube_dl/extractor/bilibili.py
+++ b/youtube_dl/extractor/bilibili.py
@@ -1,22 +1,15 @@
 # coding: utf-8
 from __future__ import unicode_literals

-import calendar
-import datetime
+import hashlib
 import re

 from .common import InfoExtractor
-from ..compat import (
-    compat_etree_fromstring,
-    compat_str,
-    compat_parse_qs,
-    compat_xml_parse_error,
-)
+from ..compat import compat_parse_qs
 from ..utils import (
-    ExtractorError,
    int_or_none,
    float_or_none,
-    xpath_text,
+    unified_timestamp,
 )


@@ -27,7 +20,7 @@ class BiliBiliIE(InfoExtractor):
        'url': 'http://www.bilibili.tv/video/av1074402/',
        'md5': '9fa226fe2b8a9a4d5a69b4c6a183417e',
        'info_dict': {
-            'id': '1554319',
+            'id': '1074402',
            'ext': 'mp4',
            'title': '【金坷垃】金泡沫',
            'description': 'md5:ce18c2a2d2193f0df2917d270f2e5923',
@@ -41,24 +34,28 @@ class BiliBiliIE(InfoExtractor):
    }, {
        'url': 'http://www.bilibili.com/video/av1041170/',
        'info_dict': {
-            'id': '1507019',
+            'id': '1041170',
            'ext': 'mp4',
            'title': '【BD1080P】刀语【诸神&异域】',
            'description': '这是个神奇的故事~每个人不留弹幕不给走哦~切利哦！~',
+            'duration': 3382.259,
            'timestamp': 1396530060,
            'upload_date': '20140403',
+            'thumbnail': 're:^https?://.+\.jpg',
            'uploader': '枫叶逝去',
            'uploader_id': '520116',
        },
    }, {
        'url': 'http://www.bilibili.com/video/av4808130/',
        'info_dict': {
-            'id': '7802182',
+            'id': '4808130',
            'ext': 'mp4',
            'title': '【长篇】哆啦A梦443【钉铛】',
            'description': '(2016.05.27)来组合客人的脸吧&amp;amp;寻母六千里锭 抱歉，又轮到周日上班现在才到家 封面www.pixiv.net/member_illust.php?mode=medium&amp;amp;illust_id=56912929',
+            'duration': 1493.995,
            'timestamp': 1464564180,
            'upload_date': '20160529',
+            'thumbnail': 're:^https?://.+\.jpg',
            'uploader': '喜欢拉面',
            'uploader_id': '151066',
        },
@@ -66,12 +63,14 @@ class BiliBiliIE(InfoExtractor):
        # Missing upload time
        'url': 'http://www.bilibili.com/video/av1867637/',
        'info_dict': {
-            'id': '2880301',
+            'id': '1867637',
            'ext': 'mp4',
            'title': '【HDTV】【喜剧】岳父岳母真难当 （2014）【法国票房冠军】',
            'description': '一个信奉天主教的法国旧式传统资产阶级家庭中有四个女儿。三个女儿却分别找了阿拉伯、犹太、中国丈夫，老夫老妻唯独期盼剩下未嫁的小女儿能找一个信奉天主教的法国白人，结果没想到小女儿找了一位非裔黑人……【这次应该不会跳帧了】',
+            'duration': 5760.0,
            'uploader': '黑夜为猫',
            'uploader_id': '610729',
+            'thumbnail': 're:^https?://.+\.jpg',
        },
        'params': {
            # Just to test metadata extraction
@@ -80,86 +79,61 @@ class BiliBiliIE(InfoExtractor):
        'expected_warnings': ['upload time'],
    }]

-    # BiliBili blocks keys from time to time. The current key is extracted from
-    # the Android client
-    # TODO: find the sign algorithm used in the flash player
-    _APP_KEY = '86385cdc024c0f6c'
+    _APP_KEY = '6f90a59ac58a4123'
+    _BILIBILI_KEY = '0bfd84cc3940035173f35e6777508326'

    def _real_extract(self, url):
-        mobj = re.match(self._VALID_URL, url)
-        video_id = mobj.group('id')
+        video_id = self._match_id(url)

        webpage = self._download_webpage(url, video_id)

-        params = compat_parse_qs(self._search_regex(
+        cid = compat_parse_qs(self._search_regex(
            [r'EmbedPlayer\([^)]+,\s*"([^"]+)"\)',
             r'<iframe[^>]+src="https://secure\.bilibili\.com/secure,([^"]+)"'],
-            webpage, 'player parameters'))
-        cid = params['cid'][0]
+            webpage, 'player parameters'))['cid'][0]

-        info_xml_str = self._download_webpage(
-            'http://interface.bilibili.com/v_cdn_play',
-            cid, query={'appkey': self._APP_KEY, 'cid': cid},
-            note='Downloading video info page')
+        payload = 'appkey=%s&cid=%s&otype=json&quality=2&type=mp4' % (self._APP_KEY, cid)
+        sign = hashlib.md5((payload + self._BILIBILI_KEY).encode('utf-8')).hexdigest()

-        err_msg = None
-        durls = None
-        info_xml = None
-        try:
-            info_xml = compat_etree_fromstring(info_xml_str.encode('utf-8'))
-        except compat_xml_parse_error:
-            info_json = self._parse_json(info_xml_str, video_id, fatal=False)
-            err_msg = (info_json or {}).get('error_text')
-        else:
-            err_msg = xpath_text(info_xml, './message')
-
-        if info_xml is not None:
-            durls = info_xml.findall('./durl')
-        if not durls:
-            if err_msg:
-                raise ExtractorError('%s said: %s' % (self.IE_NAME, err_msg), expected=True)
-            else:
-                raise ExtractorError('No videos found!')
+        video_info = self._download_json(
+            'http://interface.bilibili.com/playurl?%s&sign=%s' % (payload, sign),
+            video_id, note='Downloading video info page')

        entries = []

-        for durl in durls:
-            size = xpath_text(durl, ['./filesize', './size'])
+        for idx, durl in enumerate(video_info['durl']):
            formats = [{
-                'url': durl.find('./url').text,
-                'filesize': int_or_none(size),
+                'url': durl['url'],
+                'filesize': int_or_none(durl['size']),
            }]
-            for backup_url in durl.findall('./backup_url/url'):
+            for backup_url in durl['backup_url']:
                formats.append({
-                    'url': backup_url.text,
+                    'url': backup_url,
                    # backup URLs have lower priorities
-                    'preference': -2 if 'hd.mp4' in backup_url.text else -3,
+                    'preference': -2 if 'hd.mp4' in backup_url else -3,
                })

            self._sort_formats(formats)

            entries.append({
-                'id': '%s_part%s' % (cid, xpath_text(durl, './order')),
-                'duration': int_or_none(xpath_text(durl, './length'), 1000),
+                'id': '%s_part%s' % (video_id, idx),
+                'duration': float_or_none(durl.get('length'), 1000),
                'formats': formats,
            })

        title = self._html_search_regex('<h1[^>]+title="([^"]+)">', webpage, 'title')
        description = self._html_search_meta('description', webpage)
-        datetime_str = self._html_search_regex(
-            r'<time[^>]+datetime="([^"]+)"', webpage, 'upload time', fatal=False)
-        timestamp = None
-        if datetime_str:
-            timestamp = calendar.timegm(datetime.datetime.strptime(datetime_str, '%Y-%m-%dT%H:%M').timetuple())
+        timestamp = unified_timestamp(self._html_search_regex(
+            r'<time[^>]+datetime="([^"]+)"', webpage, 'upload time', fatal=False))

        # TODO 'view_count' requires deobfuscating Javascript
        info = {
-            'id': compat_str(cid),
+            'id': video_id,
            'title': title,
            'description': description,
            'timestamp': timestamp,
            'thumbnail': self._html_search_meta('thumbnailUrl', webpage),
-            'duration': float_or_none(xpath_text(info_xml, './timelength'), scale=1000),
+            'duration': float_or_none(video_info.get('timelength'), scale=1000),
        }

        uploader_mobj = re.search(
--- a/youtube_dl/extractor/cartoonnetwork.py
+++ b/youtube_dl/extractor/cartoonnetwork.py
@@ -0,0 +1,36 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+import re
+
+from .turner import TurnerBaseIE
+
+
+class CartoonNetworkIE(TurnerBaseIE):
+    _VALID_URL = r'https?://(?:www\.)?cartoonnetwork\.com/video/(?:[^/]+/)+(?P<id>[^/?#]+)-(?:clip|episode)\.html'
+    _TEST = {
+        'url': 'http://www.cartoonnetwork.com/video/teen-titans-go/starfire-the-cat-lady-clip.html',
+        'info_dict': {
+            'id': '8a250ab04ed07e6c014ef3f1e2f9016c',
+            'ext': 'mp4',
+            'title': 'Starfire the Cat Lady',
+            'description': 'Robin decides to become a cat so that Starfire will finally love him.',
+        },
+        'params': {
+            # m3u8 download
+            'skip_download': True,
+        },
+    }
+
+    def _real_extract(self, url):
+        display_id = self._match_id(url)
+        webpage = self._download_webpage(url, display_id)
+        id_type, video_id = re.search(r"_cnglobal\.cvp(Video|Title)Id\s*=\s*'([^']+)';", webpage).groups()
+        query = ('id' if id_type == 'Video' else 'titleId') + '=' + video_id
+        return self._extract_cvp_info(
+            'http://www.cartoonnetwork.com/video-seo-svc/episodeservices/getCvpPlaylist?networkName=CN2&' + query, video_id, {
+                'secure': {
+                    'media_src': 'http://apple-secure.cdn.turner.com/toon/big',
+                    'tokenizer_src': 'http://www.cartoonnetwork.com/cntv/mvpd/processors/services/token_ipadAdobe.do',
+                },
+            })
--- a/youtube_dl/extractor/cbc.py
+++ b/youtube_dl/extractor/cbc.py
@@ -9,10 +9,19 @@ from ..utils import (
    js_to_json,
    smuggle_url,
    try_get,
+    xpath_text,
+    xpath_element,
+    xpath_with_ns,
+    find_xpath_attr,
+    parse_iso8601,
+    parse_age_limit,
+    int_or_none,
+    ExtractorError,
 )


 class CBCIE(InfoExtractor):
+    IE_NAME = 'cbc.ca'
    _VALID_URL = r'https?://(?:www\.)?cbc\.ca/(?!player/)(?:[^/]+/)+(?P<id>[^/?#]+)'
    _TESTS = [{
        # with mediaId
@@ -114,6 +123,7 @@ class CBCIE(InfoExtractor):


 class CBCPlayerIE(InfoExtractor):
+    IE_NAME = 'cbc.ca:player'
    _VALID_URL = r'(?:cbcplayer:|https?://(?:www\.)?cbc\.ca/(?:player/play/|i/caffeine/syndicate/\?mediaId=))(?P<id>\d+)'
    _TESTS = [{
        'url': 'http://www.cbc.ca/player/play/2683190193',
@@ -167,3 +177,165 @@ class CBCPlayerIE(InfoExtractor):
                }),
            'id': video_id,
        }
+
+
+class CBCWatchBaseIE(InfoExtractor):
+    _device_id = None
+    _device_token = None
+    _API_BASE_URL = 'https://api-cbc.cloud.clearleap.com/cloffice/client/'
+    _NS_MAP = {
+        'media': 'http://search.yahoo.com/mrss/',
+        'clearleap': 'http://www.clearleap.com/namespace/clearleap/1.0/',
+    }
+
+    def _call_api(self, path, video_id):
+        url = path if path.startswith('http') else self._API_BASE_URL + path
+        result = self._download_xml(url, video_id, headers={
+            'X-Clearleap-DeviceId': self._device_id,
+            'X-Clearleap-DeviceToken': self._device_token,
+        })
+        error_message = xpath_text(result, 'userMessage') or xpath_text(result, 'systemMessage')
+        if error_message:
+            raise ExtractorError('%s said: %s' % (self.IE_NAME, error_message))
+        return result
+
+    def _real_initialize(self):
+        if not self._device_id or not self._device_token:
+            device = self._downloader.cache.load('cbcwatch', 'device') or {}
+            self._device_id, self._device_token = device.get('id'), device.get('token')
+            if not self._device_id or not self._device_token:
+                result = self._download_xml(
+                    self._API_BASE_URL + 'device/register',
+                    None, data=b'<device><type>web</type></device>')
+                self._device_id = xpath_text(result, 'deviceId', fatal=True)
+                self._device_token = xpath_text(result, 'deviceToken', fatal=True)
+                self._downloader.cache.store(
+                    'cbcwatch', 'device', {
+                        'id': self._device_id,
+                        'token': self._device_token,
+                    })
+
+    def _parse_rss_feed(self, rss):
+        channel = xpath_element(rss, 'channel', fatal=True)
+
+        def _add_ns(path):
+            return xpath_with_ns(path, self._NS_MAP)
+
+        entries = []
+        for item in channel.findall('item'):
+            guid = xpath_text(item, 'guid', fatal=True)
+            title = xpath_text(item, 'title', fatal=True)
+
+            media_group = xpath_element(item, _add_ns('media:group'), fatal=True)
+            content = xpath_element(media_group, _add_ns('media:content'), fatal=True)
+            content_url = content.attrib['url']
+
+            thumbnails = []
+            for thumbnail in media_group.findall(_add_ns('media:thumbnail')):
+                thumbnail_url = thumbnail.get('url')
+                if not thumbnail_url:
+                    continue
+                thumbnails.append({
+                    'id': thumbnail.get('profile'),
+                    'url': thumbnail_url,
+                    'width': int_or_none(thumbnail.get('width')),
+                    'height': int_or_none(thumbnail.get('height')),
+                })
+
+            timestamp = None
+            release_date = find_xpath_attr(
+                item, _add_ns('media:credit'), 'role', 'releaseDate')
+            if release_date is not None:
+                timestamp = parse_iso8601(release_date.text)
+
+            entries.append({
+                '_type': 'url_transparent',
+                'url': content_url,
+                'id': guid,
+                'title': title,
+                'description': xpath_text(item, 'description'),
+                'timestamp': timestamp,
+                'duration': int_or_none(content.get('duration')),
+                'age_limit': parse_age_limit(xpath_text(item, _add_ns('media:rating'))),
+                'episode': xpath_text(item, _add_ns('clearleap:episode')),
+                'episode_number': int_or_none(xpath_text(item, _add_ns('clearleap:episodeInSeason'))),
+                'series': xpath_text(item, _add_ns('clearleap:series')),
+                'season_number': int_or_none(xpath_text(item, _add_ns('clearleap:season'))),
+                'thumbnails': thumbnails,
+                'ie_key': 'CBCWatchVideo',
+            })
+
+        return self.playlist_result(
+            entries, xpath_text(channel, 'guid'),
+            xpath_text(channel, 'title'),
+            xpath_text(channel, 'description'))
+
+
+class CBCWatchVideoIE(CBCWatchBaseIE):
+    IE_NAME = 'cbc.ca:watch:video'
+    _VALID_URL = r'https?://api-cbc\.cloud\.clearleap\.com/cloffice/client/web/play/?\?.*?\bcontentId=(?P<id>[\da-f]{8}-[\da-f]{4}-[\da-f]{4}-[\da-f]{4}-[\da-f]{12})'
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        result = self._call_api(url, video_id)
+
+        m3u8_url = xpath_text(result, 'url', fatal=True)
+        formats = self._extract_m3u8_formats(re.sub(r'/([^/]+)/[^/?]+\.m3u8', r'/\1/\1.m3u8', m3u8_url), video_id, 'mp4', fatal=False)
+        if len(formats) < 2:
+            formats = self._extract_m3u8_formats(m3u8_url, video_id, 'mp4')
+        # Despite metadata in m3u8 all video+audio formats are
+        # actually video-only (no audio)
+        for f in formats:
+            if f.get('acodec') != 'none' and f.get('vcodec') != 'none':
+                f['acodec'] = 'none'
+        self._sort_formats(formats)
+
+        info = {
+            'id': video_id,
+            'title': video_id,
+            'formats': formats,
+        }
+
+        rss = xpath_element(result, 'rss')
+        if rss:
+            info.update(self._parse_rss_feed(rss)['entries'][0])
+            del info['url']
+            del info['_type']
+            del info['ie_key']
+        return info
+
+
+class CBCWatchIE(CBCWatchBaseIE):
+    IE_NAME = 'cbc.ca:watch'
+    _VALID_URL = r'https?://watch\.cbc\.ca/(?:[^/]+/)+(?P<id>[0-9a-f-]+)'
+    _TESTS = [{
+        'url': 'http://watch.cbc.ca/doc-zone/season-6/customer-disservice/38e815a-009e3ab12e4',
+        'info_dict': {
+            'id': '38e815a-009e3ab12e4',
+            'ext': 'mp4',
+            'title': 'Customer (Dis)Service',
+            'description': 'md5:8bdd6913a0fe03d4b2a17ebe169c7c87',
+            'upload_date': '20160219',
+            'timestamp': 1455840000,
+        },
+        'params': {
+            # m3u8 download
+            'skip_download': True,
+            'format': 'bestvideo',
+        },
+        'skip': 'Geo-restricted to Canada',
+    }, {
+        'url': 'http://watch.cbc.ca/arthur/all/1ed4b385-cd84-49cf-95f0-80f004680057',
+        'info_dict': {
+            'id': '1ed4b385-cd84-49cf-95f0-80f004680057',
+            'title': 'Arthur',
+            'description': 'Arthur, the sweetest 8-year-old aardvark, and his pals solve all kinds of problems with humour, kindness and teamwork.',
+        },
+        'playlist_mincount': 30,
+        'skip': 'Geo-restricted to Canada',
+    }]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        rss = self._call_api('web/browse/' + video_id, video_id)
+        return self._parse_rss_feed(rss)
--- a/youtube_dl/extractor/cnn.py
+++ b/youtube_dl/extractor/cnn.py
@@ -3,14 +3,11 @@ from __future__ import unicode_literals
 import re

 from .common import InfoExtractor
-from ..utils import (
-    int_or_none,
-    parse_duration,
-    url_basename,
-)
+from .turner import TurnerBaseIE
+from ..utils import url_basename


-class CNNIE(InfoExtractor):
+class CNNIE(TurnerBaseIE):
    _VALID_URL = r'''(?x)https?://(?:(?P<sub_domain>edition|www|money)\.)?cnn\.com/(?:video/(?:data/.+?|\?)/)?videos?/
        (?P<path>.+?/(?P<title>[^/]+?)(?:\.(?:[a-z\-]+)|(?=&)))'''

@@ -25,6 +22,7 @@ class CNNIE(InfoExtractor):
            'duration': 135,
            'upload_date': '20130609',
        },
+        'expected_warnings': ['Failed to download m3u8 information'],
    }, {
        'url': 'http://edition.cnn.com/video/?/video/us/2013/08/21/sot-student-gives-epic-speech.georgia-institute-of-technology&utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+rss%2Fcnn_topstories+%28RSS%3A+Top+Stories%29',
        'md5': 'b5cc60c60a3477d185af8f19a2a26f4e',
@@ -34,7 +32,8 @@ class CNNIE(InfoExtractor):
            'title': "Student's epic speech stuns new freshmen",
            'description': "A Georgia Tech student welcomes the incoming freshmen with an epic speech backed by music from \"2001: A Space Odyssey.\"",
            'upload_date': '20130821',
-        }
+        },
+        'expected_warnings': ['Failed to download m3u8 information'],
    }, {
        'url': 'http://www.cnn.com/video/data/2.0/video/living/2014/12/22/growing-america-nashville-salemtown-board-episode-1.hln.html',
        'md5': 'f14d02ebd264df951feb2400e2c25a1b',
@@ -44,7 +43,8 @@ class CNNIE(InfoExtractor):
            'title': 'Nashville Ep. 1: Hand crafted skateboards',
            'description': 'md5:e7223a503315c9f150acac52e76de086',
            'upload_date': '20141222',
-        }
+        },
+        'expected_warnings': ['Failed to download m3u8 information'],
    }, {
        'url': 'http://money.cnn.com/video/news/2016/08/19/netflix-stunning-stats.cnnmoney/index.html',
        'md5': '52a515dc1b0f001cd82e4ceda32be9d1',
@@ -54,7 +54,11 @@ class CNNIE(InfoExtractor):
            'title': '5 stunning stats about Netflix',
            'description': 'Did you know that Netflix has more than 80 million members? Here are five facts about the online video distributor that you probably didn\'t know.',
            'upload_date': '20160819',
-        }
+        },
+        'params': {
+            # m3u8 download
+            'skip_download': True,
+        },
    }, {
        'url': 'http://cnn.com/video/?/video/politics/2015/03/27/pkg-arizona-senator-church-attendance-mandatory.ktvk',
        'only_matching': True,
@@ -79,72 +83,21 @@ class CNNIE(InfoExtractor):
        },
    }

+    def _extract_timestamp(self, video_data):
+        # TODO: fix timestamp extraction
+        return None
+
    def _real_extract(self, url):
        sub_domain, path, page_title = re.match(self._VALID_URL, url).groups()
        if sub_domain not in ('money', 'edition'):
            sub_domain = 'edition'
        config = self._CONFIG[sub_domain]
-        info_url = config['data_src'] % path
-        info = self._download_xml(info_url, page_title)
-
-        formats = []
-        rex = re.compile(r'''(?x)
-            (?P<width>[0-9]+)x(?P<height>[0-9]+)
-            (?:_(?P<bitrate>[0-9]+)k)?
-        ''')
-        for f in info.findall('files/file'):
-            video_url = config['media_src'] + f.text.strip()
-            fdct = {
-                'format_id': f.attrib['bitrate'],
-                'url': video_url,
-            }
-
-            mf = rex.match(f.attrib['bitrate'])
-            if mf:
-                fdct['width'] = int(mf.group('width'))
-                fdct['height'] = int(mf.group('height'))
-                fdct['tbr'] = int_or_none(mf.group('bitrate'))
-            else:
-                mf = rex.search(f.text)
-                if mf:
-                    fdct['width'] = int(mf.group('width'))
-                    fdct['height'] = int(mf.group('height'))
-                    fdct['tbr'] = int_or_none(mf.group('bitrate'))
-                else:
-                    mi = re.match(r'ios_(audio|[0-9]+)$', f.attrib['bitrate'])
-                    if mi:
-                        if mi.group(1) == 'audio':
-                            fdct['vcodec'] = 'none'
-                            fdct['ext'] = 'm4a'
-                        else:
-                            fdct['tbr'] = int(mi.group(1))
-
-            formats.append(fdct)
-
-        self._sort_formats(formats)
-
-        thumbnails = [{
-            'height': int(t.attrib['height']),
-            'width': int(t.attrib['width']),
-            'url': t.text,
-        } for t in info.findall('images/image')]
-
-        metas_el = info.find('metas')
-        upload_date = (
-            metas_el.attrib.get('version') if metas_el is not None else None)
-
-        duration_el = info.find('length')
-        duration = parse_duration(duration_el.text)
-
-        return {
-            'id': info.attrib['id'],
-            'title': info.find('headline').text,
-            'formats': formats,
-            'thumbnails': thumbnails,
-            'description': info.find('description').text,
-            'duration': duration,
-            'upload_date': upload_date,
-        }
+        return self._extract_cvp_info(
+            config['data_src'] % path, page_title, {
+                'default': {
+                    'media_src': config['media_src'],
+                }
+            })


 class CNNBlogsIE(InfoExtractor):
@@ -159,6 +112,7 @@ class CNNBlogsIE(InfoExtractor):
            'description': 'Glenn Greenwald responds to comments made this week on Capitol Hill that journalists could be criminal accessories.',
            'upload_date': '20140209',
        },
+        'expected_warnings': ['Failed to download m3u8 information'],
        'add_ie': ['CNN'],
    }

@@ -181,9 +135,10 @@ class CNNArticleIE(InfoExtractor):
            'id': 'bestoftv/2014/12/21/ip-north-korea-obama.cnn',
            'ext': 'mp4',
            'title': 'Obama: Cyberattack not an act of war',
-            'description': 'md5:51ce6750450603795cad0cdfbd7d05c5',
+            'description': 'md5:0a802a40d2376f60e6b04c8d5bcebc4b',
            'upload_date': '20141221',
        },
+        'expected_warnings': ['Failed to download m3u8 information'],
        'add_ie': ['CNN'],
    }

--- a/youtube_dl/extractor/common.py
+++ b/youtube_dl/extractor/common.py
@@ -1202,30 +1202,45 @@ class InfoExtractor(object):
                'preference': preference,
            }]
        last_info = None
-        last_media = None
        for line in m3u8_doc.splitlines():
            if line.startswith('#EXT-X-STREAM-INF:'):
                last_info = parse_m3u8_attributes(line)
            elif line.startswith('#EXT-X-MEDIA:'):
-                last_media = parse_m3u8_attributes(line)
+                media = parse_m3u8_attributes(line)
+                media_type = media.get('TYPE')
+                if media_type in ('VIDEO', 'AUDIO'):
+                    media_url = media.get('URI')
+                    if media_url:
+                        format_id = []
+                        for v in (media.get('GROUP-ID'), media.get('NAME')):
+                            if v:
+                                format_id.append(v)
+                        formats.append({
+                            'format_id': '-'.join(format_id),
+                            'url': format_url(media_url),
+                            'language': media.get('LANGUAGE'),
+                            'vcodec': 'none' if media_type == 'AUDIO' else None,
+                            'ext': ext,
+                            'protocol': entry_protocol,
+                            'preference': preference,
+                        })
            elif line.startswith('#') or not line.strip():
                continue
            else:
                if last_info is None:
                    formats.append({'url': format_url(line)})
                    continue
-                tbr = int_or_none(last_info.get('BANDWIDTH'), scale=1000)
+                tbr = int_or_none(last_info.get('AVERAGE-BANDWIDTH') or last_info.get('BANDWIDTH'), scale=1000)
                format_id = []
                if m3u8_id:
                    format_id.append(m3u8_id)
-                last_media_name = last_media.get('NAME') if last_media and last_media.get('TYPE') not in ('SUBTITLES', 'CLOSED-CAPTIONS') else None
-                # Despite specification does not mention NAME attribute for
-                # EXT-X-STREAM-INF it still sometimes may be present
-                stream_name = last_info.get('NAME') or last_media_name
                # Bandwidth of live streams may differ over time thus making
                # format_id unpredictable. So it's better to keep provided
                # format_id intact.
                if not live:
+                    # Despite specification does not mention NAME attribute for
+                    # EXT-X-STREAM-INF it still sometimes may be present
+                    stream_name = last_info.get('NAME')
                    format_id.append(stream_name if stream_name else '%d' % (tbr if tbr else len(formats)))
                f = {
                    'format_id': '-'.join(format_id),
@@ -1252,9 +1267,6 @@ class InfoExtractor(object):
                        'abr': abr,
                    })
                f.update(parse_codecs(last_info.get('CODECS')))
-                if last_media is not None:
-                    f['m3u8_media'] = last_media
-                    last_media = None
                formats.append(f)
                last_info = {}
        return formats
--- a/youtube_dl/extractor/crackle.py
+++ b/youtube_dl/extractor/crackle.py
@@ -1,5 +1,5 @@
 # coding: utf-8
-from __future__ import unicode_literals
+from __future__ import unicode_literals, division

 from .common import InfoExtractor
 from ..utils import int_or_none
@@ -8,12 +8,22 @@ from ..utils import int_or_none
 class CrackleIE(InfoExtractor):
    _VALID_URL = r'(?:crackle:|https?://(?:www\.)?crackle\.com/(?:playlist/\d+/|(?:[^/]+/)+))(?P<id>\d+)'
    _TEST = {
-        'url': 'http://www.crackle.com/the-art-of-more/2496419',
+        'url': 'http://www.crackle.com/comedians-in-cars-getting-coffee/2498934',
        'info_dict': {
-            'id': '2496419',
+            'id': '2498934',
            'ext': 'mp4',
-            'title': 'Heavy Lies the Head',
-            'description': 'md5:bb56aa0708fe7b9a4861535f15c3abca',
+            'title': 'Everybody Respects A Bloody Nose',
+            'description': 'Jerry is kaffeeklatsching in L.A. with funnyman J.B. Smoove (Saturday Night Live, Real Husbands of Hollywood). They’re headed for brew at 10 Speed Coffee in a 1964 Studebaker Avanti.',
+            'thumbnail': 're:^https?://.*\.jpg',
+            'duration': 906,
+            'series': 'Comedians In Cars Getting Coffee',
+            'season_number': 8,
+            'episode_number': 4,
+            'subtitles': {
+                'en-US': [{
+                    'ext': 'ttml',
+                }]
+            },
        },
        'params': {
            # m3u8 download
@@ -21,12 +31,8 @@ class CrackleIE(InfoExtractor):
        }
    }

-    # extracted from http://legacyweb-us.crackle.com/flash/QueryReferrer.ashx
-    _SUBTITLE_SERVER = 'http://web-us-az.crackle.com'
-    _UPLYNK_OWNER_ID = 'e8773f7770a44dbd886eee4fca16a66b'
-    _THUMBNAIL_TEMPLATE = 'http://images-us-am.crackle.com/%stnl_1920x1080.jpg?ts=20140107233116?c=635333335057637614'
-
    # extracted from http://legacyweb-us.crackle.com/flash/ReferrerRedirect.ashx
+    _THUMBNAIL_TEMPLATE = 'http://images-us-am.crackle.com/%stnl_1920x1080.jpg?ts=20140107233116?c=635333335057637614'
    _MEDIA_FILE_SLOTS = {
        'c544.flv': {
            'width': 544,
@@ -48,16 +54,21 @@ class CrackleIE(InfoExtractor):

    def _real_extract(self, url):
        video_id = self._match_id(url)
+
+        config_doc = self._download_xml(
+            'http://legacyweb-us.crackle.com/flash/QueryReferrer.ashx?site=16',
+            video_id, 'Downloading config')
+
        item = self._download_xml(
            'http://legacyweb-us.crackle.com/app/revamp/vidwallcache.aspx?flags=-1&fm=%s' % video_id,
            video_id).find('i')
        title = item.attrib['t']

-        thumbnail = None
        subtitles = {}
        formats = self._extract_m3u8_formats(
-            'http://content.uplynk.com/ext/%s/%s.m3u8' % (self._UPLYNK_OWNER_ID, video_id),
+            'http://content.uplynk.com/ext/%s/%s.m3u8' % (config_doc.attrib['strUplynkOwnerId'], video_id),
            video_id, 'mp4', m3u8_id='hls', fatal=None)
+        thumbnail = None
        path = item.attrib.get('p')
        if path:
            thumbnail = self._THUMBNAIL_TEMPLATE % path
@@ -76,7 +87,7 @@ class CrackleIE(InfoExtractor):
                    if locale not in subtitles:
                        subtitles[locale] = []
                    subtitles[locale] = [{
-                        'url': '%s/%s%s_%s.xml' % (self._SUBTITLE_SERVER, path, locale, v),
+                        'url': '%s/%s%s_%s.xml' % (config_doc.attrib['strSubtitleServer'], path, locale, v),
                        'ext': 'ttml',
                    }]
        self._sort_formats(formats, ('width', 'height', 'tbr', 'format_id'))
@@ -85,7 +96,7 @@ class CrackleIE(InfoExtractor):
            'id': video_id,
            'title': title,
            'description': item.attrib.get('d'),
-            'duration': int(item.attrib.get('r'), 16) if item.attrib.get('r') else None,
+            'duration': int(item.attrib.get('r'), 16) / 1000 if item.attrib.get('r') else None,
            'series': item.attrib.get('sn'),
            'season_number': int_or_none(item.attrib.get('se')),
            'episode_number': int_or_none(item.attrib.get('ep')),
--- a/youtube_dl/extractor/ctv.py
+++ b/youtube_dl/extractor/ctv.py
@@ -1,11 +1,13 @@
 # coding: utf-8
 from __future__ import unicode_literals

+import re
+
 from .common import InfoExtractor


 class CTVIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?ctv\.ca/video/player\?vid=(?P<id>[0-9.]+)'
+    _VALID_URL = r'https?://(?:www\.)?(?P<domain>ctv|tsn|bnn|thecomedynetwork)\.ca/.*?(?:\bvid=|-vid|~|%7E)(?P<id>[0-9.]+)'
    _TESTS = [{
        'url': 'http://www.ctv.ca/video/player?vid=706966',
        'md5': 'ff2ebbeae0aa2dcc32a830c3fd69b7b0',
@@ -18,13 +20,27 @@ class CTVIE(InfoExtractor):
            'timestamp': 1442624700,
        },
        'expected_warnings': ['HTTP Error 404'],
+    }, {
+        'url': 'http://www.thecomedynetwork.ca/video/player?vid=923582',
+        'only_matching': True,
+    }, {
+        'url': 'http://www.tsn.ca/video/expectations-high-for-milos-raonic-at-us-open~939549',
+        'only_matching': True,
+    }, {
+        'url': 'http://www.bnn.ca/video/berman-s-call-part-two-viewer-questions~939654',
+        'only_matching': True,
+    }, {
+        'url': 'http://www.ctv.ca/YourMorning/Video/S1E6-Monday-August-29-2016-vid938009',
+        'only_matching': True,
    }]

    def _real_extract(self, url):
-        video_id = self._match_id(url)
+        domain, video_id = re.match(self._VALID_URL, url).groups()
+        if domain == 'thecomedynetwork':
+            domain = 'comedy'
        return {
            '_type': 'url_transparent',
            'id': video_id,
-            'url': '9c9media:ctv_web:%s' % video_id,
+            'url': '9c9media:%s_web:%s' % (domain, video_id),
            'ie_key': 'NineCNineMedia',
        }
--- a/youtube_dl/extractor/discoverygo.py
+++ b/youtube_dl/extractor/discoverygo.py
@@ -7,6 +7,7 @@ from ..utils import (
    int_or_none,
    parse_age_limit,
    unescapeHTML,
+    ExtractorError,
 )


@@ -53,7 +54,14 @@ class DiscoveryGoIE(InfoExtractor):

        title = video['name']

-        stream = video['stream']
+        stream = video.get('stream')
+        if not stream:
+            if video.get('authenticated') is True:
+                raise ExtractorError(
+                    'This video is only available via cable service provider subscription that'
+                    ' is not currently supported. You may want to use --cookies.', expected=True)
+            else:
+                raise ExtractorError('Unable to find stream')
        STREAM_URL_SUFFIX = 'streamUrl'
        formats = []
        for stream_kind in ('', 'hds'):
--- a/youtube_dl/extractor/douyutv.py
+++ b/youtube_dl/extractor/douyutv.py
@@ -3,9 +3,17 @@ from __future__ import unicode_literals

 import hashlib
 import time
+import uuid
+
 from .common import InfoExtractor
-from ..utils import (ExtractorError, unescapeHTML)
-from ..compat import (compat_str, compat_basestring)
+from ..compat import (
+    compat_str,
+    compat_urllib_parse_urlencode,
+)
+from ..utils import (
+    ExtractorError,
+    unescapeHTML,
+)


 class DouyuTVIE(InfoExtractor):
@@ -21,7 +29,6 @@ class DouyuTVIE(InfoExtractor):
            'description': 're:.*m7show@163\.com.*',
            'thumbnail': 're:^https?://.*\.jpg$',
            'uploader': '7师傅',
-            'uploader_id': '431925',
            'is_live': True,
        },
        'params': {
@@ -37,7 +44,6 @@ class DouyuTVIE(InfoExtractor):
            'description': 'md5:746a2f7a253966a06755a912f0acc0d2',
            'thumbnail': 're:^https?://.*\.jpg$',
            'uploader': 'douyu小漠',
-            'uploader_id': '3769985',
            'is_live': True,
        },
        'params': {
@@ -54,7 +60,6 @@ class DouyuTVIE(InfoExtractor):
            'description': 're:.*m7show@163\.com.*',
            'thumbnail': 're:^https?://.*\.jpg$',
            'uploader': '7师傅',
-            'uploader_id': '431925',
            'is_live': True,
        },
        'params': {
@@ -65,6 +70,10 @@ class DouyuTVIE(InfoExtractor):
        'only_matching': True,
    }]

+    # Decompile core.swf in webpage by ffdec "Search SWFs in memory". core.swf
+    # is encrypted originally, but ffdec can dump memory to get the decrypted one.
+    _API_KEY = 'A12Svb&%1UUmf@hC'
+
    def _real_extract(self, url):
        video_id = self._match_id(url)

@@ -75,74 +84,56 @@ class DouyuTVIE(InfoExtractor):
            room_id = self._html_search_regex(
                r'"room_id"\s*:\s*(\d+),', page, 'room id')

-        config = None
-        # Douyu API sometimes returns error "Unable to load the requested class: eticket_redis_cache"
-        # Retry with different parameters - same parameters cause same errors
-        for i in range(5):
-            prefix = 'room/%s?aid=android&client_sys=android&time=%d' % (
-                room_id, int(time.time()))
-            auth = hashlib.md5((prefix + '1231').encode('ascii')).hexdigest()
+        room = self._download_json(
+            'http://m.douyu.com/html5/live?roomId=%s' % room_id, video_id,
+            note='Downloading room info')['data']

-            config_page = self._download_webpage(
-                'http://www.douyutv.com/api/v1/%s&auth=%s' % (prefix, auth),
-                video_id)
-            try:
-                config = self._parse_json(config_page, video_id, fatal=False)
-            except ExtractorError:
-                # Wait some time before retrying to get a different time() value
-                self._sleep(1, video_id, msg_template='%(video_id)s: Error occurs. '
-                                                      'Waiting for %(timeout)s seconds before retrying')
-                continue
-            else:
-                break
-        if config is None:
-            raise ExtractorError('Unable to fetch API result')
-
-        data = config['data']
-
-        error_code = config.get('error', 0)
-        if error_code is not 0:
-            error_desc = 'Server reported error %i' % error_code
-            if isinstance(data, (compat_str, compat_basestring)):
-                error_desc += ': ' + data
-            raise ExtractorError(error_desc, expected=True)
-
-        show_status = data.get('show_status')
        # 1 = live, 2 = offline
-        if show_status == '2':
+        if room.get('show_status') == '2':
+            raise ExtractorError('Live stream is offline', expected=True)
+
+        tt = compat_str(int(time.time() / 60))
+        did = uuid.uuid4().hex.upper()
+
+        sign_content = ''.join((room_id, did, self._API_KEY, tt))
+        sign = hashlib.md5((sign_content).encode('utf-8')).hexdigest()
+
+        flv_data = compat_urllib_parse_urlencode({
+            'cdn': 'ws',
+            'rate': '0',
+            'tt': tt,
+            'did': did,
+            'sign': sign,
+        })
+
+        video_info = self._download_json(
+            'http://www.douyu.com/lapi/live/getPlay/%s' % room_id, video_id,
+            data=flv_data, note='Downloading video info',
+            headers={'Content-Type': 'application/x-www-form-urlencoded'})
+
+        error_code = video_info.get('error', 0)
+        if error_code is not 0:
            raise ExtractorError(
-                'Live stream is offline', expected=True)
+                '%s reported error %i' % (self.IE_NAME, error_code),
+                expected=True)

-        base_url = data['rtmp_url']
-        live_path = data['rtmp_live']
+        base_url = video_info['data']['rtmp_url']
+        live_path = video_info['data']['rtmp_live']

-        title = self._live_title(unescapeHTML(data['room_name']))
-        description = data.get('show_details')
-        thumbnail = data.get('room_src')
+        video_url = '%s/%s' % (base_url, live_path)

-        uploader = data.get('nickname')
-        uploader_id = data.get('owner_uid')
-
-        multi_formats = data.get('rtmp_multi_bitrate')
-        if not isinstance(multi_formats, dict):
-            multi_formats = {}
-        multi_formats['live'] = live_path
-
-        formats = [{
-            'url': '%s/%s' % (base_url, format_path),
-            'format_id': format_id,
-            'preference': 1 if format_id == 'live' else 0,
-        } for format_id, format_path in multi_formats.items()]
-        self._sort_formats(formats)
+        title = self._live_title(unescapeHTML(room['room_name']))
+        description = room.get('notice')
+        thumbnail = room.get('room_src')
+        uploader = room.get('nickname')

        return {
            'id': room_id,
            'display_id': video_id,
+            'url': video_url,
            'title': title,
            'description': description,
            'thumbnail': thumbnail,
            'uploader': uploader,
-            'uploader_id': uploader_id,
-            'formats': formats,
            'is_live': True,
        }
--- a/youtube_dl/extractor/extractors.py
+++ b/youtube_dl/extractor/extractors.py
@@ -127,9 +127,12 @@ from .carambatv import (
    CarambaTVIE,
    CarambaTVPageIE,
 )
+from .cartoonnetwork import CartoonNetworkIE
 from .cbc import (
    CBCIE,
    CBCPlayerIE,
+    CBCWatchVideoIE,
+    CBCWatchIE,
 )
 from .cbs import CBSIE
 from .cbslocal import CBSLocalIE
@@ -539,6 +542,7 @@ from .nextmedia import (
 )
 from .nfb import NFBIE
 from .nfl import NFLIE
+from .nhk import NhkVodIE
 from .nhl import (
    NHLVideocenterIE,
    NHLNewsIE,
@@ -550,7 +554,10 @@ from .nick import (
    NickDeIE,
 )
 from .niconico import NiconicoIE, NiconicoPlaylistIE
-from .ninecninemedia import NineCNineMediaIE
+from .ninecninemedia import (
+    NineCNineMediaStackIE,
+    NineCNineMediaIE,
+)
 from .ninegag import NineGagIE
 from .ninenow import NineNowIE
 from .nintendo import NintendoIE
@@ -632,7 +639,6 @@ from .phoenix import PhoenixIE
 from .photobucket import PhotobucketIE
 from .pinkbike import PinkbikeIE
 from .pladform import PladformIE
-from .played import PlayedIE
 from .playfm import PlayFMIE
 from .plays import PlaysTVIE
 from .playtvak import PlaytvakIE
@@ -820,6 +826,7 @@ from .tagesschau import (
    TagesschauIE,
 )
 from .tass import TassIE
+from .tbs import TBSIE
 from .tdslifeway import TDSLifewayIE
 from .teachertube import (
    TeacherTubeIE,
@@ -871,7 +878,6 @@ from .toypics import ToypicsUserIE, ToypicsIE
 from .traileraddict import TrailerAddictIE
 from .trilulilu import TriluliluIE
 from .trollvids import TrollvidsIE
-from .trutube import TruTubeIE
 from .tube8 import Tube8IE
 from .tubitv import TubiTvIE
 from .tudou import (
@@ -945,6 +951,7 @@ from .uplynk import (
 )
 from .urort import UrortIE
 from .urplay import URPlayIE
+from .usanetwork import USANetworkIE
 from .usatoday import USATodayIE
 from .ustream import UstreamIE, UstreamChannelIE
 from .ustudio import (
--- a/youtube_dl/extractor/generic.py
+++ b/youtube_dl/extractor/generic.py
@@ -104,7 +104,8 @@ class GenericIE(InfoExtractor):
            },
            'expected_warnings': [
                'URL could be a direct video link, returning it as such.'
-            ]
+            ],
+            'skip': 'URL invalid',
        },
        # Direct download with broken HEAD
        {
@@ -268,7 +269,8 @@ class GenericIE(InfoExtractor):
            'params': {
                # m3u8 downloads
                'skip_download': True,
-            }
+            },
+            'skip': 'video gone',
        },
        # m3u8 served with Content-Type: text/plain
        {
@@ -283,7 +285,8 @@ class GenericIE(InfoExtractor):
            'params': {
                # m3u8 downloads
                'skip_download': True,
-            }
+            },
+            'skip': 'video gone',
        },
        # google redirect
        {
@@ -368,6 +371,7 @@ class GenericIE(InfoExtractor):
                'description': 'Mississauga resident David Farmer is still out of power as a result of the ice storm a month ago. To keep the house warm, Farmer cuts wood from his property for a wood burning stove downstairs.',
            },
            'add_ie': ['BrightcoveLegacy'],
+            'skip': 'video gone',
        },
        {
            'url': 'http://www.championat.com/video/football/v/87/87499.html',
@@ -421,6 +425,7 @@ class GenericIE(InfoExtractor):
            'params': {
                'skip_download': True,
            },
+            'skip': 'movie expired',
        },
        # embed.ly video
        {
@@ -448,6 +453,8 @@ class GenericIE(InfoExtractor):
                'title': 'Between Two Ferns with Zach Galifianakis: President Barack Obama',
                'description': 'Episode 18: President Barack Obama sits down with Zach Galifianakis for his most memorable interview yet.',
            },
+            # HEAD requests lead to endless 301, while GET is OK
+            'expected_warnings': ['301'],
        },
        # RUTV embed
        {
@@ -522,6 +529,9 @@ class GenericIE(InfoExtractor):
                'title': '[NSFL] [FM15] which pumiscer was this ( vid ) ( alfa as fuck srx )',
            },
            'playlist_mincount': 7,
+            # This forum does not allow <iframe> syntaxes anymore
+            # Now HTML tags are displayed as-is
+            'skip': 'No videos on this page',
        },
        # Embedded TED video
        {
@@ -570,7 +580,8 @@ class GenericIE(InfoExtractor):
            },
            'params': {
                'skip_download': 'Requires rtmpdump'
-            }
+            },
+            'skip': 'video gone',
        },
        # francetv embed
        {
@@ -2232,11 +2243,11 @@ class GenericIE(InfoExtractor):

        # Look for VODPlatform embeds
        mobj = re.search(
-            r'<iframe[^>]+src=[\'"]((?:https?:)?//(?:www\.)?vod-platform\.net/embed/[^/?#]+)',
+            r'<iframe[^>]+src=(["\'])(?P<url>(?:https?:)?//(?:www\.)?vod-platform\.net/[eE]mbed/.+?)\1',
            webpage)
        if mobj is not None:
            return self.url_result(
-                self._proto_relative_url(unescapeHTML(mobj.group(1))), 'VODPlatform')
+                self._proto_relative_url(unescapeHTML(mobj.group('url'))), 'VODPlatform')

        # Look for Instagram embeds
        instagram_embed_url = InstagramIE._extract_embed_url(webpage)
--- a/youtube_dl/extractor/kickstarter.py
+++ b/youtube_dl/extractor/kickstarter.py
@@ -37,7 +37,6 @@ class KickStarterIE(InfoExtractor):
            'ext': 'mp4',
            'title': 'Power Drive 2000',
        },
-        'expected_warnings': ['OpenGraph description'],
    }]

    def _real_extract(self, url):
@@ -67,6 +66,6 @@ class KickStarterIE(InfoExtractor):
            'id': video_id,
            'url': video_url,
            'title': title,
-            'description': self._og_search_description(webpage),
+            'description': self._og_search_description(webpage, default=None),
            'thumbnail': thumbnail,
        }
--- a/youtube_dl/extractor/kusi.py
+++ b/youtube_dl/extractor/kusi.py
@@ -18,31 +18,20 @@ from ..utils import (
 class KUSIIE(InfoExtractor):
    _VALID_URL = r'https?://(?:www\.)?kusi\.com/(?P<path>story/.+|video\?clipId=(?P<clipId>\d+))'
    _TESTS = [{
-        'url': 'http://www.kusi.com/story/31183873/turko-files-case-closed-put-on-hold',
-        'md5': 'f926e7684294cf8cb7bdf8858e1b3988',
+        'url': 'http://www.kusi.com/story/32849881/turko-files-refused-to-help-it-aint-right',
+        'md5': '4e76ce8e53660ce9697d06c0ba6fc47d',
        'info_dict': {
-            'id': '12203019',
+            'id': '12689020',
            'ext': 'mp4',
-            'title': 'Turko Files: Case Closed! & Put On Hold!',
-            'duration': 231.0,
-            'upload_date': '20160210',
-            'timestamp': 1455087571,
+            'title': "Turko Files: Refused to Help, It Ain't Right!",
+            'duration': 223.586,
+            'upload_date': '20160826',
+            'timestamp': 1472233118,
            'thumbnail': 're:^https?://.*\.jpg$'
        },
    }, {
        'url': 'http://kusi.com/video?clipId=12203019',
-        'info_dict': {
-            'id': '12203019',
-            'ext': 'mp4',
-            'title': 'Turko Files: Case Closed! & Put On Hold!',
-            'duration': 231.0,
-            'upload_date': '20160210',
-            'timestamp': 1455087571,
-            'thumbnail': 're:^https?://.*\.jpg$'
-        },
-        'params': {
-            'skip_download': True,  # Same as previous one
-        },
+        'only_matching': True,
    }]

    def _real_extract(self, url):
--- a/youtube_dl/extractor/mtv.py
+++ b/youtube_dl/extractor/mtv.py
@@ -4,7 +4,6 @@ import re

 from .common import InfoExtractor
 from ..compat import (
-    compat_urllib_parse_urlencode,
    compat_str,
    compat_xpath,
 )
@@ -14,12 +13,13 @@ from ..utils import (
    fix_xml_ampersands,
    float_or_none,
    HEADRequest,
+    RegexNotFoundError,
    sanitized_Request,
    strip_or_none,
    timeconvert,
    unescapeHTML,
+    update_url_query,
    url_basename,
-    RegexNotFoundError,
    xpath_text,
 )

@@ -36,6 +36,11 @@ class MTVServicesInfoExtractor(InfoExtractor):
    def _id_from_uri(uri):
        return uri.split(':')[-1]

+    @staticmethod
+    def _remove_template_parameter(url):
+        # Remove the templates, like &device={device}
+        return re.sub(r'&[^=]*?={.*?}(?=(&|$))', '', url)
+
    # This was originally implemented for ComedyCentral, but it also works here
    @classmethod
    def _transform_rtmp_url(cls, rtmp_video_url):
@@ -117,9 +122,7 @@ class MTVServicesInfoExtractor(InfoExtractor):
        video_id = self._id_from_uri(uri)
        self.report_extraction(video_id)
        content_el = itemdoc.find('%s/%s' % (_media_xml_tag('group'), _media_xml_tag('content')))
-        mediagen_url = content_el.attrib['url']
-        # Remove the templates, like &device={device}
-        mediagen_url = re.sub(r'&[^=]*?={.*?}(?=(&|$))', '', mediagen_url)
+        mediagen_url = self._remove_template_parameter(content_el.attrib['url'])
        if 'acceptMethods' not in mediagen_url:
            mediagen_url += '&' if '?' in mediagen_url else '?'
            mediagen_url += 'acceptMethods=fms'
@@ -178,12 +181,12 @@ class MTVServicesInfoExtractor(InfoExtractor):
        data = {'uri': uri}
        if self._LANG:
            data['lang'] = self._LANG
-        return compat_urllib_parse_urlencode(data)
+        return data

    def _get_videos_info(self, uri):
        video_id = self._id_from_uri(uri)
        feed_url = self._get_feed_url(uri)
-        info_url = feed_url + '?' + self._get_feed_query(uri)
+        info_url = update_url_query(feed_url, self._get_feed_query(uri))
        return self._get_videos_info_from_url(info_url, video_id)

    def _get_videos_info_from_url(self, url, video_id):
@@ -256,13 +259,9 @@ class MTVServicesEmbeddedIE(MTVServicesInfoExtractor):

    def _get_feed_url(self, uri):
        video_id = self._id_from_uri(uri)
-        site_id = uri.replace(video_id, '')
-        config_url = ('http://media.mtvnservices.com/pmt-arc/e1/players/{0}/'
-                      'context52/config.xml'.format(site_id))
-        config_doc = self._download_xml(config_url, video_id)
-        feed_node = config_doc.find('.//feed')
-        feed_url = feed_node.text.strip().split('?')[0]
-        return feed_url
+        config = self._download_json(
+            'http://media.mtvnservices.com/pmt/e1/access/index.html?uri=%s&configtype=edge' % uri, video_id)
+        return self._remove_template_parameter(config['feedWithQueryParams'])

    def _real_extract(self, url):
        mobj = re.match(self._VALID_URL, url)
--- a/youtube_dl/extractor/nba.py
+++ b/youtube_dl/extractor/nba.py
@@ -1,25 +1,20 @@
 from __future__ import unicode_literals

 import functools
-import os.path
 import re

-from .common import InfoExtractor
+from .turner import TurnerBaseIE
 from ..compat import (
    compat_urllib_parse_urlencode,
    compat_urlparse,
 )
 from ..utils import (
-    int_or_none,
    OnDemandPagedList,
-    parse_duration,
    remove_start,
-    xpath_text,
-    xpath_attr,
 )


-class NBAIE(InfoExtractor):
+class NBAIE(TurnerBaseIE):
    _VALID_URL = r'https?://(?:watch\.|www\.)?nba\.com/(?P<path>(?:[^/]+/)+(?P<id>[^?]*?))/?(?:/index\.html)?(?:\?.*)?$'
    _TESTS = [{
        'url': 'http://www.nba.com/video/games/nets/2012/12/04/0021200253-okc-bkn-recap.nba/index.html',
@@ -44,28 +39,30 @@ class NBAIE(InfoExtractor):
        'url': 'http://watch.nba.com/video/channels/playoffs/2015/05/20/0041400301-cle-atl-recap.nba',
        'md5': 'b2b39b81cf28615ae0c3360a3f9668c4',
        'info_dict': {
-            'id': '0041400301-cle-atl-recap',
+            'id': 'channels/playoffs/2015/05/20/0041400301-cle-atl-recap.nba',
            'ext': 'mp4',
            'title': 'Hawks vs. Cavaliers Game 1',
            'description': 'md5:8094c3498d35a9bd6b1a8c396a071b4d',
            'duration': 228,
            'timestamp': 1432134543,
            'upload_date': '20150520',
-        }
+        },
+        'expected_warnings': ['Unable to download f4m manifest'],
    }, {
        'url': 'http://www.nba.com/clippers/news/doc-rivers-were-not-trading-blake',
        'info_dict': {
-            'id': '1455672027478-Doc_Feb16_720',
+            'id': 'teams/clippers/2016/02/17/1455672027478-Doc_Feb16_720.mov-297324',
            'ext': 'mp4',
            'title': 'Practice: Doc Rivers - 2/16/16',
            'description': 'Head Coach Doc Rivers addresses the media following practice.',
-            'upload_date': '20160217',
+            'upload_date': '20160216',
            'timestamp': 1455672000,
        },
        'params': {
            # m3u8 download
            'skip_download': True,
        },
+        'expected_warnings': ['Unable to download f4m manifest'],
    }, {
        'url': 'http://www.nba.com/timberwolves/wiggins-shootaround#',
        'info_dict': {
@@ -80,7 +77,7 @@ class NBAIE(InfoExtractor):
    }, {
        'url': 'http://www.nba.com/timberwolves/wiggins-shootaround#',
        'info_dict': {
-            'id': 'Wigginsmp4',
+            'id': 'teams/timberwolves/2014/12/12/Wigginsmp4-3462601',
            'ext': 'mp4',
            'title': 'Shootaround Access - Dec. 12 | Andrew Wiggins',
            'description': 'Wolves rookie Andrew Wiggins addresses the media after Friday\'s shootaround.',
@@ -92,6 +89,7 @@ class NBAIE(InfoExtractor):
            # m3u8 download
            'skip_download': True,
        },
+        'expected_warnings': ['Unable to download f4m manifest'],
    }]

    _PAGE_SIZE = 30
@@ -145,53 +143,12 @@ class NBAIE(InfoExtractor):
            if path.startswith('video/teams'):
                path = 'video/channels/proxy/' + path[6:]

-        video_info = self._download_xml('http://www.nba.com/%s.xml' % path, video_id)
-        video_id = os.path.splitext(xpath_text(video_info, 'slug'))[0]
-        title = xpath_text(video_info, 'headline')
-        description = xpath_text(video_info, 'description')
-        duration = parse_duration(xpath_text(video_info, 'length'))
-        timestamp = int_or_none(xpath_attr(video_info, 'dateCreated', 'uts'))
-
-        thumbnails = []
-        for image in video_info.find('images'):
-            thumbnails.append({
-                'id': image.attrib.get('cut'),
-                'url': image.text,
-                'width': int_or_none(image.attrib.get('width')),
-                'height': int_or_none(image.attrib.get('height')),
+        return self._extract_cvp_info(
+            'http://www.nba.com/%s.xml' % path, video_id, {
+                'default': {
+                    'media_src': 'http://nba.cdn.turner.com/nba/big',
+                },
+                'm3u8': {
+                    'media_src': 'http://nbavod-f.akamaihd.net',
+                },
            })
-
-        formats = []
-        for video_file in video_info.findall('.//file'):
-            video_url = video_file.text
-            if video_url.startswith('/'):
-                continue
-            if video_url.endswith('.m3u8'):
-                formats.extend(self._extract_m3u8_formats(video_url, video_id, ext='mp4', m3u8_id='hls', fatal=False))
-            elif video_url.endswith('.f4m'):
-                formats.extend(self._extract_f4m_formats(video_url + '?hdcore=3.4.1.1', video_id, f4m_id='hds', fatal=False))
-            else:
-                key = video_file.attrib.get('bitrate')
-                format_info = {
-                    'format_id': key,
-                    'url': video_url,
-                }
-                mobj = re.search(r'(\d+)x(\d+)(?:_(\d+))?', key)
-                if mobj:
-                    format_info.update({
-                        'width': int(mobj.group(1)),
-                        'height': int(mobj.group(2)),
-                        'tbr': int_or_none(mobj.group(3)),
-                    })
-                formats.append(format_info)
-        self._sort_formats(formats)
-
-        return {
-            'id': video_id,
-            'title': title,
-            'description': description,
-            'duration': duration,
-            'timestamp': timestamp,
-            'thumbnails': thumbnails,
-            'formats': formats,
-        }
--- a/youtube_dl/extractor/nhk.py
+++ b/youtube_dl/extractor/nhk.py
@@ -0,0 +1,50 @@
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+
+
+class NhkVodIE(InfoExtractor):
+    _VALID_URL = r'https?://www3\.nhk\.or\.jp/nhkworld/en/vod/(?P<id>.+?)\.html'
+    _TEST = {
+        # Videos available only for a limited period of time. Visit
+        # http://www3.nhk.or.jp/nhkworld/en/vod/ for working samples.
+        'url': 'http://www3.nhk.or.jp/nhkworld/en/vod/tokyofashion/20160815.html',
+        'info_dict': {
+            'id': 'A1bnNiNTE6nY3jLllS-BIISfcC_PpvF5',
+            'ext': 'flv',
+            'title': 'TOKYO FASHION EXPRESS - The Kimono as Global Fashion',
+            'description': 'md5:db338ee6ce8204f415b754782f819824',
+            'series': 'TOKYO FASHION EXPRESS',
+            'episode': 'The Kimono as Global Fashion',
+        },
+        'skip': 'Videos available only for a limited period of time',
+    }
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+
+        webpage = self._download_webpage(url, video_id)
+
+        embed_code = self._search_regex(
+            r'nw_vod_ooplayer\([^,]+,\s*(["\'])(?P<id>(?:(?!\1).)+)\1',
+            webpage, 'ooyala embed code', group='id')
+
+        title = self._search_regex(
+            r'<div[^>]+class=["\']episode-detail["\']>\s*<h\d+>([^<]+)',
+            webpage, 'title', default=None)
+        description = self._html_search_regex(
+            r'(?s)<p[^>]+class=["\']description["\'][^>]*>(.+?)</p>',
+            webpage, 'description', default=None)
+        series = self._search_regex(
+            r'<h2[^>]+class=["\']detail-top-player-title[^>]+><a[^>]+>([^<]+)',
+            webpage, 'series', default=None)
+
+        return {
+            '_type': 'url_transparent',
+            'ie_key': 'Ooyala',
+            'url': 'ooyala:%s' % embed_code,
+            'title': '%s - %s' % (series, title) if series and title else title,
+            'description': description,
+            'series': series,
+            'episode': title,
+        }
--- a/youtube_dl/extractor/nick.py
+++ b/youtube_dl/extractor/nick.py
@@ -2,7 +2,6 @@
 from __future__ import unicode_literals

 from .mtv import MTVServicesInfoExtractor
-from ..compat import compat_urllib_parse_urlencode
 from ..utils import update_url_query


@@ -59,10 +58,10 @@ class NickIE(MTVServicesInfoExtractor):
    }]

    def _get_feed_query(self, uri):
-        return compat_urllib_parse_urlencode({
+        return {
            'feed': 'nick_arc_player_prime',
            'mgid': uri,
-        })
+        }

    def _extract_mgid(self, webpage):
        return self._search_regex(r'data-contenturi="([^"]+)', webpage, 'mgid')
--- a/youtube_dl/extractor/ninecninemedia.py
+++ b/youtube_dl/extractor/ninecninemedia.py
@@ -4,40 +4,36 @@ from __future__ import unicode_literals
 import re

 from .common import InfoExtractor
+from ..compat import compat_str
 from ..utils import (
    parse_iso8601,
-    parse_duration,
-    ExtractorError
+    float_or_none,
+    ExtractorError,
+    int_or_none,
 )


-class NineCNineMediaIE(InfoExtractor):
-    _VALID_URL = r'9c9media:(?P<destination_code>[^:]+):(?P<id>\d+)'
+class NineCNineMediaBaseIE(InfoExtractor):
+    _API_BASE_TEMPLATE = 'http://capi.9c9media.com/destinations/%s/platforms/desktop/contents/%s/'
+
+
+class NineCNineMediaStackIE(NineCNineMediaBaseIE):
+    IE_NAME = '9c9media:stack'
+    _VALID_URL = r'9c9media:stack:(?P<destination_code>[^:]+):(?P<content_id>\d+):(?P<content_package>\d+):(?P<id>\d+)'

    def _real_extract(self, url):
-        destination_code, video_id = re.match(self._VALID_URL, url).groups()
-        api_base_url = 'http://capi.9c9media.com/destinations/%s/platforms/desktop/contents/%s/' % (destination_code, video_id)
-        content = self._download_json(api_base_url, video_id, query={
-            '$include': '[contentpackages]',
-        })
-        title = content['Name']
-        if len(content['ContentPackages']) > 1:
-            raise ExtractorError('multiple content packages')
-        content_package = content['ContentPackages'][0]
-        stacks_base_url = api_base_url + 'contentpackages/%s/stacks/' % content_package['Id']
-        stacks = self._download_json(stacks_base_url, video_id)['Items']
-        if len(stacks) > 1:
-            raise ExtractorError('multiple stacks')
-        stack = stacks[0]
-        stack_base_url = '%s%s/manifest.' % (stacks_base_url, stack['Id'])
+        destination_code, content_id, package_id, stack_id = re.match(self._VALID_URL, url).groups()
+        stack_base_url_template = self._API_BASE_TEMPLATE + 'contentpackages/%s/stacks/%s/manifest.'
+        stack_base_url = stack_base_url_template % (destination_code, content_id, package_id, stack_id)
+
        formats = []
        formats.extend(self._extract_m3u8_formats(
-            stack_base_url + 'm3u8', video_id, 'mp4',
+            stack_base_url + 'm3u8', stack_id, 'mp4',
            'm3u8_native', m3u8_id='hls', fatal=False))
        formats.extend(self._extract_f4m_formats(
-            stack_base_url + 'f4m', video_id,
+            stack_base_url + 'f4m', stack_id,
            f4m_id='hds', fatal=False))
-        mp4_url = self._download_webpage(stack_base_url + 'pd', video_id, fatal=False)
+        mp4_url = self._download_webpage(stack_base_url + 'pd', stack_id, fatal=False)
        if mp4_url:
            formats.append({
                'url': mp4_url,
@@ -46,10 +42,86 @@ class NineCNineMediaIE(InfoExtractor):
        self._sort_formats(formats)

        return {
-            'id': video_id,
-            'title': title,
-            'description': content.get('Desc') or content.get('ShortDesc'),
-            'timestamp': parse_iso8601(content.get('BroadcastDateTime')),
-            'duration': parse_duration(content.get('BroadcastTime')),
+            'id': stack_id,
            'formats': formats,
        }
+
+
+class NineCNineMediaIE(NineCNineMediaBaseIE):
+    IE_NAME = '9c9media'
+    _VALID_URL = r'9c9media:(?P<destination_code>[^:]+):(?P<id>\d+)'
+
+    def _real_extract(self, url):
+        destination_code, content_id = re.match(self._VALID_URL, url).groups()
+        api_base_url = self._API_BASE_TEMPLATE % (destination_code, content_id)
+        content = self._download_json(api_base_url, content_id, query={
+            '$include': '[Media,Season,ContentPackages]',
+        })
+        title = content['Name']
+        if len(content['ContentPackages']) > 1:
+            raise ExtractorError('multiple content packages')
+        content_package = content['ContentPackages'][0]
+        package_id = content_package['Id']
+        content_package_url = api_base_url + 'contentpackages/%s/' % package_id
+        content_package = self._download_json(content_package_url, content_id)
+
+        if content_package.get('Constraints', {}).get('Security', {}).get('Type') == 'adobe-drm':
+            raise ExtractorError('This video is DRM protected.', expected=True)
+
+        stacks = self._download_json(content_package_url + 'stacks/', package_id)['Items']
+        multistacks = len(stacks) > 1
+
+        thumbnails = []
+        for image in content.get('Images', []):
+            image_url = image.get('Url')
+            if not image_url:
+                continue
+            thumbnails.append({
+                'url': image_url,
+                'width': int_or_none(image.get('Width')),
+                'height': int_or_none(image.get('Height')),
+            })
+
+        tags, categories = [], []
+        for source_name, container in (('Tags', tags), ('Genres', categories)):
+            for e in content.get(source_name, []):
+                e_name = e.get('Name')
+                if not e_name:
+                    continue
+                container.append(e_name)
+
+        description = content.get('Desc') or content.get('ShortDesc')
+        season = content.get('Season', {})
+        base_info = {
+            'description': description,
+            'timestamp': parse_iso8601(content.get('BroadcastDateTime')),
+            'episode_number': int_or_none(content.get('Episode')),
+            'season': season.get('Name'),
+            'season_number': season.get('Number'),
+            'season_id': season.get('Id'),
+            'series': content.get('Media', {}).get('Name'),
+            'tags': tags,
+            'categories': categories,
+        }
+
+        entries = []
+        for stack in stacks:
+            stack_id = compat_str(stack['Id'])
+            entry = {
+                '_type': 'url_transparent',
+                'url': '9c9media:stack:%s:%s:%s:%s' % (destination_code, content_id, package_id, stack_id),
+                'id': stack_id,
+                'title': '%s_part%s' % (title, stack['Name']) if multistacks else title,
+                'duration': float_or_none(stack.get('Duration')),
+                'ie_key': 'NineCNineMediaStack',
+            }
+            entry.update(base_info)
+            entries.append(entry)
+
+        return {
+            '_type': 'multi_video',
+            'id': content_id,
+            'title': title,
+            'description': description,
+            'entries': entries,
+        }
--- a/youtube_dl/extractor/openload.py
+++ b/youtube_dl/extractor/openload.py
@@ -55,10 +55,12 @@ class OpenloadIE(InfoExtractor):

        video_url_chars = []

-        for c in enc_data:
+        for idx, c in enumerate(enc_data):
            j = compat_ord(c)
            if j >= 33 and j <= 126:
                j = ((j + 14) % 94) + 33
+            if idx == len(enc_data) - 1:
+                j += 1
            video_url_chars += compat_chr(j)

        video_url = 'https://openload.co/stream/%s?mime=true' % ''.join(video_url_chars)
--- a/youtube_dl/extractor/periscope.py
+++ b/youtube_dl/extractor/periscope.py
@@ -8,7 +8,14 @@ from ..utils import (
 )


-class PeriscopeIE(InfoExtractor):
+class PeriscopeBaseIE(InfoExtractor):
+    def _call_api(self, method, query, item_id):
+        return self._download_json(
+            'https://api.periscope.tv/api/v2/%s' % method,
+            item_id, query=query)
+
+
+class PeriscopeIE(PeriscopeBaseIE):
    IE_DESC = 'Periscope'
    IE_NAME = 'periscope'
    _VALID_URL = r'https?://(?:www\.)?periscope\.tv/[^/]+/(?P<id>[^/?#]+)'
@@ -34,14 +41,11 @@ class PeriscopeIE(InfoExtractor):
        'only_matching': True,
    }]

-    def _call_api(self, method, value):
-        return self._download_json(
-            'https://api.periscope.tv/api/v2/%s?broadcast_id=%s' % (method, value), value)
-
    def _real_extract(self, url):
        token = self._match_id(url)

-        broadcast_data = self._call_api('getBroadcastPublic', token)
+        broadcast_data = self._call_api(
+            'getBroadcastPublic', {'broadcast_id': token}, token)
        broadcast = broadcast_data['broadcast']
        status = broadcast['status']

@@ -61,7 +65,8 @@ class PeriscopeIE(InfoExtractor):
            'url': broadcast[image],
        } for image in ('image_url', 'image_url_small') if broadcast.get(image)]

-        stream = self._call_api('getAccessPublic', token)
+        stream = self._call_api(
+            'getAccessPublic', {'broadcast_id': token}, token)

        formats = []
        for format_id in ('replay', 'rtmp', 'hls', 'https_hls'):
@@ -88,7 +93,7 @@ class PeriscopeIE(InfoExtractor):
        }


-class PeriscopeUserIE(InfoExtractor):
+class PeriscopeUserIE(PeriscopeBaseIE):
    _VALID_URL = r'https?://www\.periscope\.tv/(?P<id>[^/]+)/?$'
    IE_DESC = 'Periscope user videos'
    IE_NAME = 'periscope:user'
@@ -106,26 +111,34 @@ class PeriscopeUserIE(InfoExtractor):
    }

    def _real_extract(self, url):
-        user_id = self._match_id(url)
+        user_name = self._match_id(url)

-        webpage = self._download_webpage(url, user_id)
+        webpage = self._download_webpage(url, user_name)

        data_store = self._parse_json(
            unescapeHTML(self._search_regex(
                r'data-store=(["\'])(?P<data>.+?)\1',
                webpage, 'data store', default='{}', group='data')),
-            user_id)
+            user_name)

-        user = data_store.get('User', {}).get('user', {})
-        title = user.get('display_name') or user.get('username')
+        user = list(data_store['UserCache']['users'].values())[0]['user']
+        user_id = user['id']
+        session_id = data_store['SessionToken']['broadcastHistory']['token']['session_id']
+
+        broadcasts = self._call_api(
+            'getUserBroadcastsPublic',
+            {'user_id': user_id, 'session_id': session_id},
+            user_name)['broadcasts']
+
+        broadcast_ids = [
+            broadcast['id'] for broadcast in broadcasts if broadcast.get('id')]
+
+        title = user.get('display_name') or user.get('username') or user_name
        description = user.get('description')

-        broadcast_ids = (data_store.get('UserBroadcastHistory', {}).get('broadcastIds') or
-                         data_store.get('BroadcastCache', {}).get('broadcastIds', []))
-
        entries = [
            self.url_result(
-                'https://www.periscope.tv/%s/%s' % (user_id, broadcast_id))
+                'https://www.periscope.tv/%s/%s' % (user_name, broadcast_id))
            for broadcast_id in broadcast_ids]

        return self.playlist_result(entries, user_id, title, description)
--- a/youtube_dl/extractor/played.py
+++ b/youtube_dl/extractor/played.py
@@ -1,60 +0,0 @@
-# coding: utf-8
-from __future__ import unicode_literals
-
-import re
-import os.path
-
-from .common import InfoExtractor
-from ..utils import (
-    ExtractorError,
-    sanitized_Request,
-    urlencode_postdata,
-)
-
-
-class PlayedIE(InfoExtractor):
-    IE_NAME = 'played.to'
-    _VALID_URL = r'https?://(?:www\.)?played\.to/(?P<id>[a-zA-Z0-9_-]+)'
-
-    _TEST = {
-        'url': 'http://played.to/j2f2sfiiukgt',
-        'md5': 'c2bd75a368e82980e7257bf500c00637',
-        'info_dict': {
-            'id': 'j2f2sfiiukgt',
-            'ext': 'flv',
-            'title': 'youtube-dl_test_video.mp4',
-        },
-        'skip': 'Removed for copyright infringement.',  # oh wow
-    }
-
-    def _real_extract(self, url):
-        video_id = self._match_id(url)
-        orig_webpage = self._download_webpage(url, video_id)
-
-        m_error = re.search(
-            r'(?s)Reason for deletion:.*?<b class="err"[^>]*>(?P<msg>[^<]+)</b>', orig_webpage)
-        if m_error:
-            raise ExtractorError(m_error.group('msg'), expected=True)
-
-        data = self._hidden_inputs(orig_webpage)
-
-        self._sleep(2, video_id)
-
-        post = urlencode_postdata(data)
-        headers = {
-            b'Content-Type': b'application/x-www-form-urlencoded',
-        }
-        req = sanitized_Request(url, post, headers)
-        webpage = self._download_webpage(
-            req, video_id, note='Downloading video page ...')
-
-        title = os.path.splitext(data['fname'])[0]
-
-        video_url = self._search_regex(
-            r'file: "?(.+?)",', webpage, 'video URL')
-
-        return {
-            'id': video_id,
-            'title': title,
-            'url': video_url,
-        }
--- a/youtube_dl/extractor/pyvideo.py
+++ b/youtube_dl/extractor/pyvideo.py
@@ -1,59 +1,72 @@
 from __future__ import unicode_literals

 import re
-import os

 from .common import InfoExtractor
+from ..compat import compat_str
+from ..utils import int_or_none


 class PyvideoIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?pyvideo\.org/video/(?P<id>\d+)/(.*)'
+    _VALID_URL = r'https?://(?:www\.)?pyvideo\.org/(?P<category>[^/]+)/(?P<id>[^/?#&.]+)'

-    _TESTS = [
-        {
-            'url': 'http://pyvideo.org/video/1737/become-a-logging-expert-in-30-minutes',
-            'md5': '520915673e53a5c5d487c36e0c4d85b5',
-            'info_dict': {
-                'id': '24_4WWkSmNo',
-                'ext': 'webm',
-                'title': 'Become a logging expert in 30 minutes',
-                'description': 'md5:9665350d466c67fb5b1598de379021f7',
-                'upload_date': '20130320',
-                'uploader': 'Next Day Video',
-                'uploader_id': 'NextDayVideo',
-            },
-            'add_ie': ['Youtube'],
+    _TESTS = [{
+        'url': 'http://pyvideo.org/pycon-us-2013/become-a-logging-expert-in-30-minutes.html',
+        'info_dict': {
+            'id': 'become-a-logging-expert-in-30-minutes',
        },
-        {
-            'url': 'http://pyvideo.org/video/2542/gloriajw-spotifywitherikbernhardsson182m4v',
-            'md5': '5fe1c7e0a8aa5570330784c847ff6d12',
-            'info_dict': {
-                'id': '2542',
-                'ext': 'm4v',
-                'title': 'Gloriajw-SpotifyWithErikBernhardsson182',
-            },
+        'playlist_count': 2,
+    }, {
+        'url': 'http://pyvideo.org/pygotham-2012/gloriajw-spotifywitherikbernhardsson182m4v.html',
+        'md5': '5fe1c7e0a8aa5570330784c847ff6d12',
+        'info_dict': {
+            'id': '2542',
+            'ext': 'm4v',
+            'title': 'Gloriajw-SpotifyWithErikBernhardsson182.m4v',
        },
-    ]
+    }]

    def _real_extract(self, url):
        mobj = re.match(self._VALID_URL, url)
+        category = mobj.group('category')
        video_id = mobj.group('id')

-        webpage = self._download_webpage(url, video_id)
+        entries = []

-        m_youtube = re.search(r'(https?://www\.youtube\.com/watch\?v=.*)', webpage)
-        if m_youtube is not None:
-            return self.url_result(m_youtube.group(1), 'Youtube')
+        data = self._download_json(
+            'https://raw.githubusercontent.com/pyvideo/data/master/%s/videos/%s.json'
+            % (category, video_id), video_id, fatal=False)

-        title = self._html_search_regex(
-            r'<div class="section">\s*<h3(?:\s+class="[^"]*"[^>]*)?>([^>]+?)</h3>',
-            webpage, 'title', flags=re.DOTALL)
-        video_url = self._search_regex(
-            [r'<source src="(.*?)"', r'<dt>Download</dt>.*?<a href="(.+?)"'],
-            webpage, 'video url', flags=re.DOTALL)
+        if data:
+            for video in data['videos']:
+                video_url = video.get('url')
+                if video_url:
+                    if video.get('type') == 'youtube':
+                        entries.append(self.url_result(video_url, 'Youtube'))
+                    else:
+                        entries.append({
+                            'id': compat_str(data.get('id') or video_id),
+                            'url': video_url,
+                            'title': data['title'],
+                            'description': data.get('description') or data.get('summary'),
+                            'thumbnail': data.get('thumbnail_url'),
+                            'duration': int_or_none(data.get('duration')),
+                        })
+        else:
+            webpage = self._download_webpage(url, video_id)
+            title = self._og_search_title(webpage)
+            media_urls = self._search_regex(
+                r'(?s)Media URL:(.+?)</li>', webpage, 'media urls')
+            for m in re.finditer(
+                    r'<a[^>]+href=(["\'])(?P<url>http.+?)\1', media_urls):
+                media_url = m.group('url')
+                if re.match(r'https?://www\.youtube\.com/watch\?v=.*', media_url):
+                    entries.append(self.url_result(media_url, 'Youtube'))
+                else:
+                    entries.append({
+                        'id': video_id,
+                        'url': media_url,
+                        'title': title,
+                    })

-        return {
-            'id': video_id,
-            'title': os.path.splitext(title)[0],
-            'url': video_url,
-        }
+        return self.playlist_result(entries, video_id)
--- a/youtube_dl/extractor/soundcloud.py
+++ b/youtube_dl/extractor/soundcloud.py
@@ -32,7 +32,7 @@ class SoundcloudIE(InfoExtractor):
    _VALID_URL = r'''(?x)^(?:https?://)?
                    (?:(?:(?:www\.|m\.)?soundcloud\.com/
                            (?P<uploader>[\w\d-]+)/
-                            (?!(?:tracks|sets(?:/[^/?#]+)?|reposts|likes|spotlight)/?(?:$|[?#]))
+                            (?!(?:tracks|sets(?:/.+?)?|reposts|likes|spotlight)/?(?:$|[?#]))
                            (?P<title>[\w\d-]+)/?
                            (?P<token>[^?]+?)?(?:[?].*)?$)
                       |(?:api\.soundcloud\.com/tracks/(?P<track_id>\d+)
@@ -265,6 +265,9 @@ class SoundcloudSetIE(SoundcloudIE):
            'title': 'The Royal Concept EP',
        },
        'playlist_mincount': 6,
+    }, {
+        'url': 'https://soundcloud.com/the-concept-band/sets/the-royal-concept-ep/token',
+        'only_matching': True,
    }]

    def _real_extract(self, url):
--- a/youtube_dl/extractor/spankbang.py
+++ b/youtube_dl/extractor/spankbang.py
@@ -14,7 +14,7 @@ class SpankBangIE(InfoExtractor):
            'id': '3vvn',
            'ext': 'mp4',
            'title': 'fantasy solo',
-            'description': 'dillion harper masturbates on a bed',
+            'description': 'Watch fantasy solo free HD porn video - 05 minutes - dillion harper masturbates on a bed free adult movies.',
            'thumbnail': 're:^https?://.*\.jpg$',
            'uploader': 'silly2587',
            'age_limit': 18,
@@ -44,12 +44,10 @@ class SpankBangIE(InfoExtractor):

        title = self._html_search_regex(
            r'(?s)<h1[^>]*>(.+?)</h1>', webpage, 'title')
-        description = self._search_regex(
-            r'class="desc"[^>]*>([^<]+)',
-            webpage, 'description', default=None)
+        description = self._og_search_description(webpage)
        thumbnail = self._og_search_thumbnail(webpage)
        uploader = self._search_regex(
-            r'class="user"[^>]*>([^<]+)',
+            r'class="user"[^>]*><img[^>]+>([^<]+)',
            webpage, 'uploader', fatal=False)

        age_limit = self._rta_search(webpage)
--- a/youtube_dl/extractor/tbs.py
+++ b/youtube_dl/extractor/tbs.py
@@ -0,0 +1,59 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+import re
+
+from .turner import TurnerBaseIE
+from ..utils import (
+    extract_attributes,
+    ExtractorError,
+)
+
+
+class TBSIE(TurnerBaseIE):
+    _VALID_URL = r'https?://(?:www\.)?(?P<site>tbs|tntdrama)\.com/videos/(?:[^/]+/)+(?P<id>[^/?#]+)\.html'
+    _TESTS = [{
+        'url': 'http://www.tbs.com/videos/people-of-earth/season-1/extras/2007318/theatrical-trailer.html',
+        'md5': '9e61d680e2285066ade7199e6408b2ee',
+        'info_dict': {
+            'id': '2007318',
+            'ext': 'mp4',
+            'title': 'Theatrical Trailer',
+            'description': 'Catch the latest comedy from TBS, People of Earth, premiering Halloween night--Monday, October 31, at 9/8c.',
+        }
+    }, {
+        'url': 'http://www.tntdrama.com/videos/good-behavior/season-1/extras/1538823/you-better-run.html',
+        'md5': 'ce53c6ead5e9f3280b4ad2031a6fab56',
+        'info_dict': {
+            'id': '1538823',
+            'ext': 'mp4',
+            'title': 'You Better Run',
+            'description': 'Letty Raines must figure out what she\'s running toward while running away from her past. Good Behavior premieres November 15 at 9/8c.',
+        }
+    }]
+
+    def _real_extract(self, url):
+        domain, display_id = re.match(self._VALID_URL, url).groups()
+        site = domain[:3]
+        webpage = self._download_webpage(url, display_id)
+        video_params = extract_attributes(self._search_regex(r'(<[^>]+id="page-video"[^>]*>)', webpage, 'video params'))
+        if video_params.get('isAuthRequired') == 'true':
+            raise ExtractorError(
+                'This video is only available via cable service provider subscription that'
+                ' is not currently supported.', expected=True)
+        query = None
+        clip_id = video_params.get('clipid')
+        if clip_id:
+            query = 'id=' + clip_id
+        else:
+            query = 'titleId=' + video_params['titleid']
+        return self._extract_cvp_info(
+            'http://www.%s.com/service/cvpXml?%s' % (domain, query), display_id, {
+                'default': {
+                    'media_src': 'http://ht.cdn.turner.com/%s/big' % site,
+                },
+                'secure': {
+                    'media_src': 'http://apple-secure.cdn.turner.com/%s/big' % site,
+                    'tokenizer_src': 'http://www.%s.com/video/processors/services/token_ipadAdobe.do' % domain,
+                },
+            })
--- a/youtube_dl/extractor/tnaflix.py
+++ b/youtube_dl/extractor/tnaflix.py
@@ -10,6 +10,7 @@ from ..utils import (
    int_or_none,
    parse_duration,
    str_to_int,
+    unescapeHTML,
    xpath_text,
 )

@@ -80,7 +81,8 @@ class TNAFlixNetworkBaseIE(InfoExtractor):

        if not cfg_url:
            inputs = self._hidden_inputs(webpage)
-            cfg_url = 'https://cdn-fck.tnaflix.com/tnaflix/%s.fid?key=%s' % (inputs['vkey'], inputs['nkey'])
+            cfg_url = ('https://cdn-fck.tnaflix.com/tnaflix/%s.fid?key=%s&VID=%s&premium=1&vip=1&alpha'
+                       % (inputs['vkey'], inputs['nkey'], video_id))

        cfg_xml = self._download_xml(
            cfg_url, display_id, 'Downloading metadata',
@@ -89,7 +91,7 @@ class TNAFlixNetworkBaseIE(InfoExtractor):
        formats = []

        def extract_video_url(vl):
-            return re.sub('speed=\d+', 'speed=', vl.text)
+            return re.sub('speed=\d+', 'speed=', unescapeHTML(vl.text))

        video_link = cfg_xml.find('./videoLink')
        if video_link is not None:
@@ -201,7 +203,7 @@ class TNAFlixIE(TNAFlixNetworkBaseIE):
    _TESTS = [{
        # anonymous uploader, no categories
        'url': 'http://www.tnaflix.com/porn-stars/Carmella-Decesare-striptease/video553878',
-        'md5': '7e569419fe6d69543d01e6be22f5f7c4',
+        'md5': 'ecf3498417d09216374fc5907f9c6ec0',
        'info_dict': {
            'id': '553878',
            'display_id': 'Carmella-Decesare-striptease',
@@ -215,11 +217,11 @@ class TNAFlixIE(TNAFlixNetworkBaseIE):
    }, {
        # non-anonymous uploader, categories
        'url': 'https://www.tnaflix.com/teen-porn/Educational-xxx-video/video6538',
-        'md5': 'fcba2636572895aba116171a899a5658',
+        'md5': '0f5d4d490dbfd117b8607054248a07c0',
        'info_dict': {
            'id': '6538',
            'display_id': 'Educational-xxx-video',
-            'ext': 'flv',
+            'ext': 'mp4',
            'title': 'Educational xxx video',
            'description': 'md5:b4fab8f88a8621c8fabd361a173fe5b8',
            'thumbnail': 're:https?://.*\.jpg$',
--- a/youtube_dl/extractor/trutube.py
+++ b/youtube_dl/extractor/trutube.py
@@ -1,26 +0,0 @@
-from __future__ import unicode_literals
-
-from .nuevo import NuevoBaseIE
-
-
-class TruTubeIE(NuevoBaseIE):
-    _VALID_URL = r'https?://(?:www\.)?trutube\.tv/(?:video/|nuevo/player/embed\.php\?v=)(?P<id>\d+)'
-    _TESTS = [{
-        'url': 'http://trutube.tv/video/14880/Ramses-II-Proven-To-Be-A-Red-Headed-Caucasoid-',
-        'md5': 'c5b6e301b0a2040b074746cbeaa26ca1',
-        'info_dict': {
-            'id': '14880',
-            'ext': 'flv',
-            'title': 'Ramses II - Proven To Be A Red Headed Caucasoid',
-            'thumbnail': 're:^http:.*\.jpg$',
-        }
-    }, {
-        'url': 'https://trutube.tv/nuevo/player/embed.php?v=14880',
-        'only_matching': True,
-    }]
-
-    def _real_extract(self, url):
-        video_id = self._match_id(url)
-        return self._extract_nuevo(
-            'https://trutube.tv/nuevo/player/config.php?v=%s' % video_id,
-            video_id)
--- a/youtube_dl/extractor/turner.py
+++ b/youtube_dl/extractor/turner.py
@@ -0,0 +1,178 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+import re
+
+from .common import InfoExtractor
+from ..compat import compat_str
+from ..utils import (
+    xpath_text,
+    int_or_none,
+    determine_ext,
+    parse_duration,
+    xpath_attr,
+    update_url_query,
+    compat_urlparse,
+)
+
+
+class TurnerBaseIE(InfoExtractor):
+    def _extract_timestamp(self, video_data):
+        return int_or_none(xpath_attr(video_data, 'dateCreated', 'uts'))
+
+    def _extract_cvp_info(self, data_src, video_id, path_data={}):
+        video_data = self._download_xml(data_src, video_id)
+        video_id = video_data.attrib['id']
+        title = xpath_text(video_data, 'headline', fatal=True)
+        # rtmp_src = xpath_text(video_data, 'akamai/src')
+        # if rtmp_src:
+        #     splited_rtmp_src = rtmp_src.split(',')
+        #     if len(splited_rtmp_src) == 2:
+        #         rtmp_src = splited_rtmp_src[1]
+        # aifp = xpath_text(video_data, 'akamai/aifp', default='')
+
+        tokens = {}
+        urls = []
+        formats = []
+        rex = re.compile(
+            r'(?P<width>[0-9]+)x(?P<height>[0-9]+)(?:_(?P<bitrate>[0-9]+))?')
+        # Possible formats locations: files/file, files/groupFiles/files
+        # and maybe others
+        for video_file in video_data.findall('.//file'):
+            video_url = video_file.text.strip()
+            if not video_url:
+                continue
+            ext = determine_ext(video_url)
+            if video_url.startswith('/mp4:protected/'):
+                continue
+                # TODO Correct extraction for these files
+                # protected_path_data = path_data.get('protected')
+                # if not protected_path_data or not rtmp_src:
+                #     continue
+                # protected_path = self._search_regex(
+                #     r'/mp4:(.+)\.[a-z0-9]', video_url, 'secure path')
+                # auth = self._download_webpage(
+                #     protected_path_data['tokenizer_src'], query={
+                #         'path': protected_path,
+                #         'videoId': video_id,
+                #         'aifp': aifp,
+                #     })
+                # token = xpath_text(auth, 'token')
+                # if not token:
+                #     continue
+                # video_url = rtmp_src + video_url + '?' + token
+            elif video_url.startswith('/secure/'):
+                secure_path_data = path_data.get('secure')
+                if not secure_path_data:
+                    continue
+                video_url = secure_path_data['media_src'] + video_url
+                secure_path = self._search_regex(r'https?://[^/]+(.+/)', video_url, 'secure path') + '*'
+                token = tokens.get(secure_path)
+                if not token:
+                    auth = self._download_xml(
+                        secure_path_data['tokenizer_src'], video_id, query={
+                            'path': secure_path,
+                            'videoId': video_id,
+                        })
+                    token = xpath_text(auth, 'token')
+                    if not token:
+                        continue
+                    tokens[secure_path] = token
+                video_url = video_url + '?hdnea=' + token
+            elif not re.match('https?://', video_url):
+                base_path_data = path_data.get(ext, path_data.get('default', {}))
+                media_src = base_path_data.get('media_src')
+                if not media_src:
+                    continue
+                video_url = media_src + video_url
+            if video_url in urls:
+                continue
+            urls.append(video_url)
+            format_id = video_file.get('bitrate')
+            if ext == 'smil':
+                formats.extend(self._extract_smil_formats(
+                    video_url, video_id, fatal=False))
+            elif ext == 'm3u8':
+                m3u8_formats = self._extract_m3u8_formats(
+                    video_url, video_id, 'mp4', m3u8_id=format_id or 'hls',
+                    fatal=False)
+                if m3u8_formats:
+                    # Sometimes final URLs inside m3u8 are unsigned, let's fix this
+                    # ourselves
+                    qs = compat_urlparse.urlparse(video_url).query
+                    if qs:
+                        query = compat_urlparse.parse_qs(qs)
+                        for m3u8_format in m3u8_formats:
+                            m3u8_format['url'] = update_url_query(m3u8_format['url'], query)
+                            m3u8_format['extra_param_to_segment_url'] = qs
+                    formats.extend(m3u8_formats)
+            elif ext == 'f4m':
+                formats.extend(self._extract_f4m_formats(
+                    update_url_query(video_url, {'hdcore': '3.7.0'}),
+                    video_id, f4m_id=format_id or 'hds', fatal=False))
+            else:
+                f = {
+                    'format_id': format_id,
+                    'url': video_url,
+                    'ext': ext,
+                }
+                mobj = rex.search(format_id + video_url)
+                if mobj:
+                    f.update({
+                        'width': int(mobj.group('width')),
+                        'height': int(mobj.group('height')),
+                        'tbr': int_or_none(mobj.group('bitrate')),
+                    })
+                elif isinstance(format_id, compat_str):
+                    if format_id.isdigit():
+                        f['tbr'] = int(format_id)
+                    else:
+                        mobj = re.match(r'ios_(audio|[0-9]+)$', format_id)
+                        if mobj:
+                            if mobj.group(1) == 'audio':
+                                f.update({
+                                    'vcodec': 'none',
+                                    'ext': 'm4a',
+                                })
+                            else:
+                                f['tbr'] = int(mobj.group(1))
+                formats.append(f)
+        self._sort_formats(formats)
+
+        subtitles = {}
+        for source in video_data.findall('closedCaptions/source'):
+            for track in source.findall('track'):
+                track_url = track.get('url')
+                if not isinstance(track_url, compat_str) or track_url.endswith('/big'):
+                    continue
+                lang = track.get('lang') or track.get('label') or 'en'
+                subtitles.setdefault(lang, []).append({
+                    'url': track_url,
+                    'ext': {
+                        'scc': 'scc',
+                        'webvtt': 'vtt',
+                        'smptett': 'tt',
+                    }.get(source.get('format'))
+                })
+
+        thumbnails = [{
+            'id': image.get('cut'),
+            'url': image.text,
+            'width': int_or_none(image.get('width')),
+            'height': int_or_none(image.get('height')),
+        } for image in video_data.findall('images/image')]
+
+        return {
+            'id': video_id,
+            'title': title,
+            'formats': formats,
+            'subtitles': subtitles,
+            'thumbnails': thumbnails,
+            'description': xpath_text(video_data, 'description'),
+            'duration': parse_duration(xpath_text(video_data, 'length') or xpath_text(video_data, 'trt')),
+            'timestamp': self._extract_timestamp(video_data),
+            'upload_date': xpath_attr(video_data, 'metas', 'version'),
+            'series': xpath_text(video_data, 'showTitle'),
+            'season_number': int_or_none(xpath_text(video_data, 'seasonNumber')),
+            'episode_number': int_or_none(xpath_text(video_data, 'episodeNumber')),
+        }
--- a/youtube_dl/extractor/uplynk.py
+++ b/youtube_dl/extractor/uplynk.py
@@ -33,9 +33,7 @@ class UplynkIE(InfoExtractor):
        formats = self._extract_m3u8_formats('http://content.uplynk.com/%s.m3u8' % path, display_id, 'mp4')
        if session_id:
            for f in formats:
-                f['extra_param_to_segment_url'] = {
-                    'pbs': session_id,
-                }
+                f['extra_param_to_segment_url'] = 'pbs=' + session_id
        self._sort_formats(formats)
        asset = self._download_json('http://content.uplynk.com/player/assetinfo/%s.json' % path, display_id)
        if asset.get('error') == 1:
--- a/youtube_dl/extractor/usanetwork.py
+++ b/youtube_dl/extractor/usanetwork.py
@@ -0,0 +1,76 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+import re
+
+from .adobepass import AdobePassIE
+from ..utils import (
+    extract_attributes,
+    smuggle_url,
+    update_url_query,
+)
+
+
+class USANetworkIE(AdobePassIE):
+    _VALID_URL = r'https?://(?:www\.)?usanetwork\.com/(?:[^/]+/videos|movies)/(?P<id>[^/?#]+)'
+    _TEST = {
+        'url': 'http://www.usanetwork.com/mrrobot/videos/hpe-cybersecurity',
+        'md5': '33c0d2ba381571b414024440d08d57fd',
+        'info_dict': {
+            'id': '3086229',
+            'ext': 'mp4',
+            'title': 'HPE Cybersecurity',
+            'description': 'The more we digitize our world, the more vulnerable we are.',
+            'upload_date': '20160818',
+            'timestamp': 1471535460,
+            'uploader': 'NBCU-USA',
+        },
+    }
+
+    def _real_extract(self, url):
+        display_id = self._match_id(url)
+        webpage = self._download_webpage(url, display_id)
+
+        player_params = extract_attributes(self._search_regex(
+            r'(<div[^>]+data-usa-tve-player-container[^>]*>)', webpage, 'player params'))
+        video_id = player_params['data-mpx-guid']
+        title = player_params['data-episode-title']
+
+        account_pid, path = re.search(
+            r'data-src="(?:https?)?//player\.theplatform\.com/p/([^/]+)/.*?/(media/guid/\d+/\d+)',
+            webpage).groups()
+
+        query = {
+            'mbr': 'true',
+        }
+        if player_params.get('data-is-full-episode') == '1':
+            query['manifest'] = 'm3u'
+
+        if player_params.get('data-entitlement') == 'auth':
+            adobe_pass = {}
+            drupal_settings = self._search_regex(
+                r'jQuery\.extend\(Drupal\.settings\s*,\s*({.+?})\);',
+                webpage, 'drupal settings', fatal=False)
+            if drupal_settings:
+                drupal_settings = self._parse_json(drupal_settings, video_id, fatal=False)
+                if drupal_settings:
+                    adobe_pass = drupal_settings.get('adobePass', {})
+            resource = self._get_mvpd_resource(
+                adobe_pass.get('adobePassResourceId', 'usa'),
+                title, video_id, player_params.get('data-episode-rating', 'TV-14'))
+            query['auth'] = self._extract_mvpd_auth(
+                url, video_id, adobe_pass.get('adobePassRequestorId', 'usa'), resource)
+
+        info = self._search_json_ld(webpage, video_id, default={})
+        info.update({
+            '_type': 'url_transparent',
+            'url': smuggle_url(update_url_query(
+                'http://link.theplatform.com/s/%s/%s' % (account_pid, path),
+                query), {'force_smil_url': True}),
+            'id': video_id,
+            'title': title,
+            'series': player_params.get('data-show-title'),
+            'episode': title,
+            'ie_key': 'ThePlatform',
+        })
+        return info
--- a/youtube_dl/extractor/vodplatform.py
+++ b/youtube_dl/extractor/vodplatform.py
@@ -6,7 +6,7 @@ from ..utils import unescapeHTML


 class VODPlatformIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?vod-platform\.net/embed/(?P<id>[^/?#]+)'
+    _VALID_URL = r'https?://(?:www\.)?vod-platform\.net/[eE]mbed/(?P<id>[^/?#]+)'
    _TEST = {
        # from http://www.lbcgroup.tv/watch/chapter/29143/52844/%D8%A7%D9%84%D9%86%D8%B5%D8%B1%D8%A9-%D9%81%D9%8A-%D8%B6%D9%8A%D8%A7%D9%81%D8%A9-%D8%A7%D9%84%D9%80-cnn/ar
        'url': 'http://vod-platform.net/embed/RufMcytHDolTH1MuKHY9Fw',
--- a/youtube_dl/utils.py
+++ b/youtube_dl/utils.py
@@ -2183,7 +2183,7 @@ def parse_codecs(codecs_str):
        if codec in ('avc1', 'avc2', 'avc3', 'avc4', 'vp9', 'vp8', 'hev1', 'hev2', 'h263', 'h264', 'mp4v'):
            if not vcodec:
                vcodec = full_codec
-        elif codec in ('mp4a', 'opus', 'vorbis', 'mp3', 'aac'):
+        elif codec in ('mp4a', 'opus', 'vorbis', 'mp3', 'aac', 'ac-3'):
            if not acodec:
                acodec = full_codec
        else:
--- a/youtube_dl/version.py
+++ b/youtube_dl/version.py
@@ -1,3 +1,3 @@
 from __future__ import unicode_literals

-__version__ = '2016.08.24.1'
+__version__ = '2016.08.31'
Author	SHA1	Message	Date
Sergey M․	4fd350611c	release 2016.08.31	2016-08-31 02:39:39 +07:00
Sergey M․	263fef43de	[ChangeLog] Actualize	2016-08-31 02:37:40 +07:00
Sergey M․	a249ab83cb	[pyvideo] Remove debugging code	2016-08-31 01:56:58 +07:00
Sergey M․	f7043ef39c	[soundcloud] Fix _VALID_URL clashes with sets (Closes #10505 )	2016-08-31 01:56:15 +07:00
Sergey M․	64fc49aba0	[bandcamp:album] Fix title extraction (Closes #10455 )	2016-08-31 00:29:49 +07:00
Sergey M․	245023a861	[pyvideo] Fix extraction (Closes #10468 )	2016-08-30 23:51:18 +07:00
Remita Amine	3c77a54d5d	[turner] keep video id intact	2016-08-30 10:46:48 +01:00
Remita Amine	da30a20a4d	[turner,cnn] move a check for wrong timestamp to CNNIE	2016-08-29 19:26:53 +01:00
Remita Amine	1fe48afea5	[cnn] update _TEST for CNNBlogsIE and CNNArticleIE(closes #10489 )	2016-08-29 18:24:16 +01:00
Remita Amine	42e05be867	[ctv] add support for (tsn,bnn,thecomedynetwork).ca websites(#10016 )	2016-08-29 18:24:16 +01:00
Remita Amine	fe45b0e060	[9c9media] fix multiple stacks extraction and extract more metadata(#10016 )	2016-08-29 18:24:16 +01:00
Sergey M․	a06e1498aa	[kusi] Update test	2016-08-29 22:54:33 +07:00
Sergey M․	5a80e7b43a	[turner] Skip invalid subtitles' URLs	2016-08-29 22:44:15 +07:00
Sergey M․	3fb2a23029	[adultswim] Extract video info from onlineOriginals (Closes #10492 )	2016-08-29 22:40:35 +07:00
Sergey M․	cd10b3ea63	[turner] Extract all formats	2016-08-29 22:13:49 +07:00
Sergey M․	547993dcd0	[turner] Fix subtitles extraction	2016-08-29 21:52:41 +07:00
Yen Chi Hsuan	6c9b71bc08	[downloader/external] Recommend --hls-prefer-native for SOCKS users Related: #10490	2016-08-29 19:05:38 +08:00
Remita Amine	93b8404599	[generic,vodplatform] improve embed regex	2016-08-29 07:57:20 +01:00
Sergey M․	9ba1e1dcc0	[played] Remove extractor (Closes #10470 )	2016-08-29 08:26:07 +07:00
Remita Amine	b8079a40bc	[turner] fix secure m3u8 formats downloading	2016-08-28 17:51:53 +01:00
Remita Amine	5bc8a73af6	[cartoonnetwork] make extraction work for more videos in the website some videos require `networkName=CN2` to be present in the feed url	2016-08-28 17:08:26 +01:00
Remita Amine	b3eaeded12	[tbs] Add new extractor(#10222 )	2016-08-28 16:51:09 +01:00
Remita Amine	ec65b391cb	[cartoonnetwork] Add new extractor(#10110 )	2016-08-28 16:51:09 +01:00
Remita Amine	2982514072	[turner,nba,cnn,adultswim] add base extractor to parse cvp feeds	2016-08-28 16:51:09 +01:00
Yen Chi Hsuan	98908bcf7c	[openload] Update algorithm again (#10408 )	2016-08-28 22:49:46 +08:00
Yen Chi Hsuan	04b32c8f96	[bilibili] Fix extraction (closes #10375 ) Thanks @gdkchan for the algorithm	2016-08-28 22:06:31 +08:00
Yen Chi Hsuan	40eec6b15c	[openload] Fix extraction (closes #10408 ) Thanks to @yokrysty again!	2016-08-28 20:27:52 +08:00
Yen Chi Hsuan	39efc6e3e0	[generic] Update some _TESTS	2016-08-28 15:46:11 +08:00
Sergey M․	1198fe14a1	release 2016.08.28	2016-08-28 07:24:08 +07:00
Sergey M․	71e90766b5	[README.md] Fix typo in download archive FAQ entry	2016-08-28 07:09:03 +07:00
Sergey M․	d7aae610f6	[ChangeLog] Actualize	2016-08-28 07:00:15 +07:00
Sergey M․	92c27a0dbf	[periscope:user] Fix extraction (Closes #10453 )	2016-08-28 02:35:49 +07:00
Yen Chi Hsuan	d181cff685	Merge branch 'steven7851-patch-2'	2016-08-27 01:17:12 +08:00
Yen Chi Hsuan	3b4b82d4ce	[douyutv] Simplify	2016-08-27 01:16:39 +08:00
Yen Chi Hsuan	545ef4f531	Merge branch 'patch-2' of https://github.com/steven7851/youtube-dl into steven7851-patch-2	2016-08-26 22:29:46 +08:00
Yen Chi Hsuan	906b87cf5f	[crackle] Revert to template-based thumbnail extraction To reduce to number of HTTP requests	2016-08-26 19:58:47 +08:00
steven7851	b281aad2dc	[douyutv] Use new api use lapi for flv info, and html5 api for room info #10153 #10318	2016-08-26 07:32:54 +08:00
Sergey M․	6b18a24e6e	[tnaflix] Fix extraction (Closes #10434 )	2016-08-26 05:57:52 +07:00
Sergey M․	c9de980106	Credit @Xender for nhk:vod (#10424 )	2016-08-26 04:49:52 +07:00
Sergey M․	f9b373afda	[nhk:vod] Improve extraction (Closes #10424 )	2016-08-26 04:48:40 +07:00
Aleksander Nitecki	298a120ab7	[nhk] Add extractor for VoD.	2016-08-26 04:15:51 +07:00
Sergey M․	e3faecde30	[trutube] Remove extractor (Closes #10438 )	2016-08-26 03:43:13 +07:00
Remita Amine	a0f071a50d	[usanetwork] Add new extractor	2016-08-25 19:41:31 +01:00
Yen Chi Hsuan	20bad91d76	[downloader/external] Clarify that ffmpeg doesn't support SOCKS Ref: #10304	2016-08-25 22:38:06 +08:00
Yen Chi Hsuan	b54a2da433	[crackle] Fix extraction and update _TESTS (closes #10333 )	2016-08-25 22:22:31 +08:00
Yen Chi Hsuan	dc2c37f316	[spankbang] Fix description and uploader (closes #10339 )	2016-08-25 20:47:35 +08:00
Philipp Hagemeister	c1f62dd338	[README] Clean up grammar in --download-archive paragraph	2016-08-25 14:45:01 +02:00
Sergey M․	5a3efcd27c	[README.md] Add FAQ entry for download archive	2016-08-25 18:57:31 +07:00
Sergey M․	4c8f9c2577	[README.md] Add comments in sample configuration for clarity	2016-08-25 18:27:15 +07:00
Sergey M․	f26a298247	[README.md] Use en-US URL in cookies FAQ entry	2016-08-25 18:19:41 +07:00
Sergey M․	ea01cdbf61	[README.md] Clarify how to export cookies from browser for cookies FAQ entry	2016-08-25 18:17:45 +07:00
Sergey M․	6a76b53355	[README.md] Quote URL in streaming to player FAQ entry	2016-08-25 18:05:01 +07:00
Remita Amine	d37708fc86	[YoutubeDL] check only for None Value in thumbnails sorting	2016-08-25 11:53:47 +01:00
Remita Amine	5c13c28566	raise unexpected error when no stream found	2016-08-25 09:55:23 +01:00
Remita Amine	f70e9229e6	[discoverygo] detect when video needs authentication(closes #10425 )	2016-08-25 09:11:23 +01:00
Remita Amine	30afe4aeb2	[cbc] Add support for watch.cbc.ca	2016-08-25 08:49:44 +01:00
Remita Amine	75fa990dc6	[YoutubeDL] add fallback value for thumbnails values in thumbnails sorting	2016-08-25 08:49:44 +01:00
Remita Amine	f39ffc5877	[common] extract formats from #EXT-X-MEDIA tags	2016-08-25 08:49:44 +01:00
Remita Amine	07ea9c9b05	[downloader/hls] fill IV with zeros for IVs shorter than 16-octet	2016-08-25 08:49:44 +01:00
Remita Amine	073ac1225f	[utils] add ac-3 to the list of audio codecs in parse_codecs	2016-08-25 08:49:44 +01:00
Sergey M․	0c6422cdd6	[README.md] Add FAQ entry for streaming to player	2016-08-25 07:34:55 +07:00
Yen Chi Hsuan	08773689f3	[kickstarter] Silent the warning for og:description Closes #10415	2016-08-25 01:29:32 +08:00
Yen Chi Hsuan	0c75abbb7b	[mtvservices:embedded] Use another endpoint to get feed URL Closes #10363 In the original mtvservices:embedded test case, config.xml is still used to get the feed URL. Some other examples, including test_Generic_40 (http://www.vulture.com/2016/06/new-key-peele-sketches-released.html), and the video mentioned in #10363, use another endpoint to get the feed URL. The 'index.html' approach works for the original test case, too. So I didn't keep the old approach.	2016-08-24 23:58:22 +08:00
Yen Chi Hsuan	97653f81b2	[bilibili] Mark as broken Bilibili now uses emscripten, which is very difficult for reverse engineering. I don't expect it to be fixed in near future, so I mark it as broken. Ref: #10375	2016-08-24 21:28:00 +08:00