ytdl/youtube_dl/extractor
Jaime Marquínez Ferrándiz a504ced097 Improve subtitles support
For each language the extractor builds a list with the available formats sorted (like for video formats), then YoutubeDL selects one of them using the '--sub-format' option which now allows giving the format preferences (for example 'ass/srt/best').
For each format the 'url' field can be set so that we only download the contents if needed, or if the contents needs to be processed (like in crunchyroll) the 'data' field can be used.

The reasons for this change are:
* We weren't checking that the format given with '--sub-format' was available, checking it in each extractor would be repetitive.
* It allows to easily support giving a format preference.
* The subtitles were automatically downloaded in the extractor, but I think that if you use for example the '--dump-json' option you want to finish as fast as possible.

Currently only the ted extractor has been updated, but the old system still works.
2015-02-16 21:51:03 +01:00
..
__init__.py [history] Add extractor (Closes #4934) 2015-02-15 04:57:52 +06:00
abc7news.py [abc7news] Add extractor (Closes #4734) 2015-01-18 08:09:18 +06:00
abc.py
academicearth.py
addanime.py
adobetv.py
adultswim.py
aftenposten.py [aftenposten] Add extractor (Closes #4863) 2015-02-07 01:46:54 +06:00
aftonbladet.py [aftonbladet] Modernize 2015-02-03 10:18:32 +01:00
aljazeera.py
allocine.py
alphaporno.py
anitube.py
anysex.py
aol.py
aparat.py Use _family_friendly_search for determining age_limit 2015-02-08 17:45:38 +02:00
appletrailers.py Remove duplicate dictionary keys 2015-01-30 20:11:51 +11:00
archiveorg.py
ard.py [ard] Remove deleted video test case 2015-02-01 12:00:47 +01:00
arte.py
atresplayer.py [utils] YoutubeDLHandler: don't use 'Youtubedl-user-agent' for overriding the default user agent 2015-01-24 18:07:21 +01:00
atttechchannel.py [atttechchannel] Add extractor (Closes #3938) 2015-01-10 19:44:29 +06:00
audiomack.py [audiomack:album] Update testcase 2015-01-25 05:15:47 +01:00
azubu.py
bambuser.py [bambuser] Fix 'uploader_id' extraction (fixes #4944) 2015-02-13 11:36:33 +01:00
bandcamp.py [bandcamp] Correct variable name 2015-02-10 01:37:14 +01:00
bbccouk.py [bbccouk] Fix fallback to legacy playlist 2015-02-15 16:32:38 +06:00
beeg.py [beeg] fix test 2015-02-14 13:42:42 +01:00
behindkink.py
bet.py [bet] Correct test IDs 2015-01-08 18:03:29 +01:00
bild.py
bilibili.py [bilibili] Fix extraction (Closes #4660) 2015-01-08 01:33:22 +06:00
blinkx.py
bliptv.py [utils] YoutubeDLHandler: don't use 'Youtubedl-user-agent' for overriding the default user agent 2015-01-24 18:07:21 +01:00
bloomberg.py
bpb.py
br.py
breakcom.py
brightcove.py [brightcove] Fix up more generically invalid XML (Fixes #4849) 2015-02-02 23:47:14 +01:00
buzzfeed.py [buzzfeed] Fix test 2015-01-08 18:03:29 +01:00
byutv.py
c56.py
camdemy.py [camdemy] Fix _VALID_URL 2015-02-13 20:10:42 +06:00
canal13cl.py
canalc2.py
canalplus.py [canalplus] Add support for itele.fr URLs (Closes #4931) 2015-02-11 16:21:52 +02:00
cbs.py
cbsnews.py
ccc.py [ccc] Add new extractor (Fixes #4890) 2015-02-10 05:42:41 +01:00
ceskatelevize.py [ceskatelevize] Fix python 2.6 format issue 2015-01-07 05:03:34 +06:00
channel9.py
chilloutzone.py
cinchcast.py [cinchcast] Wrap overly long lines (#4820) 2015-01-30 10:59:07 +01:00
clipfish.py
cliphunter.py [cliphunter] Fix extraction and update test (Fixes #4362) 2015-01-23 21:23:40 +02:00
clipsyndicate.py
cloudy.py
clubic.py
cmt.py
cnet.py
cnn.py [cnn] Use edition.cnn.com for getting the information (fixes #4757) 2015-01-21 10:31:57 +01:00
collegehumor.py
collegerama.py [collegerama] Add extractor (#4540) 2015-01-11 00:40:46 +06:00
comcarcoff.py
comedycentral.py Fix flake8 errors 2015-01-31 10:51:39 +01:00
common.py Improve subtitles support 2015-02-16 21:51:03 +01:00
commonmistakes.py [commonmistakes] Detect BOMs at the beginning of URLs 2015-02-10 01:40:55 +01:00
condenast.py
cracked.py
criterion.py
crunchyroll.py [crunchyroll] Fix format extraction 2015-01-02 21:17:10 +01:00
cspan.py
ctsnews.py [ctsnews] Remove unused import 2015-01-30 03:09:51 +01:00
dailymotion.py
daum.py
dbtv.py
dctp.py [dctptv] Skip rtmp download 2015-02-11 22:10:33 +06:00
deezer.py
defense.py Remove unused imports 2015-02-01 15:08:50 +01:00
dfb.py
discovery.py [discovery] Fix extractor 2015-01-08 18:03:29 +01:00
divxstage.py
dotsub.py [dotsub] Fix extraction and modernize 2015-02-11 22:33:03 +06:00
drbonanza.py [drbonanza] Simplify and fix duration (#4687) 2015-01-11 23:41:55 +01:00
dreisat.py
dropbox.py
drtuber.py [drtuber] Add one more title regex 2015-02-14 18:50:13 +06:00
drtv.py [drtv] Extract material id (Closes #4814) 2015-02-02 21:11:25 +06:00
dump.py
dvtv.py
ebaumsworld.py
echomsk.py
ehow.py
eighttracks.py [eighttracks] PEP8 2015-01-10 14:25:11 +01:00
einthusan.py
eitb.py
ellentv.py [ellentv] Improve extraction 2015-01-03 21:54:18 +06:00
elpais.py [elpais] Modernize 2015-01-09 22:43:49 +01:00
empflix.py
engadget.py
eporner.py
eroprofile.py
escapist.py [escapist] Filter video differently (Fixes #4919) 2015-02-10 15:55:51 +01:00
everyonesmixtape.py
exfm.py
expotv.py
extremetube.py
facebook.py
faz.py
fc2.py Merge remote-tracking branch 'h-collector/master' 2015-01-25 03:48:26 +01:00
firedrive.py
firstpost.py [firstpost] Modernize 2015-02-10 05:28:48 +01:00
firsttv.py Remove unused imports 2015-02-14 22:19:58 +01:00
fivemin.py
fktv.py [fktv] Fix download URL 2015-01-08 16:14:50 +01:00
flickr.py
folketinget.py Test rtmpdump on travis (Fixes #1601) 2015-01-25 04:56:32 +01:00
fourtube.py [fourtube] Fix extraction 2015-01-16 13:44:44 +01:00
foxgay.py
foxnews.py
franceculture.py [franceculture] Rewrite for new HTML scheme (Fixes #4853) 2015-02-03 10:17:13 +01:00
franceinter.py
francetv.py [france2.fr:generation-quoi] Modernize 2015-02-01 15:06:55 +01:00
freesound.py
freespeech.py
freevideo.py
funnyordie.py
gamekings.py [gamekings] Support videos from news pages 2015-02-08 23:12:59 +06:00
gameone.py [gameone] Modernize 2015-01-07 07:37:21 +01:00
gamespot.py
gamestar.py [gamestar] Modernize slightly 2015-01-23 01:34:24 +01:00
gametrailers.py
gdcvault.py [gdcvault] Skip test that is now restricted 2015-01-08 18:03:29 +01:00
generic.py [generic] Improve SBS detection (Fixes #4899) 2015-02-09 14:46:10 +01:00
giantbomb.py
giga.py [giga] Add extractor (Closes #4090) 2015-01-06 06:54:31 +06:00
glide.py
globo.py [globo] Properly extract m3u8 formats (#4346 #4832) 2015-02-01 04:36:24 +06:00
godtube.py
goldenmoustache.py
golem.py
googleplus.py
googlesearch.py
gorillavid.py
goshgay.py Use _family_friendly_search for determining age_limit 2015-02-08 17:45:38 +02:00
grooveshark.py The opening curly brace { is a regex reserved [control character](http://stackoverflow.com/a/400316/1106367), so it needs to be escaped. 2015-01-30 18:41:40 +11:00
groupon.py
hark.py
hearthisat.py [hearthisat] Correct error message 2015-01-22 18:15:04 +01:00
heise.py
hellporno.py
helsinki.py
hentaistigma.py
historicfilms.py [historicfilms] Add extractor (Closes #4825) 2015-02-02 20:52:37 +06:00
history.py [history] Add extractor (Closes #4934) 2015-02-15 04:57:52 +06:00
hitbox.py
hornbunny.py
hostingbulk.py
hotnewhiphop.py
howcast.py
howstuffworks.py
huffpost.py [huffpost] Make extraction more robust (Closes #4663) 2015-01-08 19:07:28 +06:00
hypem.py
iconosquare.py
ign.py
imdb.py [imdb] Remove test md5 2015-01-08 16:14:50 +01:00
ina.py
infoq.py
instagram.py
internetvideoarchive.py
iprima.py
ivi.py [ivi] Modernize 2015-01-28 23:58:14 +06:00
izlesene.py Use _family_friendly_search for determining age_limit 2015-02-08 17:45:38 +02:00
jadorecettepub.py
jeuxvideo.py
jove.py
jpopsukitv.py
jukebox.py
kankan.py [kankan] Modernize 2015-02-01 15:03:55 +01:00
karaoketv.py [karaoketv] Remove unused import 2015-01-11 10:48:20 +01:00
keek.py
keezmovies.py [keezmovies] Fix extraction and modernize test 2015-02-01 15:13:44 +01:00
khanacademy.py [khanacademy] Update test 2015-01-05 12:28:35 +01:00
kickstarter.py
kontrtube.py [kontrtube] Extract display_id 2015-01-02 22:28:48 +06:00
krasview.py [krasview] Fix extraction 2015-01-25 05:21:39 +06:00
ku6.py
la7.py Remove unused imports 2015-02-01 15:08:50 +01:00
laola1tv.py
lifenews.py
liveleak.py [liveleak] Add original videos (Fixes #4768) 2015-01-23 17:22:14 +01:00
livestream.py
lnkgo.py [lnkgo] Adapt to website changes 2015-01-27 15:38:54 +02:00
lrt.py [lrt] Fix missing provider key 2015-01-05 02:55:12 +02:00
lynda.py [lynda] Pre-test video URLs for HTTP errors (Closes #2185, closes #4782) 2015-01-26 00:33:42 +06:00
m6.py
macgamestore.py [macgamestore] Modernize 2015-02-01 15:08:33 +01:00
mailru.py
malemotion.py
mdr.py
metacafe.py
metacritic.py
mgoon.py
minhateca.py
ministrygrid.py
mit.py [mit] Amend test definitions 2015-01-08 16:14:50 +01:00
mitele.py
mixcloud.py [mixcloud] Fix extraction (Closes #4862) 2015-02-04 19:47:55 +06:00
mlb.py
moevideo.py [moevideo] Skip removed video test 2015-01-01 00:46:03 +06:00
mofosex.py
mojvideo.py
moniker.py
mooshare.py
morningstar.py
motherless.py
motorsport.py [motorsport] Fix extraction and make trailing '/' optional 2015-01-05 19:19:01 +01:00
movieclips.py
moviezine.py
movshare.py
mpora.py [mpora] Modernize 2015-02-01 11:58:37 +01:00
mtv.py [mtv] Modernize and clean up test 2015-02-01 12:08:21 +01:00
muenchentv.py
musicplayon.py
musicvault.py
muzu.py
myspace.py
myspass.py
myvideo.py
myvidster.py
naver.py
nba.py
nbc.py Remove unused imports 2015-02-14 22:19:58 +01:00
ndr.py
ndtv.py [ndtv] Modernize 2015-01-19 10:10:05 +01:00
nerdcubed.py [nerdcubed] Modernize test definition 2015-02-01 12:11:20 +01:00
nerdist.py [nerdist] Add new extractor (Fixes #4851) 2015-02-02 23:38:35 +01:00
netzkino.py [netzkino] Add new extractor (Fixes #4669) 2015-01-09 23:59:18 +01:00
newgrounds.py
newstube.py
nextmedia.py [NextMedia] Add new extractor 2015-01-29 03:18:53 +08:00
nfb.py
nfl.py [nfl] Add support for articles pages (fixes #4848) 2015-02-02 23:17:00 +01:00
nhl.py [nhl:news] Add extractor (Closes #4805) 2015-01-30 23:12:27 +06:00
niconico.py
ninegag.py
noco.py
normalboots.py [normalboots] Remove unused import 2015-02-02 23:38:45 +01:00
nosvideo.py
novamov.py
nowness.py
nowvideo.py
npo.py [npo:radio] Move to extractor to common npo place and add extractor for fragments 2015-02-13 01:36:54 +06:00
nrk.py [nrk] Improve subtitle support (#3092) 2015-01-10 17:46:01 +01:00
ntvde.py [n-tv.de] Use native m3u8 as best format 2015-02-02 21:57:48 +01:00
ntvru.py [ntvru] Adapt to new direct delivery and modernize (Closes #4918) 2015-02-10 21:35:34 +06:00
nuvid.py
nytimes.py
oktoberfesttv.py
ooyala.py
openfilm.py
orf.py [orf:oe1] Add konsole URL schema (Fixes #4675) 2015-01-10 14:27:27 +01:00
parliamentliveuk.py
patreon.py
pbs.py [pbs] Catch geoblocking errors (closes #4516) 2014-12-31 17:43:49 +01:00
phoenix.py
photobucket.py
planetaplay.py
played.py [played] Skip test 2015-01-02 22:31:55 +06:00
playfm.py
playvid.py
podomatic.py
pornhd.py [pornhd] Fix extraction (fixes #4915) 2015-02-10 03:41:31 +01:00
pornhub.py [pornhub] Detect private videos and emit an error message (Closes #4764) 2015-01-22 23:48:58 +01:00
pornotube.py
pornoxo.py
promptfile.py
prosiebensat1.py
pyvideo.py
quickvid.py
radiobremen.py [radiobremen] Make code more readable and more resilient to failures 2015-01-05 18:17:03 +01:00
radiode.py
radiofrance.py
rai.py
rbmaradio.py
redtube.py
restudy.py
reverbnation.py
ringtv.py [ringtv] Modernize test definition 2015-02-01 15:21:02 +01:00
ro220.py
rottentomatoes.py [rottentomatoes] Modernize test definition 2015-02-01 12:11:14 +01:00
roxwel.py
rtbf.py
rte.py [rte] PEP8 2015-01-10 17:59:07 +01:00
rtl2.py [rtl2] PEP8, simplify, make rtmp tests run (#470) 2015-01-25 18:09:48 +01:00
rtlnl.py Update rtlnl.py 2015-01-05 11:51:24 +01:00
rtlnow.py [rtlnow] Add test for @mmue's extension (#4908) 2015-02-09 10:47:19 +01:00
rtp.py [rtp] Construct regular HTTP download URLs (#4882) 2015-02-06 23:00:54 +02:00
rts.py [rts] Fix f4m and m3u8 extraction (Closes #4873) 2015-02-05 22:17:50 +06:00
rtve.py [rtve] Recognize mobile urls (fixes #4823) 2015-01-30 23:46:55 +01:00
ruhd.py
rutube.py [vk] Add support for rutube embeds (Fixes #4514) 2015-01-04 03:15:27 +01:00
rutv.py [rutv] Extract all m3u8 formats 2015-02-01 23:48:23 +06:00
sapo.py
savefrom.py
sbs.py
scivee.py
screencast.py
screencastomatic.py
screenwavemedia.py
servingsys.py [servingsys] Modernize 2015-02-01 15:18:52 +01:00
sexu.py
sexykarma.py [sexykarma] Add age_limit designation 2015-01-08 16:14:50 +01:00
shared.py
sharesix.py
sina.py [sina] Modernize and simplify 2015-02-01 15:16:35 +01:00
slideshare.py [slideshare] Fix extraction 2015-01-01 00:26:19 +06:00
slutload.py
smotri.py Smotri info extractor: removed unreachable code and updated old md5 for test video 2015-01-30 20:35:20 +11:00
snotr.py
sockshare.py
sohu.py
soundcloud.py [soundcloud:set] Fix test definition 2015-02-01 15:24:38 +01:00
soundgasm.py
southpark.py
space.py
spankwire.py
spiegel.py [spiegel] Use generalized formats pre-testing 2015-01-26 00:34:31 +06:00
spiegeltv.py
spike.py Fix flake8 errors 2015-01-31 10:51:39 +01:00
sport5.py
sportbox.py
sportdeutschland.py
srmediathek.py fix srmediathek description 2015-01-29 01:36:15 +01:00
stanfordoc.py
steam.py
streamcloud.py
streamcz.py [streamcz] Fix extraction (Closes #4940) 2015-02-14 17:48:04 +02:00
streetvoice.py [streetvoice] Improve 2015-01-21 22:53:51 +06:00
subtitles.py
sunporno.py [sunporno] Keep old video regex just in case 2015-02-14 18:33:52 +06:00
svtplay.py [svtplay] Correct test case 2015-02-09 16:05:01 +01:00
swrmediathek.py
syfy.py
sztvhu.py
tagesschau.py
tapely.py
tass.py
teachertube.py [teachertube] Modernize 2015-01-08 16:14:50 +01:00
teachingchannel.py
teamcoco.py [teamcoco] Fix video id extraction 2015-02-11 15:47:19 +02:00
techtalks.py
ted.py Improve subtitles support 2015-02-16 21:51:03 +01:00
telebruxelles.py
telecinco.py
telemb.py
teletask.py [teletask] Fix test definition 2015-02-01 15:25:33 +01:00
tenplay.py
testtube.py Add --list-thumbnails 2015-01-25 02:43:19 +01:00
testurl.py
tf1.py [tf1] Remove unused import 2015-01-04 14:06:23 +01:00
theonion.py
theplatform.py [theplatform] Add URL sign capability 2015-02-15 04:56:12 +06:00
thesixtyone.py
thisav.py
thvideo.py
tinypic.py [tinypic] Tweak VALID_URL regex (Closes #4754) 2015-01-21 02:15:28 +06:00
tlc.py
tmz.py
tnaflix.py [tnaflix] Make sure config URL has correct scheme 2015-01-01 00:12:41 +06:00
toutv.py [toutv] Modernize test definition 2015-02-01 15:01:33 +01:00
toypics.py
traileraddict.py
trilulilu.py [trilulilu] Add support for videos without category in the URL (Closes #4067) 2015-02-09 17:00:05 +02:00
trutube.py
tube8.py
tudou.py [tudou] Fix extraction 2015-01-08 18:03:29 +01:00
tumblr.py
tunein.py [tunein] Ignore reliability if it's >90% (#4097) 2015-01-08 16:14:50 +01:00
turbo.py
tutv.py
tvigle.py [tvigle] Add support for cloud URLs (Closes #4887) 2015-02-06 21:15:01 +06:00
tvp.py [tvp] Fix extraction 2015-01-19 23:00:22 +06:00
tvplay.py
tweakers.py [tweakers] Switch extraction to xspf playlist, extract all formats and meta (#4881) 2015-02-06 03:23:42 +06:00
twentyfourvideo.py
twitch.py [twitch] PEP8 2015-01-23 21:05:07 +01:00
ubu.py [ubu] Fix test and modernize 2015-01-25 05:23:21 +01:00
udemy.py
unistra.py
urort.py
ustream.py
vbox7.py
veehd.py [veehd] Update test 2015-01-11 16:20:39 +06:00
veoh.py
vesti.py
vevo.py [vevo] Restore SMIL support (#3656) 2015-02-02 22:48:12 +01:00
vgtv.py
vh1.py
vice.py
viddler.py [viddler] PEP8 2015-01-30 02:13:37 +01:00
videobam.py
videodetective.py
videofyme.py
videolecturesnet.py
videomega.py [extractors] Use http_headers for setting the User-Agent and the Referer 2015-01-24 18:23:53 +01:00
videopremium.py
videott.py [videott] Improve _VALID_URL and add test 2015-01-24 16:11:40 +06:00
videoweed.py
vidme.py
vidzi.py
vier.py [vier:videos] Tune _VALID_URL not to match single videos 2015-01-02 22:09:00 +06:00
viki.py [viki] Modernize 2015-01-07 07:21:24 +01:00
vimeo.py [vimeo:album] Add support for album passwords (Fixes #4917) 2015-02-10 04:53:21 +01:00
vimple.py [vimple] Replace tests 2015-01-05 11:54:14 +01:00
vine.py
vk.py [vk] Remove debug assert statement (fixes #4672, fixes #4514) 2015-01-09 20:13:53 +01:00
vodlocker.py
vporn.py
vrt.py
vube.py
vuclip.py
vulture.py
walla.py
washingtonpost.py [washingtonpost] Catch more UUIDs 2015-01-30 15:53:58 +01:00
wat.py
wayofthemaster.py
wdr.py [wdr] Complete test definition 2015-02-01 15:27:16 +01:00
webofstories.py [webofstories] Add new extractor (Closes #4585) 2015-01-05 02:22:01 +02:00
weibo.py
wimp.py
wistia.py
worldstarhiphop.py
wrzuta.py
wsj.py [wsj] Add new extractor (Fixes #4854) 2015-02-03 10:58:28 +01:00
xbef.py
xboxclips.py [xboxclips] Fix extraction 2015-01-11 15:25:29 +06:00
xhamster.py [xhamster] Remove identical tests 2015-01-02 21:12:25 +06:00
xminus.py
xnxx.py
xtube.py Respect age_limit when listing extractors (Fixes #4653) 2015-01-07 07:20:20 +01:00
xuite.py [xuite] Fix _VALID_URL 2015-01-29 23:11:13 +06:00
xvideos.py
xxxymovies.py
yahoo.py
yesjapan.py
ynet.py
youjizz.py
youku.py
youporn.py
yourupload.py
youtube.py [youtube] Extract average rating (closes #2362) 2015-02-11 18:39:31 +01:00
zdf.py [zdf] Fix test case 2015-01-08 16:14:50 +01:00
zingmp3.py