Browse Source

Add support for multiple TV info sources.

Changelog
---------
Change improve loading speed of shows at startup.
Change improve main execution loop speed.
Add force cast update to view show page.
Add person view.
Add character view.
Add characters, person to clean-up cache (30 days).
Add reload person, character images every 7 days.
Add suppress UI notification for scheduled people updates during show updates and during switching ids.
Add resume support of switched shows after restart that switched id but not finished updating.
Add failed TV info switches to show tasks page.
Add remove item from queue and clear queue test buttons to mange/show-tasks and manage/search-tasks.
Change improve show update logic.
Add to view-show page a notification message if a show fails to switch.
Add check for existing show with new id pair before switching.
Change prioritize first episode start year over the start year set at the tv info source.
Change delete non existing episodes when switching indexer.
Change "exists in db" link on search results page to support any info source.
Add TMDB person pics as fallback.
Add use person fallback for character images.
Add logic to add start, end year in case of multiple characters per person.
Change improve speed getting list in the import view.
Add abort people cast update when show is deleted, also remove show from any queued show item or search item
Support list of names to search for in show search.
Change assist user search terms when the actual title of a show is unknown.
Add support URLs in search.
Change remove year from search term ... year is still used for relevancy order.
Add updating show.nfo when a cast changes.
Change use longest biography available for output.
Add UI requests of details for feb 28 will also return feb 29 in years without feb 29.
Add fetch extra data fallback from TMDB for persons.

technical commit messages (combined commits)
--------------------------------------------
Add tmdb get_trending, get_popular, get_top_rated, discover to tvinfoapi.
Add home/get_persons ajax endpoint, currently supports: birthday, deathday, names.
Change daily-schedule to new get_episode_time - More TODO
Change view-show to use get_episode_time.
Add get show updates list in show updater.
Add get_episode_time to network_timezones.
Add airtime for episode and show to get_episode_time.
Small ep obj load performance improvement.
Add handle special episodes and assign numbers based on airdate.
Add handle tvmaze specials without airdate.
Change during switch tv info source, specials are removed from db because of non-existing generic numbering.
Change add first/latest regular episode, use first_aired_regular_episode in all places that have airdate of first episode.
Add IMDb to person interface.
Add akas to person.
Add IMDb bio parser.
Add TMDB api for people.
Add character role start/end year to IMDb api and tv character.
Fix updating characters with multiple persons, by limiting to one.
Add cache to imdb_api.py.
Add cache to tmdb_api: get_person, _search_person.
Add cache to trakt get_person, _search_person cache.
Improve main execution loop speed https://stackify.com/20-simple-python-performance-tuning-tips/ point: 12
Add network to episode for tvdb_api (from show data).
Add fallback for network/timezone for tv episode to tv show.
Add skip retrieve_exceptions for tv info sources that don't have 'scene_url'.
Change move network load before show load to make sure the tvshow obj timezone can be set (startup).
Add datetime.time to/from integer to sbdatetime (hour, minute only).
Add load all indexer mapping at once from db during startup.
Add load the failed count during startup.
Change move sanitize_filename to sg_helpers.
Change move download_file to sg_helpers.
Add list_tables, list_indexes to db.py.
Add multi db column add function.
Restore backup tables during upgrade.
Use lib.tvinfo_base import everywhere.
Add new properties to tvepisode.
Add show_updates to indexer endpoint.
Add new db schema.
Add Character and Persons tables.
Add tvmaze_api lib.
Add pytvmaze lib.
Add debug __repr__ to people/show queue.
Add crew to show tvinfo_base.
Drop backup tables for now.
Don't save switch refresh, update show queue items (since they are sub queues of switch).
Remove show from switched_shows in case it is in it when deleting the show.
Use switch queue for manual switch
Load/save updated time for person/Character.
Add show_queue table.
Add _get_item_sql, _delete_item_from_db_sql to search search_queue.
Add people_queue type.
Add people scheduler.
Add people queue.
Add tvinfo source switch queue item.
Alternate naming for person/character images.
Add load and save image/thumb urls for persons.
Add load character pics.
Add save images for characters.
Add save image urls for characters to db.
Change limit for person searches to 100 instead of 10 default.
Add person verification.
Add people_url and character_url to sources that support them.
Add save castlist changes.
Add remove old characters from castlist.
Change optimize find_show_by_id.
Change improve debug info for person, character obj.
Add db_support_upsert support flag in db.py.
Add cast list objs and load from db.
Add parse and add additional images to TVInfo.
Add optional loading of images and actors/crew.
Fix _make_timestamp in py2.
Save and load _src_update_time.
Change use update time for show updates.
Add _set_network optimization.
Add src_update_timestamp to tvshow tbl.
Add updated_timestamp to TVInfoShow.
Add _indexer_update_time TVShow obj.
Add support to search for external ids on TV info sources: [TVINFO_TVDB, TVINFO_IMDb, TVINFO_TMDB, TVINFO_TRAKT].
Change show tasks page, keep remove button for failed switches always visible.
Add use get_url for tvmaze_api interface (to support failure handling), without changing original lib.
Add missing settings in switch show.
Add new switch error:  TVSWITCH_ID_CONFLICT: 'new id conflicts with existing show'.
Add messages for manual switch id.
Add connection skip handling.
Add get_url failure handling to Trakt lib.
Add real search_person to tvinfo interface.
Change move TMDB api key to tmdb_api.
Add get_url usage via tmdb_api.
Change join/split of akas/join and sql group_concat to `;;;`  to prevent issues with names that may contain comma (`,`).
Add Trakt api specific failure times.
Add warning about reassigning MEMCACHE
Add calc_age to sg_helpers for person page to remove dupe code.
Add return age in ajax for persons.
Change format dates on person page to honour config/General.
Add convert_to_inch_fraction_html to sg_helpers.
Change direct assign method instead of wrapper (faster).
Change force disk_pickle_protocol=2 for py2 compatibility in tvinfo cache.
Add episode rename info during switch.
Add save deleted episodes when switching tvinfo source.
Add get_switch_changed page.
Add force=True to QueueItemRefresh during switch of tvinfo to force rewriting of metadata.
Add filter character dupes in tvdb_api only take images for unique character, role combos on tvdb.
Fix humanize lib in py27.
Add diskcache to tvinfo_base.
Add doc files for diskcache lib.
Add clean tvinfo to show_updater.
Add switch_ep_errors table to sickbeard.db
Add debug log message when extra data is fetched for a person.
Change make character loading more efficient.
Add placeholder image for characters and persons.
Change start, end year moved to new extra table to support multiple person per character.
Change replace start_year, end_year with persons_years in Character class.
Add new table character_person_years, change table structure characters.
Add check if show is found for switch pages.
Add missing scheduled to cache.db people queue table.
Change trakt show search now will ignore failure handling, to make sure it's always tried when searching for shows.
Change add/improve tvmaze id cross search.
Change improve search, if only ids are given, resulting seriesnames on source will be used as text search on following tvinfo sources.
Add new parameter: prefer_person to imagecache/character endpoint.
Add if prefer_person is set and person_id is set to valid person and the character has more then 1 person assigned the character image will not be returned, instead the actors image or the placeholder.
Add only take external ids if character is confirmed in logic.
Add match name instead of person id for checking same person in show when adding cast for sources without person id (tvdb)
Add cache tmdb genres directly in dict for performance.
tags/release_0.25.1
Prinz23 4 years ago
committed by JackDandy
parent
commit
47305d30d5
  1. BIN
      gui/slick/images/facebook16.png
  2. BIN
      gui/slick/images/instagram16.png
  3. BIN
      gui/slick/images/twitter16.png
  4. BIN
      gui/slick/images/wikipedia16.png
  5. 65
      gui/slick/interfaces/default/character.tmpl
  6. 23
      gui/slick/interfaces/default/displayShow.tmpl
  7. 12
      gui/slick/interfaces/default/home_massAddTable.tmpl
  8. 3
      gui/slick/interfaces/default/inc_displayShow.tmpl
  9. 5
      gui/slick/interfaces/default/inc_top.tmpl
  10. 12
      gui/slick/interfaces/default/manage_manageSearches.tmpl
  11. 109
      gui/slick/interfaces/default/manage_showProcesses.tmpl
  12. 81
      gui/slick/interfaces/default/person.tmpl
  13. 23
      gui/slick/interfaces/default/show_switch_errors.tmpl
  14. 25
      gui/slick/interfaces/default/show_switch_errors_episodes.tmpl
  15. 3
      gui/slick/js/editShow.js
  16. 14
      gui/slick/js/manageSearches.js
  17. 32
      gui/slick/js/manageShowProcesses.js
  18. 5
      lib/diskcache/__init__.pyi
  19. 0
      lib/diskcache/cli.pyi
  20. 112
      lib/diskcache/core.pyi
  21. 33
      lib/diskcache/djangocache.pyi
  22. 52
      lib/diskcache/fanout.pyi
  23. 81
      lib/diskcache/persistent.pyi
  24. 51
      lib/diskcache/recipes.pyi
  25. 4
      lib/exceptions_helper.py
  26. 0
      lib/imdb_api/__init__.py
  27. 156
      lib/imdb_api/imdb_api.py
  28. 62
      lib/imdb_api/imdb_exceptions.py
  29. 4
      lib/libtrakt/exceptions.py
  30. 269
      lib/libtrakt/indexerapiinterface.py
  31. 30
      lib/libtrakt/trakt.py
  32. 4
      lib/pytvmaze/__init__.py
  33. 36
      lib/pytvmaze/endpoints.py
  34. 166
      lib/pytvmaze/exceptions.py
  35. 1516
      lib/pytvmaze/tvmaze.py
  36. 204
      lib/sg_helpers.py
  37. 0
      lib/tmdb_api/__init__.py
  38. 348
      lib/tmdb_api/tmdb_api.py
  39. 62
      lib/tmdb_api/tmdb_exceptions.py
  40. 96
      lib/tvdb_api/tvdb_api.py
  41. 2
      lib/tvdb_api/tvdb_exceptions.py
  42. 563
      lib/tvinfo_base/base.py
  43. 12
      lib/tvinfo_base/exceptions.py
  44. 0
      lib/tvmaze_api/__init__.py
  45. 503
      lib/tvmaze_api/tvmaze_api.py
  46. 62
      lib/tvmaze_api/tvmaze_exceptions.py
  47. 42
      sickbeard/__init__.py
  48. 38
      sickbeard/databases/cache_db.py
  49. 190
      sickbeard/databases/mainDB.py
  50. 57
      sickbeard/db.py
  51. 183
      sickbeard/generic_queue.py
  52. 76
      sickbeard/helpers.py
  53. 103
      sickbeard/image_cache.py
  54. 34
      sickbeard/indexermapper.py
  55. 16
      sickbeard/indexers/indexer_api.py
  56. 84
      sickbeard/indexers/indexer_config.py
  57. 2
      sickbeard/indexers/indexer_exceptions.py
  58. 52
      sickbeard/metadata/generic.py
  59. 2
      sickbeard/metadata/kodi.py
  60. 2
      sickbeard/metadata/mede8er.py
  61. 2
      sickbeard/metadata/mediabrowser.py
  62. 2
      sickbeard/metadata/tivo.py
  63. 2
      sickbeard/metadata/wdtv.py
  64. 2
      sickbeard/metadata/xbmc_12plus.py
  65. 2
      sickbeard/name_parser/parser.py
  66. 106
      sickbeard/network_timezones.py
  67. 3
      sickbeard/notifiers/trakt.py
  68. 208
      sickbeard/people_queue.py
  69. 4
      sickbeard/scene_exceptions.py
  70. 7
      sickbeard/scheduler.py
  71. 223
      sickbeard/search_queue.py
  72. 9
      sickbeard/sgdatetime.py
  73. 813
      sickbeard/show_queue.py
  74. 26
      sickbeard/show_updater.py
  75. 2594
      sickbeard/tv.py
  76. 8
      sickbeard/tv_base.py
  77. 2
      sickbeard/webapi.py
  78. 912
      sickbeard/webserve.py
  79. 92
      sickgear.py

BIN
gui/slick/images/facebook16.png

Binary file not shown.

After

Width:  |  Height:  |  Size: 265 B

BIN
gui/slick/images/instagram16.png

Binary file not shown.

After

Width:  |  Height:  |  Size: 753 B

BIN
gui/slick/images/twitter16.png

Binary file not shown.

After

Width:  |  Height:  |  Size: 374 B

BIN
gui/slick/images/wikipedia16.png

Binary file not shown.

After

Width:  |  Height:  |  Size: 142 B

65
gui/slick/interfaces/default/character.tmpl

@ -0,0 +1,65 @@
#import sickbeard
#from sickbeard import TVInfoAPI
#from sickbeard.helpers import anon_url
#from six import iteritems
<% def sg_var(varname, default=False): return getattr(sickbeard, varname, default) %>#slurp#
<% def sg_str(varname, default=''): return getattr(sickbeard, varname, default) %>#slurp#
##
#set global $title = 'Character'
#set global $header = 'Character'
#set global $sbPath = '../..'
#set global $topmenu = 'Character'
##
#import os.path
#include $os.path.join($sickbeard.PROG_DIR, 'gui/slick/interfaces/default/inc_top.tmpl')
#if $varExists('header')
<h1 class="header">$header</h1>
#else
<h1 class="title">$title</h1>
#end if
##
#set $html_selected = ' selected="selected"'
#set $html_checked = ' checked="checked"'
<div id="character">
<div id="character-content" class="linefix">
<a href="$sbRoot/imagecache/character?character_id=$character.id&amp;show=$show_obj.tvid_prodid&amp;thumb=0" rel="dialog"><img src="$sbRoot/imagecache/character?character_id=$character.id&amp;show=$show_obj.tvid_prodid&amp;thumb=1" style="height: 300px;"></a>
<div style="font-weight: bolder;font-size: x-large;">$character.name</div>
#if $character.person
<div style="font-weight:lighter;font-size: large;">(
#set $p_count = len($character.person)
#for $p_nb, $person in enumerate($character.person, 1)
<a href="$sbRoot/home/person?person_id=$person.id" >$person.name</a>
#if $p_nb < $p_count
<span>, </span>
#end if
#end for
)</div>
#end if
<br />
<div><span style="font-weight: bolder; font-size: larger;">Bio:</span><br />$character.biography</div>
<br />
<div><span style="font-weight: bolder; font-size: larger;">Links:</span><br />
#for $src, $sid in sorted(iteritems($character.ids))
#if $TVInfoAPI($src).config.get('character_url')
<a href="$anon_url($TVInfoAPI($src).config['character_url'] % $sid)" target="_blank">
#if $TVInfoAPI($src).config.get('icon')
<img alt="$TVInfoAPI($src).name" height="16" width="16" src="$sbRoot/images/$TVInfoAPI($src).config['icon']">
#end if
$TVInfoAPI($src).name</a><br />
#end if
#end for
</div>
<br />
<div><span style="font-weight: bolder; font-size: x-large;">Shows featuring this Character in db:</span><br />
#for $character in $characters
<div style="margin: 10px;font-size: x-large;"><span style="width: 210px;display: inline-block;"><a href="$sbRoot/imagecache/character?character_id=$character['character_id']&amp;show=$character['show_obj'].tvid_prodid&amp;thumb=0" rel="dialog"><img src="$sbRoot/imagecache/character?character_id=$character['character_id']&amp;show=$character['show_obj'].tvid_prodid&amp;thumb=1" style="height: 200px;"></a></span>$character['character_name'] <span style="font-weight: lighter;">(<a href="$sbRoot/home/view-show?tvid_prodid=$character['show_obj'].tvid_prodid">$character['show_obj'].name</a>)</span></div>
#end for
</div>
</div>
</div>
<div></div>
#include $os.path.join($sickbeard.PROG_DIR, 'gui/slick/interfaces/default/inc_bottom.tmpl')

23
gui/slick/interfaces/default/displayShow.tmpl

@ -330,6 +330,29 @@
No plot overview available
#end if
#end if
#if $show_obj.cast_list
<div>
<br />
<p><span style="font-weight: bolder;">Cast:</span></p>
<div style="white-space: nowrap;overflow-x: auto;overflow-y: hidden;">
#for $character in $show_obj.cast_list
<div style="top:0;width:200px;height:auto;position:relative;display: inline-block;text-align: center;background-color: rgba(255,255,255,0.2)">
<a href="$sbRoot/imagecache/character?character_id=$character.id&amp;show=$show_obj.tvid_prodid&amp;thumb=0" rel="dialog">
<img src="$sbRoot/imagecache/character?character_id=$character.id&amp;show=$show_obj.tvid_prodid" height="150" alt="$character.name" style="border: 2px solid #555;display: block;margin-left: auto;margin-right: auto;" >
</a><br /><span style="font-weight:bolder;"><a href="$sbRoot/home/character?character_id=$character.id&amp;show=$show_obj.tvid_prodid" >#if $character.name#$character.name#else#Unknown Name#end if#</a></span><br />
#set $p_count = len($character.person)
#for $p_nb, $person in enumerate($character.person, 1)
<span style="color: #000000"><a href="$sbRoot/home/person?person_id=$person.id" >$person.name</a>
#if $p_nb < $p_count
<span>, </span>
#end if
</span>
#end for
</div>
#end for
</div>
</div>
#end if
</div>
<div id="details-bottom">

12
gui/slick/interfaces/default/home_massAddTable.tmpl

@ -29,12 +29,6 @@
#continue
#end if
#set $show_id = $curDir['dir']
#if $curDir['existing_info'][1]
#set $show_id = $show_id + '|' + $str($curDir['existing_info'][1]) + '|' + $str($curDir['existing_info'][2])
#set $indexer = $curDir['existing_info'][0]
#end if
#set $indexer = 0
#if $curDir['existing_info'][1] and $sickbeard.TVInfoAPI($curDir['existing_info'][0]).config.get('active')
#set $indexer = $curDir['existing_info'][0]
@ -42,6 +36,12 @@
#set $indexer = $sickbeard.TVINFO_DEFAULT
#end if
#set $show_id = $curDir['dir']
#if $curDir['existing_info'][1]
#set $show_id = $show_id + '|' + $str($indexer) + ':' + $str($curDir['existing_info'][1]) + '|' + $str($curDir['existing_info'][2])
#set $indexer = $curDir['existing_info'][0]
#end if
<tr>
<td class="col-checkbox">
<input type="checkbox" id="$show_id" class="dirCheck"$state_checked>

3
gui/slick/interfaces/default/inc_displayShow.tmpl

@ -80,8 +80,9 @@
#if not $ep['name'] or 'TBA' == $ep['name']#<em${cls}>TBA</em>#else#$ep['name']#end if#
</td>
#set $ep_dt = $network_timezones.get_episode_time($ep['airdate'], $network_time, $show_obj.network, $show_obj.airtime, $network_timezone, $ep['timestamp'], $ep['network'], $ep['airtime'], $ep['timezone'])
<td class="col-airdate">
<span class="$fuzzydate" data-airdate="$ep['airdate']"#if $sg_var('FUZZY_DATING')# data-fulldate="$SGDatetime.sbfdate(dt=$datetime.date.fromordinal($ep['airdate']), d_preset='%A, %B %d, %Y')"#end if#>#if 1 == int($ep['airdate']) then 'never' else $SGDatetime.sbfdate($SGDatetime.convert_to_setting($network_timezones.parse_date_time($ep['airdate'], $network_time, $network_timezone)))#</span>
<span class="$fuzzydate addQTip" title="$SGDatetime.sbfdatetime($SGDatetime.convert_to_setting($ep_dt))" data-airdate="$ep['airdate']"#if $sg_var('FUZZY_DATING')# data-fulldate="$SGDatetime.sbfdate(dt=$ep_dt), d_preset='%A, %B %d, %Y')"#end if#>#if 1 == int($ep['airdate']) then 'never' else $SGDatetime.sbfdate($SGDatetime.convert_to_setting($ep_dt))#</span>
</td>
#if $sg_var('USE_SUBTITLES') and $show_obj.subtitles

5
gui/slick/interfaces/default/inc_top.tmpl

@ -352,6 +352,11 @@
#end if
</div>
#end if
#if $tvinfo_switch_running
<div class="alert alert-danger" role="alert">
<span>Shows are in the process of being switched to a new tv info source. Expect Links and elements not to work temporally.</span>
</div>
#end if
#if $sg_str('NEWEST_VERSION_STRING')
<div class="alert alert-success upgrade-notification" role="alert">
<span>$sg_str('NEWEST_VERSION_STRING')</span>

12
gui/slick/interfaces/default/manage_manageSearches.tmpl

@ -1,9 +1,11 @@
#import sickbeard
#import os
##
#set global $title = 'Search Tasks'
#set global $header = 'Search Tasks'
#set global $sbPath = '..'
#set global $topmenu = 'manage'
#set $dev_mode = os.environ.get('SG_DEV_MODE')
##
#import os.path
#include $os.path.join($sickbeard.PROG_DIR, 'gui/slick/interfaces/default/inc_top.tmpl')
@ -110,7 +112,7 @@
<div id="queue-backlog" class="section">
Backlog: <i>$len($queue_length['backlog']) item$sickbeard.helpers.maybe_plural($len($queue_length['backlog']))</i>
Backlog: <i>$len($queue_length['backlog']) item$sickbeard.helpers.maybe_plural($len($queue_length['backlog']))</i>#if $dev_mode #<input type="button" class="btn" id="clear-btn-backlog" value="Clear" data-action="$sickbeard.search_queue.BACKLOG_SEARCH">#end if#
#if $queue_length['backlog']
<input type="button" class="shows-more btn" id="backlog-btn-more" value="Expand" #if not $queue_length['backlog']# style="display:none" #end if#><input type="button" class="shows-less btn" id="backlog-btn-less" value="Collapse" style="display:none"><br>
<table class="sickbeardTable manageTable" cellspacing="1" border="0" cellpadding="0" style="display:none">
@ -138,7 +140,7 @@
<td style="width:80%;text-align:left;color:white">
<a class="whitelink" href="$sbRoot/home/view-show?tvid_prodid=$cur_item['tvid_prodid']">$cur_item['name']</a> - $sickbeard.helpers.make_search_segment_html_string($cur_item['segment'])
</td>
<td style="width:20%;text-align:center;color:white">$search_type</td>
<td style="width:20%;text-align:center;color:white">$search_type#if $dev_mode #<input type="button" class="btn" id="remove-btn-$cur_item['uid']" value="Remove" data-uid="$cur_item['uid']">#end if#</td>
</tr>
#end for
</tbody>
@ -148,7 +150,7 @@
<div id="queue-manual" class="section">
Manual: <i>$len($queue_length['manual']) item$sickbeard.helpers.maybe_plural($len($queue_length['manual']))</i>
Manual: <i>$len($queue_length['manual']) item$sickbeard.helpers.maybe_plural($len($queue_length['manual']))</i> #if $dev_mode #<input type="button" class="btn" id="clear-btn-manual" value="Clear" data-action="$sickbeard.search_queue.MANUAL_SEARCH">#end if#
#if $queue_length['manual']
<input type="button" class="shows-more btn" id="manual-btn-more" value="Expand" #if not $queue_length['manual']# style="display:none" #end if#><input type="button" class="shows-less btn" id="manual-btn-less" value="Collapse" style="display:none"><br>
<table class="sickbeardTable manageTable" cellspacing="1" border="0" cellpadding="0" style="display:none">
@ -159,6 +161,7 @@
<tr class="#echo ('odd', 'even')[$row % 2]##set $row+=1#">
<td style="width:100%;text-align:left;color:white">
<a class="whitelink" href="$sbRoot/home/view-show?tvid_prodid=$cur_item['tvid_prodid']">$cur_item['name']</a> - $sickbeard.helpers.make_search_segment_html_string($cur_item['segment'])
#if $dev_mode #<input type="button" class="btn" id="remove-btn-$cur_item['uid']" value="Remove" data-uid="$cur_item['uid']">#end if#
</td>
</tr>
#end for
@ -169,7 +172,7 @@
<div id="queue-failed" class="section">
Failed: <i>$len($queue_length['failed']) item$sickbeard.helpers.maybe_plural($len($queue_length['failed']))</i>
Failed: <i>$len($queue_length['failed']) item$sickbeard.helpers.maybe_plural($len($queue_length['failed']))</i> #if $dev_mode #<input type="button" class="btn" id="clear-btn-failed" value="Clear" data-action="$sickbeard.search_queue.FAILED_SEARCH">#end if#
#if $queue_length['failed']
<input type="button" class="shows-more btn" id="failed-btn-more" value="Expand" #if not $queue_length['failed']# style="display:none" #end if#><input type="button" class="shows-less btn" id="failed-btn-less" value="Collapse" style="display:none"><br>
<table class="sickbeardTable manageTable" cellspacing="1" border="0" cellpadding="0" style="display:none">
@ -180,6 +183,7 @@
<tr class="#echo ('odd', 'even')[$row % 2]##set $row+=1#">
<td style="width:100%;text-align:left;color:white">
<a class="whitelink" href="$sbRoot/home/view-show?tvid_prodid=$cur_item['tvid_prodid']">$cur_item['name']</a> - $sickbeard.helpers.make_search_segment_html_string($cur_item['segment'])
#if $dev_mode #<input type="button" class="btn" id="remove-btn-$cur_item['uid']" value="Remove" data-uid="$cur_item['uid']">#end if#
</td>
</tr>
#end for

109
gui/slick/interfaces/default/manage_showProcesses.tmpl

@ -6,6 +6,7 @@
#set global $header = 'Show Tasks'
#set global $sbPath = '..'
#set global $topmenu = 'manage'
#set $dev_mode = os.environ.get('SG_DEV_MODE')
##
#import os.path
#include $os.path.join($sickbeard.PROG_DIR, 'gui/slick/interfaces/default/inc_top.tmpl')
@ -95,7 +96,7 @@
<input type="button" class="show-all-more btn" id="all-btn-more" value="Expand All"><input type="button" class="show-all-less btn" id="all-btn-less" value="Collapse All"><br>
#end if
<br>
Add: <i>$len($queue_length['add']) show$sickbeard.helpers.maybe_plural($len($queue_length['add']))</i>
Add: <i>$len($queue_length['add']) show$sickbeard.helpers.maybe_plural($len($queue_length['add']))</i> #if $dev_mode #<input type="button" class="btn" id="clear-btn-add" value="Clear" data-action="$sickbeard.show_queue.ShowQueueActions.ADD">#end if#
#if $queue_length['add']
<input type="button" class="shows-more btn" id="add-btn-more" value="Expand" #if not $queue_length['add']# style="display:none" #end if#><input type="button" class="shows-less btn" id="add-btn-less" value="Collapse" style="display:none"><br>
<table class="sickbeardTable manageTable" cellspacing="1" border="0" cellpadding="0" style="display:none">
@ -111,7 +112,7 @@
#set $show_name = str($cur_show['name'])
<tr class="#echo ('odd', 'even')[$row % 2]##set $row+=1#">
<td style="text-align:left;color:white">$show_name</td>
<td style="text-align:center;color:white">#if $cur_show['scheduled_update']#Scheduled#end if#</td>
<td style="text-align:center;color:white">#if $cur_show['scheduled_update']#Scheduled#end if##if $dev_mode #<input type="button" class="btn" id="remove-btn-$cur_show['uid']" value="Remove" data-uid="$cur_show['uid']">#end if#</td>
</tr>
#end for
</tbody>
@ -120,7 +121,7 @@
<br>
#end if
<br>
Update <span class="grey-text">(Forced / Forced Web)</span>: <i>$len($queue_length['update']) <span class="grey-text">($len($queue_length['forceupdate']) / $len($queue_length['forceupdateweb']))</span> show$sickbeard.helpers.maybe_plural($len($queue_length['update']))</i>
Update <span class="grey-text">(Forced / Forced Web)</span>: <i>$len($queue_length['update']) <span class="grey-text">($len($queue_length['forceupdate']) / $len($queue_length['forceupdateweb']))</span> show$sickbeard.helpers.maybe_plural($len($queue_length['update']))</i> #if $dev_mode #<input type="button" class="btn" id="clear-btn-update" value="Clear" data-action="$sickbeard.show_queue.ShowQueueActions.UPDATE">#end if#
#if $queue_length['update']
<input type="button" class="shows-more btn" id="update-btn-more" value="Expand" #if not $queue_length['update']# style="display:none" #end if#><input type="button" class="shows-less btn" id="update-btn-less" value="Collapse" style="display:none"><br>
<table class="sickbeardTable manageTable" cellspacing="1" border="0" cellpadding="0" style="display:none">
@ -144,7 +145,7 @@
<td style="text-align:left">
<a class="whitelink" href="$sbRoot/home/view-show?tvid_prodid=$cur_show['tvid_prodid']">$show_name</a>
</td>
<td style="text-align:center;color:white">#if $cur_show['scheduled_update']#Scheduled, #end if#$cur_show['update_type']</td>
<td style="text-align:center;color:white">#if $cur_show['scheduled_update']#Scheduled, #end if#$cur_show['update_type']#if $dev_mode #<input type="button" class="btn" id="remove-btn-$cur_show['uid']" value="Remove" data-uid="$cur_show['uid']">#end if#</td>
</tr>
#end for
</tbody>
@ -153,7 +154,7 @@
<br>
#end if
<br>
Refresh: <i>$len($queue_length['refresh']) show$sickbeard.helpers.maybe_plural($len($queue_length['refresh']))</i>
Refresh: <i>$len($queue_length['refresh']) show$sickbeard.helpers.maybe_plural($len($queue_length['refresh']))</i> #if $dev_mode #<input type="button" class="btn" id="clear-btn-refreah" value="Clear" data-action="$sickbeard.show_queue.ShowQueueActions.REFRESH">#end if#
#if $queue_length['refresh']
<input type="button" class="shows-more btn" id="refresh-btn-more" value="Expand" #if not $queue_length['refresh']# style="display:none" #end if#><input type="button" class="shows-less btn" id="refresh-btn-less" value="Collapse" style="display:none"><br>
<table class="sickbeardTable manageTable" cellspacing="1" border="0" cellpadding="0" style="display:none">
@ -170,7 +171,7 @@
<td style="text-align:left">
<a class="whitelink" href="$sbRoot/home/view-show?tvid_prodid=$cur_show['tvid_prodid']">$cur_show['name']</a>
</td>
<td style="text-align:center;color:white">#if $cur_show['scheduled_update']#Scheduled#end if#</td>
<td style="text-align:center;color:white">#if $cur_show['scheduled_update']#Scheduled#end if##if $dev_mode #<input type="button" class="btn" id="remove-btn-$cur_show['uid']" value="Remove" data-uid="$cur_show['uid']">#end if#</td>
</tr>
#end for
</tbody>
@ -179,7 +180,94 @@
<br>
#end if
<br>
Rename: <i>$len($queue_length['rename']) show$sickbeard.helpers.maybe_plural($len($queue_length['rename']))</i>
Switch Source: <i>$len($queue_length['switch']) show$sickbeard.helpers.maybe_plural($len($queue_length['switch']))</i> #if $dev_mode #<input type="button" class="btn" id="clear-btn-swtich" value="Clear" data-action="$sickbeard.show_queue.ShowQueueActions.SWITCH">#end if#
#if $queue_length['switch']
<input type="button" class="shows-more btn" id="switch-btn-more" value="Expand" #if not $queue_length['switch']# style="display:none" #end if#><input type="button" class="shows-less btn" id="switch-btn-less" value="Collapse" style="display:none"><br>
<table class="sickbeardTable manageTable" cellspacing="1" border="0" cellpadding="0" style="display:none">
<thead>
<tr>
<th style="width:80%;text-align:left">Show name</th>
<th style="width:20%">New TV Source</th>
</tr>
</thead>
<tbody>
#set $row = 0
#for $cur_show in $queue_length['switch']:
<tr class="#echo ('odd', 'even')[$row % 2]##set $row+=1#">
<td style="text-align:left">
<a class="whitelink" href="$sbRoot/home/view-show?tvid_prodid=$cur_show['tvid_prodid']">$cur_show['name']</a>
</td>
<td style="text-align:center;color:white">$sickbeard.TVInfoAPI($cur_show['new_tvid']).name#if $dev_mode #<input type="button" class="btn" id="remove-btn-$cur_show['uid']" value="Remove" data-uid="$cur_show['uid']">#end if#</td>
</tr>
#end for
</tbody>
</table>
#else
<br>
#end if
<br>
Switch Failed: <i>$len($failed_switch) show$sickbeard.helpers.maybe_plural($len($failed_switch))</i>
#if $failed_switch
<input type="button" class="shows-more btn" id="switch-failed-btn-more" value="Expand" #if not $failed_switch# style="display:none" #end if#><input type="button" class="shows-less btn" id="switch-failed-btn-less" value="Collapse" style="display:none"><br>
<table class="sickbeardTable manageTable" cellspacing="1" border="0" cellpadding="0" style="display:none">
<thead>
<tr>
<th style="width:60%;text-align:left">Show name</th>
<th style="width:20%">New TV Source</th>
<th style="width:20%">Fail Reason</th>
</tr>
</thead>
<tbody>
#set $row = 0
#for $cur_show in $failed_switch:
<tr class="#echo ('odd', 'even')[$row % 2]##set $row+=1#">
<td style="text-align:left">
#if $cur_show['show_obj']
<a class="whitelink" href="$sbRoot/home/view-show?tvid_prodid=$cur_show['show_obj'].tvid_prodid">$cur_show['show_obj'].name</a>
#else
<span>Unknown Show: $cur_show['old_tvid']:$cur_show['old_prodid']</span>
#end if
</td>
<td style="text-align:center;color:white">$sickbeard.TVInfoAPI($cur_show['new_tvid']).name</td>
<td style="text-align: center;color:white">
$cur_show['status'] <input type="button" class="btn" id="remove-btn-$cur_show['uid']" value="Remove" data-uid="$cur_show['uid']" data-force="1">
</td>
</tr>
#end for
</tbody>
</table>
#else
<br>
#end if
<br>
People: <i>$len($people_queue['main_cast']) show$sickbeard.helpers.maybe_plural($len($people_queue['main_cast']))</i> #if $dev_mode #<input type="button" class="btn" id="clear-people-btn" value="Clear" data-action="$sickbeard.people_queue.PeopleQueueActions.SHOWCAST">#end if#
#if $people_queue['main_cast']
<input type="button" class="shows-more btn" id="main_cast-btn-more" value="Expand" #if not $people_queue['main_cast']# style="display:none" #end if#><input type="button" class="shows-less btn" id="main_cast-btn-less" value="Collapse" style="display:none"><br>
<table class="sickbeardTable manageTable" cellspacing="1" border="0" cellpadding="0" style="display:none">
<thead>
<tr>
<th style="width:80%;text-align:left">Show name</th>
<th style="width:20%">People type</th>
</tr>
</thead>
<tbody>
#set $row = 0
#for $cur_show in $people_queue['main_cast']:
<tr class="#echo ('odd', 'even')[$row % 2]##set $row+=1#">
<td style="text-align:left">
<a class="whitelink" href="$sbRoot/home/view-show?tvid_prodid=$cur_show['tvid_prodid']">$cur_show['name']</a>
</td>
<td style="text-align:center;color:white">Main Cast#if $dev_mode #<input type="button" class="btn" id="remove-people-btn-$cur_show['uid']" value="Remove" data-uid="$cur_show['uid']">#end if#</td>
</tr>
#end for
</tbody>
</table>
#else
<br>
#end if
<br>
Rename: <i>$len($queue_length['rename']) show$sickbeard.helpers.maybe_plural($len($queue_length['rename']))</i> #if $dev_mode #<input type="button" class="btn" id="clear-btn-rename" value="Clear" data-action="$sickbeard.show_queue.ShowQueueActions.RENAME">#end if#
#if $queue_length['rename']
<input type="button" class="shows-more btn" id="rename-btn-more" value="Expand" #if not $queue_length['rename']# style="display:none" #end if#><input type="button" class="shows-less btn" id="rename-btn-less" value="Collapse" style="display:none"><br>
@ -197,7 +285,7 @@
<td style="text-align:left">
<a class="whitelink" href="$sbRoot/home/view-show?tvid_prodid=$cur_show['tvid_prodid']">$cur_show['name']</a>
</td>
<td style="text-align:center;color:white">#if $cur_show['scheduled_update']#Scheduled#end if#</td>
<td style="text-align:center;color:white">#if $cur_show['scheduled_update']#Scheduled#end if##if $dev_mode #<input type="button" class="btn" id="remove-btn-$cur_show['uid']" value="Remove" data-uid="$cur_show['uid']">#end if#</td>
</tr>
#end for
</tbody>
@ -207,7 +295,7 @@
#end if
#if $sickbeard.USE_SUBTITLES
<br>
Subtitle: <i>$len($queue_length['subtitle']) show$sickbeard.helpers.maybe_plural($len($queue_length['subtitle']))</i>
Subtitle: <i>$len($queue_length['subtitle']) show$sickbeard.helpers.maybe_plural($len($queue_length['subtitle']))</i> #if $dev_mode #<input type="button" class="btn" id="clear-btn-subtitles" value="Clear" data-action="$sickbeard.show_queue.ShowQueueActions.SUBTITLE">#end if#
#if $queue_length['subtitle']
<input type="button" class="shows-more btn" id="subtitle-btn-more" value="Expand" #if not $queue_length['subtitle']# style="display:none" #end if#><input type="button" class="shows-less btn" id="subtitle-btn-less" value="Collapse" style="display:none"><br>
<table class="sickbeardTable manageTable" cellspacing="1" border="0" cellpadding="0" style="display:none">
@ -224,7 +312,7 @@
<td style="text-align:left">
<a class="whitelink" href="$sbRoot/home/view-show?tvid_prodid=$cur_show['tvid_prodid']">$cur_show['name']</a>
</td>
<td style="text-align:center;color:white">#if $cur_show['scheduled_update']#Scheduled#end if#</td>
<td style="text-align:center;color:white">#if $cur_show['scheduled_update']#Scheduled#end if##if $dev_mode #<input type="button" class="btn" id="remove-btn-$cur_show['uid']" value="Remove" data-uid="$cur_show['uid']">#end if#</td>
</tr>
#end for
</tbody>
@ -233,6 +321,7 @@
<br>
#end if
#end if
</div>
</div>
#include $os.path.join($sickbeard.PROG_DIR, 'gui/slick/interfaces/default/inc_bottom.tmpl')

81
gui/slick/interfaces/default/person.tmpl

@ -0,0 +1,81 @@
#import sickbeard
#from sickbeard import TVInfoAPI
#from sickbeard.helpers import anon_url
#from sickbeard.tv import PersonGenders
#from six import iteritems
<% def sg_var(varname, default=False): return getattr(sickbeard, varname, default) %>#slurp#
<% def sg_str(varname, default=''): return getattr(sickbeard, varname, default) %>#slurp#
##
#set global $title = 'Person'
#set global $header = 'Person'
#set global $sbPath = '../..'
#set global $topmenu = 'person'
##
#import os.path
#include $os.path.join($sickbeard.PROG_DIR, 'gui/slick/interfaces/default/inc_top.tmpl')
#if $varExists('header')
<h1 class="header">$header</h1>
#else
<h1 class="title">$title</h1>
#end if
##
#set $html_selected = ' selected="selected"'
#set $html_checked = ' checked="checked"'
#set $age = $person.age
<div id="person">
<div id="person-content" class="linefix">
<a href="$sbRoot/imagecache/person?person_id=$person.id&amp;thumb=0" rel="dialog"><img src="$sbRoot/imagecache/person?person_id=$person.id&amp;thumb=1" style="height: 300px;"></a>
<div style="font-weight: bolder;font-size: x-large;">$person.name#if $age #<span> ($age)</span>#end if#</div>
#if $PersonGenders.UNKNOWN != $person.gender
<div><span style="font-weight: bolder;">Gender:</span> $PersonGenders.names.get($person.gender, 'unknown')</div>
#end if
#if $person.birthday
<div><span style="font-weight: bolder;">Birthdate:</span> $person.birthday</div>
#end if
#if $person.birthplace
<div><span style="font-weight: bolder;">Birthplace:</span> $person.birthplace</div>
#end if
#if $person.deathday
<div><span style="font-weight: bolder;">Deathdate:</span> $person.deathday</div>
#end if
<br />
<div><span style="font-weight: bolder; font-size: larger;">Bio:</span><br />$person.biography</div>
<br />
<div><span style="font-weight: bolder; font-size: larger;">Links:</span><br />
#for $src, $sid in sorted(iteritems($person.ids))
#if $TVInfoAPI($src).config.get('people_url')
<a href="$anon_url($TVInfoAPI($src).config['people_url'] % $sid)" target="_blank">
#if $TVInfoAPI($src).config.get('icon')
<img alt="$TVInfoAPI($src).name" height="16" width="16" src="$sbRoot/images/$TVInfoAPI($src).config['icon']">
#end if
$TVInfoAPI($src).name</a><br />
#end if
#end for
</div>
<br />
<div><span style="font-weight: bolder; font-size: x-large;">Characters in db:</span><br />
#for $character in $characters
#if $character.get('show_obj')
<div style="margin: 10px;font-size: x-large;"><span style="width: 210px;display: inline-block;"><a href="$sbRoot/imagecache/character?character_id=$character['character_id']&amp;show=$character['show_obj'].tvid_prodid&amp;thumb=0" rel="dialog"><img src="$sbRoot/imagecache/character?character_id=$character['character_id']&amp;show=$character['show_obj'].tvid_prodid&amp;thumb=1" style="height: 200px;"></a></span><a href="$sbRoot/home/character?character_id=$character['character_id']&amp;show=$character['show_obj'].tvid_prodid&amp;thumb=0" style="font-weight: bolder;">$character['character_name']</a> <span style="font-weight: lighter;">(<a href="$sbRoot/home/view-show?tvid_prodid=$character['show_obj'].tvid_prodid">$character['show_obj'].name</a>)</span>
#if $character.get('show_obj')
#set $first_aired = $character['show_obj'].first_aired_episode
#set $latest_aired = $character['show_obj'].latest_aired_episode
#if $first_aired
#set $from_age = $person.calc_age($first_aired.airdate)
#if $from_age
($from_age#if $latest_aired# - $person.calc_age($latest_aired.airdate)#end if# years)
#end if
#end if
#end if
</div>
#end if
#end for
</div>
</div>
</div>
<div></div>
#include $os.path.join($sickbeard.PROG_DIR, 'gui/slick/interfaces/default/inc_bottom.tmpl')

23
gui/slick/interfaces/default/show_switch_errors.tmpl

@ -0,0 +1,23 @@
#import datetime
#import sickbeard
#from sickbeard import TVInfoAPI
#from sickbeard.helpers import anon_url
#from sickbeard.tv import PersonGenders
#from six import iteritems
<% def sg_var(varname, default=False): return getattr(sickbeard, varname, default) %>#slurp#
<% def sg_str(varname, default=''): return getattr(sickbeard, varname, default) %>#slurp#
##
#set global $title = 'Show Switch Errors'
#set global $header = 'Show Switch Errors'
#set global $sbPath = '../..'
#set global $topmenu = 'show'
##
#import os.path
#include $os.path.join($sickbeard.PROG_DIR, 'gui/slick/interfaces/default/inc_top.tmpl')
<h4>Shows with changed episodes</h4>
#for $show_obj, $show_data in $iteritems($show_list)
<div><a href="$sbRoot/home/view-show?tvid_prodid=$show_obj.tvid_prodid">$show_obj.name</a> - <a href="$sbRoot/home/get-switch-changed-episodes?tvid_prodid=$show_obj.tvid_prodid">($show_data.get('changed', 0) changed, $show_data.get('deleted', 0) deleted episodes)</a></div>
#end for
#include $os.path.join($sickbeard.PROG_DIR, 'gui/slick/interfaces/default/inc_bottom.tmpl')

25
gui/slick/interfaces/default/show_switch_errors_episodes.tmpl

@ -0,0 +1,25 @@
#import datetime
#import sickbeard
#from sickbeard import TVInfoAPI
#from sickbeard.helpers import anon_url
#from sickbeard.tv import PersonGenders
#from six import iteritems
<% def sg_var(varname, default=False): return getattr(sickbeard, varname, default) %>#slurp#
<% def sg_str(varname, default=''): return getattr(sickbeard, varname, default) %>#slurp#
##
#set global $title = 'Show Switch Changes'
#set global $header = 'Show Switch Changes'
#set global $sbPath = '../..'
#set global $topmenu = 'show'
##
#import os.path
#include $os.path.join($sickbeard.PROG_DIR, 'gui/slick/interfaces/default/inc_top.tmpl')
<h4>$show_obj.name - Changes</h4>
#for $episode in $ep_list
#set $ep_obj = $episode['ep_obj']
#set $ep_name = ($ep_obj and $ep_obj.name) or ''
<div>$episode['reason'] <a href="$sbRoot/home/view-show?tvid_prodid=$show_obj.tvid_prodid#season-$episode['season']">$ep_name #echo '%sx%s' % ($episode['season'], $episode['episode'])#</a></div>
#end for
#include $os.path.join($sickbeard.PROG_DIR, 'gui/slick/interfaces/default/inc_bottom.tmpl')

3
gui/slick/js/editShow.js

@ -261,7 +261,8 @@ $(document).ready(function () {
/** @namespace data.switch */
/** @namespace data.switch.mtvid_prodid */
if (!isMaster && data.hasOwnProperty('switch') && data.switch.hasOwnProperty('Success')) {
window.location.replace(sbRoot + '/home/view-show?tvid_prodid=' + data.mtvid_prodid);
window.location.replace(sbRoot + '/home/view-show?tvid_prodid=' + $('#tvid_prodid').val());
// window.location.replace(sbRoot + '/home/view-show?tvid_prodid=' + data.mtvid_prodid);
} else if ((0 < $('*[data-maybe-master=1]').length)
&& (((0 === $('[name^=set-master]').length) && (0 < $('*[data-maybe-master=1]').val()))
|| ((0 < $('[name^=set-master]').length) && (0 === $('*[data-maybe-master=1]').val())))) {

14
gui/slick/js/manageSearches.js

@ -34,4 +34,18 @@ $(function(){
$(this).hide();
$(this).nextAll('input:first').show();
});
$('input[id^="remove-btn-"]').click(function() {
var param = {'to_remove': $(this).data('uid')};
$.getJSON(sbRoot + '/manage/search-tasks/remove-from-search-queue', param)
.done(function(){
location.reload();
})
});
$('input[id^="clear-btn-"]').click(function() {
var param = {'search_type': $(this).data('action')};
$.getJSON(sbRoot + '/manage/search-tasks/clear-search-queue', param)
.done(function(){
location.reload();
})
});
});

32
gui/slick/js/manageShowProcesses.js

@ -25,6 +25,38 @@ $(document).ready(function() {
$(this).nextAll('input:first').show();
});
$('input[id^="remove-btn-"]').click(function() {
var param = {'to_remove': $(this).data('uid'), 'force': $(this).data('force') !== undefined};
$.getJSON(sbRoot + '/manage/show-tasks/remove-from-show-queue', param)
.done(function(){
location.reload();
})
});
$('input[id^="remove-people-btn-"]').click(function() {
var param = {'to_remove': $(this).data('uid')};
$.getJSON(sbRoot + '/manage/show-tasks/remove-from-people-queue', param)
.done(function(){
location.reload();
})
});
$('input[id^="clear-btn-"]').click(function() {
var param = {'show_type': $(this).data('action')};
$.getJSON(sbRoot + '/manage/show-tasks/clear-show-queue', param)
.done(function(){
location.reload();
})
});
$('input[id^="clear-people-btn"]').click(function() {
var param = {'people_type': $(this).data('action')};
$.getJSON(sbRoot + '/manage/show-tasks/clear-people-queue', param)
.done(function(){
location.reload();
})
});
function disableSaveBtn(state){
$('#save-nowarnicon').prop('disabled', state)
}

5
lib/diskcache/__init__.pyi

@ -0,0 +1,5 @@
from .core import Cache as Cache, DEFAULT_SETTINGS as DEFAULT_SETTINGS, Disk as Disk, ENOVAL as ENOVAL, EVICTION_POLICY as EVICTION_POLICY, EmptyDirWarning as EmptyDirWarning, JSONDisk as JSONDisk, Timeout as Timeout, UNKNOWN as UNKNOWN, UnknownFileWarning as UnknownFileWarning
from .djangocache import DjangoCache as DjangoCache
from .fanout import FanoutCache as FanoutCache
from .persistent import Deque as Deque, Index as Index
from .recipes import Averager as Averager, BoundedSemaphore as BoundedSemaphore, Lock as Lock, RLock as RLock, barrier as barrier, memoize_stampede as memoize_stampede, throttle as throttle

0
lib/diskcache/cli.pyi

112
lib/diskcache/core.pyi

@ -0,0 +1,112 @@
from typing import Any, Optional
def full_name(func: Any): ...
class WindowsError(Exception): ...
class Constant(tuple):
def __new__(cls, name: Any): ...
def __repr__(self): ...
DBNAME: str
ENOVAL: Any
UNKNOWN: Any
MODE_NONE: int
MODE_RAW: int
MODE_BINARY: int
MODE_TEXT: int
MODE_PICKLE: int
DEFAULT_SETTINGS: Any
METADATA: Any
EVICTION_POLICY: Any
class Disk:
_directory: Any = ...
min_file_size: Any = ...
pickle_protocol: Any = ...
def __init__(self, directory: Any, min_file_size: int = ..., pickle_protocol: int = ...) -> None: ...
def hash(self, key: Any): ...
def put(self, key: Any): ...
def get(self, key: Any, raw: Any): ...
def store(self, value: Any, read: Any, key: Any = ...): ...
def fetch(self, mode: Any, filename: Any, value: Any, read: Any): ...
def filename(self, key: Any = ..., value: Any = ...): ...
def remove(self, filename: Any) -> None: ...
class JSONDisk(Disk):
compress_level: Any = ...
def __init__(self, directory: Any, compress_level: int = ..., **kwargs: Any) -> None: ...
def put(self, key: Any): ...
def get(self, key: Any, raw: Any): ...
def store(self, value: Any, read: Any, key: Any = ...): ...
def fetch(self, mode: Any, filename: Any, value: Any, read: Any): ...
class Timeout(Exception): ...
class UnknownFileWarning(UserWarning): ...
class EmptyDirWarning(UserWarning): ...
def args_to_key(base: Any, args: Any, kwargs: Any, typed: Any): ...
class Cache:
_directory: Any = ...
_timeout: int = ...
_local: Any = ...
_txn_id: Any = ...
_disk: Any = ...
def __init__(self, directory: Optional[Any] = ..., timeout: int = ..., disk: Any = ..., **settings: Any) -> None: ...
@property
def directory(self): ...
@property
def timeout(self): ...
@property
def disk(self): ...
@property
def _con(self): ...
@property
def _sql(self): ...
@property
def _sql_retry(self): ...
def transact(self, retry: bool = ...) -> None: ...
def _transact(self, retry: bool = ..., filename: Optional[Any] = ...) -> None: ...
def set(self, key: Any, value: Any, expire: Optional[Any] = ..., read: bool = ..., tag: Optional[Any] = ..., retry: bool = ...): ...
def __setitem__(self, key: Any, value: Any) -> None: ...
def _row_update(self, rowid: Any, now: Any, columns: Any) -> None: ...
def _row_insert(self, key: Any, raw: Any, now: Any, columns: Any) -> None: ...
def _cull(self, now: Any, sql: Any, cleanup: Any, limit: Optional[Any] = ...) -> None: ...
def touch(self, key: Any, expire: Optional[Any] = ..., retry: bool = ...): ...
def add(self, key: Any, value: Any, expire: Optional[Any] = ..., read: bool = ..., tag: Optional[Any] = ..., retry: bool = ...): ...
def incr(self, key: Any, delta: int = ..., default: int = ..., retry: bool = ...): ...
def decr(self, key: Any, delta: int = ..., default: int = ..., retry: bool = ...): ...
def get(self, key: Any, default: Optional[Any] = ..., read: bool = ..., expire_time: bool = ..., tag: bool = ..., retry: bool = ...): ...
def __getitem__(self, key: Any): ...
def read(self, key: Any, retry: bool = ...): ...
def __contains__(self, key: Any): ...
def pop(self, key: Any, default: Optional[Any] = ..., expire_time: bool = ..., tag: bool = ..., retry: bool = ...): ...
def __delitem__(self, key: Any, retry: bool = ...): ...
def delete(self, key: Any, retry: bool = ...): ...
def push(self, value: Any, prefix: Optional[Any] = ..., side: str = ..., expire: Optional[Any] = ..., read: bool = ..., tag: Optional[Any] = ..., retry: bool = ...): ...
def pull(self, prefix: Optional[Any] = ..., default: Any = ..., side: str = ..., expire_time: bool = ..., tag: bool = ..., retry: bool = ...): ...
def peek(self, prefix: Optional[Any] = ..., default: Any = ..., side: str = ..., expire_time: bool = ..., tag: bool = ..., retry: bool = ...): ...
def peekitem(self, last: bool = ..., expire_time: bool = ..., tag: bool = ..., retry: bool = ...): ...
def memoize(self, name: Optional[Any] = ..., typed: bool = ..., expire: Optional[Any] = ..., tag: Optional[Any] = ...): ...
def check(self, fix: bool = ..., retry: bool = ...): ...
def create_tag_index(self) -> None: ...
def drop_tag_index(self) -> None: ...
def evict(self, tag: Any, retry: bool = ...): ...
def expire(self, now: Optional[Any] = ..., retry: bool = ...): ...
def cull(self, retry: bool = ...): ...
def clear(self, retry: bool = ...): ...
def _select_delete(self, select: Any, args: Any, row_index: int = ..., arg_index: int = ..., retry: bool = ...): ...
def iterkeys(self, reverse: bool = ...) -> None: ...
def _iter(self, ascending: bool = ...) -> None: ...
def __iter__(self) -> Any: ...
def __reversed__(self): ...
def stats(self, enable: bool = ..., reset: bool = ...): ...
def volume(self): ...
def close(self) -> None: ...
def __enter__(self): ...
def __exit__(self, *exception: Any) -> None: ...
def __len__(self): ...
def __getstate__(self): ...
def __setstate__(self, state: Any) -> None: ...
def reset(self, key: Any, value: Any = ..., update: bool = ...): ...

33
lib/diskcache/djangocache.pyi

@ -0,0 +1,33 @@
from .core import ENOVAL as ENOVAL, args_to_key as args_to_key, full_name as full_name
from .fanout import FanoutCache as FanoutCache
from django.core.cache.backends.base import BaseCache
from typing import Any, Optional
class DjangoCache(BaseCache):
_cache: Any = ...
def __init__(self, directory: Any, params: Any) -> None: ...
@property
def directory(self): ...
def cache(self, name: Any): ...
def deque(self, name: Any): ...
def index(self, name: Any): ...
def add(self, key: Any, value: Any, timeout: Any = ..., version: Optional[Any] = ..., read: bool = ..., tag: Optional[Any] = ..., retry: bool = ...): ...
def get(self, key: Any, default: Optional[Any] = ..., version: Optional[Any] = ..., read: bool = ..., expire_time: bool = ..., tag: bool = ..., retry: bool = ...): ...
def read(self, key: Any, version: Optional[Any] = ...): ...
def set(self, key: Any, value: Any, timeout: Any = ..., version: Optional[Any] = ..., read: bool = ..., tag: Optional[Any] = ..., retry: bool = ...): ...
def touch(self, key: Any, timeout: Any = ..., version: Optional[Any] = ..., retry: bool = ...): ...
def pop(self, key: Any, default: Optional[Any] = ..., version: Optional[Any] = ..., expire_time: bool = ..., tag: bool = ..., retry: bool = ...): ...
def delete(self, key: Any, version: Optional[Any] = ..., retry: bool = ...) -> None: ...
def incr(self, key: Any, delta: int = ..., version: Optional[Any] = ..., default: Optional[Any] = ..., retry: bool = ...): ...
def decr(self, key: Any, delta: int = ..., version: Optional[Any] = ..., default: Optional[Any] = ..., retry: bool = ...): ...
def has_key(self, key: Any, version: Optional[Any] = ...): ...
def expire(self): ...
def stats(self, enable: bool = ..., reset: bool = ...): ...
def create_tag_index(self) -> None: ...
def drop_tag_index(self) -> None: ...
def evict(self, tag: Any): ...
def cull(self): ...
def clear(self): ...
def close(self, **kwargs: Any) -> None: ...
def get_backend_timeout(self, timeout: Any = ...): ...
def memoize(self, name: Optional[Any] = ..., timeout: Any = ..., version: Optional[Any] = ..., typed: bool = ..., tag: Optional[Any] = ...): ...

52
lib/diskcache/fanout.pyi

@ -0,0 +1,52 @@
from .core import Cache as Cache, DEFAULT_SETTINGS as DEFAULT_SETTINGS, Disk as Disk, ENOVAL as ENOVAL, Timeout as Timeout
from .persistent import Deque as Deque, Index as Index
from typing import Any, Optional
class FanoutCache:
_count: Any = ...
_directory: Any = ...
_shards: Any = ...
_hash: Any = ...
_caches: Any = ...
_deques: Any = ...
_indexes: Any = ...
def __init__(self, directory: Optional[Any] = ..., shards: int = ..., timeout: float = ..., disk: Any = ..., **settings: Any) -> None: ...
@property
def directory(self): ...
def __getattr__(self, name: Any): ...
def transact(self, retry: bool = ...) -> None: ...
def set(self, key: Any, value: Any, expire: Optional[Any] = ..., read: bool = ..., tag: Optional[Any] = ..., retry: bool = ...): ...
def __setitem__(self, key: Any, value: Any) -> None: ...
def touch(self, key: Any, expire: Optional[Any] = ..., retry: bool = ...): ...
def add(self, key: Any, value: Any, expire: Optional[Any] = ..., read: bool = ..., tag: Optional[Any] = ..., retry: bool = ...): ...
def incr(self, key: Any, delta: int = ..., default: int = ..., retry: bool = ...): ...
def decr(self, key: Any, delta: int = ..., default: int = ..., retry: bool = ...): ...
def get(self, key: Any, default: Optional[Any] = ..., read: bool = ..., expire_time: bool = ..., tag: bool = ..., retry: bool = ...): ...
def __getitem__(self, key: Any): ...
def read(self, key: Any): ...
def __contains__(self, key: Any): ...
def pop(self, key: Any, default: Optional[Any] = ..., expire_time: bool = ..., tag: bool = ..., retry: bool = ...): ...
def delete(self, key: Any, retry: bool = ...): ...
def __delitem__(self, key: Any) -> None: ...
def check(self, fix: bool = ..., retry: bool = ...): ...
def expire(self, retry: bool = ...): ...
def create_tag_index(self) -> None: ...
def drop_tag_index(self) -> None: ...
def evict(self, tag: Any, retry: bool = ...): ...
def cull(self, retry: bool = ...): ...
def clear(self, retry: bool = ...): ...
def _remove(self, name: Any, args: Any = ..., retry: bool = ...): ...
def stats(self, enable: bool = ..., reset: bool = ...): ...
def volume(self): ...
def close(self) -> None: ...
def __enter__(self): ...
def __exit__(self, *exception: Any) -> None: ...
def __getstate__(self): ...
def __setstate__(self, state: Any) -> None: ...
def __iter__(self) -> Any: ...
def __reversed__(self): ...
def __len__(self): ...
def reset(self, key: Any, value: Any = ...): ...
def cache(self, name: Any): ...
def deque(self, name: Any): ...
def index(self, name: Any): ...

81
lib/diskcache/persistent.pyi

@ -0,0 +1,81 @@
from .core import Cache as Cache, ENOVAL as ENOVAL
from collections.abc import MutableMapping, Sequence
from typing import Any, Optional
def _make_compare(seq_op: Any, doc: Any): ...
class Deque(Sequence):
_cache: Any = ...
def __init__(self, iterable: Any = ..., directory: Optional[Any] = ...) -> None: ...
@classmethod
def fromcache(cls, cache: Any, iterable: Any = ...): ...
@property
def cache(self): ...
@property
def directory(self): ...
def _index(self, index: Any, func: Any): ...
def __getitem__(self, index: Any): ...
def __setitem__(self, index: Any, value: Any): ...
def __delitem__(self, index: Any) -> None: ...
def __repr__(self): ...
__eq__: Any = ...
__ne__: Any = ...
__lt__: Any = ...
__gt__: Any = ...
__le__: Any = ...
__ge__: Any = ...
def __iadd__(self, iterable: Any): ...
def __iter__(self) -> Any: ...
def __len__(self): ...
def __reversed__(self) -> None: ...
def __getstate__(self): ...
def __setstate__(self, state: Any) -> None: ...
def append(self, value: Any) -> None: ...
def appendleft(self, value: Any) -> None: ...
def clear(self) -> None: ...
def count(self, value: Any): ...
def extend(self, iterable: Any) -> None: ...
def extendleft(self, iterable: Any) -> None: ...
def peek(self): ...
def peekleft(self): ...
def pop(self): ...
def popleft(self): ...
def remove(self, value: Any) -> None: ...
def reverse(self) -> None: ...
def rotate(self, steps: int = ...) -> None: ...
__hash__: Any = ...
def transact(self) -> None: ...
class Index(MutableMapping):
_cache: Any = ...
def __init__(self, *args: Any, **kwargs: Any) -> None: ...
@classmethod
def fromcache(cls, cache: Any, *args: Any, **kwargs: Any): ...
@property
def cache(self): ...
@property
def directory(self): ...
def __getitem__(self, key: Any): ...
def __setitem__(self, key: Any, value: Any) -> None: ...
def __delitem__(self, key: Any) -> None: ...
def setdefault(self, key: Any, default: Optional[Any] = ...): ...
def peekitem(self, last: bool = ...): ...
def pop(self, key: Any, default: Any = ...): ...
def popitem(self, last: bool = ...): ...
def push(self, value: Any, prefix: Optional[Any] = ..., side: str = ...): ...
def pull(self, prefix: Optional[Any] = ..., default: Any = ..., side: str = ...): ...
def clear(self) -> None: ...
def __iter__(self) -> Any: ...
def __reversed__(self): ...
def __len__(self): ...
def keys(self): ...
def values(self): ...
def items(self): ...
__hash__: Any = ...
def __getstate__(self): ...
def __setstate__(self, state: Any) -> None: ...
def __eq__(self, other: Any) -> Any: ...
def __ne__(self, other: Any) -> Any: ...
def memoize(self, name: Optional[Any] = ..., typed: bool = ...): ...
def transact(self) -> None: ...
def __repr__(self): ...

51
lib/diskcache/recipes.pyi

@ -0,0 +1,51 @@
from .core import ENOVAL as ENOVAL, args_to_key as args_to_key, full_name as full_name
from typing import Any, Optional
class Averager:
_cache: Any = ...
_key: Any = ...
_expire: Any = ...
_tag: Any = ...
def __init__(self, cache: Any, key: Any, expire: Optional[Any] = ..., tag: Optional[Any] = ...) -> None: ...
def add(self, value: Any) -> None: ...
def get(self): ...
def pop(self): ...
class Lock:
_cache: Any = ...
_key: Any = ...
_expire: Any = ...
_tag: Any = ...
def __init__(self, cache: Any, key: Any, expire: Optional[Any] = ..., tag: Optional[Any] = ...) -> None: ...
def acquire(self) -> None: ...
def release(self) -> None: ...
def locked(self): ...
def __enter__(self) -> None: ...
def __exit__(self, *exc_info: Any) -> None: ...
class RLock:
_cache: Any = ...
_key: Any = ...
_expire: Any = ...
_tag: Any = ...
def __init__(self, cache: Any, key: Any, expire: Optional[Any] = ..., tag: Optional[Any] = ...) -> None: ...
def acquire(self) -> None: ...
def release(self) -> None: ...
def __enter__(self) -> None: ...
def __exit__(self, *exc_info: Any) -> None: ...
class BoundedSemaphore:
_cache: Any = ...
_key: Any = ...
_value: Any = ...
_expire: Any = ...
_tag: Any = ...
def __init__(self, cache: Any, key: Any, value: int = ..., expire: Optional[Any] = ..., tag: Optional[Any] = ...) -> None: ...
def acquire(self) -> None: ...
def release(self) -> None: ...
def __enter__(self) -> None: ...
def __exit__(self, *exc_info: Any) -> None: ...
def throttle(cache: Any, count: Any, seconds: Any, name: Optional[Any] = ..., expire: Optional[Any] = ..., tag: Optional[Any] = ..., time_func: Any = ..., sleep_func: Any = ...): ...
def barrier(cache: Any, lock_factory: Any, name: Optional[Any] = ..., expire: Optional[Any] = ..., tag: Optional[Any] = ...): ...
def memoize_stampede(cache: Any, expire: Any, name: Optional[Any] = ..., typed: bool = ..., tag: Optional[Any] = ..., beta: int = ...): ...

4
lib/exceptions_helper.py

@ -129,6 +129,10 @@ class CantUpdateException(SickBeardException):
"""The show can't be updated right now"""
class CantSwitchException(SickBeardException):
"""The show can't be switched right now"""
class PostProcessingFailed(SickBeardException):
"""Post-processing the episode failed"""

0
lib/imdb_api/__init__.py

156
lib/imdb_api/imdb_api.py

@ -0,0 +1,156 @@
# encoding:utf-8
# author:Prinz23
# project:imdb_api
__author__ = 'Prinz23'
__version__ = '1.0'
__api_version__ = '1.0.0'
import logging
import re
from .imdb_exceptions import *
from exceptions_helper import ex
from six import iteritems
from bs4_parser import BS4Parser
from lib import imdbpie
from lib.tvinfo_base.exceptions import BaseTVinfoShownotfound
from lib.tvinfo_base import TVInfoBase, TVINFO_TRAKT, TVINFO_TMDB, TVINFO_TVDB, TVINFO_TVRAGE, TVINFO_IMDB, \
Person, PersonGenders, TVINFO_TWITTER, TVINFO_FACEBOOK, TVINFO_WIKIPEDIA, TVINFO_INSTAGRAM, Character, TVInfoShow
from sg_helpers import get_url, try_int
from lib.dateutil.parser import parser
# noinspection PyUnreachableCode
if False:
from typing import Any, AnyStr, Dict, List, Optional, Union
from six import integer_types
tz_p = parser()
log = logging.getLogger('imdb.api')
log.addHandler(logging.NullHandler())
class IMDbIndexer(TVInfoBase):
# supported_id_searches = [TVINFO_IMDB]
supported_person_id_searches = [TVINFO_IMDB]
# noinspection PyUnusedLocal
# noinspection PyDefaultArgument
def __init__(self, *args, **kwargs):
super(IMDbIndexer, self).__init__(*args, **kwargs)
@staticmethod
def _convert_person(person_obj, filmography=None, bio=None):
if isinstance(person_obj, dict) and 'imdb_id' in person_obj:
imdb_id = try_int(re.search(r'(\d+)', person_obj['imdb_id']).group(1))
return Person(p_id=imdb_id, name=person_obj['name'], ids={TVINFO_IMDB: imdb_id})
characters = []
for known_for in (filmography and filmography['filmography']) or []:
if known_for['titleType'] not in ('tvSeries', 'tvMiniSeries'):
continue
for character in known_for.get('characters') or []:
show = TVInfoShow()
show.id = try_int(re.search(r'(\d+)', known_for.get('id')).group(1))
show.ids.imdb = show.id
show.seriesname = known_for.get('title')
show.firstaired = known_for.get('year')
characters.append(
Character(name=character, show=show, start_year=known_for.get('startYear'),
end_year=known_for.get('endYear'))
)
try:
birthdate = person_obj['base']['birthDate'] and tz_p.parse(person_obj['base']['birthDate']).date()
except (BaseException, Exception):
birthdate = None
try:
deathdate = person_obj['base']['deathDate'] and tz_p.parse(person_obj['base']['deathDate']).date()
except (BaseException, Exception):
deathdate = None
imdb_id = try_int(re.search(r'(\d+)', person_obj['id']).group(1))
return Person(p_id=imdb_id, name=person_obj['base'].get('name'), ids={TVINFO_IMDB: imdb_id},
gender=PersonGenders.imdb_map.get(person_obj['base'].get('gender'), PersonGenders.unknown),
image=person_obj['base'].get('image', {}).get('url'),
birthplace=person_obj['base'].get('birthPlace'), birthdate=birthdate, deathdate=deathdate,
height=person_obj['base'].get('heightCentimeters'), characters=characters,
deathplace=person_obj['base'].get('deathPlace'),
nicknames=set((person_obj['base'].get('nicknames') and person_obj['base'].get('nicknames'))
or []),
real_name=person_obj['base'].get('realName'),
akas=set((person_obj['base'].get('akas') and person_obj['base'].get('akas')) or []), bio=bio
)
def _search_person(self, name=None, ids=None):
# type: (AnyStr, Dict[integer_types, integer_types]) -> List[Person]
"""
search for person by name
:param name: name to search for
:param ids: dict of ids to search
:return: list of found person's
"""
results, ids = [], ids or {}
for tv_src in self.supported_person_id_searches:
if tv_src in ids:
if TVINFO_IMDB == tv_src:
try:
p = self.get_person(ids[tv_src])
except (BaseException, Exception):
p = None
if p:
results.append(p)
if name:
cache_name_key = 'p-name-%s' % name
is_none, ps = self._get_cache_entry(cache_name_key)
if None is ps and not is_none:
try:
ps = imdbpie.Imdb().search_for_name(name)
except (BaseException, Exception):
ps = None
self._set_cache_entry(cache_name_key, ps)
if ps:
for cp in ps:
if not any(1 for c in results if cp['imdb_id'] == 'nm%07d' % c.id):
results.append(self._convert_person(cp))
return results
def _get_bio(self, p_id):
try:
bio = get_url('https://www.imdb.com/name/nm%07d/bio' % p_id, headers={'Accept-Language': 'en'})
if not bio:
return
with BS4Parser(bio) as bio_item:
bv = bio_item.find(string='Mini Bio', recursive=True).find_next('p')
for a in bv.findAll('a'):
a.replaceWithChildren()
for b in bv.findAll('br'):
b.replaceWith('\n')
return bv.get_text().strip()
except (BaseException, Exception):
return
def get_person(self, p_id, get_show_credits=False, get_images=False, **kwargs):
# type: (integer_types, bool, bool, Any) -> Optional[Person]
if not p_id:
return
cache_main_key, cache_bio_key, cache_credits_key = 'p-main-%s' % p_id, 'p-bio-%s' % p_id, 'p-credits-%s' % p_id
is_none, p = self._get_cache_entry(cache_main_key)
if None is p and not is_none:
try:
p = imdbpie.Imdb().get_name(imdb_id='nm%07d' % p_id)
except (BaseException, Exception):
p = None
self._set_cache_entry(cache_main_key, p)
is_none, bio = self._get_cache_entry(cache_bio_key)
if None is bio and not is_none:
bio = self._get_bio(p_id)
self._set_cache_entry(cache_bio_key, bio)
fg = None
if get_show_credits:
is_none, fg = self._get_cache_entry(cache_credits_key)
if None is fg and not is_none:
try:
fg = imdbpie.Imdb().get_name_filmography(imdb_id='nm%07d' % p_id)
except (BaseException, Exception):
fg = None
self._set_cache_entry(cache_credits_key, fg)
if p:
return self._convert_person(p, filmography=fg, bio=bio)

62
lib/imdb_api/imdb_exceptions.py

@ -0,0 +1,62 @@
# encoding:utf-8
"""Custom exceptions used or raised by tvmaze_api
"""
__author__ = 'Prinz23'
__version__ = '1.0'
__all__ = ['IMDbException', 'IMDbError', 'IMDbUserabort', 'IMDbShownotfound',
'IMDbSeasonnotfound', 'IMDbEpisodenotfound', 'IMDbAttributenotfound', 'IMDbTokenexpired']
from lib.tvinfo_base.exceptions import *
class IMDbException(BaseTVinfoException):
"""Any exception generated by tvdb_api
"""
pass
class IMDbError(BaseTVinfoError, IMDbException):
"""An error with thetvdb.com (Cannot connect, for example)
"""
pass
class IMDbUserabort(BaseTVinfoUserabort, IMDbError):
"""User aborted the interactive selection (via
the q command, ^c etc)
"""
pass
class IMDbShownotfound(BaseTVinfoShownotfound, IMDbError):
"""Show cannot be found on thetvdb.com (non-existant show)
"""
pass
class IMDbSeasonnotfound(BaseTVinfoSeasonnotfound, IMDbError):
"""Season cannot be found on thetvdb.com
"""
pass
class IMDbEpisodenotfound(BaseTVinfoEpisodenotfound, IMDbError):
"""Episode cannot be found on thetvdb.com
"""
pass
class IMDbAttributenotfound(BaseTVinfoAttributenotfound, IMDbError):
"""Raised if an episode does not have the requested
attribute (such as a episode name)
"""
pass
class IMDbTokenexpired(BaseTVinfoAuthenticationerror, IMDbError):
"""token expired or missing thetvdb.com
"""
pass

4
lib/libtrakt/exceptions.py

@ -43,3 +43,7 @@ class TraktServerError(TraktException):
class TraktLockedUserAccount(TraktException):
pass
class TraktInvalidGrant(TraktException):
pass

269
lib/libtrakt/indexerapiinterface.py

@ -1,29 +1,55 @@
import logging
import re
from .exceptions import TraktException
from exceptions_helper import ex
from exceptions_helper import ConnectionSkipException, ex
from six import iteritems
from .trakt import TraktAPI
from tvinfo_base.exceptions import BaseTVinfoShownotfound
from tvinfo_base import TVInfoBase
from lib.tvinfo_base.exceptions import BaseTVinfoShownotfound
from lib.tvinfo_base import TVInfoBase, TVINFO_TRAKT, TVINFO_TMDB, TVINFO_TVDB, TVINFO_TVRAGE, TVINFO_IMDB, \
TVINFO_SLUG, Person, TVINFO_TWITTER, TVINFO_FACEBOOK, TVINFO_WIKIPEDIA, TVINFO_INSTAGRAM, Character, TVInfoShow, \
TVInfoIDs, TVINFO_TRAKT_SLUG
from sg_helpers import try_int
from lib.dateutil.parser import parser
# noinspection PyUnreachableCode
if False:
from typing import Any, AnyStr, List, Optional
from tvinfo_base import TVInfoShow
from typing import Any, AnyStr, Dict, List, Optional, Union
from six import integer_types
id_map = {
'trakt': TVINFO_TRAKT,
'slug': TVINFO_SLUG,
'tvdb': TVINFO_TVDB,
'imdb': TVINFO_IMDB,
'tmdb': TVINFO_TMDB,
'tvrage': TVINFO_TVRAGE
}
id_map_reverse = {v: k for k, v in iteritems(id_map)}
tz_p = parser()
log = logging.getLogger('libtrakt.api')
log.addHandler(logging.NullHandler())
def _convert_imdb_id(src, s_id):
if TVINFO_IMDB == src:
try:
return try_int(re.search(r'(\d+)', s_id).group(1), s_id)
except (BaseException, Exception):
pass
return s_id
class TraktSearchTypes(object):
text = 1
trakt_id = 'trakt'
trakt_slug = 'trakt_slug'
tvdb_id = 'tvdb'
imdb_id = 'imdb'
tmdb_id = 'tmdb'
tvrage_id = 'tvrage'
all = [text, trakt_id, tvdb_id, imdb_id, tmdb_id, tvrage_id]
all = [text, trakt_id, tvdb_id, imdb_id, tmdb_id, tvrage_id, trakt_slug]
def __init__(self):
pass
@ -42,6 +68,9 @@ class TraktResultTypes(object):
class TraktIndexer(TVInfoBase):
supported_id_searches = [TVINFO_TVDB, TVINFO_IMDB, TVINFO_TMDB, TVINFO_TRAKT, TVINFO_TRAKT_SLUG]
supported_person_id_searches = [TVINFO_TRAKT, TVINFO_IMDB, TVINFO_TMDB]
# noinspection PyUnusedLocal
# noinspection PyDefaultArgument
def __init__(self, custom_ui=None, sleep_retry=None, search_type=TraktSearchTypes.text,
@ -64,28 +93,81 @@ class TraktIndexer(TVInfoBase):
[x in TraktResultTypes.all for x in result_types]) else [TraktResultTypes.show],
})
def _search_show(self, name, **kwargs):
# type: (AnyStr, Optional[Any]) -> List[TVInfoShow]
@staticmethod
def _make_result_obj(shows, results):
if shows:
try:
for s in shows:
if s['ids']['trakt'] not in [i['ids'].trakt for i in results]:
s['id'] = s['ids']['trakt']
s['ids'] = TVInfoIDs(
trakt=s['ids']['trakt'], tvdb=s['ids']['tvdb'], tmdb=s['ids']['tmdb'],
imdb=s['ids']['imdb'] and try_int(s['ids']['imdb'].replace('tt', ''), None))
results.append(s)
except (BaseException, Exception) as e:
log.debug('Error creating result dict: %s' % ex(e))
def _search_show(self, name=None, ids=None, **kwargs):
# type: (AnyStr, Dict[integer_types, integer_types], Optional[Any]) -> List[TVInfoShow]
"""This searches Trakt for the series name,
If a custom_ui UI is configured, it uses this to select the correct
series.
"""
all_series = self.search(name)
if not isinstance(all_series, list):
all_series = [all_series]
results = []
if ids:
for t, p in iteritems(ids):
if t in self.supported_id_searches:
if t == TVINFO_TVDB:
try:
show = self.search(p, search_type=TraktSearchTypes.tvdb_id)
except (BaseException, Exception):
continue
elif t == TVINFO_IMDB:
try:
show = self.search(p, search_type=TraktSearchTypes.imdb_id)
except (BaseException, Exception):
continue
elif t == TVINFO_TMDB:
try:
show = self.search(p, search_type=TraktSearchTypes.tmdb_id)
except (BaseException, Exception):
continue
elif t == TVINFO_TRAKT:
try:
show = self.search(p, search_type=TraktSearchTypes.trakt_id)
except (BaseException, Exception):
continue
elif t == TVINFO_TRAKT_SLUG:
try:
show = self.search(p, search_type=TraktSearchTypes.trakt_slug)
except (BaseException, Exception):
continue
else:
continue
self._make_result_obj(show, results)
if name:
names = ([name], name)[isinstance(name, list)]
len_names = len(names)
for i, n in enumerate(names, 1):
all_series = self.search(n)
if not isinstance(all_series, list):
all_series = [all_series]
if 0 == len(all_series):
log.debug('Series result returned zero')
raise BaseTVinfoShownotfound('Show-name search returned zero results (cannot find show on TVDB)')
if i == len_names and 0 == len(all_series) and not results:
log.debug('Series result returned zero')
raise BaseTVinfoShownotfound('Show-name search returned zero results (cannot find show on TVDB)')
if None is not self.config['custom_ui']:
log.debug('Using custom UI %s' % self.config['custom_ui'].__name__)
custom_ui = self.config['custom_ui']
ui = custom_ui(config=self.config)
if all_series:
if None is not self.config['custom_ui']:
log.debug('Using custom UI %s' % self.config['custom_ui'].__name__)
custom_ui = self.config['custom_ui']
ui = custom_ui(config=self.config)
self._make_result_obj(ui.select_series(all_series), results)
return ui.select_series(all_series)
else:
self._make_result_obj(all_series, results)
return all_series
return results
@staticmethod
def _dict_prevent_none(d, key, default):
@ -94,10 +176,14 @@ class TraktIndexer(TVInfoBase):
v = d.get(key, default)
return (v, default)[None is v]
def search(self, series):
# type: (AnyStr) -> List
if TraktSearchTypes.text != self.config['search_type']:
url = '/search/%s/%s?type=%s&extended=full&limit=100' % (self.config['search_type'], series,
def search(self, series, search_type=None):
# type: (AnyStr, Union[int, AnyStr]) -> List
search_type = search_type or self.config['search_type']
if TraktSearchTypes.trakt_slug == search_type:
url = '/shows/%s?extended=full' % series
elif TraktSearchTypes.text != search_type:
url = '/search/%s/%s?type=%s&extended=full&limit=100' % (search_type, (series, 'tt%07d' % series)[
TraktSearchTypes.imdb_id == search_type and not str(series).startswith('tt')],
','.join(self.config['result_types']))
else:
url = '/search/%s?query=%s&extended=full&limit=100' % (','.join(self.config['result_types']), series)
@ -107,8 +193,10 @@ class TraktIndexer(TVInfoBase):
kwargs['sleep_retry'] = self.config['sleep_retry']
try:
from sickbeard.helpers import clean_data
resp = TraktAPI().trakt_request(url, **kwargs)
resp = TraktAPI().trakt_request(url, failure_monitor=False, raise_skip_exception=False, **kwargs)
if len(resp):
if isinstance(resp, dict):
resp = [{'type': 'show', 'score': 1, 'show': resp}]
for d in resp:
if isinstance(d, dict) and 'type' in d and d['type'] in self.config['result_types']:
for k, v in iteritems(d):
@ -122,7 +210,136 @@ class TraktIndexer(TVInfoBase):
d['firstaired'] = (d.get('first_aired') and
re.sub(r'T.*$', '', str(d.get('first_aired'))) or d.get('year'))
filtered.append(d)
except TraktException as e:
except (ConnectionSkipException, TraktException) as e:
log.debug('Could not connect to Trakt service: %s' % ex(e))
return filtered
@staticmethod
def _convert_person_obj(person_obj):
# type: (Dict) -> Person
try:
birthdate = person_obj['birthday'] and tz_p.parse(person_obj['birthday']).date()
except (BaseException, Exception):
birthdate = None
try:
deathdate = person_obj['death'] and tz_p.parse(person_obj['death']).date()
except (BaseException, Exception):
deathdate = None
return Person(p_id=person_obj['ids']['trakt'],
name=person_obj['name'],
bio=person_obj['biography'],
birthdate=birthdate,
deathdate=deathdate,
homepage=person_obj['homepage'],
birthplace=person_obj['birthplace'],
social_ids={TVINFO_TWITTER: person_obj['social_ids']['twitter'],
TVINFO_FACEBOOK: person_obj['social_ids']['facebook'],
TVINFO_INSTAGRAM: person_obj['social_ids']['instagram'],
TVINFO_WIKIPEDIA: person_obj['social_ids']['wikipedia']
},
ids={TVINFO_TRAKT: person_obj['ids']['trakt'], TVINFO_SLUG: person_obj['ids']['slug'],
TVINFO_IMDB:
person_obj['ids']['imdb'] and
try_int(person_obj['ids']['imdb'].replace('nm', ''), None),
TVINFO_TMDB: person_obj['ids']['tmdb'],
TVINFO_TVRAGE: person_obj['ids']['tvrage']})
def get_person(self, p_id, get_show_credits=False, get_images=False, **kwargs):
# type: (integer_types, bool, bool, Any) -> Optional[Person]
"""
get person's data for id or list of matching persons for name
:param p_id: persons id
:param get_show_credits: get show credits (only for native id)
:param get_images: get images for person
:return: person object
"""
if not p_id:
return
urls = [('/people/%s?extended=full' % p_id, False)]
if get_show_credits:
urls.append(('/people/%s/shows?extended=full' % p_id, True))
if not urls:
return
result = None
for url, show_credits in urls:
try:
cache_key_name = 'p-%s-%s' % (('main', 'credits')[show_credits], p_id)
is_none, resp = self._get_cache_entry(cache_key_name)
if None is resp and not is_none:
resp = TraktAPI().trakt_request(url, **kwargs)
self._set_cache_entry(cache_key_name, resp)
if resp:
if show_credits:
pc = []
for c in resp.get('cast') or []:
show = TVInfoShow()
show.id = c['show']['ids'].get('trakt')
show.seriesname = c['show']['title']
show.ids = TVInfoIDs(ids={id_map[src]: _convert_imdb_id(id_map[src], sid)
for src, sid in iteritems(c['show']['ids']) if src in id_map})
show.network = c['show']['network']
show.firstaired = c['show']['first_aired']
show.overview = c['show']['overview']
show.status = c['show']['status']
show.imdb_id = c['show']['ids'].get('imdb')
show.runtime = c['show']['runtime']
show.genre_list = c['show']['genres']
for ch in c.get('characters') or []:
pc.append(
Character(
name=ch, regular=c.get('series_regular'),
show=show
)
)
result.characters = pc
else:
result = self._convert_person_obj(resp)
except ConnectionSkipException as e:
raise e
except TraktException as e:
log.debug('Could not connect to Trakt service: %s' % ex(e))
return result
def _search_person(self, name=None, ids=None):
# type: (AnyStr, Dict[integer_types, integer_types]) -> List[Person]
urls, result, ids = [], [], ids or {}
for tv_src in self.supported_person_id_searches:
if tv_src in ids:
if TVINFO_TRAKT == tv_src:
url = '/people/%s?extended=full' % ids.get(tv_src)
elif tv_src in (TVINFO_IMDB, TVINFO_TMDB):
url = '/search/%s/%s?type=person&extended=full&limit=100' % \
(id_map_reverse[tv_src], (ids.get(tv_src), 'nm%07d' % ids.get(tv_src))[TVINFO_IMDB == tv_src])
else:
continue
urls.append((tv_src, ids.get(tv_src), url))
if name:
urls.append(('text', name, '/search/person?query=%s&extended=full&limit=100' % name))
for src, s_id, url in urls:
try:
cache_key_name = 'p-src-%s-%s' % (src, s_id)
is_none, resp = self._get_cache_entry(cache_key_name)
if None is resp and not is_none:
resp = TraktAPI().trakt_request(url)
self._set_cache_entry(cache_key_name, resp)
if resp:
for per in (resp, [{'person': resp, 'type': 'person'}])[url.startswith('/people')]:
if 'person' != per['type']:
continue
person = per['person']
if not any(1 for p in result if person['ids']['trakt'] == p.id):
result.append(self._convert_person_obj(person))
except ConnectionSkipException as e:
raise e
except TraktException as e:
log.debug('Could not connect to Trakt service: %s' % ex(e))
return result

30
lib/libtrakt/trakt.py

@ -5,8 +5,8 @@ import sickbeard
import time
import datetime
import logging
from exceptions_helper import ex
from sg_helpers import try_int
from exceptions_helper import ex, ConnectionSkipException
from sg_helpers import get_url, try_int
from .exceptions import *
@ -194,6 +194,13 @@ class TraktAPI(object):
now = datetime.datetime.now()
resp = self.trakt_request('oauth/token', data=data, headers=headers, url=self.auth_url,
count=count, sleep_retry=0)
except TraktInvalidGrant:
if None is not account and account in sickbeard.TRAKT_ACCOUNTS:
sickbeard.TRAKT_ACCOUNTS[account].token = ''
sickbeard.TRAKT_ACCOUNTS[account].refresh_token = ''
sickbeard.TRAKT_ACCOUNTS[account].token_valid_date = None
sickbeard.save_config()
return False
except (TraktAuthException, TraktException):
return False
@ -207,8 +214,8 @@ class TraktAPI(object):
return False
def trakt_request(self, path, data=None, headers=None, url=None, count=0, sleep_retry=60,
send_oauth=None, method=None, **kwargs):
# type: (AnyStr, Dict, Dict, AnyStr, int, int, AnyStr, AnyStr, Any) -> Dict
send_oauth=None, method=None, raise_skip_exception=True, failure_monitor=True, **kwargs):
# type: (AnyStr, Dict, Dict, AnyStr, int, int, AnyStr, AnyStr, bool, bool, Any) -> Dict
if method not in ['GET', 'POST', 'PUT', 'DELETE', None]:
return {}
@ -230,7 +237,8 @@ class TraktAPI(object):
if sickbeard.TRAKT_ACCOUNTS[send_oauth].active:
if sickbeard.TRAKT_ACCOUNTS[send_oauth].needs_refresh:
self.trakt_token(refresh=True, count=0, account=send_oauth)
if sickbeard.TRAKT_ACCOUNTS[send_oauth].token_expired:
if sickbeard.TRAKT_ACCOUNTS[send_oauth].token_expired or \
not sickbeard.TRAKT_ACCOUNTS[send_oauth].active:
return {}
headers['Authorization'] = 'Bearer %s' % sickbeard.TRAKT_ACCOUNTS[send_oauth].token
else:
@ -242,7 +250,10 @@ class TraktAPI(object):
url = url or self.api_url
try:
resp = self.session.request(method, '%s%s' % (url, path), **kwargs)
resp = get_url('%s%s' % (url, path), session=self.session, use_method=method, return_response=True,
raise_exceptions=True, raise_status_code=True, raise_skip_exception=raise_skip_exception,
failure_monitor=failure_monitor, **kwargs)
# resp = self.session.request(method, '%s%s' % (url, path), **kwargs)
if 'DELETE' == method:
result = None
@ -340,9 +351,16 @@ class TraktAPI(object):
' that triggers a Trakt access lock.'
' SickGear may only send a notification on a media process completion if set up for it.')
raise TraktLockedUserAccount()
elif 400 == code and 'invalid_grant' in getattr(e, 'text', ''):
raise TraktInvalidGrant('Error: invalid_grant. The provided authorization grant is invalid, expired, '
'revoked, does not match the redirection URI used in the authorization request,'
' or was issued to another client.')
else:
log.error(u'Could not connect to Trakt. Code error: {0}'.format(code))
raise TraktException('Could not connect to Trakt. Code error: %s' % code)
except ConnectionSkipException as e:
log.error('Failure handling error')
raise e
except ValueError as e:
log.error(u'Value Error: %s' % ex(e))
raise TraktValueError(u'Value Error: %s' % ex(e))

4
lib/pytvmaze/__init__.py

@ -0,0 +1,4 @@
import logging
logger = logging.getLogger('pytvmaze')
logger.addHandler(logging.NullHandler())

36
lib/pytvmaze/endpoints.py

@ -0,0 +1,36 @@
#!/usr/bin/python
# TVMaze Free endpoints
show_search = 'https://api.tvmaze.com/search/shows?q={0}'
show_single_search = 'https://api.tvmaze.com/singlesearch/shows?q={0}'
lookup_tvrage = 'https://api.tvmaze.com/lookup/shows?tvrage={0}'
lookup_tvdb = 'https://api.tvmaze.com/lookup/shows?thetvdb={0}'
lookup_imdb = 'https://api.tvmaze.com/lookup/shows?imdb={0}'
get_schedule = 'https://api.tvmaze.com/schedule?country={0}&date={1}'
get_full_schedule = 'https://api.tvmaze.com/schedule/full'
show_main_info = 'https://api.tvmaze.com/shows/{0}'
episode_list = 'https://api.tvmaze.com/shows/{0}/episodes'
episode_by_number = 'https://api.tvmaze.com/shows/{0}/episodebynumber?season={1}&number={2}'
episodes_by_date = 'https://api.tvmaze.com/shows/{0}/episodesbydate?date={1}'
show_cast = 'https://api.tvmaze.com/shows/{0}/cast'
show_index = 'https://api.tvmaze.com/shows?page={0}'
people_search = 'https://api.tvmaze.com/search/people?q={0}'
person_main_info = 'https://api.tvmaze.com/people/{0}'
person_cast_credits = 'https://api.tvmaze.com/people/{0}/castcredits'
person_crew_credits = 'https://api.tvmaze.com/people/{0}/crewcredits'
show_crew = 'https://api.tvmaze.com/shows/{0}/crew'
show_updates = 'https://api.tvmaze.com/updates/shows'
show_akas = 'https://api.tvmaze.com/shows/{0}/akas'
show_seasons = 'https://api.tvmaze.com/shows/{0}/seasons'
season_by_id = 'https://api.tvmaze.com/seasons/{0}'
episode_by_id = 'https://api.tvmaze.com/episodes/{0}'
show_images = 'https://api.tvmaze.com/shows/{0}/images'
# TVMaze Premium endpoints
followed_shows = 'https://api.tvmaze.com/v1/user/follows/shows{0}'
followed_people = 'https://api.tvmaze.com/v1/user/follows/people{0}'
followed_networks = 'https://api.tvmaze.com/v1/user/follows/networks{0}'
followed_web_channels = 'https://api.tvmaze.com/v1/user/follows/webchannels{0}'
marked_episodes = 'https://api.tvmaze.com/v1/user/episodes{0}'
voted_shows = 'https://api.tvmaze.com/v1/user/votes/shows{0}'
voted_episodes = 'https://api.tvmaze.com/v1/user/votes/episodes{0}'

166
lib/pytvmaze/exceptions.py

@ -0,0 +1,166 @@
import sys
from . import logger
class BaseError(Exception):
def __init__(self, value):
self.value = value
logger.error(self.__str__())
def __str__(self):
if sys.version_info > (3,):
return self.value
else:
return unicode(self.value).encode('utf-8')
class ShowNotFound(BaseError):
pass
class IDNotFound(BaseError):
pass
class ScheduleNotFound(BaseError):
pass
class EpisodeNotFound(BaseError):
pass
class NoEpisodesForAirdate(BaseError):
pass
class CastNotFound(BaseError):
pass
class ShowIndexError(BaseError):
pass
class PersonNotFound(BaseError):
pass
class CreditsNotFound(BaseError):
pass
class UpdateNotFound(BaseError):
pass
class AKASNotFound(BaseError):
pass
class SeasonNotFound(BaseError):
pass
class GeneralError(BaseError):
pass
class MissingParameters(BaseError):
pass
class IllegalAirDate(BaseError):
pass
class ConnectionError(BaseError):
pass
class BadRequest(BaseError):
pass
class NoFollowedShows(BaseError):
pass
class ShowNotFollowed(BaseError):
pass
class NoFollowedPeople(BaseError):
pass
class PersonNotFollowed(BaseError):
pass
class NoMarkedEpisodes(BaseError):
pass
class EpisodeNotMarked(BaseError):
pass
class InvalidMarkedEpisodeType(BaseError):
pass
class InvalidEmbedValue(BaseError):
pass
class NetworkNotFollowed(BaseError):
pass
class NoFollowedWebChannels(BaseError):
pass
class NoVotedShows(BaseError):
pass
class ShowNotVotedFor(BaseError):
pass
class InvalidVoteValue(BaseError):
pass
class NoVotedEpisodes(BaseError):
pass
class EpisodeNotVotedFor(BaseError):
pass
class CrewNotFound(BaseError):
pass
class ShowImagesNotFound(BaseError):
pass
class NoFollowedNetworks(BaseError):
pass
class NetworkNotFound(BaseError):
pass
class WebChannelNotFound(BaseError):
pass
class WebChannelNotFollowed(BaseError):
pass

1516
lib/pytvmaze/tvmaze.py

File diff suppressed because it is too large

204
lib/sg_helpers.py

@ -2,6 +2,7 @@
# ---------------
# functions are placed here to remove cyclic import issues from placement in helpers
#
from __future__ import division
import ast
import codecs
import datetime
@ -24,6 +25,7 @@ import traceback
from exceptions_helper import ex, ConnectionSkipException
from lib.cachecontrol import CacheControl, caches
from lib.tmdbsimple.configuration import Configuration
from lib.tmdbsimple.genres import Genres
from cfscrape import CloudflareScraper
from send2trash import send2trash
@ -33,7 +35,7 @@ import requests
from _23 import decode_bytes, filter_list, html_unescape, list_range, \
ordered_dict, Popen, scandir, urlparse, urlsplit, urlunparse
from six import integer_types, iteritems, iterkeys, itervalues, PY2, string_types, text_type
from six import integer_types, iteritems, iterkeys, itervalues, moves, PY2, string_types, text_type
import zipfile
# py7z hardwired removed, see comment below
@ -62,6 +64,8 @@ if False:
# global tmdb_info cache
_TMDB_INFO_CACHE = {'date': datetime.datetime(2000, 1, 1), 'data': None}
html_convert_fractions = {0: '', 25: '&frac14;', 50: '&frac12;', 75: '&frac34;', 100: 1}
PROG_DIR = ek.ek(os.path.join, os.path.dirname(os.path.normpath(os.path.abspath(__file__))), '..')
# Mapping error status codes to official W3C names
@ -112,6 +116,9 @@ CACHE_DIR = None
DATA_DIR = None
PROXY_SETTING = None
TRASH_REMOVE_SHOW = False
REMOVE_FILENAME_CHARS = None
MEMCACHE = {}
FLARESOLVERR_HOST = None
# noinspection PyRedeclaration
db = None
@ -205,6 +212,12 @@ class ConnectionFailDict(object):
DOMAIN_FAILURES = ConnectionFailDict()
sp = 8
trakt_fail_times = {(i * sp) + m: s for m in range(1, 1 + sp) for i, s in
enumerate([(0, 15), (0, 30), (1, 0), (2, 0)])}
trakt_fail_times.update({i: s for i, s in enumerate([(3, 0), (6, 0), (12, 0), (24, 0)], len(trakt_fail_times))})
domain_fail_times = {'api.trakt.tv': trakt_fail_times}
default_fail_times = {1: (0, 15), 2: (0, 30), 3: (1, 0), 4: (2, 0), 5: (3, 0), 6: (6, 0), 7: (12, 0), 8: (24, 0)}
class ConnectionFailList(object):
@ -223,7 +236,7 @@ class ConnectionFailList(object):
self._tmr_limit_wait = None # type: Optional[datetime.timedelta]
self._last_fail_type = None # type: Optional[ConnectionFail]
self.has_limit = False # type: bool
self.fail_times = {1: (0, 15), 2: (0, 30), 3: (1, 0), 4: (2, 0), 5: (3, 0), 6: (6, 0), 7: (12, 0), 8: (24, 0)}
self.fail_times = domain_fail_times.get(url, default_fail_times) # type: Dict[integer_types, Tuple[int, int]]
self._load_fail_values()
self.dirty = False # type: bool
@ -741,8 +754,10 @@ def get_url(url, # type: AnyStr
exclude_client_http_codes=True, # type: bool
exclude_http_codes=(404, 429), # type: Tuple[integer_types]
exclude_no_data=True, # type: bool
use_method=None, # type: Optional[AnyStr]
return_response=False, # type: bool
**kwargs):
# type: (...) -> Optional[Union[AnyStr, bool, bytes, Dict, Tuple[Union[Dict, List], requests.Session]]]
# type: (...) -> Optional[Union[AnyStr, bool, bytes, Dict, Tuple[Union[Dict, List], requests.Session], requests.Response]]
"""
Return data from a URI with a possible check for authentication prior to the data fetch.
Raised errors and no data in responses are tracked for making future logic decisions.
@ -773,6 +788,8 @@ def get_url(url, # type: AnyStr
:param exclude_client_http_codes: if True, exclude client http codes 4XX from failure monitor
:param exclude_http_codes: http codes to exclude from failure monitor, default: (404, 429)
:param exclude_no_data: exclude no data as failure
:param use_method: force any supported method by Session(): get, put, post, delete
:param return_response: return response object
:param kwargs: keyword params to passthru to Requests
:return: None or data fetched from address
"""
@ -842,6 +859,10 @@ def get_url(url, # type: AnyStr
# don't trust os environments (auth, proxies, ...)
session.trust_env = False
method = None
if None is not use_method:
method = getattr(session, use_method.strip().lower())
result = response = raised = connection_fail_params = log_failure_url = None
try:
# sanitise url
@ -860,20 +881,21 @@ def get_url(url, # type: AnyStr
logger.debug('Using %s' % msg)
session.proxies = {'http': proxy_address, 'https': proxy_address}
# decide if we get or post data to server
if post_data or post_json:
if True is post_data:
post_data = None
if not method:
# decide if we get or post data to server
if post_data or post_json:
if True is post_data:
post_data = None
if post_data:
kwargs.setdefault('data', post_data)
if post_data:
kwargs.setdefault('data', post_data)
if post_json:
kwargs.setdefault('json', post_json)
if post_json:
kwargs.setdefault('json', post_json)
method = session.post
else:
method = session.get
method = session.post
else:
method = session.get
for r in range(0, 5):
response = method(url, timeout=timeout, **kwargs)
@ -958,10 +980,17 @@ def get_url(url, # type: AnyStr
if isinstance(raised, Exception):
if raise_exceptions or raise_status_code:
try:
if not hasattr(raised, 'text') and hasattr(response, 'text'):
raised.text = response.text
except (BaseException, Exception):
pass
raise raised
return
if None is result and None is not response and response.ok:
if return_response:
result = response
elif None is result and None is not response and response.ok:
if isinstance(memcache_cookies, dict):
parsed_url = urlparse(url)
domain = parsed_url.netloc
@ -997,8 +1026,8 @@ def get_url(url, # type: AnyStr
raise raised
if failure_monitor:
if result and not isinstance(result, tuple) \
or isinstance(result, tuple) and result[0]:
if return_response or (result and not isinstance(result, tuple)
or isinstance(result, tuple) and result[0]):
domain = DOMAIN_FAILURES.get_domain(url)
if 0 != DOMAIN_FAILURES.domain_list[domain].failure_count:
logger.info('Unblocking: %s' % domain)
@ -1364,6 +1393,40 @@ def maybe_plural(subject=1):
return ('s', '')[1 == number]
def time_to_int(dt):
# type: (Union[datetime.time, None]) -> Optional[integer_types]
"""
converts datetime.time to integer (hour + minute only)
:param dt: datetime.time obj
:return: integer of hour + min
"""
if None is dt:
return None
try:
return dt.hour * 100 + dt.minute
except (BaseException, Exception):
return 0
def int_to_time(d_int):
# type: (Union[integer_types, None]) -> Optional[datetime.time]
"""
convert integer from dt_to_int back to datetime.time
:param d_int: integer
:return: datetime.time
"""
if None is d_int:
return None
if isinstance(d_int, integer_types):
try:
return datetime.time(*divmod(d_int, 100))
except (BaseException, Exception):
pass
return datetime.time(hour=0, minute=0)
def indent_xml(elem, level=0):
"""
Does our pretty printing, makes Matt very happy
@ -1386,13 +1449,19 @@ def indent_xml(elem, level=0):
elem.tail = i
def get_tmdb_info():
# type: (...) -> Dict
def get_tmdb_info(get_tv_genres=False):
# type: (bool) -> Dict
"""return tmdbsimple Configuration().info() or cached copy"""
global _TMDB_INFO_CACHE
# only retrieve info data if older then 3 days
if 3 < (datetime.datetime.now() - _TMDB_INFO_CACHE['date']).days or not _TMDB_INFO_CACHE['data']:
_TMDB_INFO_CACHE = {'date': datetime.datetime.now(), 'data': Configuration().info()}
try:
tv_genres = {g['id']: g['name'] for g in Genres().tv_list()['genres']}
except (BaseException, Exception):
tv_genres = {}
_TMDB_INFO_CACHE = {'date': datetime.datetime.now(), 'data': Configuration().info(), 'genres': tv_genres}
if get_tv_genres:
return _TMDB_INFO_CACHE['genres']
return _TMDB_INFO_CACHE['data']
@ -1520,3 +1589,98 @@ def ast_eval(value, default=None):
value = default
return value
def sanitize_filename(name):
"""
:param name: filename
:type name: AnyStr
:return: sanitized filename
:rtype: AnyStr
"""
# remove bad chars from the filename
name = re.sub(r'[\\/*]', '-', name)
name = re.sub(r'[:"<>|?]', '', name)
# remove leading/trailing periods and spaces
name = name.strip(' .')
for char in REMOVE_FILENAME_CHARS or []:
name = name.replace(char, '')
return name
def download_file(url, filename, session=None, **kwargs):
"""
download given url to given filename
:param url: url to download
:type url: AnyStr
:param filename: filename to save the data to
:type filename: AnyStr
:param session: optional requests session object
:type session: requests.Session or None
:param kwargs:
:return: success of download
:rtype: bool
"""
MEMCACHE.setdefault('cookies', {})
if None is get_url(url, session=session, savename=filename,
url_solver=FLARESOLVERR_HOST, memcache_cookies=MEMCACHE['cookies'],
**kwargs):
remove_file_perm(filename)
return False
return True
def calc_age(birthday, deathday=None, date=None):
# type: (datetime.date, datetime.date, Optional[datetime.date]) -> Optional[int]
"""
returns age based on current date or given date
:param birthday: birth date
:param deathday: death date
:param date:
"""
if isinstance(birthday, datetime.date):
today = (datetime.date.today(), date)[isinstance(date, datetime.date)]
today = (today, deathday)[isinstance(deathday, datetime.date) and today > deathday]
try:
b_d = birthday.replace(year=today.year)
# raised when birth date is February 29
# and the current year is not a leap year
except ValueError:
b_d = birthday.replace(year=today.year, month=birthday.month + 1, day=1)
if b_d > today:
return today.year - birthday.year - 1
else:
return today.year - birthday.year
def convert_to_inch_faction_html(height):
# type: (float) -> AnyStr
"""
returns html string in foot and inches including fractions
:param height: height in cm
"""
total_inches = round(height / float(2.54), 2)
foot, inches = divmod(total_inches, 12)
_, fraction = '{0:.2f}'.format(total_inches).split('.')
fraction = int(fraction)
# fix rounding errors
fraction = next((html_convert_fractions.get(fraction + round_error)
or html_convert_fractions.get(fraction - round_error)
for round_error in moves.xrange(0, 25) if fraction + round_error in html_convert_fractions
or fraction - round_error in html_convert_fractions), '')
if 1 == fraction:
inches += 1
fraction = ''
if 12 <= inches:
foot += 1
inches = 0
inches = str(inches).split('.')[0]
return '%s\' %s%s%s' % (int(foot), (inches, '')['0' == inches], fraction,
('', '"')['0' != inches or '' != fraction])

0
lib/tmdb_api/__init__.py

348
lib/tmdb_api/tmdb_api.py

@ -0,0 +1,348 @@
# encoding:utf-8
# author:Prinz23
# project:tmdb_api
__author__ = 'Prinz23'
__version__ = '1.0'
__api_version__ = '1.0.0'
import json
import logging
import datetime
from six import iteritems
from sg_helpers import get_tmdb_info, get_url, try_int
from lib.dateutil.parser import parser
from lib.dateutil.tz.tz import _datetime_to_timestamp
from lib.exceptions_helper import ConnectionSkipException, ex
from .tmdb_exceptions import *
from lib.tvinfo_base import TVInfoBase, TVInfoImage, TVInfoImageSize, TVInfoImageType, Character, Crew, \
crew_type_names, Person, RoleTypes, TVInfoEpisode, TVInfoIDs, TVInfoSeason, PersonGenders, TVINFO_TVMAZE, \
TVINFO_TVDB, TVINFO_IMDB, TVINFO_TMDB, TVINFO_TWITTER, TVINFO_INSTAGRAM, TVINFO_FACEBOOK, TVInfoShow
from lib import tmdbsimple
# noinspection PyUnreachableCode
if False:
from typing import Any, AnyStr, Dict, List, Optional, Union
from six import integer_types
log = logging.getLogger('tmdb.api')
log.addHandler(logging.NullHandler())
tz_p = parser()
tmdbsimple.API_KEY = 'edc5f123313769de83a71e157758030b'
id_map = {TVINFO_IMDB: 'imdb_id', TVINFO_TVDB: 'tvdb_id', TVINFO_FACEBOOK: 'facebook_id', TVINFO_TWITTER: 'twitter_id',
TVINFO_INSTAGRAM: 'instagram_id'}
def tmdb_GET(self, path, params=None):
url = self._get_complete_url(path)
params = self._get_params(params)
return get_url(url=url, params=params, json=True, raise_skip_exception=True)
def tmdb_POST(self, path, params=None, payload=None):
url = self._get_complete_url(path)
params = self._get_params(params)
data = json.dumps(payload) if payload else payload
return get_url(url=url, params=params, post_data=data, json=True, raise_skip_exception=True)
tmdbsimple.base.TMDB._GET = tmdb_GET
tmdbsimple.base.TMDB._POST = tmdb_POST
class TmdbIndexer(TVInfoBase):
API_KEY = tmdbsimple.API_KEY
# supported_id_searches = [TVINFO_TVDB, TVINFO_IMDB, TVINFO_TMDB, TVINFO_TRAKT]
supported_person_id_searches = [TVINFO_TMDB, TVINFO_IMDB, TVINFO_TWITTER, TVINFO_INSTAGRAM, TVINFO_FACEBOOK]
# noinspection PyUnusedLocal
# noinspection PyDefaultArgument
def __init__(self, *args, **kwargs):
super(TmdbIndexer, self).__init__(*args, **kwargs)
response = get_tmdb_info()
self.img_base_url = response['images']['base_url']
self.img_sizes = response['images']['profile_sizes']
self.tv_genres = get_tmdb_info(get_tv_genres=True)
def _convert_person_obj(self, person_obj):
gender = PersonGenders.tmdb_map.get(person_obj.get('gender'), PersonGenders.unknown)
try:
birthdate = person_obj.get('birthday') and tz_p.parse(person_obj.get('birthday')).date()
except (BaseException, Exception):
birthdate = None
try:
deathdate = person_obj.get('deathday') and tz_p.parse(person_obj.get('deathday')).date()
except (BaseException, Exception):
deathdate = None
cast = person_obj.get('cast') or person_obj.get('tv_credits', {}).get('cast')
characters = []
for character in cast or []:
show = TVInfoShow()
show.id = character.get('id')
show.ids = TVInfoIDs(ids={TVINFO_TMDB: show.id})
show.seriesname = character.get('original_name')
show.overview = character.get('overview')
show.firstaired = character.get('first_air_date')
characters.append(
Character(name=character.get('character'), show=show)
)
pi = person_obj.get('images')
image_url, thumb_url = None, None
if pi:
def size_str_to_int(x):
return float('inf') if 'original' == x else int(x[1:])
thumb_size = next(s for s in sorted(self.img_sizes, key=size_str_to_int)
if 150 < size_str_to_int(s))
for i in sorted(pi['profiles'], key=lambda a: a['vote_average'] or 0, reverse=True):
if 500 < i['height'] and not image_url:
image_url = '%s%s%s' % (self.img_base_url, 'original', i['file_path'])
thumb_url = '%s%s%s' % (self.img_base_url, thumb_size, i['file_path'])
elif not thumb_url:
thumb_url = '%s%s%s' % (self.img_base_url, 'original', i['file_path'])
if image_url and thumb_url:
break
return Person(p_id=person_obj.get('id'), gender=gender, name=person_obj.get('name'), birthdate=birthdate,
deathdate=deathdate, bio=person_obj.get('biography'), birthplace=person_obj.get('place_of_birth'),
homepage=person_obj.get('homepage'), characters=characters, image=image_url, thumb_url=thumb_url,
ids={TVINFO_TMDB: person_obj.get('id'),
TVINFO_IMDB:
person_obj.get('imdb_id') and try_int(person_obj['imdb_id'].replace('nm', ''), None)})
def _search_person(self, name=None, ids=None):
# type: (AnyStr, Dict[integer_types, integer_types]) -> List[Person]
"""
search for person by name
:param name: name to search for
:param ids: dict of ids to search
:return: list of found person's
"""
results, ids = [], ids or {}
search_text_obj = tmdbsimple.Search()
for tv_src in self.supported_person_id_searches:
if tv_src in ids:
if TVINFO_TMDB == tv_src:
try:
people_obj = self.get_person(ids[tv_src])
except ConnectionSkipException as e:
raise e
except (BaseException, Exception):
people_obj = None
if people_obj and not any(1 for r in results if r.id == people_obj.id):
results.append(people_obj)
elif tv_src in (TVINFO_IMDB, TVINFO_TMDB):
try:
cache_key_name = 'p-src-%s-%s' % (tv_src, ids.get(tv_src))
is_none, result_objs = self._get_cache_entry(cache_key_name)
if None is result_objs and not is_none:
result_objs = tmdbsimple.Find(id=(ids.get(tv_src),
'nm%07d' % ids.get(tv_src))[TVINFO_IMDB == tv_src]).info(
external_source=id_map[tv_src]).get('person_results')
self._set_cache_entry(cache_key_name, result_objs)
except ConnectionSkipException as e:
raise e
except (BaseException, Exception):
result_objs = None
if result_objs:
for person_obj in result_objs:
if not any(1 for r in results if r.id == person_obj['id']):
results.append(self._convert_person_obj(person_obj))
else:
continue
if name:
cache_key_name = 'p-src-text-%s' % name
is_none, people_objs = self._get_cache_entry(cache_key_name)
if None is people_objs and not is_none:
try:
people_objs = search_text_obj.person(query=name, include_adult=True)
self._set_cache_entry(cache_key_name, people_objs)
except ConnectionSkipException as e:
raise e
except (BaseException, Exception):
people_objs = None
if people_objs and people_objs.get('results'):
for person_obj in people_objs['results']:
if not any(1 for r in results if r.id == person_obj['id']):
results.append(self._convert_person_obj(person_obj))
return results
def get_person(self, p_id, get_show_credits=False, get_images=False, **kwargs):
# type: (integer_types, bool, bool, Any) -> Optional[Person]
kw = {}
to_append = []
if get_show_credits:
to_append.append('tv_credits')
if get_images:
to_append.append('images')
if to_append:
kw['append_to_response'] = ','.join(to_append)
cache_key_name = 'p-%s-%s' % (p_id, '-'.join(to_append))
is_none, people_obj = self._get_cache_entry(cache_key_name)
if None is people_obj and not is_none:
try:
people_obj = tmdbsimple.People(id=p_id).info(**kw)
except ConnectionSkipException as e:
raise e
except (BaseException, Exception):
people_obj = None
self._set_cache_entry(cache_key_name, people_obj)
if people_obj:
return self._convert_person_obj(people_obj)
def _convert_show(self, show_dict):
# type: (Dict) -> TVInfoShow
tv_s = TVInfoShow()
if show_dict:
tv_s.seriesname = show_dict.get('name') or show_dict.get('original_name') or show_dict.get('original_title')
org_title = show_dict.get('original_name') or show_dict.get('original_title')
if org_title != tv_s.seriesname:
tv_s.aliases = [org_title]
tv_s.id = show_dict.get('id')
tv_s.seriesid = tv_s.id
tv_s.language = show_dict.get('original_language')
tv_s.overview = show_dict.get('overview')
tv_s.firstaired = show_dict.get('first_air_date')
tv_s.vote_count = show_dict.get('vote_count')
tv_s.vote_average = show_dict.get('vote_average')
tv_s.popularity = show_dict.get('popularity')
tv_s.origin_countries = show_dict.get('origin_country') or []
tv_s.genre_list = []
for g in show_dict.get('genre_ids') or []:
if g in self.tv_genres:
tv_s.genre_list.append(self.tv_genres.get(g))
tv_s.genre = ', '.join(tv_s.genre_list)
image_url = show_dict.get('poster_path') and '%s%s%s' % (self.img_base_url, 'original',
show_dict.get('poster_path'))
backdrop_url = show_dict.get('backdrop_path') and '%s%s%s' % (self.img_base_url, 'original',
show_dict.get('backdrop_path'))
tv_s.poster = image_url
tv_s.fanart = backdrop_url
tv_s.ids = TVInfoIDs(tmdb=tv_s.id)
return tv_s
def _get_show_list(self, src_method, result_count, **kwargs):
result = []
try:
c_page = 1
while len(result) < result_count:
results = src_method(page=c_page, **kwargs)
t_pages = results.get('total_pages')
if c_page != results.get('page') or c_page >= t_pages:
break
c_page += 1
if results and 'results' in results:
result += [self._convert_show(t) for t in results['results']]
else:
break
except (BaseException, Exception):
pass
return result[:result_count]
def get_trending(self, result_count=100, time_window='day', **kwargs):
"""
list of trending tv shows for day or week
:param result_count:
:param time_window: valid values: 'day', 'week'
"""
t_windows = ('day', 'week')['week' == time_window]
return self._get_show_list(tmdbsimple.Trending(media_type='tv', time_window=t_windows).info, result_count)
def get_popular(self, result_count=100, **kwargs):
return self._get_show_list(tmdbsimple.TV().popular, result_count)
def get_top_rated(self, result_count=100, **kwargs):
return self._get_show_list(tmdbsimple.TV().top_rated, result_count)
def discover(self, result_count=100, **kwargs):
"""
Discover TV shows by different types of data like average rating,
number of votes, genres, the network they aired on and air dates.
Discover also supports a nice list of sort options. See below for all
of the available options.
Also note that a number of filters support being comma (,) or pipe (|)
separated. Comma's are treated like an AND and query while pipe's are
an OR.
Some examples of what can be done with discover can be found at
https://www.themoviedb.org/documentation/api/discover.
kwargs:
language: (optional) ISO 639-1 code.
sort_by: (optional) Available options are 'vote_average.desc',
'vote_average.asc', 'first_air_date.desc',
'first_air_date.asc', 'popularity.desc', 'popularity.asc'
sort_by: (optional) Allowed values: vote_average.desc,
vote_average.asc, first_air_date.desc, first_air_date.asc,
popularity.desc, popularity.asc
Default: popularity.desc
air_date.gte: (optional) Filter and only include TV shows that have
a air date (by looking at all episodes) that is greater or
equal to the specified value.
air_date.lte: (optional) Filter and only include TV shows that have
a air date (by looking at all episodes) that is less than or
equal to the specified value.
first_air_date.gte: (optional) Filter and only include TV shows
that have a original air date that is greater or equal to the
specified value. Can be used in conjunction with the
"include_null_first_air_dates" filter if you want to include
items with no air date.
first_air_date.lte: (optional) Filter and only include TV shows
that have a original air date that is less than or equal to the
specified value. Can be used in conjunction with the
"include_null_first_air_dates" filter if you want to include
items with no air date.
first_air_date_year: (optional) Filter and only include TV shows
that have a original air date year that equal to the specified
value. Can be used in conjunction with the
"include_null_first_air_dates" filter if you want to include
items with no air date.
timezone: (optional) Used in conjunction with the air_date.gte/lte
filter to calculate the proper UTC offset. Default
America/New_York.
vote_average.gte: (optional) Filter and only include movies that
have a rating that is greater or equal to the specified value.
Minimum 0.
vote_count.gte: (optional) Filter and only include movies that have
a rating that is less than or equal to the specified value.
Minimum 0.
with_genres: (optional) Comma separated value of genre ids that you
want to include in the results.
with_networks: (optional) Comma separated value of network ids that
you want to include in the results.
without_genres: (optional) Comma separated value of genre ids that
you want to exclude from the results.
with_runtime.gte: (optional) Filter and only include TV shows with
an episode runtime that is greater than or equal to a value.
with_runtime.lte: (optional) Filter and only include TV shows with
an episode runtime that is less than or equal to a value.
include_null_first_air_dates: (optional) Use this filter to include
TV shows that don't have an air date while using any of the
"first_air_date" filters.
with_original_language: (optional) Specify an ISO 639-1 string to
filter results by their original language value.
without_keywords: (optional) Exclude items with certain keywords.
You can comma and pipe seperate these values to create an 'AND'
or 'OR' logic.
screened_theatrically: (optional) Filter results to include items
that have been screened theatrically.
with_companies: (optional) A comma separated list of production
company ID's. Only include movies that have one of the ID's
added as a production company.
with_keywords: (optional) A comma separated list of keyword ID's.
Only includes TV shows that have one of the ID's added as a
keyword.
:param result_count:
"""
return self._get_show_list(tmdbsimple.Discover().tv, result_count, **kwargs)

62
lib/tmdb_api/tmdb_exceptions.py

@ -0,0 +1,62 @@
# encoding:utf-8
"""Custom exceptions used or raised by tmdb_api
"""
__author__ = 'Prinz23'
__version__ = '1.0'
__all__ = ['TmdbException', 'TmdbError', 'TmdbUserabort', 'TmdbShownotfound',
'TmdbSeasonnotfound', 'TmdbEpisodenotfound', 'TmdbAttributenotfound', 'TmdbTokenexpired']
from lib.tvinfo_base.exceptions import *
class TmdbException(BaseTVinfoException):
"""Any exception generated by tvdb_api
"""
pass
class TmdbError(BaseTVinfoError, TmdbException):
"""An error with thetvdb.com (Cannot connect, for example)
"""
pass
class TmdbUserabort(BaseTVinfoUserabort, TmdbError):
"""User aborted the interactive selection (via
the q command, ^c etc)
"""
pass
class TmdbShownotfound(BaseTVinfoShownotfound, TmdbError):
"""Show cannot be found on thetvdb.com (non-existant show)
"""
pass
class TmdbSeasonnotfound(BaseTVinfoSeasonnotfound, TmdbError):
"""Season cannot be found on thetvdb.com
"""
pass
class TmdbEpisodenotfound(BaseTVinfoEpisodenotfound, TmdbError):
"""Episode cannot be found on thetvdb.com
"""
pass
class TmdbAttributenotfound(BaseTVinfoAttributenotfound, TmdbError):
"""Raised if an episode does not have the requested
attribute (such as a episode name)
"""
pass
class TmdbTokenexpired(BaseTVinfoAuthenticationerror, TmdbError):
"""token expired or missing thetvdb.com
"""
pass

96
lib/tvdb_api/tvdb_api.py

@ -26,7 +26,8 @@ from six import integer_types, string_types, iteritems, PY2
from _23 import list_values, map_list
from sg_helpers import clean_data, try_int, get_url
from collections import OrderedDict
from tvinfo_base import TVInfoBase, CastList, Character, CrewList, Person, RoleTypes
from lib.tvinfo_base import TVInfoBase, TVInfoIDs, CastList, Character, CrewList, Person, RoleTypes, TVINFO_TVDB, \
TVINFO_TVDB_SLUG
from lib.dateutil.parser import parse
from lib.cachecontrol import CacheControl, caches
@ -39,7 +40,7 @@ from .tvdb_exceptions import TvdbError, TvdbShownotfound, TvdbTokenexpired
if False:
# noinspection PyUnresolvedReferences
from typing import Any, AnyStr, Dict, List, Optional
from tvinfo_base import TVInfoShow
from lib.tvinfo_base import TVInfoShow
THETVDB_V2_API_TOKEN = {'token': None, 'datetime': datetime.datetime.fromordinal(1)}
@ -134,6 +135,7 @@ class Tvdb(TVInfoBase):
>> t['Scrubs'][1][24]['episodename']
u'My Last Day'
"""
supported_id_searches = [TVINFO_TVDB, TVINFO_TVDB_SLUG]
# noinspection PyUnusedLocal
def __init__(self,
@ -305,13 +307,44 @@ class Tvdb(TVInfoBase):
self.config['url_series_images'] = '%(base_url)sseries/%%s/images/query?keyType=%%s' % self.config
self.config['url_artworks'] = 'https://artworks.thetvdb.com/banners/%s'
def _search_show(self, name, **kwargs):
# type: (AnyStr, Optional[Any]) -> List[TVInfoShow]
def _search_show(self, name=None, ids=None, **kwargs):
# type: (AnyStr, Dict[integer_types, integer_types], Optional[Any]) -> List[TVInfoShow]
def map_data(data):
data['poster'] = data.get('image')
data['ids'] = TVInfoIDs(
tvdb=data.get('id'), imdb=data.get('imdb_id') and
try_int(data.get('imdb_id', '').replace('tt', ''), None))
return data
return map_list(map_data, self.get_series(name))
results = []
if ids:
if ids.get(TVINFO_TVDB):
try:
d_m = self._get_show_data(ids.get(TVINFO_TVDB), self.config['language'], direct_data=True)
if d_m:
results = map_list(map_data, [d_m['data']])
except (BaseException, Exception) as e:
pass
if ids.get(TVINFO_TVDB_SLUG):
try:
d_m = self.get_series(ids.get(TVINFO_TVDB_SLUG).replace('-', ' '))
if d_m:
for r in d_m:
if ids.get(TVINFO_TVDB_SLUG) == r['slug']:
results = map_list(map_data, [r])
break
except (BaseException, Exception):
pass
if name:
for n in ([name], name)[isinstance(name, list)]:
try:
r = self.get_series(n)
if r:
results.extend(map_list(map_data, r))
except (BaseException, Exception):
pass
return results
def get_new_token(self):
global THETVDB_V2_API_TOKEN
@ -599,17 +632,28 @@ class Tvdb(TVInfoBase):
a = []
cast = CastList()
try:
for n in sorted(actor_list, key=lambda x: x['sortorder']):
role_image = (None, self.config['url_artworks'] % n.get('image'))[any([n.get('image')])]
unique_c_p, c_p_list, new_actor_list = set(), [], []
for actor in sorted(actor_list, key=lambda x: x.get('lastupdated'), reverse=True):
c_p_list.append((actor['name'], actor['role']))
if (actor['name'], actor['role']) not in unique_c_p:
unique_c_p.add((actor['name'], actor['role']))
new_actor_list.append(actor)
for n in sorted(new_actor_list, key=lambda x: x['sortorder']):
role_image = (None, self.config['url_artworks'] % n.get('image'))[
any([n.get('image')]) and 1 == c_p_list.count((n['name'], n['role']))]
character_name = n.get('role', '').strip()
person_name = n.get('name', '').strip()
try:
person_id = try_int(re.search(r'^person/(\d+)/', n.get('image', '')).group(1), None)
except (BaseException, Exception):
person_id = None
character_id = n.get('id', None)
a.append({'character': {'id': character_id,
'name': character_name,
'url': None, # not supported by tvdb
'image': role_image,
},
'person': {'id': None,
'person': {'id': person_id,
'name': person_name,
'url': None, # not supported by tvdb
'image': None, # not supported by tvdb
@ -621,7 +665,7 @@ class Tvdb(TVInfoBase):
})
cast[RoleTypes.ActorMain].append(
Character(p_id=character_id, name=character_name,
person=Person(name=person_name), image=role_image))
person=[Person(p_id=person_id, name=person_name)], image=role_image))
except (BaseException, Exception):
pass
self._set_show_data(sid, 'actors', a)
@ -650,11 +694,11 @@ class Tvdb(TVInfoBase):
return data
def _parse_images(self, sid, language, show_data, image_type, enabled_type):
def _parse_images(self, sid, language, show_data, image_type, enabled_type, type_bool):
mapped_img_types = {'banner': 'series'}
excluded_main_data = enabled_type in ['seasons_enabled', 'seasonwides_enabled']
loaded_name = '%s_loaded' % image_type
if self.config[enabled_type] and not getattr(self.shows.get(sid), loaded_name, False):
if (type_bool or self.config[enabled_type]) and not getattr(self.shows.get(sid), loaded_name, False):
image_data = self._getetsrc(self.config['url_series_images'] %
(sid, mapped_img_types.get(image_type, image_type)), language=language)
if image_data and 0 < len(image_data.get('data', '') or ''):
@ -674,8 +718,9 @@ class Tvdb(TVInfoBase):
self._set_show_data(sid, u'%s_thumb' % image_type,
re.sub(r'\.jpg$', '_t.jpg', show_data['data'][image_type], flags=re.I))
def _get_show_data(self, sid, language, get_ep_info=False, **kwargs):
# type: (integer_types, AnyStr, bool, Optional[Any]) -> bool
def _get_show_data(self, sid, language, get_ep_info=False, banners=False, posters=False, seasons=False,
seasonwides=False, fanart=False, actors=False, direct_data=False, **kwargs):
# type: (integer_types, AnyStr, bool, bool, bool, bool, bool, bool, bool, bool, bool, Optional[Any]) -> bool
"""Takes a series ID, gets the epInfo URL and parses the TVDB
XML file into the shows dict in layout:
shows[series_id][season_number][episode_number]
@ -683,9 +728,11 @@ class Tvdb(TVInfoBase):
# Parse show information
url = self.config['url_series_info'] % sid
if sid not in self.shows or None is self.shows[sid].id:
if direct_data or sid not in self.shows or None is self.shows[sid].id:
log.debug('Getting all series data for %s' % sid)
show_data = self._getetsrc(url, language=language)
if direct_data:
return show_data
# check and make sure we have data to process and that it contains a series name
if not (show_data and 'seriesname' in show_data.get('data', {}) or {}):
@ -693,15 +740,22 @@ class Tvdb(TVInfoBase):
for k, v in iteritems(show_data['data']):
self._set_show_data(sid, k, v)
self._set_show_data(sid, 'ids',
TVInfoIDs(
tvdb=show_data['data'].get('id'),
imdb=show_data['data'].get('imdb_id') and
try_int(show_data['data'].get('imdb_id', '').replace('tt', ''), None)))
else:
show_data = {'data': {}}
for img_type, en_type in [(u'poster', 'posters_enabled'), (u'banner', 'banners_enabled'),
(u'fanart', 'fanart_enabled'), (u'season', 'seasons_enabled'),
(u'seasonwide', 'seasonwides_enabled')]:
self._parse_images(sid, language, show_data, img_type, en_type)
for img_type, en_type, p_type in [(u'poster', 'posters_enabled', posters),
(u'banner', 'banners_enabled', banners),
(u'fanart', 'fanart_enabled', fanart),
(u'season', 'seasons_enabled', seasons),
(u'seasonwide', 'seasonwides_enabled', seasonwides)]:
self._parse_images(sid, language, show_data, img_type, en_type, p_type)
if self.config['actors_enabled'] and not getattr(self.shows.get(sid), 'actors_loaded', False):
if (actors or self.config['actors_enabled']) and not getattr(self.shows.get(sid), 'actors_loaded', False):
actor_data = self._getetsrc(self.config['url_actors_info'] % sid, language=language)
if actor_data and 0 < len(actor_data.get('data', '') or ''):
self._parse_actors(sid, actor_data['data'])
@ -759,6 +813,8 @@ class Tvdb(TVInfoBase):
seas_no = int(float(elem_seasnum))
ep_no = int(float(elem_epno))
if 'network' not in cur_ep:
cur_ep['network'] = self.shows[sid].network
for k, v in iteritems(cur_ep):
k = k.lower()
@ -781,7 +837,7 @@ class Tvdb(TVInfoBase):
pass
try:
for guest in cur_ep.get('gueststars_list', []):
cast[RoleTypes.ActorGuest].append(Character(person=Person(name=guest)))
cast[RoleTypes.ActorGuest].append(Character(person=[Person(name=guest)]))
except (BaseException, Exception):
pass
try:

2
lib/tvdb_api/tvdb_exceptions.py

@ -13,7 +13,7 @@ __version__ = '1.9'
__all__ = ['TvdbException', 'TvdbError', 'TvdbUserabort', 'TvdbShownotfound',
'TvdbSeasonnotfound', 'TvdbEpisodenotfound', 'TvdbAttributenotfound', 'TvdbTokenexpired']
from tvinfo_base.exceptions import *
from lib.tvinfo_base.exceptions import *
class TvdbException(BaseTVinfoException):

563
lib/tvinfo_base/base.py

@ -1,15 +1,43 @@
import copy
import datetime
import diskcache
import logging
import threading
import shutil
import time
from exceptions_helper import ex
from six import integer_types, iteritems, iterkeys, string_types, text_type
from _23 import list_items, list_values
from .exceptions import *
from lib.tvinfo_base.exceptions import *
from sg_helpers import calc_age, make_dirs
# noinspection PyUnreachableCode
if False:
from typing import Any, AnyStr, Dict, List, Optional, Tuple, Union
from typing import Any, AnyStr, Dict, List, Optional, Set, Tuple, Union
TVINFO_TVDB = 1
TVINFO_TVRAGE = 2
TVINFO_TVMAZE = 3
# old tvdb api - version 1
# TVINFO_TVDB_V1 = 10001
# mapped only source
TVINFO_IMDB = 100
TVINFO_TRAKT = 101
TVINFO_TMDB = 102
# end mapped only source
TVINFO_TVDB_SLUG = 1001
TVINFO_TRAKT_SLUG = 1101
# generic stuff
TVINFO_SLUG = 100000
# social media sources
TVINFO_TWITTER = 250000
TVINFO_FACEBOOK = 250001
TVINFO_INSTAGRAM = 250002
TVINFO_WIKIPEDIA = 250003
log = logging.getLogger('TVInfo')
log.addHandler(logging.NullHandler())
@ -58,8 +86,82 @@ class ShowContainer(dict):
nr_shows = len(self)
return '<ShowContainer (containing %s Show%s)>' % (nr_shows, ('s', '')[1 == nr_shows])
def __repr__(self):
return self.__str__()
__repr__ = __str__
class TVInfoIDs(object):
def __init__(self, tvdb=None, tmdb=None, tvmaze=None, imdb=None, trakt=None, rage=None, ids=None):
# type: (integer_types, integer_types, integer_types, integer_types, integer_types, integer_types, Dict[int, integer_types]) -> TVInfoIDs
ids = ids or {}
self.tvdb = tvdb or ids.get(TVINFO_TVDB)
self.tmdb = tmdb or ids.get(TVINFO_TMDB)
self.tvmaze = tvmaze or ids.get(TVINFO_TVMAZE)
self.imdb = imdb or ids.get(TVINFO_IMDB)
self.trakt = trakt or ids.get(TVINFO_TRAKT)
self.rage = rage or ids.get(TVINFO_TVRAGE)
def __getitem__(self, key):
return {TVINFO_TVDB: self.tvdb, TVINFO_TMDB: self.tmdb, TVINFO_TVMAZE: self.tvmaze,
TVINFO_IMDB: self.imdb, TVINFO_TRAKT: self.trakt, TVINFO_TVRAGE: self.rage}.get(key)
def __iter__(self):
for s, v in [(TVINFO_TVDB, self.tvdb), (TVINFO_TMDB, self.tmdb), (TVINFO_TVMAZE, self.tvmaze),
(TVINFO_IMDB, self.imdb), (TVINFO_TRAKT, self.trakt), (TVINFO_TVRAGE, self.rage)]:
yield s, v
def __str__(self):
return ', '.join('%s:%s' % v for v in self.__iter__())
__repr__ = __str__
iteritems = __iter__
items = __iter__
class TVInfoImageType(object):
poster = 1
banner = 2
# fanart/background
fanart = 3
typography = 4
other = 10
reverse_str = {
1: 'poster',
2: 'banner',
# fanart/background
3: 'fanart',
4: 'typography',
10: 'other'
}
class TVInfoImageSize(object):
original = 1
medium = 2
small = 3
reverse_str = {
1: 'original',
2: 'medium',
3: 'small'
}
class TVInfoImage(object):
def __init__(self, image_type, sizes, img_id=None, main_image=False, type_str='', rating=None, votes=None):
self.img_id = img_id # type: Optional[integer_types]
self.image_type = image_type # type: integer_types
self.sizes = sizes # type: Dict[TVInfoImageSize, AnyStr]
self.type_str = type_str # type: AnyStr
self.main_image = main_image # type: bool
self.rating = rating # type: Optional[Union[float, integer_types]]
self.votes = votes # type: Optional[integer_types]
def __str__(self):
return '<TVInfoImage %s [%s]>' % (TVInfoImageType.reverse_str.get(self.image_type, 'unknown'),
', '.join(TVInfoImageSize.reverse_str.get(s, 'unkown') for s in self.sizes))
__repr__ = __str__
class TVInfoShow(dict):
@ -79,7 +181,7 @@ class TVInfoShow(dict):
self.actors_loaded = False # type: bool
self.show_not_found = False # type: bool
self.id = None # type: integer_types
self.ids = {} # type: Dict[AnyStr, Optional[integer_types, AnyStr]]
self.ids = TVInfoIDs() # type: TVInfoIDs
self.slug = None # type: Optional[AnyStr]
self.seriesid = None # type: integer_types
self.seriesname = None # type: Optional[AnyStr]
@ -89,7 +191,8 @@ class TVInfoShow(dict):
self.genre = None # type: Optional[AnyStr]
self.genre_list = [] # type: List[AnyStr]
self.actors = [] # type: List[Dict]
self.cast = CastList() # type: Dict[integer_types, Character]
self.cast = CastList() # type: CastList
self.crew = CrewList() # type: CrewList
self.show_type = [] # type: List[AnyStr]
self.network = None # type: Optional[AnyStr]
self.network_id = None # type: integer_types
@ -104,11 +207,11 @@ class TVInfoShow(dict):
self.zap2itid = None # type: Optional[AnyStr]
self.airs_dayofweek = None # type: Optional[AnyStr]
self.airs_time = None # type: Optional[AnyStr]
self.time = None # type: Optional[datetime.time]
self.firstaired = None # type: Optional[AnyStr]
self.added = None # type: Optional[AnyStr]
self.addedby = None # type: Union[integer_types, AnyStr]
self.siteratingcount = None # type: integer_types
self.slug = None # type: Optional[AnyStr]
self.lastupdated = None # type: integer_types
self.contentrating = None # type: Optional[AnyStr]
self.rating = None # type: integer_types
@ -120,14 +223,18 @@ class TVInfoShow(dict):
self.banner_thumb = None # type: Optional[AnyStr]
self.fanart = None # type: Optional[AnyStr]
self.banners = {} # type: Dict
self.images = {} # type: Dict[TVInfoImageType, List[TVInfoImage]]
self.updated_timestamp = None # type: Optional[integer_types]
# special properties for trending, popular, ...
self.popularity = None # type: Optional[Union[integer_types, float]]
self.vote_count = None # type: Optional[integer_types]
self.vote_average = None # type: Optional[Union[integer_types, float]]
self.origin_countries = [] # type: List[AnyStr]
def __str__(self):
nr_seasons = len(self)
return '<Show %r (containing %s season%s)>' % (self.seriesname, nr_seasons, ('s', '')[1 == nr_seasons])
def __repr__(self):
return self.__str__()
def __getattr__(self, key):
if key in self:
# Key is an episode, return it
@ -140,6 +247,9 @@ class TVInfoShow(dict):
raise AttributeError
def __getitem__(self, key):
if isinstance(key, string_types) and key in self.__dict__:
return self.__dict__[key]
if key in self:
# Key is an episode, return it
return dict.__getitem__(self, key)
@ -170,8 +280,9 @@ class TVInfoShow(dict):
setattr(result[k], 'show', result)
return result
def __nonzero__(self):
return any(self.data.keys())
def __bool__(self):
# type: (...) -> bool
return bool(self.id) or any(iterkeys(self.data))
def aired_on(self, date):
ret = self.search(str(date), 'firstaired')
@ -195,6 +306,9 @@ class TVInfoShow(dict):
return results
__repr__ = __str__
__nonzero__ = __bool__
class TVInfoSeason(dict):
def __init__(self, show=None, **kwargs):
@ -218,15 +332,13 @@ class TVInfoSeason(dict):
self.end_date = None # type: Optional[AnyStr]
self.poster = None # type: Optional[AnyStr]
self.summery = None # type: Optional[AnyStr]
self.episode_order = None # type: Optional[integer_types]
def __str__(self):
nr_episodes = len(self)
return '<Season %s instance (containing %s episode%s)>' % \
(self.number, nr_episodes, ('s', '')[1 == nr_episodes])
def __repr__(self):
return self.__str__()
def __getattr__(self, episode_number):
if episode_number in self:
return self[episode_number]
@ -262,6 +374,8 @@ class TVInfoSeason(dict):
results.append(searchresult)
return results
__repr__ = __str__
class TVInfoEpisode(dict):
def __init__(self, season=None, **kwargs):
@ -282,7 +396,7 @@ class TVInfoEpisode(dict):
self.directors = [] # type: List[AnyStr]
self.writer = None # type: Optional[AnyStr]
self.writers = [] # type: List[AnyStr]
self.crew = CrewList() # type: Dict[integer_types, Person]
self.crew = CrewList() # type: CrewList
self.episodename = None # type: Optional[AnyStr]
self.overview = None # type: Optional[AnyStr]
self.language = {'episodeName': None, 'overview': None} # type: Dict[AnyStr, Optional[AnyStr]]
@ -294,7 +408,9 @@ class TVInfoEpisode(dict):
self.dvd_episodenumber = None # type: integer_types
self.dvdchapter = None # type: integer_types
self.firstaired = None # type: Optional[AnyStr]
self.airtime = None # type: Optional[AnyStr]
self.airtime = None # type: Optional[datetime.time]
self.runtime = 0 # type: integer_types
self.timestamp = None # type: Optional[integer_types]
self.network = None # type: Optional[AnyStr]
self.network_id = None # type: integer_types
self.network_timezone = None # type: Optional[AnyStr]
@ -320,9 +436,6 @@ class TVInfoEpisode(dict):
else:
return '<Episode %02dx%02d>' % (seasno, epno)
def __repr__(self):
return self.__str__()
def __getattr__(self, key):
if key in self:
return self[key]
@ -345,6 +458,10 @@ class TVInfoEpisode(dict):
result[k] = copy.deepcopy(v)
return result
def __bool__(self):
# type: (...) -> bool
return bool(self.id) or bool(self.episodename)
def search(self, term=None, key=None):
"""Search episode data for term, if it matches, return the Episode (self).
The key parameter can be used to limit the search to a specific element,
@ -362,6 +479,9 @@ class TVInfoEpisode(dict):
if cur_value.find(text_type(term).lower()) > -1:
return self
__repr__ = __str__
__nonzero__ = __bool__
class Persons(dict):
"""Holds all Persons instances for a show
@ -370,8 +490,7 @@ class Persons(dict):
persons_count = len(self)
return '<Persons (containing %s Person%s)>' % (persons_count, ('', 's')[1 != persons_count])
def __repr__(self):
return self.__str__()
__repr__ = __str__
class CastList(Persons):
@ -391,8 +510,7 @@ class CastList(Persons):
persons_text = ('0', '(%s)' % persons_text)['' != persons_text]
return '<Cast (containing %s Person%s)>' % (persons_text, ('', 's')['' != persons_text])
def __repr__(self):
return self.__str__()
__repr__ = __str__
class CrewList(Persons):
@ -400,7 +518,7 @@ class CrewList(Persons):
super(CrewList, self).__init__(**kwargs)
for t in iterkeys(RoleTypes.reverse):
if t >= RoleTypes.crew_limit:
self[t] = [] # type: List[Person]
self[t] = [] # type: List[Crew]
def __str__(self):
persons_count = []
@ -412,8 +530,7 @@ class CrewList(Persons):
persons_text = ('0', '(%s)' % persons_text)['' != persons_text]
return '<Crew (containing %s Person%s)>' % (persons_text, ('', 's')['' != persons_text])
def __repr__(self):
return self.__str__()
__repr__ = __str__
class PersonBase(dict):
@ -425,13 +542,14 @@ class PersonBase(dict):
role,
sortorder
"""
def __init__(self, p_id=None, name=None, image=None, gender=None, bio=None, birthdate=None, deathdate=None,
country=None, country_code=None, country_timezone=None, **kwargs):
country=None, country_code=None, country_timezone=None, ids=None, thumb_url=None, **kwargs):
# type: (integer_types, AnyStr, AnyStr, int, AnyStr, datetime.date, datetime.date, AnyStr, AnyStr, AnyStr, Dict, AnyStr, Dict) -> PersonBase
super(PersonBase, self).__init__(**kwargs)
self.id = p_id # type: Optional[integer_types]
self.name = name # type: Optional[AnyStr]
self.image = image # type: Optional[AnyStr]
self.thumb_url = thumb_url # type: Optional[AnyStr]
self.gender = gender # type: Optional[int]
self.bio = bio # type: Optional[AnyStr]
self.birthdate = birthdate # type: Optional[datetime.date]
@ -439,25 +557,11 @@ class PersonBase(dict):
self.country = country # type: Optional[AnyStr]
self.country_code = country_code # type: Optional[AnyStr]
self.country_timezone = country_timezone # type: Optional[AnyStr]
self.ids = ids or {} # type: Dict[int, integer_types]
def calc_age(self, date=None):
# type: (Optional[datetime.date]) -> Optional[int]
if isinstance(self.birthdate, datetime.date):
today = (datetime.date.today(), date)[isinstance(date, datetime.date)]
today = (today, self.deathdate)[isinstance(self.deathdate, datetime.date) and today > self.deathdate]
try:
birthday = self.birthdate.replace(year=today.year)
# raised when birth date is February 29
# and the current year is not a leap year
except ValueError:
birthday = self.birthdate.replace(year=today.year,
month=self.birthdate.month + 1, day=1)
if birthday > today:
return today.year - birthday.year - 1
else:
return today.year - birthday.year
return calc_age(self.birthdate, self.deathdate, date)
@property
def age(self):
@ -467,50 +571,90 @@ class PersonBase(dict):
"""
return self.calc_age()
def __bool__(self):
# type: (...) -> bool
return bool(self.name)
def __str__(self):
return '<Person "%s">' % self.name
def __repr__(self):
return self.__str__()
__repr__ = __str__
__nonzero__ = __bool__
class PersonGenders(object):
unknown = 0
male = 1
female = 2
reverse = {1: 'Male', 2: 'Female'}
named = {'unknown': 0, 'male': 1, 'female': 2}
reverse = {v: k for k, v in iteritems(named)}
tmdb_map = {0: unknown, 1: female, 2: male}
imdb_map = {'female': female, 'male': male}
class Crew(PersonBase):
def __init__(self, crew_type_name=None, **kwargs):
super(Crew, self).__init__(**kwargs)
self.crew_type_name = crew_type_name
def __str__(self):
return '<Crew%s "%s)">' % (('', ('/%s' % self.crew_type_name))[isinstance(self.crew_type_name, string_types)],
self.name)
__repr__ = __str__
class Person(PersonBase):
def __init__(self, p_id=None, name=None, image=None, gender=None, bio=None, birthdate=None, deathdate=None,
country=None, country_code=None, country_timezone=None, **kwargs):
super(Person, self).__init__(p_id=p_id, name=name, image=image, gender=gender, bio=bio, birthdate=birthdate,
deathdate=deathdate, country=country, country_code=country_code,
country_timezone=country_timezone, **kwargs)
def __init__(self, p_id=None, name=None, image=None, thumb_url=None, gender=None, bio=None, birthdate=None, deathdate=None,
country=None, country_code=None, country_timezone=None, ids=None, homepage=None, social_ids=None,
birthplace=None, url=None, characters=None, height=None, deathplace=None, nicknames=None,
real_name=None, akas=None, **kwargs):
# type: (integer_types, AnyStr, AnyStr, AnyStr, int, AnyStr, datetime.date, datetime.date, AnyStr, AnyStr, AnyStr, Dict, AnyStr, Dict, AnyStr, AnyStr, List[Character], Union[integer_types, float], AnyStr, Set[AnyStr], AnyStr, Set[AnyStr], Dict) -> Person
super(Person, self).__init__(p_id=p_id, name=name, image=image, thumb_url=thumb_url, gender=gender, bio=bio,
birthdate=birthdate, deathdate=deathdate, country=country,
country_code=country_code, country_timezone=country_timezone, ids=ids, **kwargs)
self.credits = [] # type: List
self.homepage = homepage # type: Optional[AnyStr]
self.social_ids = social_ids or {} # type: Dict
self.birthplace = birthplace # type: Optional[AnyStr]
self.deathplace = deathplace # type: Optional[AnyStr]
self.nicknames = nicknames or set() # type: Set[AnyStr]
self.real_name = real_name # type: AnyStr
self.url = url # type: Optional[AnyStr]
self.height = height # type: Optional[Union[integer_types, float]]
self.akas = akas or set() # type: Set[AnyStr]
self.characters = characters or [] # type: List[Character]
def __str__(self):
return '<Person "%s">' % self.name
def __repr__(self):
return self.__str__()
__repr__ = __str__
class Character(PersonBase):
def __init__(self, person=None, voice=None, plays_self=None, **kwargs):
def __init__(self, person=None, voice=None, plays_self=None, regular=None, show=None, start_year=None,
end_year=None, **kwargs):
# type: (List[Person], bool, bool, bool, TVInfoShow, int, int, Dict) -> Character
super(Character, self).__init__(**kwargs)
self.person = person # type: Optional[Person]
self.person = person # type: List[Person]
self.voice = voice # type: Optional[bool]
self.plays_self = plays_self # type: Optional[bool]
self.regular = regular # type: Optional[bool]
self.show = show # type: Optional[TVInfoShow]
self.start_year = start_year # type: Optional[integer_types]
self.end_year = end_year # type: Optional[integer_types]
def __str__(self):
pn = ''
if None is not self.person and getattr(self.person, 'name', None):
pn = ' - (%s)' % getattr(self.person, 'name', '')
return '<Character "%s%s">' % (self.name, pn)
pn = []
if None is not self.person:
for p in self.person:
if getattr(p, 'name', None):
pn.append(p.name)
return '<Character "%s%s">' % (self.name, ('', ' - (%s)' % ', '.join(pn))[bool(pn)])
def __repr__(self):
return self.__str__()
__repr__ = __str__
class RoleTypes(object):
@ -523,22 +667,50 @@ class RoleTypes(object):
CrewDirector = 50
CrewWriter = 51
CrewProducer = 52
reverse = {1: 'Main', 2: 'Recurring', 3: 'Guest', 4: 'Special Guest', 50: 'Director', 51: 'Writer', 52: 'Producer'}
CrewExecutiveProducer = 53
CrewCreator = 60
CrewEditor = 61
CrewCamera = 62
CrewMusic = 63
CrewStylist = 64
CrewMakeup = 65
CrewPhotography = 66
CrewSound = 67
CrewDesigner = 68
CrewDeveloper = 69
CrewAnimation = 70
CrewVisualEffects = 71
CrewOther = 100
reverse = {1: 'Main', 2: 'Recurring', 3: 'Guest', 4: 'Special Guest', 50: 'Director', 51: 'Writer', 52: 'Producer',
53: 'Executive Producer', 60: 'Creator', 61: 'Editor', 62: 'Camera', 63: 'Music', 64: 'Stylist',
65: 'Makeup', 66: 'Photography', 67: 'Sound', 68: 'Designer', 69: 'Developer', 70: 'Animation',
71: 'Visual Effects', 100: 'Other'}
crew_limit = 50
crew_type_names = {c.lower(): v for v, c in iteritems(RoleTypes.reverse) if v >= RoleTypes.crew_limit}
class TVInfoBase(object):
def __init__(self, *args, **kwargs):
supported_id_searches = []
supported_person_id_searches = []
def __init__(self, banners=False, posters=False, seasons=False, seasonwides=False, fanart=False, actors=False,
*args, **kwargs):
global TVInfoShowContainer
if self.__class__.__name__ not in TVInfoShowContainer:
TVInfoShowContainer[self.__class__.__name__] = ShowContainer()
self.shows = TVInfoShowContainer[self.__class__.__name__] # type: ShowContainer
self.shows = TVInfoShowContainer[self.__class__.__name__] # type: ShowContainer[integer_types, TVInfoShow]
self.shows.cleanup_old()
self.lang = None # type: Optional[AnyStr]
self.corrections = {} # type: Dict
self.show_not_found = False # type: bool
self.not_found = False # type: bool
self._old_config = None
self._cachedir = kwargs.get('diskcache_dir') # type: AnyStr
self.diskcache = diskcache.Cache(directory=self._cachedir, disk_pickle_protocol=2) # type: diskcache.Cache
self.cache_expire = 60 * 60 * 18 # type: integer_types
self.config = {
'apikey': '',
'debug_enabled': False,
@ -550,68 +722,190 @@ class TVInfoBase(object):
'langabbv_to_id': {},
'language': 'en',
'base_url': '',
'banners_enabled': False,
'posters_enabled': False,
'seasons_enabled': False,
'seasonwides_enabled': False,
'fanart_enabled': False,
'actors_enabled': False,
'banners_enabled': banners,
'posters_enabled': posters,
'seasons_enabled': seasons,
'seasonwides_enabled': seasonwides,
'fanart_enabled': fanart,
'actors_enabled': actors,
} # type: Dict[AnyStr, Any]
def _must_load_data(self, sid, load_episodes):
# type: (integer_types, bool) -> bool
def _must_load_data(self, sid, load_episodes, banners, posters, seasons, seasonwides, fanart, actors):
# type: (integer_types, bool, bool, bool, bool, bool, bool, bool) -> bool
"""
returns if show data has to be fetched for (extra) data (episodes, images, ...)
or can taken from self.shows cache
:param sid: show id
:param load_episodes: should episodes be loaded
:param banners: should load banners
:param posters: should load posters
:param seasons: should load season images
:param seasonwides: should load season wide images
:param fanart: should load fanart
:param actors: should load actors
"""
if sid not in self.shows or None is self.shows[sid].id or \
(load_episodes and not getattr(self.shows[sid], 'ep_loaded', False)):
return True
for data_type, en_type in [(u'poster', 'posters_enabled'), (u'banner', 'banners_enabled'),
(u'fanart', 'fanart_enabled'), (u'season', 'seasons_enabled'),
(u'seasonwide', 'seasonwides_enabled'), (u'actors', 'actors_enabled')]:
if self.config.get(en_type, False) and not getattr(self.shows[sid], '%s_loaded' % data_type, False):
for data_type, en_type, p_type in [(u'poster', 'posters_enabled', posters),
(u'banner', 'banners_enabled', banners),
(u'fanart', 'fanart_enabled', fanart),
(u'season', 'seasons_enabled', seasons),
(u'seasonwide', 'seasonwides_enabled', seasonwides),
(u'actors', 'actors_enabled', actors)]:
if (p_type or self.config.get(en_type, False)) and \
not getattr(self.shows[sid], '%s_loaded' % data_type, False):
return True
return False
def get_person(self, p_id, **kwargs):
# type: (integer_types, Optional[Any]) -> Optional[Person]
def clear_cache(self):
"""
Clear cache.
"""
try:
with self.diskcache as dc:
dc.clear()
except (BaseException, Exception):
pass
def clean_cache(self):
"""
Remove expired items from cache.
"""
try:
with self.diskcache as dc:
dc.expire()
except (BaseException, Exception):
pass
def check_cache(self):
"""
checks cache
"""
try:
with self.diskcache as dc:
dc.check()
except (BaseException, Exception):
pass
def _get_cache_entry(self, key, retry=False):
# type: (Any, bool) -> Tuple[bool, Any]
"""
get person's data
returns tuple of is_None and value
:param key:
:param retry:
"""
with self.diskcache as dc:
try:
v = dc.get(key)
return 'None' == v, (v, None)['None' == v]
except ValueError as e:
if not retry:
dc.close()
try:
shutil.rmtree(self._cachedir)
except (BaseException, Exception) as e:
log.error(ex(e))
pass
try:
make_dirs(self._cachedir)
except (BaseException, Exception):
pass
return self._get_cache_entry(key, retry=True)
else:
log.error('Error getting %s from cache: %s' % (key, ex(e)))
except (BaseException, Exception) as e:
log.error('Error getting %s from cache: %s' % (key, ex(e)))
return False, None
def _set_cache_entry(self, key, value, tag=None):
# type: (Any, Any, AnyStr) -> None
try:
with self.diskcache as dc:
dc.set(key, (value, 'None')[None is value], expire=self.cache_expire, tag=tag)
except (BaseException, Exception) as e:
log.error('Error setting %s to cache: %s' % (key, ex(e)))
def get_person(self, p_id, get_show_credits=False, get_images=False, **kwargs):
# type: (integer_types, bool, bool, Any) -> Optional[Person]
"""
get person's data for id or list of matching persons for name
:param p_id: persons id
:param get_show_credits: get show credits
:param get_images: get images for person
:return: person object
"""
pass
def search_person(self, name):
# type: (AnyStr) -> List[Person]
def _search_person(self, name=None, ids=None):
# type: (AnyStr, Dict[integer_types, integer_types]) -> List[Person]
"""
search for person by name
:param name: name to search for
:param ids: dict of ids to search
:return: list of found person's
"""
pass
return []
def _get_show_data(self, sid, language, get_ep_info=False, **kwargs):
# type: (integer_types, AnyStr, bool, Optional[Any]) -> bool
def search_person(self, name=None, ids=None):
# type: (AnyStr, Dict[integer_types, integer_types]) -> List[Person]
"""
search for person by name
:param name: name to search for
:param ids: dict of ids to search
:return: list of found person's
"""
if not name and not ids:
log.debug('Nothing to search')
raise BaseTVinfoPersonNotFound('Nothing to search')
found_persons = []
if ids:
if not any(1 for i in ids if i in self.supported_person_id_searches) and not name:
log.debug('Id type not supported')
raise BaseTVinfoPersonNotFound('Id type not supported')
found_persons = self._search_person(name=name, ids=ids)
elif name:
found_persons = self._search_person(name=name, ids=ids)
return found_persons
def _get_show_data(self, sid, language, get_ep_info=False, banners=False, posters=False, seasons=False,
seasonwides=False, fanart=False, actors=False, **kwargs):
# type: (integer_types, AnyStr, bool, bool, bool, bool, bool, bool, bool, Optional[Any]) -> bool
"""
internal function that should be overwritten in class to get data for given show id
:param sid: show id
:param language: language
:param get_ep_info: get episodes
:param banners: load banners
:param posters: load posters
:param seasons: load seasons
:param seasonwides: load seasonwides
:param fanart: load fanard
:param actors: load actors
"""
pass
def get_show(self, show_id, load_episodes=True, **kwargs):
# type: (integer_types, bool, Optional[Any]) -> Optional[TVInfoShow]
def get_show(self, show_id, load_episodes=True, banners=False, posters=False, seasons=False,
seasonwides=False, fanart=False, actors=False, old_call=False, **kwargs):
# type: (integer_types, bool, bool, bool, bool, bool, bool, bool, bool, Optional[Any]) -> Optional[TVInfoShow]
"""
get data for show id
:param show_id: id of show
:param load_episodes: load episodes
:param banners: load banners
:param posters: load posters
:param seasons: load season images
:param seasonwides: load season wide images
:param fanart: load fanart
:param actors: load actors
:param old_call: load legacy call
:return: show object
"""
if not old_call and None is self._old_config:
self._old_config = self.config.copy()
self.config.update({'banners_enabled': banners, 'posters_enabled': posters, 'seasons_enabled': seasons,
'seasonwides_enabled': seasonwides, 'fanart_enabled': fanart, 'actors_enabled': actors})
self.shows.lock.acquire()
try:
if show_id not in self.shows:
@ -619,8 +913,10 @@ class TVInfoBase(object):
with self.shows[show_id].lock:
self.shows.lock.release()
try:
if self._must_load_data(show_id, load_episodes):
self._get_show_data(show_id, self.config['language'], load_episodes)
if self._must_load_data(show_id, load_episodes, banners, posters, seasons, seasonwides, fanart,
actors):
self._get_show_data(show_id, self.config['language'], load_episodes, banners, posters, seasons,
seasonwides, fanart, actors)
if None is self.shows[show_id].id:
with self.shows.lock:
del self.shows[show_id]
@ -637,27 +933,51 @@ class TVInfoBase(object):
self.shows.lock.release()
except RuntimeError:
pass
if not old_call and None is not self._old_config:
self.config = self._old_config
self._old_config = None
# noinspection PyMethodMayBeStatic
def _search_show(self, name, **kwargs):
# type: (AnyStr, Optional[Any]) -> List[Dict]
def _search_show(self, name=None, ids=None, **kwargs):
# type: (Union[AnyStr, List[AnyStr]], Dict[integer_types, integer_types], Optional[Any]) -> List[Dict]
"""
internal search function to find shows, should be overwritten in class
:param name: name to search for
:param ids: dict of ids {tvid: prodid} to search for
"""
return []
def search_show(self, name, **kwargs):
# type: (AnyStr, Optional[Any]) -> List[Dict]
@staticmethod
def _convert_search_names(name):
if name:
names = ([name], name)[isinstance(name, list)]
for i, n in enumerate(names):
if not isinstance(n, string_types):
names[i] = text_type(n)
names[i] = names[i].lower()
return names
return name
def search_show(self, name=None, ids=None, **kwargs):
# type: (Union[AnyStr, List[AnyStr]], Dict[integer_types, integer_types], Optional[Any]) -> List[Dict]
"""
search for series with name
:param name: series name to search for
:return: list of series
search for series with name(s) or ids
:param name: series name or list of names to search for
:param ids: dict of ids {tvid: prodid} to search for
:return: combined list of series results
"""
if not isinstance(name, string_types):
name = text_type(name)
name = name.lower()
selected_series = self._search_show(name)
if not name and not ids:
log.debug('Nothing to search')
raise BaseTVinfoShownotfound('Nothing to search')
name, selected_series = self._convert_search_names(name), []
if ids:
if not name and not any(1 for i in ids if i in self.supported_id_searches):
log.debug('Id type not supported')
raise BaseTVinfoShownotfound('Id type not supported')
selected_series = self._search_show(name=name, ids=ids)
elif name:
selected_series = self._search_show(name)
if isinstance(selected_series, dict):
selected_series = [selected_series]
if not isinstance(selected_series, list) or 0 == len(selected_series):
@ -717,6 +1037,40 @@ class TVInfoBase(object):
else:
self.shows[sid].__dict__[key] = value
def get_updated_shows(self):
# type: (...) -> Dict[integer_types, integer_types]
"""
gets all ids and timestamp of updated shows
returns dict of id: timestamp
"""
return {}
def get_trending(self, result_count=100, **kwargs):
# type: (...) -> List[TVInfoShow]
"""
get trending shows
:param result_count:
"""
return []
def get_popular(self, result_count=100, **kwargs):
# type: (...) -> List[TVInfoShow]
"""
get all popular shows
"""
return []
def get_top_rated(self, result_count=100, **kwargs):
# type: (...) -> List[TVInfoShow]
"""
get all latest shows
"""
return []
def discover(self, result_count=100, **kwargs):
# type: (...) -> List[TVInfoShow]
return []
def __getitem__(self, item):
# type: (Union[AnyStr, integer_types, Tuple[integer_types, bool]]) -> Union[TVInfoShow, List[Dict], None]
"""Legacy handler (use get_show or search_show instead)
@ -731,7 +1085,7 @@ class TVInfoBase(object):
if isinstance(item, integer_types):
# Item is integer, treat as show id
return self.get_show(item, (True, arg)[None is not arg])
return self.get_show(item, (True, arg)[None is not arg], old_call=True)
return self.search_show(item)
@ -746,5 +1100,4 @@ class TVInfoBase(object):
def __str__(self):
return '<TVInfo(%s) (containing: %s)>' % (self.__class__.__name__, text_type(self.shows))
def __repr__(self):
return self.__str__()
__repr__ = __str__

12
lib/tvinfo_base/exceptions.py

@ -60,3 +60,15 @@ class BaseTVinfoAuthenticationerror(BaseTVinfoError):
class BaseTVinfoIndexerInitError(BaseTVinfoError):
pass
class BaseTVinfoPersonError(BaseTVinfoError):
"""
"""
pass
class BaseTVinfoPersonNotFound(BaseTVinfoPersonError):
"""Raised when Person is not found
"""
pass

0
lib/tvmaze_api/__init__.py

503
lib/tvmaze_api/tvmaze_api.py

@ -0,0 +1,503 @@
# encoding:utf-8
# author:Prinz23
# project:tvmaze_api
__author__ = 'Prinz23'
__version__ = '1.0'
__api_version__ = '1.0.0'
import logging
import datetime
import requests
from requests.packages.urllib3.util.retry import Retry
from requests.adapters import HTTPAdapter
from six import iteritems
from sg_helpers import get_url, try_int
from lib.dateutil.parser import parser
from lib.dateutil.tz.tz import _datetime_to_timestamp
from lib.exceptions_helper import ConnectionSkipException, ex
from .tvmaze_exceptions import *
from lib.tvinfo_base import TVInfoBase, TVInfoImage, TVInfoImageSize, TVInfoImageType, Character, Crew, \
crew_type_names, Person, RoleTypes, TVInfoShow, TVInfoEpisode, TVInfoIDs, TVInfoSeason, PersonGenders, \
TVINFO_TVMAZE, TVINFO_TVDB, TVINFO_IMDB
from lib.pytvmaze import tvmaze
# noinspection PyUnreachableCode
if False:
from typing import Any, AnyStr, Dict, List, Optional, Union
from six import integer_types
log = logging.getLogger('tvmaze.api')
log.addHandler(logging.NullHandler())
# Query TVMaze free endpoints
def tvmaze_endpoint_standard_get(url):
s = requests.Session()
retries = Retry(total=5,
backoff_factor=0.1,
status_forcelist=[429])
s.mount('http://', HTTPAdapter(max_retries=retries))
s.mount('https://', HTTPAdapter(max_retries=retries))
return get_url(url, json=True, session=s, hooks={'response': tvmaze._record_hook}, raise_skip_exception=True)
tvmaze.TVMaze.endpoint_standard_get = staticmethod(tvmaze_endpoint_standard_get)
tvm_obj = tvmaze.TVMaze()
empty_ep = TVInfoEpisode()
empty_se = TVInfoSeason()
tz_p = parser()
img_type_map = {
'poster': TVInfoImageType.poster,
'banner': TVInfoImageType.banner,
'background': TVInfoImageType.fanart,
'typography': TVInfoImageType.typography,
}
img_size_map = {
'original': TVInfoImageSize.original,
'medium': TVInfoImageSize.medium,
}
show_map = {
'id': 'maze_id',
'ids': 'externals',
# 'slug': '',
'seriesid': 'maze_id',
'seriesname': 'name',
'aliases': 'akas',
# 'season': '',
'classification': 'type',
# 'genre': '',
'genre_list': 'genres',
# 'actors': '',
# 'cast': '',
# 'show_type': '',
# 'network': 'network',
# 'network_id': '',
# 'network_timezone': '',
# 'network_country': '',
# 'network_country_code': '',
# 'network_is_stream': '',
'runtime': 'runtime',
'language': 'language',
'official_site': 'official_site',
# 'imdb_id': '',
# 'zap2itid': '',
# 'airs_dayofweek': '',
# 'airs_time': '',
# 'time': '',
'firstaired': 'premiered',
# 'added': '',
# 'addedby': '',
# 'siteratingcount': '',
# 'lastupdated': '',
# 'contentrating': '',
'rating': 'rating',
'status': 'status',
'overview': 'summary',
# 'poster': 'image',
# 'poster_thumb': '',
# 'banner': '',
# 'banner_thumb': '',
# 'fanart': '',
# 'banners': '',
'updated_timestamp': 'updated',
}
season_map = {
'id': 'id',
'number': 'season_number',
'name': 'name',
# 'actors': '',
# 'cast': '',
# 'network': '',
# 'network_id': '',
# 'network_timezone': '',
# 'network_country': '',
# 'network_country_code': '',
# 'network_is_stream': '',
'ordered': '',
'start_date': 'premiere_date',
'end_date': 'end_date',
# 'poster': '',
'summery': 'summary',
'episode_order': 'episode_order',
}
class TvMaze(TVInfoBase):
supported_id_searches = [TVINFO_TVMAZE, TVINFO_TVDB, TVINFO_IMDB]
supported_person_id_searches = [TVINFO_TVMAZE]
def __init__(self, *args, **kwargs):
super(TvMaze, self).__init__(*args, **kwargs)
def _search_show(self, name=None, ids=None, **kwargs):
def _make_result_dict(s):
return {'seriesname': s.name, 'id': s.id, 'firstaired': s.premiered,
'network': s.network and s.network.name,
'genres': s.genres, 'overview': s.summary,
'aliases': [a.name for a in s.akas], 'image': s.image and s.image.get('original'),
'ids': TVInfoIDs(tvdb=s.externals.get('thetvdb'), rage=s.externals.get('tvrage'), tvmaze=s.id,
imdb=s.externals.get('imdb') and try_int(s.externals.get('imdb').replace('tt', ''),
None))}
results = []
if ids:
for t, p in iteritems(ids):
if t in self.supported_id_searches:
if t == TVINFO_TVDB:
try:
show = tvmaze.lookup_tvdb(p)
except (BaseException, Exception):
continue
elif t == TVINFO_IMDB:
try:
show = tvmaze.lookup_imdb((p, 'tt%07d' % p)[not str(p).startswith('tt')])
except (BaseException, Exception):
continue
elif t == TVINFO_TVMAZE:
try:
show = tvm_obj.get_show(maze_id=p)
except (BaseException, Exception):
continue
else:
continue
if show:
try:
if show.id not in [i['id'] for i in results]:
results.append(_make_result_dict(show))
except (BaseException, Exception) as e:
log.debug('Error creating result dict: %s' % ex(e))
if name:
for n in ([name], name)[isinstance(name, list)]:
try:
shows = tvmaze.show_search(n)
results = [_make_result_dict(s) for s in shows]
except (BaseException, Exception) as e:
log.debug('Error searching for show: %s' % ex(e))
return results
def _set_episode(self, sid, ep_obj):
for _k, _s in [('seasonnumber', 'season_number'), ('episodenumber', 'episode_number'),
('episodename', 'title'), ('overview', 'summary'), ('firstaired', 'airdate'),
('airtime', 'airtime'), ('runtime', 'runtime'),
('seriesid', 'maze_id'), ('id', 'maze_id'), ('is_special', 'special'),
('filename', 'image')]:
if 'filename' == _k:
image = getattr(ep_obj, _s, {}) or {}
image = image.get('original') or image.get('medium')
self._set_item(sid, ep_obj.season_number, ep_obj.episode_number, _k, image)
else:
self._set_item(sid, ep_obj.season_number, ep_obj.episode_number, _k,
getattr(ep_obj, _s, getattr(empty_ep, _k)))
if ep_obj.airstamp:
try:
at = _datetime_to_timestamp(tz_p.parse(ep_obj.airstamp))
self._set_item(sid, ep_obj.season_number, ep_obj.episode_number, 'timestamp', at)
except (BaseException, Exception):
pass
def _set_network(self, show_obj, network, is_stream):
show_obj['network'] = network.name
show_obj['network_timezone'] = network.timezone
show_obj['network_country'] = network.country
show_obj['network_country_code'] = network.code
show_obj['network_id'] = network.maze_id
show_obj['network_is_stream'] = is_stream
def _get_show_data(self, sid, language, get_ep_info=False, banners=False, posters=False, seasons=False,
seasonwides=False, fanart=False, actors=False, **kwargs):
log.debug('Getting all series data for %s' % sid)
try:
self.show_not_found = False
show_data = tvm_obj.get_show(maze_id=sid, embed='cast%s' % ('', ',episodes')[get_ep_info])
except tvmaze.ShowNotFound:
self.show_not_found = True
return False
except (BaseException, Exception) as e:
log.debug('Error getting data for tvmaze show id: %s' % sid)
return False
show_obj = self.shows[sid].__dict__
for k, v in iteritems(show_obj):
if k not in ('cast', 'crew', 'images'):
show_obj[k] = getattr(show_data, show_map.get(k, k), show_obj[k])
if show_data.image:
show_obj['poster'] = show_data.image.get('original')
show_obj['poster_thumb'] = show_data.image.get('medium')
if (banners or posters or fanart or
any(self.config.get('%s_enabled' % t, False) for t in ('banners', 'posters', 'fanart'))) and \
not all(getattr(self.shows[sid], '%s_loaded' % t, False) for t in ('poster', 'banner', 'fanart')):
if show_data.images:
b_set, f_set = False, False
self.shows[sid].poster_loaded = True
self.shows[sid].banner_loaded = True
self.shows[sid].fanart_loaded = True
for img in show_data.images:
img_type = img_type_map.get(img.type, TVInfoImageType.other)
img_src = {}
for res, img_url in iteritems(img.resolutions):
img_size = img_size_map.get(res)
if img_size:
img_src[img_size] = img_url.get('url')
show_obj['images'].setdefault(img_type, []).append(
TVInfoImage(image_type=img_type, sizes=img_src, img_id=img.id, main_image=img.main,
type_str=img.type))
if not b_set and 'banner' == img.type:
b_set = True
show_obj['banner'] = img.resolutions.get('original')['url']
show_obj['banner_thumb'] = img.resolutions.get('medium')['url']
elif not f_set and 'background' == img.type:
f_set = True
show_obj['fanart'] = img.resolutions.get('original')['url']
if show_data.schedule:
if 'time' in show_data.schedule:
show_obj['airs_time'] = show_data.schedule['time']
try:
h, m = show_data.schedule['time'].split(':')
h, m = try_int(h, None), try_int(m, None)
if None is not h and None is not m:
show_obj['time'] = datetime.time(hour=h, minute=m)
except (BaseException, Exception):
pass
if 'days' in show_data.schedule:
show_obj['airs_dayofweek'] = ', '.join(show_data.schedule['days'])
if show_data.genres:
show_obj['genre'] = ','.join(show_data.genres)
if (actors or self.config['actors_enabled']) and not getattr(self.shows.get(sid), 'actors_loaded', False):
if show_data.cast:
character_person_ids = {}
for ch in show_obj['cast'][RoleTypes.ActorMain]:
character_person_ids.setdefault(ch.id, []).extend([p.id for p in ch.person])
for ch in show_data.cast.characters:
existing_character = next((c for c in show_obj['cast'][RoleTypes.ActorMain] if c.id == ch.id),
None) # type: Optional[Character]
person = self._convert_person(ch.person)
if existing_character:
existing_person = next((p for p in existing_character.person
if person.id == p.ids.get(TVINFO_TVMAZE)),
None) # type: Person
if existing_person:
try:
character_person_ids[ch.id].remove(existing_person.id)
except (BaseException, Exception):
print('error')
pass
existing_person.p_id, existing_person.name, existing_person.image, existing_person.gender, \
existing_person.birthdate, existing_person.deathdate, existing_person.country, \
existing_person.country_code, existing_person.country_timezone, existing_person.thumb_url, \
existing_person.url, existing_person.ids = \
ch.person.id, ch.person.name, ch.person.image and ch.person.image.get('original'), \
PersonGenders.named.get(ch.person.gender and ch.person.gender.lower(),
PersonGenders.unknown),\
person.birthdate, person.deathdate,\
ch.person.country and ch.person.country.get('name'),\
ch.person.country and ch.person.country.get('code'),\
ch.person.country and ch.person.country.get('timezone'),\
ch.person.image and ch.person.image.get('medium'),\
ch.person.url, {TVINFO_TVMAZE: ch.person.id}
else:
existing_character.person.append(person)
else:
show_obj['cast'][RoleTypes.ActorMain].append(
Character(p_id=ch.id, name=ch.name, image=ch.image and ch.image.get('original'),
person=[person],
plays_self=ch.plays_self, thumb_url=ch.image and ch.image.get('medium')
))
if character_person_ids:
for c, p_ids in iteritems(character_person_ids):
if p_ids:
char = next((mc for mc in show_obj['cast'][RoleTypes.ActorMain] if mc.id == c),
None) # type: Optional[Character]
if char:
char.person = [p for p in char.person if p.id not in p_ids]
if show_data.cast:
show_obj['actors'] = [
{'character': {'id': ch.id,
'name': ch.name,
'url': 'https://www.tvmaze.com/character/view?id=%s' % ch.id,
'image': ch.image and ch.image.get('original'),
},
'person': {'id': ch.person and ch.person.id,
'name': ch.person and ch.person.name,
'url': ch.person and 'https://www.tvmaze.com/person/view?id=%s' % ch.person.id,
'image': ch.person and ch.person.image and ch.person.image.get('original'),
'birthday': None, # not sure about format
'deathday': None, # not sure about format
'gender': ch.person and ch.person.gender and ch.person.gender,
'country': ch.person and ch.person.country and ch.person.country.get('name'),
},
} for ch in show_data.cast.characters]
if show_data.crew:
for cw in show_data.crew:
rt = crew_type_names.get(cw.type.lower(), RoleTypes.CrewOther)
show_obj['crew'][rt].append(
Crew(p_id=cw.person.id, name=cw.person.name,
image=cw.person.image and cw.person.image.get('original'),
gender=cw.person.gender, birthdate=cw.person.birthday, deathdate=cw.person.death_day,
country=cw.person.country and cw.person.country.get('name'),
country_code=cw.person.country and cw.person.country.get('code'),
country_timezone=cw.person.country and cw.person.country.get('timezone'),
crew_type_name=cw.type,
)
)
if show_data.externals:
show_obj['ids'] = TVInfoIDs(tvdb=show_data.externals.get('thetvdb'),
rage=show_data.externals.get('tvrage'),
imdb=show_data.externals.get('imdb') and
try_int(show_data.externals.get('imdb').replace('tt', ''), None))
if show_data.network:
self._set_network(show_obj, show_data.network, False)
elif show_data.web_channel:
self._set_network(show_obj, show_data.web_channel, True)
if get_ep_info and not getattr(self.shows.get(sid), 'ep_loaded', False):
log.debug('Getting all episodes of %s' % sid)
if None is show_data:
try:
self.show_not_found = False
show_data = tvm_obj.get_show(maze_id=sid, embed='cast%s' % ('', ',episodes')[get_ep_info])
except tvmaze.ShowNotFound:
self.show_not_found = True
return False
except (BaseException, Exception) as e:
log.debug('Error getting data for tvmaze show id: %s' % sid)
return False
if show_data.episodes:
specials = []
for cur_ep in show_data.episodes:
if cur_ep.is_special():
specials.append(cur_ep)
else:
self._set_episode(sid, cur_ep)
if specials:
specials.sort(key=lambda ep: ep.airstamp or 'Last')
for ep_n, cur_sp in enumerate(specials, start=1):
cur_sp.season_number, cur_sp.episode_number = 0, ep_n
self._set_episode(sid, cur_sp)
if show_data.seasons:
for cur_s_k, cur_s_v in iteritems(show_data.seasons):
season_obj = None
if cur_s_v.season_number not in self.shows[sid]:
if all(_e.is_special() for _e in cur_s_v.episodes or []):
season_obj = self.shows[sid][0].__dict__
else:
log.error('error episodes have no numbers')
season_obj = season_obj or self.shows[sid][cur_s_v.season_number].__dict__
for k, v in iteritems(season_map):
season_obj[k] = getattr(cur_s_v, v, None) or empty_se.get(v)
if cur_s_v.network:
self._set_network(season_obj, cur_s_v.network, False)
elif cur_s_v.web_channel:
self._set_network(season_obj, cur_s_v.web_channel, True)
if cur_s_v.image:
season_obj['poster'] = cur_s_v.image.get('original')
self.shows[sid].season_images_loaded = True
self.shows[sid].ep_loaded = True
return True
def get_updated_shows(self):
# type: (...) -> Dict[integer_types, integer_types]
return {sid: v.seconds_since_epoch for sid, v in iteritems(tvmaze.show_updates().updates)}
@staticmethod
def _convert_person(person_obj):
# type: (tvmaze.Person) -> Person
ch = []
for c in person_obj.castcredits or []:
show = TVInfoShow()
show.seriesname = c.show.name
show.id = c.show.id
show.firstaired = c.show.premiered
show.ids = TVInfoIDs(ids={TVINFO_TVMAZE: show.id})
show.overview = c.show.summary
show.status = c.show.status
net = c.show.network or c.show.web_channel
show.network = net.name
show.network_id = net.maze_id
show.network_country = net.country
show.network_timezone = net.timezone
show.network_country_code = net.code
show.network_is_stream = None is not c.show.web_channel
ch.append(Character(name=c.character.name, show=show))
try:
birthdate = person_obj.birthday and tz_p.parse(person_obj.birthday).date()
except (BaseException, Exception):
birthdate = None
try:
deathdate = person_obj.death_day and tz_p.parse(person_obj.death_day).date()
except (BaseException, Exception):
deathdate = None
return Person(p_id=person_obj.id, name=person_obj.name,
image=person_obj.image and person_obj.image.get('original'),
gender=PersonGenders.named.get(person_obj.gender and person_obj.gender.lower(),
PersonGenders.unknown),
birthdate=birthdate, deathdate=deathdate,
country=person_obj.country and person_obj.country.get('name'),
country_code=person_obj.country and person_obj.country.get('code'),
country_timezone=person_obj.country and person_obj.country.get('timezone'),
thumb_url=person_obj.image and person_obj.image.get('medium'),
url=person_obj.url, ids={TVINFO_TVMAZE: person_obj.id}, characters=ch
)
def _search_person(self, name=None, ids=None):
# type: (AnyStr, Dict[integer_types, integer_types]) -> List[Person]
urls, result, ids = [], [], ids or {}
for tv_src in self.supported_person_id_searches:
if tv_src in ids:
if TVINFO_TVMAZE == tv_src:
try:
r = self.get_person(ids[tv_src])
except ConnectionSkipException as e:
raise e
except (BaseException, Exception):
r = None
if r:
result.append(r)
if name:
try:
r = tvmaze.people_search(name)
except ConnectionSkipException as e:
raise e
except (BaseException, Exception):
r = None
if r:
for p in r:
if not any(1 for ep in result if p.id == ep.id):
result.append(self._convert_person(p))
return result
def get_person(self, p_id, get_show_credits=False, get_images=False, **kwargs):
# type: (integer_types, bool, bool, Any) -> Optional[Person]
if not p_id:
return
kw = {}
to_embed = []
if get_show_credits:
to_embed.append('castcredits')
if to_embed:
kw['embed'] = ','.join(to_embed)
try:
p = tvmaze.person_main_info(p_id, **kw)
except ConnectionSkipException as e:
raise e
except (BaseException, Exception):
p = None
if p:
return self._convert_person(p)

62
lib/tvmaze_api/tvmaze_exceptions.py

@ -0,0 +1,62 @@
# encoding:utf-8
"""Custom exceptions used or raised by tvmaze_api
"""
__author__ = 'Prinz23'
__version__ = '1.0'
__all__ = ['TvMazeException', 'TvMazeError', 'TvMazeUserabort', 'TvMazeShownotfound',
'TvMazeSeasonnotfound', 'TvMazeEpisodenotfound', 'TvMazeAttributenotfound', 'TvMazeTokenexpired']
from lib.tvinfo_base.exceptions import *
class TvMazeException(BaseTVinfoException):
"""Any exception generated by tvdb_api
"""
pass
class TvMazeError(BaseTVinfoError, TvMazeException):
"""An error with thetvdb.com (Cannot connect, for example)
"""
pass
class TvMazeUserabort(BaseTVinfoUserabort, TvMazeError):
"""User aborted the interactive selection (via
the q command, ^c etc)
"""
pass
class TvMazeShownotfound(BaseTVinfoShownotfound, TvMazeError):
"""Show cannot be found on thetvdb.com (non-existant show)
"""
pass
class TvMazeSeasonnotfound(BaseTVinfoSeasonnotfound, TvMazeError):
"""Season cannot be found on thetvdb.com
"""
pass
class TvMazeEpisodenotfound(BaseTVinfoEpisodenotfound, TvMazeError):
"""Episode cannot be found on thetvdb.com
"""
pass
class TvMazeAttributenotfound(BaseTVinfoAttributenotfound, TvMazeError):
"""Raised if an episode does not have the requested
attribute (such as a episode name)
"""
pass
class TvMazeTokenexpired(BaseTVinfoAuthenticationerror, TvMazeError):
"""token expired or missing thetvdb.com
"""
pass

42
sickbeard/__init__.py

@ -38,7 +38,7 @@ import zlib
# noinspection PyPep8Naming
import encodingKludge as ek
from . import classes, db, helpers, image_cache, indexermapper, logger, metadata, naming, providers, \
from . import classes, db, helpers, image_cache, indexermapper, logger, metadata, naming, people_queue, providers, \
scene_exceptions, scene_numbering, scheduler, search_backlog, search_propers, search_queue, search_recent, \
show_queue, show_updater, subtitles, trakt_helpers, traktChecker, version_checker, watchedstate_queue
from . import auto_post_processer, properFinder # must come after the above imports
@ -46,7 +46,7 @@ from .common import SD, SKIPPED, USER_AGENT
from .config import check_section, check_setting_int, check_setting_str, ConfigMigrator, minimax
from .databases import cache_db, failed_db, mainDB
from .indexers.indexer_api import TVInfoAPI
from .indexers.indexer_config import TVINFO_IMDB, TVINFO_TVDB
from .indexers.indexer_config import TVINFO_IMDB, TVINFO_TVDB, TmdbIndexer
from .providers.generic import GenericProvider
from .providers.newznab import NewznabConstants
from .tv import TVidProdid
@ -69,6 +69,7 @@ if False:
from adba import Connection
from .event_queue import Events
from .tv import TVShow
from lib.libtrakt.trakt import TraktAccount
PID = None
ENV = {}
@ -95,6 +96,7 @@ events = None # type: Events
recent_search_scheduler = None
backlog_search_scheduler = None
show_update_scheduler = None
people_queue_scheduler = None
update_software_scheduler = None
update_packages_scheduler = None
show_queue_scheduler = None
@ -113,6 +115,7 @@ provider_ping_thread_pool = {}
showList = [] # type: List[TVShow]
showDict = {} # type: Dict[int, TVShow]
switched_shows = {} # type: Dict[AnyStr, AnyStr]
UPDATE_SHOWS_ON_START = False
SHOW_UPDATE_HOUR = 3
@ -586,7 +589,7 @@ WANTEDLIST_CACHE = None
CALENDAR_UNPROTECTED = False
TMDB_API_KEY = 'edc5f123313769de83a71e157758030b'
TMDB_API_KEY = TmdbIndexer.API_KEY
FANART_API_KEY = '3728ca1a2a937ba0c93b6e63cc86cecb'
# to switch between staging and production TRAKT environment
@ -595,7 +598,7 @@ TRAKT_STAGING = False
TRAKT_TIMEOUT = 60
TRAKT_VERIFY = True
TRAKT_CONNECTED_ACCOUNT = None
TRAKT_ACCOUNTS = {}
TRAKT_ACCOUNTS = {} # type: Dict[int, TraktAccount]
TRAKT_MRU = ''
if TRAKT_STAGING:
@ -622,7 +625,10 @@ CACHE_IMAGE_URL_LIST = classes.ImageUrlList()
__INITIALIZED__ = False
__INIT_STAGE__ = 0
# don't reassign MEMCACHE var without reassigning sg_helpers.MEMCACHE
# as long as the pointer is the same (dict only modified) all is fine
MEMCACHE = {}
sg_helpers.MEMCACHE = MEMCACHE
MEMCACHE_FLAG_IMAGES = {}
@ -651,7 +657,7 @@ def initialize(console_logging=True):
def init_stage_1(console_logging):
# Misc
global showList, showDict, providerList, newznabProviderList, torrentRssProviderList, \
global showList, showDict, switched_shows, providerList, newznabProviderList, torrentRssProviderList, \
WEB_HOST, WEB_ROOT, ACTUAL_CACHE_DIR, CACHE_DIR, ZONEINFO_DIR, ADD_SHOWS_WO_DIR, ADD_SHOWS_METALANG, \
CREATE_MISSING_SHOW_DIRS, SHOW_DIRS_WITH_DOTS, \
RECENTSEARCH_STARTUP, NAMING_FORCE_FOLDERS, SOCKET_TIMEOUT, DEBUG, TVINFO_DEFAULT, \
@ -1004,6 +1010,7 @@ def init_stage_1(console_logging):
SEARCH_UNAIRED = bool(check_setting_int(CFG, 'General', 'search_unaired', 0))
UNAIRED_RECENT_SEARCH_ONLY = bool(check_setting_int(CFG, 'General', 'unaired_recent_search_only', 1))
FLARESOLVERR_HOST = check_setting_str(CFG, 'General', 'flaresolverr_host', '')
sg_helpers.FLARESOLVERR_HOST = FLARESOLVERR_HOST
NZB_DIR = check_setting_str(CFG, 'Blackhole', 'nzb_dir', '')
TORRENT_DIR = check_setting_str(CFG, 'Blackhole', 'torrent_dir', '')
@ -1026,6 +1033,7 @@ def init_stage_1(console_logging):
ADD_SHOWS_WO_DIR = bool(check_setting_int(CFG, 'General', 'add_shows_wo_dir', 0))
ADD_SHOWS_METALANG = check_setting_str(CFG, 'General', 'add_shows_metalang', 'en')
REMOVE_FILENAME_CHARS = check_setting_str(CFG, 'General', 'remove_filename_chars', '')
sg_helpers.REMOVE_FILENAME_CHARS = REMOVE_FILENAME_CHARS
IMPORT_DEFAULT_CHECKED_SHOWS = bool(check_setting_int(CFG, 'General', 'import_default_checked_shows', 0))
SAB_USERNAME = check_setting_str(CFG, 'SABnzbd', 'sab_username', '')
@ -1493,6 +1501,9 @@ def init_stage_1(console_logging):
showList = []
showDict = {}
# dict of switched shows for web redirects
switched_shows = {}
def init_stage_2():
@ -1500,7 +1511,7 @@ def init_stage_2():
global __INITIALIZED__, MEMCACHE, MEMCACHE_FLAG_IMAGES, RECENTSEARCH_STARTUP
# Schedulers
# global trakt_checker_scheduler
global recent_search_scheduler, backlog_search_scheduler, show_update_scheduler, \
global recent_search_scheduler, backlog_search_scheduler, people_queue_scheduler, show_update_scheduler, \
update_software_scheduler, update_packages_scheduler, show_queue_scheduler, search_queue_scheduler, \
proper_finder_scheduler, media_process_scheduler, subtitles_finder_scheduler, \
background_mapping_task, \
@ -1579,6 +1590,12 @@ def init_stage_2():
threadName='SHOWUPDATER',
prevent_cycle_run=show_queue_scheduler.action.isShowUpdateRunning) # 3AM
people_queue_scheduler = scheduler.Scheduler(
people_queue.PeopleQueue(),
cycleTime=datetime.timedelta(seconds=3),
threadName='PEOPLEQUEUE'
)
# searchers
search_queue_scheduler = scheduler.Scheduler(
search_queue.SearchQueue(),
@ -1650,7 +1667,8 @@ def init_stage_2():
threadName='FINDSUBTITLES',
silent=not USE_SUBTITLES)
background_mapping_task = threading.Thread(name='MAPPINGSUPDATER', target=indexermapper.load_mapped_ids)
background_mapping_task = threading.Thread(name='MAPPINGSUPDATER', target=indexermapper.load_mapped_ids,
kwargs={'load_all': True})
watched_state_queue_scheduler = scheduler.Scheduler(
watchedstate_queue.WatchedStateQueue(),
@ -1686,7 +1704,7 @@ def init_stage_2():
def enabled_schedulers(is_init=False):
# ([], [trakt_checker_scheduler])[USE_TRAKT] + \
return ([], [events])[is_init] \
+ ([], [recent_search_scheduler, backlog_search_scheduler, show_update_scheduler,
+ ([], [recent_search_scheduler, backlog_search_scheduler, show_update_scheduler, people_queue_scheduler,
update_software_scheduler, update_packages_scheduler,
show_queue_scheduler, search_queue_scheduler, proper_finder_scheduler,
media_process_scheduler, subtitles_finder_scheduler,
@ -1703,7 +1721,8 @@ def start():
# Load all Indexer mappings in background
indexermapper.defunct_indexer = [
i for i in TVInfoAPI().all_sources if TVInfoAPI(i).config.get('defunct')]
indexermapper.indexer_list = [i for i in TVInfoAPI().all_sources]
indexermapper.indexer_list = [i for i in TVInfoAPI().all_sources if TVInfoAPI(i).config.get('show_url')
and True is not TVInfoAPI(i).config.get('people_only')]
background_mapping_task.start()
for p in providers.sortedProviderList():
@ -1779,6 +1798,11 @@ def halt():
for thread in enabled_schedulers(): # type: scheduler.Scheduler
try:
thread.stopit()
if getattr(thread, 'action', None) and getattr(thread.action, 'save_queue', None):
try:
thread.action.save_queue()
except (BaseException, Exception):
pass
except Exception as e:
logger.log('Thread %s stop failed with: %s' % (thread.name, e))

38
sickbeard/databases/cache_db.py

@ -21,8 +21,8 @@ from collections import OrderedDict
from .. import db
MIN_DB_VERSION = 1
MAX_DB_VERSION = 6
TEST_BASE_VERSION = None # the base production db version, only needed for TEST db versions (>=100000)
MAX_DB_VERSION = 100002
TEST_BASE_VERSION = 6 # the base production db version, only needed for TEST db versions (>=100000)
# Add new migrations at the bottom of the list; subclass the previous migration.
@ -70,6 +70,31 @@ class InitialSchema(db.SchemaUpgrade):
' failure_count NUMERIC, failure_time NUMERIC,'
' tmr_limit_count NUMERIC, tmr_limit_time NUMERIC, tmr_limit_wait NUMERIC)'
]),
('save_queues', [
'CREATE TABLE people_queue(indexer NUMERIC NOT NULL, indexer_id NUMERIC NOT NULL,'
' action_id NUMERIC NOT NULL, forced INTEGER DEFAULT 0, scheduled INTEGER DEFAULT 0,'
' uid NUMERIC NOT NULL)',
'CREATE UNIQUE INDEX idx_people_queue ON people_queue (indexer,indexer_id,action_id)',
'CREATE UNIQUE INDEX idx_people_queue_uid ON people_queue (uid)',
'CREATE TABLE search_queue(indexer NUMERIC NOT NULL, indexer_id NUMERIC NOT NULL,'
' segment TEXT NOT NULL, standard_backlog INTEGER DEFAULT 0, limited_backlog INTEGER DEFAULT 0,'
' forced INTEGER DEFAULT 0, torrent_only INTEGER DEFAULT 0, action_id INTEGER NOT NULL,'
' uid NUMERIC NOT NULL )',
'CREATE UNIQUE INDEX idx_search_queue ON search_queue'
' (indexer, indexer_id, segment, standard_backlog, limited_backlog, forced, torrent_only, action_id)',
'CREATE UNIQUE INDEX idx_search_queue_uid ON search_queue (uid)',
'CREATE TABLE show_queue(tvid NUMERIC NOT NULL, prodid NUMERIC NOT NULL, priority INTEGER DEFAULT 20,'
' force INTEGER DEFAULT 0, scheduled_update INTEGER DEFAULT 0, after_update INTEGER DEFAULT 0,'
' force_image_cache INTEGER DEFAULT 0, show_dir TEXT, default_status NUMERIC, quality NUMERIC,'
' flatten_folders INTEGER, lang TEXT, subtitles INTEGER DEFAULT 0, anime INTEGER,'
' scene INTEGER, paused INTEGER, blocklist TEXT, allowlist TEXT,'
' wanted_begin NUMERIC, wanted_latest NUMERIC, prune NUMERIC DEFAULT 0, tag TEXT,'
' new_show INTEGER DEFAULT 0, show_name TEXT, upgrade_once INTEGER DEFAULT 0,'
' pausestatus_after INTEGER, skip_refresh INTEGER DEFAULT 0, action_id INTEGER NOT NULL,'
' uid NUMERIC NOT NULL)',
'CREATE UNIQUE INDEX idx_show_queue_uid ON show_queue(uid)',
'CREATE UNIQUE INDEX idx_show_queue ON show_queue(tvid, prodid, action_id)'
])
])
def test(self):
@ -131,3 +156,12 @@ class AddGenericFailureHandling(AddBacklogParts):
def execute(self):
self.do_query(self.queries['connection_fails'])
self.finish()
class AddSaveQueues(AddGenericFailureHandling):
def test(self):
return 6 < self.checkDBVersion()
def execute(self):
self.do_query(self.queries['save_queues'])
self.setDBVersion(100002, check_db_version=False)

190
sickbeard/databases/mainDB.py

@ -28,8 +28,8 @@ import encodingKludge as ek
from six import iteritems
MIN_DB_VERSION = 9 # oldest db version we support migrating from
MAX_DB_VERSION = 20014
TEST_BASE_VERSION = None # the base production db version, only needed for TEST db versions (>=100000)
MAX_DB_VERSION = 100008
TEST_BASE_VERSION = 20014 # the base production db version, only needed for TEST db versions (>=100000)
class MainSanityCheck(db.DBSanityCheck):
@ -1627,13 +1627,14 @@ class AddShowExludeGlobals(db.SchemaUpgrade):
def execute(self):
if not self.hasColumn('tv_shows', 'rls_global_exclude_ignore'):
logger.log('Adding rls_global_exclude_ignore, rls_global_exclude_require to tv_shows')
self.upgrade_log('Adding rls_global_exclude_ignore, rls_global_exclude_require to tv_shows')
db.backup_database('sickbeard.db', self.checkDBVersion())
self.addColumn('tv_shows', 'rls_global_exclude_ignore', data_type='TEXT', default='')
self.addColumn('tv_shows', 'rls_global_exclude_require', data_type='TEXT', default='')
if self.hasTable('tv_shows_exclude_backup'):
self.upgrade_log('Adding rls_global_exclude_ignore, rls_global_exclude_require from backup to tv_shows')
self.connection.mass_action([['UPDATE tv_shows SET rls_global_exclude_ignore = '
'(SELECT te.rls_global_exclude_ignore FROM tv_shows_exclude_backup te WHERE '
'te.show_id = tv_shows.show_id AND te.indexer = tv_shows.indexer), '
@ -1690,3 +1691,186 @@ class AddHistoryHideColumn(db.SchemaUpgrade):
])
return self.setDBVersion(20014)
# 20014 -> 100008
class ChangeShowData(db.SchemaUpgrade):
def execute(self):
self.upgrade_log('Adding new data columns to tv_shows')
self.addColumns('tv_shows', [('timezone', 'TEXT', ''), ('airtime', 'NUMERIC'),
('network_country', 'TEXT', ''), ('network_country_code', 'TEXT', ''),
('network_id', 'NUMERIC'), ('network_is_stream', 'INTEGER'),
('src_update_timestamp', 'INTEGER')])
self.upgrade_log('Adding new data columns to tv_episodes')
self.addColumns('tv_episodes', [('timezone', 'TEXT', ''), ('airtime', 'NUMERIC'),
('runtime', 'NUMERIC', 0), ('timestamp', 'NUMERIC'),
('network', 'TEXT', ''), ('network_country', 'TEXT', ''),
('network_country_code', 'TEXT', ''), ('network_id', 'NUMERIC'),
('network_is_stream', 'INTEGER')])
self.upgrade_log('Adding Character and Persons tables')
table_create_sql = {
'castlist': [
['CREATE TABLE castlist'
' ('
' id INTEGER PRIMARY KEY AUTOINCREMENT,'
' indexer NUMERIC NOT NULL,'
' indexer_id NUMERIC NOT NULL,'
' character_id NUMERIC NOT NULL,'
' sort_order NUMERIC DEFAULT 0 NOT NULL,'
' updated NUMERIC'
' );'],
],
'idx_castlist': [
['CREATE INDEX idx_castlist ON castlist (indexer, indexer_Id);'],
['CREATE UNIQUE INDEX idx_unique_castlist ON castlist (indexer, indexer_id, character_id);']
],
'characters': [
['CREATE TABLE characters'
' ('
' id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,'
' name TEXT,'
' bio TEXT,'
' thumb_url TEXT,'
' image_url TEXT,'
' updated NUMERIC'
' );']
],
'character_ids': [
['CREATE TABLE character_ids'
' ('
' id INTEGER PRIMARY KEY AUTOINCREMENT,'
' src NUMERIC NOT NULL,'
' src_id NUMERIC NOT NULL,'
' character_id NUMERIC NOT NULL'
' );'],
],
'idx_character_ids': [
['CREATE UNIQUE INDEX idx_unique_character_ids ON character_ids (src, character_id);'],
['CREATE INDEX idx_character_ids ON character_ids (character_id);']
],
'character_person_map': [
['CREATE TABLE character_person_map'
' ('
' id INTEGER PRIMARY KEY AUTOINCREMENT,'
' character_id NUMERIC NOT NULL,'
' person_id NUMERIC NOT NULL'
' );'],
],
'idx_character_person_map': [
['CREATE INDEX idx_character_person_map_character ON character_person_map (character_id);'],
['CREATE INDEX idx_character_person_map_person ON character_person_map (person_id);'],
['CREATE UNIQUE INDEX idx_unique_character_person ON character_person_map (character_id, person_id);']
],
'character_person_years': [
[
"""CREATE TABLE character_person_years
(
character_id NUMERIC NOT NULL,
person_id NUMERIC NOT NULL,
start_year NUMERIC,
end_year NUMERIC
);
"""
],
],
'idx_character_person_years': [
['CREATE UNIQUE INDEX idx_unique_character_person_years '
'ON character_person_years (character_id, person_id)'],
['CREATE INDEX idx_character_person_years ON character_person_years (character_id)'],
],
'persons': [
['CREATE TABLE persons'
' ('
' id INTEGER PRIMARY KEY AUTOINCREMENT,'
' name TEXT,'
' gender INTEGER,'
' birthdate NUMERIC,'
' deathdate NUMERIC,'
' birthplace TEXT,'
' deathplace TEXT,'
' height NUMERIC,'
' realname TEXT,'
' nicknames TEXT,'
' akas TEXT,'
' homepage TEXT,'
' bio TEXT,'
' thumb_url TEXT,'
' image_url TEXT,'
' updated NUMERIC'
' );']
],
'person_ids': [
['CREATE TABLE person_ids'
' ('
' id INTEGER PRIMARY KEY AUTOINCREMENT,'
' src INTEGER NOT NULL,'
' src_id TEXT NOT NULL,'
' person_id NUMERIC NOT NULL'
' );'],
],
'idx_person_ids': [
['CREATE UNIQUE INDEX idx_unique_person_ids ON person_ids (src, person_id);'],
['CREATE INDEX idx_person_ids ON person_ids (person_id);']
],
'tv_src_switch': [
['CREATE TABLE tv_src_switch'
' ('
' old_indexer NUMERIC NOT NULL,'
' old_indexer_id NUMERIC NOT NULL,'
' new_indexer NUMERIC NOT NULL,'
' new_indexer_id NUMERIC,'
' force_id NUMERIC DEFAULT 0 NOT NULL,'
' set_pause INTEGER DEFAULT 0 NOT NULL,'
' mark_wanted INTEGER DEFAULT 0 NOT NULL,'
' status NUMERIC DEFAULT 0 NOT NULL,'
' action_id INTEGER NOT NULL,'
' uid NUMERIC NOT NULL'
' );'
],
],
'idx_tv_src_switch': [
['CREATE UNIQUE INDEX idx_unique_tv_src_switch ON tv_src_switch (old_indexer, old_indexer_id);']
],
'switch_ep_errors': [
['CREATE TABLE switch_ep_errors'
' ('
' old_indexer NUMERIC NOT NULL,'
' old_indexer_id NUMERIC NOT NULL,'
' new_indexer NUMERIC NOT NULL,'
' new_indexer_id NUMERIC NOT NULL,'
' season NUMERIC NOT NULL,'
' episode NUMERIC NOT NULL,'
' reason NUMERIC DEFAULT 0'
' );'
],
],
'idx_switch_ep_errors': [
['CREATE INDEX idx_switch_ep_errors ON switch_ep_errors (new_indexer, new_indexer_id)'],
['CREATE INDEX idx_unique_switch_ep_errors'
' ON switch_ep_errors (new_indexer, new_indexer_id, season, episode, reason)'],
['CREATE INDEX idx_switch_ep_errors_old ON switch_ep_errors (old_indexer, old_indexer_id)'],
],
}
cl = []
tables = self.list_tables()
for t in ('castlist', 'characters', 'character_ids', 'persons', 'person_ids', 'character_person_map',
'character_person_years', 'tv_src_switch', 'switch_ep_errors'):
if 'backup_%s' % t in tables:
cl.append(['ALTER TABLE backup_%s RENAME TO %s' % (t, t)])
elif t not in tables:
cl.extend(table_create_sql[t])
if 'idx_%s' % t in table_create_sql:
cl.extend(table_create_sql['idx_%s' % t])
cl.extend(sickbeard.tv.TVShow.orphaned_cast_sql())
if cl:
self.connection.mass_action(cl)
self.connection.action('VACUUM')
self.setDBVersion(100008)
return self.checkDBVersion()

57
sickbeard/db.py

@ -36,7 +36,7 @@ from .sgdatetime import timestamp_near
from sg_helpers import make_dirs, compress_file, remove_file_perm, scantree
from _23 import filter_iter, list_values, scandir
from _23 import filter_iter, filter_list, list_values, scandir
from six import iterkeys, iteritems, itervalues
# noinspection PyUnreachableCode
@ -45,7 +45,9 @@ if False:
db_lock = threading.Lock()
db_support_multiple_insert = (3, 7, 11) <= sqlite3.sqlite_version_info # type: bool
db_support_column_rename = (3, 25, 0) <= sqlite3.sqlite_version_info # type: bool
db_support_upsert = (3, 25, 0) <= sqlite3.sqlite_version_info # type: bool
db_supports_backup = hasattr(sqlite3.Connection, 'backup') and (3, 6, 11) <= sqlite3.sqlite_version_info # type: bool
@ -498,16 +500,57 @@ class SchemaUpgrade(object):
def hasColumn(self, table_name, column):
return column in self.connection.tableInfo(table_name)
def list_tables(self):
# type: (...) -> List[AnyStr]
"""
returns list of all table names in db
"""
return [s['name'] for s in self.connection.select('SELECT name FROM main.sqlite_master WHERE type = ?;',
['table'])]
def list_indexes(self):
# type: (...) -> List[AnyStr]
"""
returns list of all index names in db
"""
return [s['name'] for s in self.connection.select('SELECT name FROM main.sqlite_master WHERE type = ?;',
['index'])]
# noinspection SqlResolve
def addColumn(self, table, column, data_type='NUMERIC', default=0, set_default=False):
self.connection.action('ALTER TABLE [%s] ADD %s %s%s' %
(table, column, data_type, ('', ' DEFAULT "%s"' % default)[set_default]))
self.connection.action('UPDATE [%s] SET %s = ?' % (table, column), (default,))
def dropColumn(self, table, column):
# noinspection SqlResolve
def addColumns(self, table, column_list=None):
# type: (AnyStr, List) -> None
if isinstance(column_list, list):
sql = []
for col in column_list:
is_list = isinstance(col, (list, tuple))
list_len = 0 if not is_list else len(col)
column = col if not is_list else col[0]
data_type = 'NUMERIC' if not is_list or 2 > list_len else col[1]
default = 0 if not is_list or 3 > list_len else col[2]
sql.append(['ALTER TABLE [%s] ADD %s %s%s' %
(table, column, data_type, '' if list_len < 3 else
' DEFAULT %s' % ('""' if 'TEXT' == data_type and '' == default else default))])
if 2 < list_len:
sql.append(['UPDATE [%s] SET %s = ?' % (table, column), (default,)])
if sql:
self.connection.mass_action(sql)
def dropColumn(self, table, columns):
# type: (AnyStr, AnyStr) -> None
self.drop_columns(table, columns)
def drop_columns(self, table, column):
# type: (AnyStr, Union[AnyStr, List[AnyStr]]) -> None
# get old table columns and store the ones we want to keep
result = self.connection.select('pragma table_info([%s])' % table)
keptColumns = [c for c in result if c['name'] != column]
columns_list = ([column], column)[isinstance(column, list)]
keptColumns = filter_list(lambda col: col['name'] not in columns_list, result)
keptColumnsNames = []
final = []
@ -583,12 +626,7 @@ class SchemaUpgrade(object):
return check_db_version and self.checkDBVersion()
def listTables(self):
tables = []
# noinspection SqlResolve
sql_result = self.connection.select('SELECT name FROM [sqlite_master] WHERE type = "table"')
for table in sql_result:
tables.append(table[0])
return tables
return self.list_tables()
def do_query(self, queries):
if not isinstance(queries, list):
@ -682,6 +720,7 @@ def MigrationCode(my_db):
20011: sickbeard.mainDB.AddShowExludeGlobals,
20012: sickbeard.mainDB.RenameAllowBlockListTables,
20013: sickbeard.mainDB.AddHistoryHideColumn,
20014: sickbeard.mainDB.ChangeShowData,
# 20002: sickbeard.mainDB.AddCoolSickGearFeature3,
}

183
sickbeard/generic_queue.py

@ -20,15 +20,16 @@ import copy
import datetime
import threading
from . import logger
from . import db, logger
from exceptions_helper import ex
from six import integer_types
# noinspection PyUnreachableCode
if False:
from typing import AnyStr, Callable, Dict, List, Tuple, Union
from typing import AnyStr, Callable, Dict, List, Optional, Tuple, Union
from .search_queue import BaseSearchQueueItem
from .show_queue import ShowQueueItem
from .people_queue import CastQueueItem
class QueuePriorities(object):
@ -39,7 +40,8 @@ class QueuePriorities(object):
class GenericQueue(object):
def __init__(self):
def __init__(self, cache_db_tables=None, main_db_tables=None):
# type: (List[AnyStr], List[AnyStr]) -> None
self.currentItem = None # type: QueueItem or None
@ -53,6 +55,163 @@ class GenericQueue(object):
self.lock = threading.RLock()
self.cache_db_tables = cache_db_tables or [] # type: List[AnyStr]
self.main_db_tables = main_db_tables or [] # type: List[AnyStr]
self._id_counter = self._load_init_id() # type: integer_types
def _load_init_id(self):
# type: (...) -> integer_types
"""
fetch highest uid for queue type to initialize the class
"""
my_db = db.DBConnection('cache.db')
cr = my_db.mass_action([['SELECT max(uid) as max_id FROM %s' % t] for t in self.cache_db_tables])
my_db = db.DBConnection()
mr = my_db.mass_action([['SELECT max(uid) as max_id FROM %s' % t] for t in self.main_db_tables])
return max([c[0]['max_id'] or 0 for c in cr] + [s[0]['max_id'] or 0 for s in mr] + [0])
def _get_new_id(self):
# type: (...) -> integer_types
self._id_counter += 1
return self._id_counter
def load_queue(self):
pass
def save_queue(self):
cl = self._clear_sql()
try:
with self.lock:
for item in (self.currentItem and [self.currentItem]) or [] + self.queue:
cl.extend(self._get_item_sql(item))
if cl:
my_db = db.DBConnection('cache.db')
my_db.mass_action(cl)
except (BaseException, Exception) as e:
logger.log('Exception saving queue %s to db: %s' % (self.__class__.__name__, ex(e)), logger.ERROR)
def _clear_sql(self):
# type: (...) -> List[List]
return []
def save_item(self, item):
try:
if item:
item_sql = self._get_item_sql(item)
if item_sql:
my_db = db.DBConnection('cache.db')
my_db.mass_action(item_sql)
except (BaseException, Exception) as e:
logger.log('Exception saving item %s to db: %s' % (item, ex(e)), logger.ERROR)
def delete_item(self, item, finished_run=False):
# type: (Union[QueueItem, CastQueueItem], bool) -> None
"""
:param item:
:param finished_run: set to True when queue item has run
"""
if item:
try:
item_sql = self._delete_item_from_db_sql(item)
if item_sql:
my_db = db.DBConnection('cache.db')
my_db.mass_action(item_sql)
except (BaseException, Exception) as e:
logger.log('Exception deleting item %s from db: %s' % (item, ex(e)), logger.ERROR)
def _get_item_sql(self, item):
# type: (Union[QueueItem, CastQueueItem]) -> List[List]
return []
def _delete_item_from_db_sql(self, item):
# type: (Union[QueueItem, CastQueueItem]) -> List[List]
pass
def remove_from_queue(self, to_remove=None, force=False):
# type: (List[AnyStr], bool) -> None
"""
remove given uid items from queue
:param to_remove: list of uids to remove from queue
:param force: force removal from db
"""
self._remove_from_queue(to_remove=to_remove, excluded_types=[], force=force)
def _remove_from_queue(self, to_remove=None, excluded_types=None, force=False):
# type: (List[AnyStr], List, bool) -> None
"""
remove given uid items from queue
:param to_remove: list of uids to remove from queue
:param force: force removal from db
"""
if to_remove:
excluded_types = excluded_types or []
with self.lock:
if not force:
to_remove = [r for r in to_remove for q in self.queue
if r == q.uid and (q.action_id not in excluded_types)]
del_sql = [
['DELETE FROM %s WHERE uid IN (%s)' % (t, ','.join(['?'] * len(to_remove))), to_remove]
for t in self.cache_db_tables
]
del_main_sql = [
['DELETE FROM %s WHERE uid IN (%s)' % (t, ','.join(['?'] * len(to_remove))), to_remove]
for t in self.main_db_tables
]
self.queue = [q for q in self.queue if q.uid not in to_remove]
if del_sql:
my_db = db.DBConnection('cache.db')
my_db.mass_action(del_sql)
if del_main_sql:
my_db = db.DBConnection()
my_db.mass_action(del_main_sql)
def clear_queue(self, action_types=None):
# type: (integer_types) -> None
"""
clear queue excluding internal defined types
:param action_types: only clear all of given action type
"""
if not isinstance(action_types, list):
action_types = [action_types]
return self._clear_queue(action_types=action_types)
def _clear_queue(self, action_types=None, excluded_types=None):
# type: (List[integer_types], List) -> None
excluded_types = excluded_types or []
with self.lock:
if action_types:
self.queue = [q for q in self.queue if q.action_id in excluded_types or q.action_id not in action_types]
del_sql = [
['DELETE FROM %s WHERE action_id IN (%s)' % (t, ','.join(['?'] * len(action_types))), action_types]
for t in self.cache_db_tables
]
del_main_sql = [
['DELETE FROM %s WHERE action_id IN (%s)' % (t, ','.join(['?'] * len(action_types))), action_types]
for t in self.main_db_tables
]
else:
self.queue = [q for q in self.queue if q.action_id in excluded_types]
del_sql = [
['DELETE FROM %s' % t] for t in self.cache_db_tables
]
del_main_sql = [
['DELETE FROM %s' % t] for t in self.main_db_tables
]
if del_sql:
my_db = db.DBConnection('cache.db')
my_db.mass_action(del_sql)
if del_main_sql:
my_db = db.DBConnection()
my_db.mass_action(del_main_sql)
def pause(self):
logger.log(u'Pausing queue')
if self.lock:
@ -63,17 +222,21 @@ class GenericQueue(object):
with self.lock:
self.min_priority = 0
def add_item(self, item):
def add_item(self, item, add_to_db=True):
"""
:param item: Queue Item
:type item: QueueItem
:param add_to_db: add to db
:return: Queue Item
:rtype: QueueItem
"""
with self.lock:
item.added = datetime.datetime.now()
item.uid = item.uid or self._get_new_id()
self.queue.append(item)
if add_to_db:
self.save_item(item)
return item
@ -117,6 +280,10 @@ class GenericQueue(object):
# if the thread is dead then the current item should be finished
if self.currentItem:
self.currentItem.finish()
try:
self.delete_item(self.currentItem, finished_run=True)
except (BaseException, Exception):
pass
self.currentItem = None
# if there's something in the queue then run it in a thread and take it out of the queue
@ -136,12 +303,13 @@ class GenericQueue(object):
class QueueItem(threading.Thread):
def __init__(self, name, action_id=0):
# type: (AnyStr, int) -> None
def __init__(self, name, action_id=0, uid=None):
# type: (AnyStr, int, integer_types) -> None
"""
:param name: name
:param action_id:
:param uid:
"""
super(QueueItem, self).__init__()
@ -150,7 +318,8 @@ class QueueItem(threading.Thread):
self.priority = QueuePriorities.NORMAL # type: int
self.action_id = action_id # type: int
self.stop = threading.Event()
self.added = None
self.added = None # type: Optional[datetime.datetime]
self.uid = uid # type: integer_types
def copy(self, deepcopy_obj=None):
"""

76
sickbeard/helpers.py

@ -42,7 +42,7 @@ from . import db, logger, notifiers
from .common import cpu_presets, mediaExtensions, Overview, Quality, statusStrings, subtitleExtensions, \
ARCHIVED, DOWNLOADED, FAILED, IGNORED, SKIPPED, SNATCHED_ANY, SUBTITLED, UNAIRED, UNKNOWN, WANTED
from .sgdatetime import timestamp_near
from tvinfo_base.exceptions import *
from lib.tvinfo_base.exceptions import *
# noinspection PyPep8Naming
import encodingKludge as ek
from exceptions_helper import ex, MultipleShowObjectsException
@ -60,9 +60,9 @@ from six.moves import zip
# the following are imported from elsewhere,
# therefore, they intentionally don't resolve and are unused in this particular file.
# noinspection PyUnresolvedReferences
from sg_helpers import chmod_as_parent, clean_data, copy_file, fix_set_group_id, get_system_temp_dir, \
from sg_helpers import chmod_as_parent, clean_data, copy_file, download_file, fix_set_group_id, get_system_temp_dir, \
get_url, indent_xml, make_dirs, maybe_plural, md5_for_text, move_file, proxy_setting, remove_file, \
remove_file_perm, replace_extension, scantree, try_int, try_ord, write_file
remove_file_perm, replace_extension, sanitize_filename, scantree, try_int, try_ord, write_file
# noinspection PyUnreachableCode
if False:
@ -71,6 +71,7 @@ if False:
from .tv import TVShow
# the following workaround hack resolves a pyc resolution bug
from .name_cache import retrieveNameFromCache
from six import integer_types
RE_XML_ENCODING = re.compile(r'^(<\?xml[^>]+)\s+(encoding\s*=\s*[\"\'][^\"\']*[\"\'])(\s*\?>|)', re.U)
RE_IMDB_ID = re.compile(r'(?i)(tt\d{4,})')
@ -182,32 +183,12 @@ def is_first_rar_volume(filename):
return None is not re.search(r'(?P<file>^(?P<base>(?:(?!\.part\d+\.rar$).)*)\.(?:(?:part0*1\.)?rar)$)', filename)
def sanitize_filename(name):
"""
:param name: filename
:type name: AnyStr
:return: sanitized filename
:rtype: AnyStr
"""
# remove bad chars from the filename
name = re.sub(r'[\\/*]', '-', name)
name = re.sub(r'[:"<>|?]', '', name)
# remove leading/trailing periods and spaces
name = name.strip(' .')
for char in sickbeard.REMOVE_FILENAME_CHARS or []:
name = name.replace(char, '')
return name
def find_show_by_id(
show_id, # type: Union[AnyStr, Dict[int, int], int]
show_list=None, # type: Optional[List[TVShow]]
no_mapped_ids=True, # type: bool
check_multishow=False # type: bool
check_multishow=False, # type: bool
no_exceptions=False # type: bool
):
# type: (...) -> Optional[TVShow]
"""
@ -215,6 +196,7 @@ def find_show_by_id(
:param show_list: (optional) TVShow objects list
:param no_mapped_ids: don't check mapped ids
:param check_multishow: check for multiple show matches
:param no_exceptions: suppress the MultipleShowObjectsException and return None instead
"""
results = []
if None is show_list:
@ -235,16 +217,14 @@ def find_show_by_id(
if isinstance(show_id, dict):
if no_mapped_ids:
sid_int_list = [sickbeard.tv.TVShow.create_sid(sk, sv) for sk, sv in iteritems(show_id) if 0 < sv
and sickbeard.tv.tvid_bitmask >= sk]
sid_int_list = [sickbeard.tv.TVShow.create_sid(sk, sv) for sk, sv in iteritems(show_id) if sv and
0 < sv and sickbeard.tv.tvid_bitmask >= sk]
if check_multishow:
results = [sickbeard.showDict.get(_show_sid_id) for _show_sid_id in sid_int_list
if sickbeard.showDict.get(_show_sid_id)]
else:
for _show_sid_id in sid_int_list:
if _show_sid_id in sickbeard.showDict:
return sickbeard.showDict.get(_show_sid_id)
return None
return next((sickbeard.showDict.get(_show_sid_id) for _show_sid_id in sid_int_list
if sickbeard.showDict.get(_show_sid_id)), None)
else:
show_id = {k: v for k, v in iteritems(show_id) if 0 < v}
results = [_show_obj for k, v in iteritems(show_id)
@ -254,7 +234,8 @@ def find_show_by_id(
if 1 == num_shows:
return results[0]
elif 1 < num_shows:
raise MultipleShowObjectsException()
if not no_exceptions:
raise MultipleShowObjectsException()
def make_dir(path):
@ -1070,29 +1051,6 @@ def _maybe_request_url(e, def_url=''):
return hasattr(e, 'request') and hasattr(e.request, 'url') and ' ' + e.request.url or def_url
def download_file(url, filename, session=None, **kwargs):
"""
download given url to given filename
:param url: url to download
:type url: AnyStr
:param filename: filename to save the data to
:type filename: AnyStr
:param session: optional requests session object
:type session: requests.Session or None
:param kwargs:
:return: success of download
:rtype: bool
"""
sickbeard.MEMCACHE.setdefault('cookies', {})
if None is get_url(url, session=session, savename=filename,
url_solver=sickbeard.FLARESOLVERR_HOST, memcache_cookies=sickbeard.MEMCACHE['cookies'],
**kwargs):
remove_file_perm(filename)
return False
return True
def clear_cache(force=False):
"""
clear sickgear cache folder
@ -1394,7 +1352,8 @@ def cleanup_cache():
Delete old cached files
"""
delete_not_changed_in([ek.ek(os.path.join, sickbeard.CACHE_DIR, 'images', 'browse', 'thumb', x) for x in [
'anidb', 'imdb', 'trakt', 'tvdb']])
'anidb', 'imdb', 'trakt', 'tvdb']] + [ek.ek(os.path.join, sickbeard.CACHE_DIR, 'images', x) for x in [
'characters', 'person']] + [ek.ek(os.path.join, sickbeard.CACHE_DIR, 'tvinfo_cache')])
def delete_not_changed_in(paths, days=30, minutes=0):
@ -1564,18 +1523,15 @@ def path_mapper(search, replace, subject):
def get_overview(ep_status, show_quality, upgrade_once, split_snatch=False):
# type: (integer_types, integer_types, Union[integer_types, bool]) -> integer_types
"""
:param ep_status: episode status
:type ep_status: int
:param show_quality: show quality
:type show_quality: int
:param upgrade_once: upgrade once
:type upgrade_once: bool
:param split_snatch:
:type split_snatch: bool
:return: constant from classes Overview
:rtype: int
"""
status, quality = Quality.splitCompositeStatus(ep_status)
if ARCHIVED == status:

103
sickbeard/image_cache.py

@ -31,26 +31,34 @@ import sg_helpers
from . import db, logger
from .metadata.generic import GenericMetadata
from .sgdatetime import timestamp_near
from .indexers.indexer_config import TVINFO_TVDB, TVINFO_TVMAZE, TVINFO_TMDB
from six import itervalues
from six import itervalues, iteritems
# noinspection PyUnreachableCode
if False:
from typing import AnyStr, Optional, Union
from .tv import TVShow
from typing import AnyStr, Optional, Tuple, Union
from .tv import TVShow, Person, Character
from six import integer_types
from lib.hachoir.parser import createParser
from lib.hachoir.metadata import extractMetadata
cache_img_base = {'tvmaze': TVINFO_TVMAZE, 'themoviedb': TVINFO_TMDB, 'thetvdb': TVINFO_TVDB}
class ImageCache(object):
base_dir = None # type: AnyStr or None
shows_dir = None # type: AnyStr or None
persons_dir = None # type: Optional[AnyStr]
characters_dir = None # type: Optional[AnyStr]
def __init__(self):
if None is ImageCache.base_dir and ek.ek(os.path.exists, sickbeard.CACHE_DIR):
ImageCache.base_dir = ek.ek(os.path.abspath, ek.ek(os.path.join, sickbeard.CACHE_DIR, 'images'))
ImageCache.shows_dir = ek.ek(os.path.abspath, ek.ek(os.path.join, self.base_dir, 'shows'))
ImageCache.persons_dir = self._persons_dir()
ImageCache.characters_dir = self._characters_dir()
def __del__(self):
pass
@ -62,6 +70,14 @@ class ImageCache(object):
# """
# return ek.ek(os.path.abspath, ek.ek(os.path.join, sickbeard.CACHE_DIR, 'images'))
def _persons_dir(self):
# type: (...) -> AnyStr
return ek.ek(os.path.join, sickbeard.CACHE_DIR, 'images', 'person')
def _characters_dir(self):
# type: (...) -> AnyStr
return ek.ek(os.path.join, sickbeard.CACHE_DIR, 'images', 'characters')
def _fanart_dir(self, tvid=None, prodid=None):
# type: (int, int) -> AnyStr
"""
@ -91,6 +107,87 @@ class ImageCache(object):
"""
return ek.ek(os.path.abspath, ek.ek(os.path.join, self.shows_dir, '%s-%s' % (tvid, prodid), 'thumbnails'))
def _person_base_name(self, person_obj):
# type: (Person) -> AnyStr
base_id = next((v for k, v in iteritems(cache_img_base)
if k in (person_obj.image_url or '') or person_obj.thumb_url), 0)
return '%s-%s' % (base_id, person_obj.ids.get(base_id) or sg_helpers.sanitize_filename(person_obj.name))
def _character_base_name(self, character_obj, show_obj, tvid=None, proid=None):
# type: (Character, TVShow, integer_types, integer_types) -> AnyStr
return '%s-%s' % (tvid or show_obj._tvid, character_obj.ids.get(tvid or show_obj._tvid)
or sg_helpers.sanitize_filename(character_obj.name))
def person_path(self, person_obj, base_path=None):
# type: (Optional[Person], AnyStr) -> AnyStr
"""
return image filename
:param person_obj:
:param base_path:
"""
filename = '%s.jpg' % base_path or self._person_base_name(person_obj)
return ek.ek(os.path.join, self.persons_dir, filename)
def person_thumb_path(self, person_obj, base_path=None):
# type: (Optional[Person], AnyStr) -> AnyStr
"""
return thumb image filename
:param person_obj:
:param base_path:
"""
filename = '%s_thumb.jpg' % base_path or self._person_base_name(person_obj)
return ek.ek(os.path.join, self.persons_dir, filename)
def person_both_paths(self, person_obj):
# type: (Person) -> Tuple[AnyStr, AnyStr]
"""
return tuple image, thumb filenames
:param person_obj:
"""
base_path = self._person_base_name(person_obj)
return self.person_path(None, base_path=base_path), self.person_thumb_path(None, base_path=base_path)
def character_path(self, character_obj, show_obj, base_path=None):
# type: (Optional[Character], Optional[TVShow], AnyStr) -> AnyStr
"""
return image filename
:param character_obj:
:param show_obj:
:param base_path:
"""
filename = '%s.jpg' % base_path or self._character_base_name(character_obj, show_obj)
return ek.ek(os.path.join, self.characters_dir, filename)
def character_thumb_path(self, character_obj, show_obj, base_path=None):
# type: (Optional[Character], Optional[TVShow], AnyStr) -> AnyStr
"""
return thumb image filename
:param character_obj:
:param show_obj:
:param base_path:
"""
filename = '%s_thumb.jpg' % base_path or self._character_base_name(character_obj, show_obj)
return ek.ek(os.path.join, self.characters_dir, filename)
def character_both_path(self, character_obj, show_obj=None, tvid=None, proid=None, person_obj=None):
# type: (Character, TVShow, integer_types, integer_types, Person) -> Tuple[AnyStr, AnyStr]
"""
returns tuple image, thumb image
:param character_obj:
:param show_obj:
:param tvid:
:param proid:
:param person_obj:
"""
base_path = self._character_base_name(character_obj, show_obj=show_obj, tvid=tvid, proid=proid)
from .tv import Person
if isinstance(person_obj, Person):
person_base = self._person_base_name(person_obj)
if person_base:
base_path = '%s-%s' % (base_path, person_base)
return self.character_path(None, None, base_path=base_path), \
self.character_thumb_path(None, None, base_path=base_path)
def poster_path(self, tvid, prodid):
# type: (int, int) -> AnyStr
"""

34
sickbeard/indexermapper.py

@ -35,6 +35,7 @@ from lib.dateutil.parser import parse
from lib.imdbpie import Imdb
from libtrakt import TraktAPI
from libtrakt.exceptions import TraktAuthException, TraktException
from exceptions_helper import ConnectionSkipException
from _23 import unidecode, urlencode
from six import iteritems, iterkeys, string_types, PY2
@ -164,7 +165,7 @@ def get_premieredate(show_obj):
:rtype: datetime.date or None
"""
try:
ep_obj = show_obj.get_episode(season=1, episode=1)
ep_obj = show_obj.first_aired_regular_episode
if ep_obj and ep_obj.airdate:
return ep_obj.airdate
except (BaseException, Exception):
@ -240,7 +241,7 @@ def get_trakt_ids(url_trakt):
break
if found:
break
except (TraktAuthException, TraktException, IndexError, KeyError):
except (ConnectionSkipException, TraktAuthException, TraktException, IndexError, KeyError):
pass
return {k: v for k, v in iteritems(ids) if v not in (None, '', 0)}
@ -317,7 +318,7 @@ def check_missing_trakt_id(n_ids, show_obj, url_trakt):
return n_ids
def map_indexers_to_show(show_obj, update=False, force=False, recheck=False):
def map_indexers_to_show(show_obj, update=False, force=False, recheck=False, sql=None):
"""
:param show_obj: TVShow Object
@ -339,11 +340,18 @@ def map_indexers_to_show(show_obj, update=False, force=False, recheck=False):
'status': (MapStatus.NONE, MapStatus.SOURCE)[int(tvid) == int(show_obj.tvid)],
'date': datetime.date.fromordinal(1)}
my_db = db.DBConnection()
sql_result = my_db.select('SELECT *'
' FROM indexer_mapping'
' WHERE indexer = ? AND indexer_id = ?',
[show_obj.tvid, show_obj.prodid])
sql_result = []
if sql:
for s in sql:
if show_obj._prodid == s['indexer_id'] and show_obj._tvid == s['indexer']:
sql_result.append(s)
if not sql_result:
my_db = db.DBConnection()
sql_result = my_db.select('SELECT *'
' FROM indexer_mapping'
' WHERE indexer = ? AND indexer_id = ?',
[show_obj.tvid, show_obj.prodid])
# for each mapped entry
for cur_result in sql_result:
@ -559,7 +567,7 @@ def should_recheck_update_ids(show_obj):
k not in defunct_indexer] or [datetime.date.fromtimestamp(1)])
if today - ids_updated >= datetime.timedelta(days=365):
return True
ep_obj = show_obj.get_episode(season=1, episode=1)
ep_obj = show_obj.first_aired_regular_episode
if ep_obj and ep_obj.airdate and ep_obj.airdate > datetime.date.fromtimestamp(1):
show_age = (today - ep_obj.airdate).days
# noinspection PyTypeChecker
@ -573,11 +581,19 @@ def should_recheck_update_ids(show_obj):
def load_mapped_ids(**kwargs):
logger.log('Start loading TV info mappings...')
if 'load_all' in kwargs:
del kwargs['load_all']
my_db = db.DBConnection()
sql_result = my_db.select('SELECT * FROM indexer_mapping ORDER BY indexer, indexer_id')
else:
sql_result = None
for cur_show_obj in sickbeard.showList:
with cur_show_obj.lock:
n_kargs = kwargs.copy()
if 'update' in kwargs and should_recheck_update_ids(cur_show_obj):
n_kargs['recheck'] = True
if sql_result:
n_kargs['sql'] = sql_result
try:
cur_show_obj.ids = sickbeard.indexermapper.map_indexers_to_show(cur_show_obj, **n_kargs)
except (BaseException, Exception):

16
sickbeard/indexers/indexer_api.py

@ -18,9 +18,10 @@
import os
from .indexer_config import init_config, tvinfo_config
from sg_helpers import proxy_setting
from sg_helpers import make_dirs, proxy_setting
import sickbeard
from tvinfo_base import TVInfoBase
from lib.tvinfo_base import TVInfoBase
import encodingKludge as ek
from _23 import list_values
@ -42,6 +43,9 @@ class TVInfoAPI(object):
if tvinfo_config[self.tvid]['active'] or ('no_dummy' in kwargs and True is kwargs['no_dummy']):
if 'no_dummy' in kwargs:
kwargs.pop('no_dummy')
indexer_cache_dir = ek.ek(os.path.join, sickbeard.CACHE_DIR, 'tvinfo_cache',
tvinfo_config[self.tvid]['name'])
kwargs['diskcache_dir'] = indexer_cache_dir
return tvinfo_config[self.tvid]['module'](*args, **kwargs)
else:
return TVInfoBase(*args, **kwargs)
@ -84,13 +88,14 @@ class TVInfoAPI(object):
def sources(self):
# type: () -> Dict[int, AnyStr]
return dict([(int(x['id']), x['name']) for x in list_values(tvinfo_config) if not x['mapped_only'] and
True is not x.get('fallback')])
True is not x.get('fallback') and True is not x.get('people_only')])
@property
def search_sources(self):
# type: () -> Dict[int, AnyStr]
return dict([(int(x['id']), x['name']) for x in list_values(tvinfo_config) if not x['mapped_only'] and
x.get('active') and not x.get('defunct') and True is not x.get('fallback')])
x.get('active') and not x.get('defunct') and True is not x.get('fallback')
and True is not x.get('people_only')])
@property
def all_sources(self):
@ -98,7 +103,8 @@ class TVInfoAPI(object):
"""
:return: return all indexers including mapped only indexers excluding fallback indexers
"""
return dict([(int(x['id']), x['name']) for x in list_values(tvinfo_config) if True is not x.get('fallback')])
return dict([(int(x['id']), x['name']) for x in list_values(tvinfo_config) if True is not x.get('fallback')
and True is not x.get('people_only')])
@property
def fallback_sources(self):

84
sickbeard/indexers/indexer_config.py

@ -1,18 +1,11 @@
from lib.tvdb_api.tvdb_api import Tvdb
from lib.libtrakt.indexerapiinterface import TraktIndexer
TVINFO_TVDB = 1
TVINFO_TVRAGE = 2
TVINFO_TVMAZE = 3
# old tvdb api - version 1
# TVINFO_TVDB_V1 = 10001
# mapped only source
TVINFO_IMDB = 100
TVINFO_TRAKT = 101
TVINFO_TMDB = 102
# end mapped only source
from lib.tvmaze_api.tvmaze_api import TvMaze
from lib.tmdb_api.tmdb_api import TmdbIndexer
from lib.imdb_api.imdb_api import IMDbIndexer
from lib.tvinfo_base import TVINFO_TWITTER, TVINFO_WIKIPEDIA, TVINFO_FACEBOOK, TVINFO_INSTAGRAM, \
TVINFO_TVDB, TVINFO_TVRAGE, TVINFO_TVMAZE, TVINFO_IMDB, TVINFO_TRAKT, TVINFO_TMDB, TVINFO_TRAKT_SLUG, \
TVINFO_TVDB_SLUG
init_config = {
'valid_languages': ['da', 'fi', 'nl', 'de', 'it', 'es', 'fr', 'pl', 'hu', 'el', 'tr',
@ -32,6 +25,7 @@ tvinfo_config = {
dupekey='',
mapped_only=False,
icon='thetvdb16.png',
people_url='https://thetvdb.com/people/%s',
),
TVINFO_TVRAGE: dict(
main_url='http://tvrage.com/',
@ -48,23 +42,26 @@ tvinfo_config = {
main_url='https://www.tvmaze.com/',
id=TVINFO_TVMAZE,
name='TVmaze', slug='tvmaze',
module=None,
module=TvMaze,
api_params={},
active=False,
active=True,
dupekey='tvm',
mapped_only=True,
mapped_only=False,
icon='tvmaze16.png',
people_url='https://www.tvmaze.com/person/view?id=%s',
character_url='https://www.tvmaze.com/character/view?id=%s',
),
TVINFO_IMDB: dict(
main_url='https://www.imdb.com/',
id=TVINFO_IMDB,
name='IMDb', slug='imdb', kodi_slug='imdb',
module=None,
module=IMDbIndexer,
api_params={},
active=False,
active=True,
dupekey='imdb',
mapped_only=True,
icon='imdb16.png',
people_url='https://www.imdb.com/name/nm%07d',
),
TVINFO_TRAKT: dict(
main_url='https://www.trakt.tv/',
@ -76,17 +73,64 @@ tvinfo_config = {
dupekey='trakt',
mapped_only=True,
icon='trakt16.png',
people_url='https://trakt.tv/people/%s',
),
TVINFO_TMDB: dict(
main_url='https://www.themoviedb.org/',
id=TVINFO_TMDB,
name='TMDb', slug='tmdb', kodi_slug='tmdb',
module=None,
module=TmdbIndexer,
api_params={},
active=False,
active=True,
dupekey='tmdb',
mapped_only=True,
icon='tmdb16.png',
people_url='https://www.themoviedb.org/person/%s',
),
# social media sources for people
TVINFO_INSTAGRAM: dict(
id=TVINFO_INSTAGRAM,
name='Instagram',
module=None,
active=False,
mapped_only=True,
people_url='https://www.instagram.com/%s',
show_url=None,
people_only=True,
icon='instagram16.png'
),
TVINFO_TWITTER: dict(
id=TVINFO_TWITTER,
name='Twitter',
module=None,
active=False,
mapped_only=True,
people_url='https://twitter.com/%s',
show_url=None,
people_only=True,
icon='twitter16.png'
),
TVINFO_FACEBOOK: dict(
id=TVINFO_FACEBOOK,
name='Facebook',
module=None,
active=False,
mapped_only=True,
people_url='https://www.facebook.com/%s',
show_url=None,
people_only=True,
icon='facebook16.png'
),
TVINFO_WIKIPEDIA: dict(
id=TVINFO_WIKIPEDIA,
name='Wikipedia',
module=None,
active=False,
mapped_only=True,
people_url='https://en.wikipedia.org/wiki/%s',
show_url=None,
people_only=True,
icon='wikipedia16.png'
)
}

2
sickbeard/indexers/indexer_exceptions.py

@ -4,4 +4,4 @@
# license:unlicense (http://unlicense.org/)
# noinspection PyUnresolvedReferences
from tvinfo_base.exceptions import *
from lib.tvinfo_base.exceptions import *

52
sickbeard/metadata/generic.py

@ -29,7 +29,8 @@ from .. import logger
import sg_helpers
from ..indexers import indexer_config
from ..indexers.indexer_config import TVINFO_TVDB
from tvinfo_base.exceptions import *
from lib.tvinfo_base import TVInfoImage, TVInfoImageType, TVInfoImageSize
from lib.tvinfo_base.exceptions import *
import sickbeard
# noinspection PyPep8Naming
import encodingKludge as ek
@ -48,6 +49,15 @@ if False:
from typing import AnyStr, Dict, List, Optional, Tuple, Union
map_image_types = {
'poster': TVInfoImageType.poster,
'banner': TVInfoImageType.banner,
'fanart': TVInfoImageType.fanart,
'poster_thumb': TVInfoImageType.poster,
'banner_thumb': TVInfoImageType.banner,
}
class GenericMetadata(object):
"""
Base class for all metadata providers. Default behavior is meant to mostly
@ -217,7 +227,10 @@ class GenericMetadata(object):
return '%s' % show_obj.startyear
if not show_obj.sxe_ep_obj.get(1, {}).get(1, None):
show_obj.get_all_episodes()
first_ep_obj = show_obj.get_episode(1, 1, no_create=True)
try:
first_ep_obj = show_obj.first_aired_regular_episode
except (BaseException, Exception):
first_ep_obj = None
if isinstance(first_ep_obj, sickbeard.tv.TVEpisode) \
and isinstance(first_ep_obj.airdate, datetime.date) and 1900 < first_ep_obj.airdate.year:
return '%s' % (first_ep_obj.airdate.year, first_ep_obj.airdate)[not year_only]
@ -323,7 +336,8 @@ class GenericMetadata(object):
"""
if not (isinstance(fetched_show_info, dict) and
isinstance(getattr(fetched_show_info, 'data', None), (list, dict)) and
'seriesname' in getattr(fetched_show_info, 'data', [])):
'seriesname' in getattr(fetched_show_info, 'data', [])) and \
not hasattr(fetched_show_info, 'seriesname'):
logger.log(u'Show %s not found on %s ' %
(show_obj.name, sickbeard.TVInfoAPI(show_obj.tvid).name), logger.WARNING)
return False
@ -906,30 +920,33 @@ class GenericMetadata(object):
elif None is not getattr(show_info, 'poster', None):
image_urls, alt_tvdb_urls = build_url(show_info, 'poster')
for item in self._fanart_urls_from_show(show_obj, image_type, show_lang, True) or []:
image_urls.append(item[2])
if 0 == len(image_urls):
for item in self._tmdb_image_url(show_obj, image_type) or []:
if return_links or not image_urls:
for item in self._fanart_urls_from_show(show_obj, image_type, show_lang, True) or []:
image_urls.append(item[2])
if 0 == len(image_urls):
for item in self._tmdb_image_url(show_obj, image_type) or []:
image_urls.append(item[2])
elif 'banner_thumb' == image_type:
if None is not getattr(show_info, image_type, None):
image_urls, alt_tvdb_urls = build_url(show_info, image_type)
elif None is not getattr(show_info, 'banner', None):
image_urls, alt_tvdb_urls = build_url(show_info, 'banner')
for item in self._fanart_urls_from_show(show_obj, image_type, show_lang, True) or []:
image_urls.append(item[2])
if return_links or not image_urls:
for item in self._fanart_urls_from_show(show_obj, image_type, show_lang, True) or []:
image_urls.append(item[2])
else:
for item in self._fanart_urls_from_show(show_obj, image_type, show_lang) or []:
image_urls.append(item[2])
if None is not getattr(show_info, image_type, None):
image_url = show_info[image_type]
if image_url:
image_urls.append(image_url)
if 'poster' == image_type:
if image_type in ('poster', 'banner', 'fanart'):
init_url = image_url
if return_links or None is init_url:
for item in self._fanart_urls_from_show(show_obj, image_type, show_lang) or []:
image_urls.append(item[2])
# check extra provided images in '_banners' key
if None is not getattr(show_info, '_banners', None) and \
isinstance(show_info['_banners'].get(image_type, None), (list, dict)):
@ -937,6 +954,15 @@ class GenericMetadata(object):
for item in itervalues(value):
image_urls.append(item['bannerpath'])
# extra images via images property
tvinfo_type = map_image_types.get(image_type)
tvinfo_size = (TVInfoImageSize.original, TVInfoImageSize.medium)['_thumb' in image_type]
if tvinfo_type and getattr(show_info, 'images', None) and show_info.images.get(tvinfo_type):
for img in show_info.images[tvinfo_type]: # type: TVInfoImage
for img_size, img_url in iteritems(img.sizes):
if tvinfo_size == img_size:
image_urls.append(img_url)
if 0 == len(image_urls) or 'fanart' == image_type:
for item in self._tmdb_image_url(show_obj, image_type) or []:
image_urls.append('%s?%s' % (item[2], item[0]))

2
sickbeard/metadata/kodi.py

@ -24,7 +24,7 @@ from . import generic
from .. import logger
import sg_helpers
from ..indexers.indexer_config import TVINFO_IMDB, TVINFO_TVDB
from tvinfo_base.exceptions import *
from lib.tvinfo_base.exceptions import *
import sickbeard
# noinspection PyPep8Naming
import encodingKludge as ek

2
sickbeard/metadata/mede8er.py

@ -22,7 +22,7 @@ import datetime
from . import mediabrowser
from .. import logger
import sg_helpers
from tvinfo_base.exceptions import *
from lib.tvinfo_base.exceptions import *
import sickbeard
import exceptions_helper
from exceptions_helper import ex

2
sickbeard/metadata/mediabrowser.py

@ -24,7 +24,7 @@ import re
from . import generic
from .. import logger
import sg_helpers
from tvinfo_base.exceptions import *
from lib.tvinfo_base.exceptions import *
import sickbeard
# noinspection PyPep8Naming
import encodingKludge as ek

2
sickbeard/metadata/tivo.py

@ -25,7 +25,7 @@ import os
from . import generic
from .. import logger
import sg_helpers
from tvinfo_base.exceptions import *
from lib.tvinfo_base.exceptions import *
import sickbeard
# noinspection PyPep8Naming
import encodingKludge as ek

2
sickbeard/metadata/wdtv.py

@ -24,7 +24,7 @@ import re
from . import generic
from .. import logger
import sg_helpers
from tvinfo_base.exceptions import *
from lib.tvinfo_base.exceptions import *
import sickbeard
# noinspection PyPep8Naming
import encodingKludge as ek

2
sickbeard/metadata/xbmc_12plus.py

@ -21,7 +21,7 @@ import datetime
from . import generic
from .. import logger
import sg_helpers
from tvinfo_base.exceptions import *
from lib.tvinfo_base.exceptions import *
import sickbeard
import exceptions_helper
from exceptions_helper import ex

2
sickbeard/name_parser/parser.py

@ -39,7 +39,7 @@ import encodingKludge as ek
from exceptions_helper import ex
import sickbeard
from .. import common, db, helpers, logger, scene_exceptions, scene_numbering
from tvinfo_base.exceptions import *
from lib.tvinfo_base.exceptions import *
from ..classes import OrderedDefaultdict
from .._legacy_classes import LegacyParseResult

106
sickbeard/network_timezones.py

@ -25,6 +25,7 @@ import threading
import sickbeard
from . import db, helpers, logger
from sg_helpers import int_to_time
# noinspection PyPep8Naming
import encodingKludge as ek
@ -32,9 +33,8 @@ from lib.dateutil import tz, zoneinfo
from lib.tzlocal import get_localzone
from sg_helpers import remove_file_perm, scantree
from six import integer_types, iteritems, string_types, PY2
from _23 import list_keys
from six import iteritems
# noinspection PyUnreachableCode
if False:
@ -437,7 +437,7 @@ def parse_time(time_of_day):
def parse_date_time(date_stamp, time_of_day, network):
# type: (int, Union[AnyStr or Tuple[int, int]], AnyStr) -> datetime.datetime
# type: (int, Union[AnyStr or Tuple[int, int]], Union[AnyStr, datetime.tzinfo]) -> datetime.datetime
"""
parse date and time string into local time
@ -445,8 +445,12 @@ def parse_date_time(date_stamp, time_of_day, network):
:param time_of_day: time as a string or as a tuple(hr, m)
:param network: network names
"""
if isinstance(time_of_day, tuple) and 2 == len(time_of_day) \
and isinstance(time_of_day[0], int) and isinstance(time_of_day[1], int):
dt_t = None
hour = mins = 0
if isinstance(time_of_day, integer_types):
dt_t = int_to_time(time_of_day)
elif isinstance(time_of_day, tuple) and 2 == len(time_of_day) and isinstance(time_of_day[0], int) \
and isinstance(time_of_day[1], int):
(hour, mins) = time_of_day
else:
(hour, mins) = parse_time(time_of_day)
@ -457,10 +461,18 @@ def parse_date_time(date_stamp, time_of_day, network):
foreign_timezone = network
else:
foreign_timezone = get_network_timezone(network)
foreign_naive = datetime.datetime(dt.year, dt.month, dt.day, hour, mins, tzinfo=foreign_timezone)
if None is not dt_t:
foreign_naive = datetime.datetime.combine(datetime.date(dt.year, dt.month, dt.day),
dt_t).replace(tzinfo=foreign_timezone)
else:
foreign_naive = datetime.datetime(dt.year, dt.month, dt.day, hour, mins, tzinfo=foreign_timezone)
return foreign_naive
except (BaseException, Exception):
return datetime.datetime(dt.year, dt.month, dt.day, hour, mins, tzinfo=SG_TIMEZONE)
if None is dt_t:
return datetime.datetime(dt.year, dt.month, dt.day, hour, mins, tzinfo=SG_TIMEZONE)
else:
return datetime.datetime.combine(datetime.datetime(dt.year, dt.month, dt.day),
dt_t).replace(tzinfo=SG_TIMEZONE)
def test_timeformat(t):
@ -544,3 +556,83 @@ def _load_network_conversions():
# change all network conversion info at once (much faster)
if 0 < len(cl):
my_db.mass_action(cl)
def get_episode_time(d, # type: int
t, # type: Union[AnyStr or Tuple[int, int]]
show_network, # type: Optional[AnyStr]
show_airtime=None, # type: Optional[integer_types, datetime.time]
show_timezone=None, # type: Union[AnyStr, datetime.tzinfo]
ep_timestamp=None, # type: Union[integer_types, float]
ep_network=None, # type: Optional[AnyStr]
ep_airtime=None, # type: Optional[integer_types, datetime.time]
ep_timezone=None # type: Union[AnyStr, datetime.tzinfo]
):
# type: (...) -> datetime.datetime
"""
parse data and time data into datetime
:param d: ordinal datetime
:param t: time as a string or as a tuple(hr, m)
:param show_network: network names of show
:param show_airtime: airtime of show as integer or time
:param show_timezone: timezone of show as string or tzinfo
:param ep_timestamp: timestamp of episode
:param ep_network: network name of episode as string
:param ep_airtime: airtime as integer or time
:param ep_timezone: timezone of episode as string or tzinfo
"""
tzinfo = None
if ep_timezone:
if isinstance(ep_timezone, datetime.tzinfo):
tzinfo = ep_timezone
elif isinstance(ep_timezone, string_types):
tzinfo = tz.gettz(ep_timezone, zoneinfo_priority=True)
if not tzinfo:
if ep_network and isinstance(ep_network, string_types):
tzinfo = get_network_timezone(ep_network)
if not tzinfo:
if show_timezone:
if isinstance(show_timezone, datetime.tzinfo):
tzinfo = show_timezone
elif isinstance(show_timezone, string_types):
tzinfo = tz.gettz(show_timezone, zoneinfo_priority=True)
if not tzinfo:
if show_network and isinstance(show_network, string_types):
tzinfo = get_network_timezone(show_network)
if not isinstance(tzinfo, datetime.tzinfo):
tzinfo = SG_TIMEZONE
if isinstance(ep_timestamp, (integer_types, float)):
from .sgdatetime import SGDatetime
return SGDatetime.from_timestamp(ep_timestamp, tzinfo=tzinfo, tz_aware=True, local_time=False)
ep_time = None
if isinstance(ep_airtime, integer_types):
ep_time = int_to_time(ep_airtime)
elif isinstance(ep_airtime, datetime.time):
ep_time = ep_airtime
if None is ep_time and show_airtime:
if isinstance(show_airtime, integer_types):
ep_time = int_to_time(show_airtime)
elif isinstance(show_airtime, datetime.time):
ep_time = show_airtime
if None is ep_time:
if isinstance(t, string_types):
ep_hr, ep_min = parse_time(t)
ep_time = datetime.time(ep_hr, ep_min)
else:
ep_time = datetime.time(0, 0)
if d and None is not ep_time and None is not tzinfo:
ep_date = datetime.date.fromordinal(helpers.try_int(d))
if PY2:
return datetime.datetime.combine(ep_date, ep_time).replace(tzinfo=tzinfo)
return datetime.datetime.combine(ep_date, ep_time, tzinfo)
return parse_date_time(d, t, tzinfo)

3
sickbeard/notifiers/trakt.py

@ -21,6 +21,7 @@ import os
from .generic import BaseNotifier
import sickbeard
from lib.libtrakt import TraktAPI, exceptions
from exceptions_helper import ConnectionSkipException
from _23 import list_keys
from six import iteritems
@ -108,7 +109,7 @@ class TraktNotifier(BaseNotifier):
msg = 'Episode not found on Trakt, not adding to'
else:
warn, msg = True, 'Could not add episode to'
except (exceptions.TraktAuthException, exceptions.TraktException):
except (ConnectionSkipException, exceptions.TraktAuthException, exceptions.TraktException):
warn, msg = True, 'Error adding episode to'
msg = 'Trakt: %s your %s collection' % (msg, sickbeard.TRAKT_ACCOUNTS[tid].name)
if not warn:

208
sickbeard/people_queue.py

@ -0,0 +1,208 @@
# Author: Nic Wolfe <nic@wolfeden.ca>
# URL: http://code.google.com/p/sickbeard/
#
# This file is part of SickGear.
#
# SickGear is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# SickGear is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with SickGear. If not, see <http://www.gnu.org/licenses/>.
from __future__ import with_statement
import traceback
# noinspection PyPep8Naming
from exceptions_helper import ex
from . import db, generic_queue, logger, helpers, ui
# noinspection PyUnreachableCode
if False:
from six import integer_types
from typing import AnyStr, Dict, List, Optional
from .tv import TVShow
from lib.tvinfo_base import CastList
class PeopleQueue(generic_queue.GenericQueue):
def __init__(self):
generic_queue.GenericQueue.__init__(self, cache_db_tables=['people_queue'])
self.queue_name = 'PEOPLEQUEUE' # type: AnyStr
def load_queue(self):
try:
my_db = db.DBConnection('cache.db')
queue_sql = my_db.select('SELECT * FROM people_queue')
for q in queue_sql:
if PeopleQueueActions.SHOWCAST == q['action_id']:
try:
show_obj = helpers.find_show_by_id({q['indexer']: q['indexer_id']})
except (BaseException, Exception):
continue
if not show_obj:
continue
self.add_cast_update(show_obj=show_obj, show_info_cast=None, uid=q['uid'], force=bool(q['forced']),
scheduled_update=bool(q['scheduled']), add_to_db=False)
except (BaseException, Exception) as e:
logger.log('Exception loading queue %s: %s' % (self.__class__.__name__, ex(e)), logger.ERROR)
def _clear_sql(self):
return [
['DELETE FROM people_queue']
]
def _get_item_sql(self, item):
# type: (PeopleQueueItem) -> List[List]
return [
['INSERT OR IGNORE INTO people_queue (indexer, indexer_id, action_id, forced, scheduled, uid)'
' VALUES (?,?,?,?,?,?)',
[item.show_obj._tvid, item.show_obj._prodid, item.action_id, int(item.force), int(item.scheduled_update),
item.uid]]
]
def _delete_item_from_db_sql(self, item):
# type: (PeopleQueueItem) -> List[List]
return [
['DELETE FROM people_queue WHERE uid = ?', [item.uid]]
]
def queue_data(self):
# type: (...) -> Dict[AnyStr, List[AnyStr, Dict]]
data = {'main_cast': []}
with self.lock:
for cur_item in [self.currentItem] + self.queue: # type: PeopleQueueItem
if not cur_item:
continue
result_item = {'name': cur_item.show_obj.name, 'tvid_prodid': cur_item.show_obj.tvid_prodid,
'uid': cur_item.uid, 'forced': cur_item.force}
if isinstance(cur_item, CastQueueItem):
data['main_cast'].append(result_item)
return data
def show_in_queue(self, show_obj):
# type: (TVShow) -> bool
with self.lock:
return any(1 for q in ((self.currentItem and [self.currentItem]) or []) + self.queue
if show_obj == q.show_obj)
def abort_cast_update(self, show_obj):
# type: (TVShow) -> None
if show_obj:
with self.lock:
to_remove = []
for c in ((self.currentItem and [self.currentItem]) or []) + self.queue:
if show_obj == c.show_obj:
try:
to_remove.append(c.uid)
except (BaseException, Exception):
pass
try:
c.stop.set()
except (BaseException, Exception):
pass
if to_remove:
try:
self.remove_from_queue(to_remove)
except (BaseException, Exception):
pass
def add_cast_update(self, show_obj, show_info_cast, uid=None, add_to_db=True, force=False, scheduled_update=False,
switch=False):
# type: (TVShow, Optional[CastList], AnyStr, bool, bool, bool, bool) -> CastQueueItem
"""
:param show_obj: TV Show object
:param show_info_cast: TV Info object
:param uid: unique id
:param add_to_db: add to queue db table
:param force:
:param scheduled_update: suppresses ui notifications
:param switch: part of id switch
"""
with self.lock:
if not self.show_in_queue(show_obj):
cast_item = CastQueueItem(show_obj=show_obj, show_info_cast=show_info_cast, uid=uid, force=force,
scheduled_update=scheduled_update, switch=switch)
self.add_item(cast_item, add_to_db=add_to_db)
return cast_item
class PeopleQueueActions(object):
SHOWCAST = 1
names = {
SHOWCAST: 'Show Cast',
}
class PeopleQueueItem(generic_queue.QueueItem):
def __init__(self, action_id, show_obj, uid=None, force=False, **kwargs):
# type: (integer_types, TVShow, AnyStr, bool, Dict) -> PeopleQueueItem
"""
:param action_id:
:param show_obj: show object
"""
generic_queue.QueueItem.__init__(self, PeopleQueueActions.names[action_id], action_id, uid=uid)
self.show_obj = show_obj # type: TVShow
self.force = force # type: bool
class CastQueueItem(PeopleQueueItem):
def __init__(self, show_obj, show_info_cast=None, uid=None, force=False, scheduled_update=False, switch=False,
**kwargs):
# type: (TVShow, CastList, AnyStr, bool, bool, bool, Dict) -> CastQueueItem
"""
:param show_obj: show obj
:param show_info_cast: show info cast list
:param scheduled_update: suppresses ui notifications
:param switch: part of id switch
"""
PeopleQueueItem.__init__(self, PeopleQueueActions.SHOWCAST, show_obj, uid=uid, force=force, **kwargs)
self.show_info_cast = show_info_cast # type: Optional[CastList]
self.scheduled_update = scheduled_update # type: bool
self.switch = switch # type: bool
def run(self):
PeopleQueueItem.run(self)
if self.show_obj:
logger.log('Starting to update show cast for %s' % self.show_obj.name)
old_cast = set((c.name, c.image_url or '', c.thumb_url or '',
hash(*(p.name for p in c.person or [] if p.name)))
for c in self.show_obj.cast_list or [] if c.name)
if not self.scheduled_update and not self.switch:
ui.notifications.message('Starting to update show cast for %s' % self.show_obj.name)
try:
self.show_obj.load_cast_from_tvinfo(self.show_info_cast, force=self.force, stop_event=self.stop)
except (BaseException, Exception) as e:
logger.error('Exception in cast update queue: %s' % ex(e))
logger.debug('Traceback: %s' % traceback.format_exc())
if old_cast != set((c.name, c.image_url or '', c.thumb_url or '',
hash(*(p.name for p in c.person or [] if p.name)))
for c in self.show_obj.cast_list or [] if c.name):
logger.debug('Update show nfo with new cast data')
self.show_obj.write_show_nfo(force=True)
logger.log('Finished cast update for show %s' % self.show_obj.name)
if not self.scheduled_update and not self.switch:
ui.notifications.message('Finished to update show cast for %s' % self.show_obj.name)
self.finish()
def __str__(self):
return '<Cast Queue Item (%s)%s>' % (self.show_obj.name, ('', ' - forced')[self.force])
def __repr__(self):
return self.__str__()

4
sickbeard/scene_exceptions.py

@ -238,7 +238,9 @@ def retrieve_exceptions():
if should_refresh(sickbeard.TVInfoAPI(tvid).name):
logger.log(u'Checking for scene exception updates for %s' % sickbeard.TVInfoAPI(tvid).name)
url = sickbeard.TVInfoAPI(tvid).config['scene_url']
url = sickbeard.TVInfoAPI(tvid).config.get('scene_url')
if not url:
continue
url_data = helpers.get_url(url)
if None is url_data:

7
sickbeard/scheduler.py

@ -79,6 +79,13 @@ class Scheduler(threading.Thread):
def run(self):
self.set_paused_state()
# load previously saved queue
try:
if getattr(self.action, 'load_queue', None):
self.action.load_queue()
except (BaseException, Exception):
pass
# if self._unpause Event() is NOT set the loop pauses
while self._unpause.wait() and not self._stopper.is_set():

223
sickbeard/search_queue.py

@ -25,8 +25,11 @@ import threading
import traceback
import exceptions_helper
# noinspection PyPep8Naming
from exceptions_helper import ex
import sickbeard
from lib.dateutil import tz
from . import common, db, failed_history, generic_queue, helpers, \
history, logger, network_timezones, properFinder, search, ui
from .classes import Proper, SimpleNamespace
@ -38,6 +41,7 @@ from _23 import filter_list
# noinspection PyUnreachableCode
if False:
from typing import Any, AnyStr, Dict, List, Optional, Union
from .tv import TVShow
search_queue_lock = threading.Lock()
@ -54,35 +58,105 @@ MANUAL_SEARCH_HISTORY_SIZE = 100
class SearchQueue(generic_queue.GenericQueue):
def __init__(self):
generic_queue.GenericQueue.__init__(self)
generic_queue.GenericQueue.__init__(self, cache_db_tables=['search_queue'])
self.queue_name = 'SEARCHQUEUE' # type: AnyStr
def load_queue(self):
try:
my_db = db.DBConnection('cache.db')
queue_sql = my_db.select('SELECT * FROM search_queue')
for q in queue_sql:
if q['action_id'] in (BACKLOG_SEARCH, FAILED_SEARCH, MANUAL_SEARCH):
show_obj = helpers.find_show_by_id({q['indexer']: q['indexer_id']})
if not show_obj:
continue
if BACKLOG_SEARCH == q['action_id']:
segments = [show_obj.get_episode(*tuple([int(x) for x in cur_ep.split('x')]))
for cur_ep in q['segment'].split(',')]
item = BacklogQueueItem(show_obj=show_obj, segment=segments,
standard_backlog=bool(q['standard_backlog']),
limited_backlog=bool(q['limited_backlog']),
forced=bool(q['forced']), torrent_only=bool(q['torrent_only']),
uid=q['uid'])
elif FAILED_SEARCH == q['action_id']:
segments = [show_obj.get_episode(*tuple([int(x) for x in cur_ep.split('x')]))
for cur_ep in q['segment'].split(',')]
item = FailedQueueItem(show_obj=show_obj, segment=segments, uid=q['uid'])
elif MANUAL_SEARCH == q['action_id']:
segment = show_obj.get_episode(*tuple([int(x) for x in q['segment'].split('x')]))
item = ManualSearchQueueItem(show_obj=show_obj, segment=segment, uid=q['uid'])
else:
continue
self.add_item(item, add_to_db=False)
except (BaseException, Exception) as e:
logger.log('Exception loading queue %s: %s' % (self.__class__.__name__, ex(e)), logger.ERROR)
def _clear_sql(self):
return [
['DELETE FROM search_queue']
]
def _get_item_sql(self, item):
# type: (BaseSearchQueueItem) -> List[List]
if isinstance(item, BacklogQueueItem):
return [
['INSERT OR IGNORE INTO search_queue (indexer, indexer_id, segment, standard_backlog, limited_backlog,'
' forced, torrent_only, action_id, uid) VALUES (?,?,?,?,?,?,?,?,?)',
[item.show_obj._tvid, item.show_obj._prodid,
','.join('%sx%s' % (i.season, i.episode) for i in item.segment), int(item.standard_backlog),
int(item.limited_backlog), int(item.forced), int(item.torrent_only), BACKLOG_SEARCH, item.uid]]
]
elif isinstance(item, FailedQueueItem):
return [
['INSERT OR IGNORE INTO search_queue (indexer, indexer_id, segment, action_id, uid)'
' VALUES (?,?,?,?,?)',
[item.show_obj._tvid, item.show_obj._prodid,
','.join('%sx%s' % (i.season, i.episode) for i in item.segment), FAILED_SEARCH, item.uid]]
]
elif isinstance(item, ManualSearchQueueItem):
return [
['INSERT OR IGNORE INTO search_queue (indexer, indexer_id, segment, action_id, uid)'
' VALUES (?,?,?,?,?)',
[item.show_obj._tvid, item.show_obj._prodid, '%sx%s' % (item.segment.season, item.segment.episode),
MANUAL_SEARCH, item.uid]]
]
return []
def _delete_item_from_db_sql(self, item):
# type: (BaseSearchQueueItem) -> List[List]
if isinstance(item, (BacklogQueueItem, FailedQueueItem, ManualSearchQueueItem)):
return [
['DELETE FROM search_queue WHERE uid = ?', [item.uid]]
]
def _clear_queue(self, action_types=None, excluded_types=None):
generic_queue.GenericQueue._clear_queue(self, action_types=action_types,
excluded_types=[RECENT_SEARCH, PROPER_SEARCH])
def remove_from_queue(self, to_remove=None, force=False):
generic_queue.GenericQueue._remove_from_queue(self, to_remove=to_remove,
excluded_types=[RECENT_SEARCH, PROPER_SEARCH], force=force)
def is_in_queue(self, show_obj, segment):
# type: (sickbeard.tv.TVShow, List[sickbeard.tv.TVEpisode]) -> bool
with self.lock:
for cur_item in self.queue:
if isinstance(cur_item, BacklogQueueItem) \
and show_obj == cur_item.show_obj \
and segment == cur_item.segment:
return True
return False
return any(1 for cur_item in self.queue
if isinstance(cur_item, BacklogQueueItem) and show_obj == cur_item.show_obj
and segment == cur_item.segment)
def is_ep_in_queue(self, segment):
# type: (List[sickbeard.tv.TVEpisode]) -> bool
with self.lock:
for cur_item in self.queue:
if isinstance(cur_item, (ManualSearchQueueItem, FailedQueueItem)) and cur_item.segment == segment:
return True
return False
return any(1 for cur_item in self.queue
if isinstance(cur_item, (ManualSearchQueueItem, FailedQueueItem))
and cur_item.segment == segment)
def is_show_in_queue(self, tvid_prodid):
# type: (AnyStr) -> bool
with self.lock:
for cur_item in self.queue:
if isinstance(cur_item, (ManualSearchQueueItem, FailedQueueItem)) and \
tvid_prodid == cur_item.show_obj.tvid_prodid:
return True
return False
return any(1 for cur_item in self.queue
if isinstance(cur_item, (ManualSearchQueueItem, FailedQueueItem))
and tvid_prodid == cur_item.show_obj.tvid_prodid)
def pause_backlog(self):
# type: (...) -> None
@ -104,10 +178,7 @@ class SearchQueue(generic_queue.GenericQueue):
def _is_in_progress(self, item_type):
# type: (Any) -> bool
with self.lock:
for cur_item in self.queue + [self.currentItem]:
if isinstance(cur_item, item_type):
return True
return False
return any(1 for cur_item in self.queue + [self.currentItem] if isinstance(cur_item, item_type))
def get_queued_manual(self, tvid_prodid):
# type: (Optional[AnyStr]) -> List[BaseSearchQueueItem]
@ -149,18 +220,14 @@ class SearchQueue(generic_queue.GenericQueue):
def is_propersearch_in_progress(self):
# type: (...) -> bool
with self.lock:
for cur_item in self.queue + [self.currentItem]:
if isinstance(cur_item, ProperSearchQueueItem) and None is cur_item.propers:
return True
return False
return any(1 for cur_item in self.queue + [self.currentItem]
if isinstance(cur_item, ProperSearchQueueItem) and None is cur_item.propers)
def is_standard_backlog_in_progress(self):
# type: (...) -> bool
with self.lock:
for cur_item in self.queue + [self.currentItem]:
if isinstance(cur_item, BacklogQueueItem) and cur_item.standard_backlog:
return True
return False
return any(1 for cur_item in self.queue + [self.currentItem]
if isinstance(cur_item, BacklogQueueItem) and cur_item.standard_backlog)
def type_of_backlog_in_progress(self):
# type: (...) -> AnyStr
@ -202,7 +269,8 @@ class SearchQueue(generic_queue.GenericQueue):
tvid=cur_item.show_obj.tvid, prodid=cur_item.show_obj.prodid,
tvid_prodid=cur_item.show_obj.tvid_prodid,
# legacy keys for api responses
indexer=cur_item.show_obj.tvid, indexerid=cur_item.show_obj.prodid
indexer=cur_item.show_obj.tvid, indexerid=cur_item.show_obj.prodid,
uid=cur_item.uid
)
if isinstance(cur_item, BacklogQueueItem):
result_item.update(dict(
@ -215,26 +283,50 @@ class SearchQueue(generic_queue.GenericQueue):
length['manual'] += [result_item]
return length
def abort_show(self, show_obj):
# type: (TVShow) -> None
if show_obj:
with self.lock:
to_remove = []
for c in ((self.currentItem and [self.currentItem]) or []) + self.queue:
if show_obj == getattr(c, 'show_obj', None):
try:
to_remove.append(c.uid)
except (BaseException, Exception):
pass
try:
c.stop.set()
except (BaseException, Exception):
pass
if to_remove:
try:
self.remove_from_queue(to_remove)
except (BaseException, Exception):
pass
def add_item(
self,
item # type: Union[RecentSearchQueueItem, ProperSearchQueueItem, BacklogQueueItem, ManualSearchQueueItem, FailedQueueItem]
item, # type: Union[RecentSearchQueueItem, ProperSearchQueueItem, BacklogQueueItem, ManualSearchQueueItem, FailedQueueItem]
add_to_db=True # type: bool
):
# type: (...) -> None
"""
:param item:
:param add_to_db:
:type item: RecentSearchQueueItem or ProperSearchQueueItem or BacklogQueueItem or ManualSearchQueueItem or
FailedQueueItem
"""
if isinstance(item, (RecentSearchQueueItem, ProperSearchQueueItem)):
# recent and proper searches
generic_queue.GenericQueue.add_item(self, item)
generic_queue.GenericQueue.add_item(self, item, add_to_db=add_to_db)
elif isinstance(item, BacklogQueueItem) and not self.is_in_queue(item.show_obj, item.segment):
# backlog searches
generic_queue.GenericQueue.add_item(self, item)
generic_queue.GenericQueue.add_item(self, item, add_to_db=add_to_db)
elif isinstance(item, (ManualSearchQueueItem, FailedQueueItem)) and not self.is_ep_in_queue(item.segment):
# manual and failed searches
generic_queue.GenericQueue.add_item(self, item)
generic_queue.GenericQueue.add_item(self, item, add_to_db=add_to_db)
else:
logger.log(u'Not adding item, it\'s already in the queue', logger.DEBUG)
@ -364,7 +456,8 @@ class RecentSearchQueueItem(generic_queue.QueueItem):
my_db = db.DBConnection()
sql_result = my_db.select(
'SELECT indexer AS tvid, showid AS prodid, airdate, season, episode'
'SELECT indexer AS tvid, showid AS prodid, airdate, season, episode, timestamp,'
' timezone, network, airtime, runtime'
' FROM tv_episodes'
' WHERE status = ? AND season > 0 AND airdate <= ? AND airdate > 1'
' ORDER BY indexer, showid', [common.UNAIRED, cur_date])
@ -383,8 +476,16 @@ class RecentSearchQueueItem(generic_queue.QueueItem):
continue
try:
end_time = (network_timezones.parse_date_time(cur_result['airdate'], show_obj.airs, show_obj.network) +
datetime.timedelta(minutes=helpers.try_int(show_obj.runtime, 60)))
end_time = network_timezones.get_episode_time(cur_result['airdate'],
cur_result['airtime'] or show_obj.airs,
show_obj.network,
show_obj.timezone,
cur_result['timestamp'],
cur_result['network'],
cur_result['timezone']
)
end_time += datetime.timedelta(minutes=helpers.try_int(cur_result['runtime'] or show_obj.runtime, 60))
# filter out any episodes that haven't aired yet
if end_time > cur_time:
continue
@ -466,18 +567,25 @@ class ProperSearchQueueItem(generic_queue.QueueItem):
finally:
self.finish()
def __str__(self):
return '<%s - %s>' % (self.__class__.__name__, ('recent', 'native')[None is self.propers])
def __repr__(self):
return self.__str__()
class BaseSearchQueueItem(generic_queue.QueueItem):
def __init__(self, show_obj, segment, name, action_id=0):
# type: (sickbeard.tv.TVShow, Union[TVEpisode, List[TVEpisode]], AnyStr, int) -> None
def __init__(self, show_obj, segment, name, action_id=0, uid=None):
# type: (sickbeard.tv.TVShow, Union[TVEpisode, List[TVEpisode]], AnyStr, int, AnyStr) -> None
"""
:param show_obj: show object
:param segment: segment
:param name: name
:param action_id:
:param uid:
"""
super(BaseSearchQueueItem, self).__init__(name, action_id)
super(BaseSearchQueueItem, self).__init__(name, action_id, uid=uid)
self.segment = segment # type: Union[TVEpisode, List[TVEpisode]]
self.show_obj = show_obj
self.added_dt = None
@ -499,16 +607,34 @@ class BaseSearchQueueItem(generic_queue.QueueItem):
quality=s.show_obj.quality, upgrade_once=s.show_obj.upgrade_once
)) for s in ([self.segment], self.segment)[isinstance(self.segment, list)]])
def __str__(self):
if self.segment:
if isinstance(self.segment, list):
show_name = self.segment[0].show_obj and self.segment[0].show_obj.name
else:
show_name = self.segment.show_obj and self.segment.show_obj.name
segment_str = ' - %s (%s)' % \
(show_name,
','.join(['%sx%s' % (s.season, s.episode)
for s in ([self.segment], self.segment)[isinstance(self.segment, list)]]))
else:
segment_str = ''
return '<%s%s>' % (self.__class__.__name__, segment_str)
def __repr__(self):
return self.__str__()
class ManualSearchQueueItem(BaseSearchQueueItem):
def __init__(self, show_obj, segment):
# type: (sickbeard.tv.TVShow, sickbeard.tv.TVEpisode) -> None
def __init__(self, show_obj, segment, uid=None):
# type: (sickbeard.tv.TVShow, sickbeard.tv.TVEpisode, AnyStr) -> None
"""
:param show_obj: show object
:param segment: segment
:param uid:
"""
super(ManualSearchQueueItem, self).__init__(show_obj, segment, 'Manual Search', MANUAL_SEARCH)
super(ManualSearchQueueItem, self).__init__(show_obj, segment, 'Manual Search', MANUAL_SEARCH, uid=uid)
self.priority = generic_queue.QueuePriorities.HIGH # type: int
self.name = 'MANUAL-%s' % show_obj.tvid_prodid # type: AnyStr
self.started = None
@ -578,7 +704,8 @@ class BacklogQueueItem(BaseSearchQueueItem):
standard_backlog=False, # type: bool
limited_backlog=False, # type: bool
forced=False, # type: bool
torrent_only=False # type: bool
torrent_only=False, # type: bool
uid=None # type: AnyStr
):
"""
@ -588,8 +715,9 @@ class BacklogQueueItem(BaseSearchQueueItem):
:param limited_backlog: is limited backlog
:param forced: forced
:param torrent_only: torrent only
:param uid:
"""
super(BacklogQueueItem, self).__init__(show_obj, segment, 'Backlog', BACKLOG_SEARCH)
super(BacklogQueueItem, self).__init__(show_obj, segment, 'Backlog', BACKLOG_SEARCH, uid=uid)
self.priority = generic_queue.QueuePriorities.LOW # type: int
self.name = 'BACKLOG-%s' % show_obj.tvid_prodid # type: AnyStr
self.standard_backlog = standard_backlog # type: bool
@ -640,14 +768,15 @@ class BacklogQueueItem(BaseSearchQueueItem):
class FailedQueueItem(BaseSearchQueueItem):
def __init__(self, show_obj, segment):
# type: (sickbeard.tv.TVShow, List[sickbeard.tv.TVEpisode]) -> None
def __init__(self, show_obj, segment, uid=None):
# type: (sickbeard.tv.TVShow, List[sickbeard.tv.TVEpisode], AnyStr) -> None
"""
:param show_obj: show object
:param segment: segment
:param uid:
"""
super(FailedQueueItem, self).__init__(show_obj, segment, 'Retry', FAILED_SEARCH)
super(FailedQueueItem, self).__init__(show_obj, segment, 'Retry', FAILED_SEARCH, uid=uid)
self.priority = generic_queue.QueuePriorities.HIGH # type: int
self.name = 'RETRY-%s' % show_obj.tvid_prodid # type: AnyStr
self.started = None

9
sickbeard/sgdatetime.py

@ -245,20 +245,23 @@ class SGDatetime(datetime.datetime):
return SGDatetime.timestamp_far(obj)
@staticmethod
def from_timestamp(ts, local_time=True, tz_aware=False):
# type: (Union[float, integer_types], bool, bool) -> datetime.datetime
def from_timestamp(ts, local_time=True, tz_aware=False, tzinfo=None):
# type: (Union[float, integer_types], bool, bool, datetime.tzinfo) -> datetime.datetime
"""
convert timestamp to datetime.datetime obj
:param ts: timestamp integer, float
:param local_time: return as local timezone (SG_TIMEZONE)
:param tz_aware: return tz aware datetime
:param tzinfo: tzinfo to be used
"""
from .network_timezones import EPOCH_START, SG_TIMEZONE
result = EPOCH_START + datetime.timedelta(seconds=ts)
if local_time and SG_TIMEZONE:
result = result.astimezone(SG_TIMEZONE)
if isinstance(tzinfo, datetime.tzinfo):
result = result.astimezone(tzinfo)
if not tz_aware:
result = result.replace(tzinfo=None)
return result.replace(tzinfo=None)
return result
@static_or_instance

813
sickbeard/show_queue.py

File diff suppressed because it is too large

26
sickbeard/show_updater.py

@ -120,6 +120,22 @@ class ShowUpdater(object):
logger.log('image cache cleanup error', logger.ERROR)
logger.log(traceback.format_exc(), logger.ERROR)
# check tvinfo cache
try:
for i in sickbeard.TVInfoAPI().all_sources:
sickbeard.TVInfoAPI(i).setup().check_cache()
except (BaseException, Exception):
logger.log('tvinfo cache check error', logger.ERROR)
logger.log(traceback.format_exc(), logger.ERROR)
# cleanup tvinfo cache
try:
for i in sickbeard.TVInfoAPI().all_sources:
sickbeard.TVInfoAPI(i).setup().clean_cache()
except (BaseException, Exception):
logger.log('tvinfo cache cleanup error', logger.ERROR)
logger.log(traceback.format_exc(), logger.ERROR)
# cleanup ignore and require lists
try:
clean_ignore_require_words()
@ -178,13 +194,21 @@ class ShowUpdater(object):
stale_should_update.append(cur_result['tvid_prodid'])
# start update process
show_updates = {}
for src in sickbeard.TVInfoAPI().search_sources:
tvinfo_config = sickbeard.TVInfoAPI(src).api_params.copy()
t = sickbeard.TVInfoAPI(src).setup(**tvinfo_config)
show_updates.update({src: t.get_updated_shows()})
pi_list = []
for cur_show_obj in sickbeard.showList: # type: sickbeard.tv.TVShow
try:
# if should_update returns True (not 'Ended') or show is selected stale 'Ended' then update,
# otherwise just refresh
if cur_show_obj.should_update(update_date=update_date) \
if cur_show_obj.should_update(update_date=update_date,
last_indexer_change=show_updates.get(cur_show_obj.tvid, {}).
get(cur_show_obj.prodid)) \
or cur_show_obj.tvid_prodid in stale_should_update:
cur_queue_item = sickbeard.show_queue_scheduler.action.updateShow(cur_show_obj,
scheduled_update=True)

2594
sickbeard/tv.py

File diff suppressed because it is too large

8
sickbeard/tv_base.py

@ -61,7 +61,7 @@ class TVShowBase(LegacyTVShow, TVBase):
self._name = ''
self._imdbid = ''
self._network = ''
self.internal_network = ''
self._genre = ''
self._classification = ''
self._runtime = 0
@ -143,14 +143,14 @@ class TVShowBase(LegacyTVShow, TVBase):
def imdbid(self, *arg):
self.dirty_setter('_imdbid')(self, *arg)
# network = property(lambda self: self._network, dirty_setter('_network'))
# network = property(lambda self: self.internal_network, dirty_setter('internal_network'))
@property
def network(self):
return self._network
return self.internal_network
@network.setter
def network(self, *arg):
self.dirty_setter('_network')(self, *arg)
self.dirty_setter('internal_network')(self, *arg)
# genre = property(lambda self: self._genre, dirty_setter('_genre'))
@property

2
sickbeard/webapi.py

@ -51,7 +51,7 @@ from .common import ARCHIVED, DOWNLOADED, IGNORED, SKIPPED, SNATCHED, SNATCHED_A
from .helpers import remove_article
from .indexers import indexer_api, indexer_config
from .indexers.indexer_config import *
from tvinfo_base.exceptions import *
from lib.tvinfo_base.exceptions import *
from .scene_numbering import set_scene_numbering_helper
from .search_backlog import FORCED_BACKLOG
from .show_updater import clean_ignore_require_words

912
sickbeard/webserve.py

File diff suppressed because it is too large

92
sickgear.py

@ -94,6 +94,7 @@ from exceptions_helper import ex
import sickbeard
from sickbeard import db, logger, name_cache, network_timezones
from sickbeard.event_queue import Events
from sickbeard.generic_queue import QueuePriorities
from sickbeard.tv import TVShow
from sickbeard.webserveInit import WebServer
@ -503,6 +504,10 @@ class SickGear(object):
else:
logger.log_error_and_exit(u'Restore FAILED!')
# refresh network timezones
sickbeard.classes.loading_msg.message = 'Checking network timezones'
network_timezones.update_network_dict()
update_arg = '--update-restart'
manual_update_arg = '--update-pkg'
if update_arg not in sickbeard.MY_ARGS and sickbeard.UPDATES_TODO \
@ -545,10 +550,6 @@ class SickGear(object):
sickbeard.classes.loading_msg.message = 'Build name cache'
name_cache.buildNameCache()
# refresh network timezones
sickbeard.classes.loading_msg.message = 'Checking network timezones'
network_timezones.update_network_dict()
# load all ids from xem
sickbeard.classes.loading_msg.message = 'Loading xem data'
startup_background_tasks = threading.Thread(name='XEMUPDATER', target=sickbeard.scene_exceptions.get_xem_ids)
@ -565,8 +566,65 @@ class SickGear(object):
if not db.DBConnection().has_flag('kodi_nfo_default_removed'):
sickbeard.metadata.kodi.remove_default_attr()
my_db = db.DBConnection()
sw = my_db.select('SELECT * FROM tv_src_switch WHERE status = 0')
if sw:
switching = True
l_msg = 'Adding Shows that switching tv source to queue'
s_count = len(sw)
sickbeard.classes.loading_msg.set_msg_progress(l_msg, '0/%s' % s_count)
for i, s in enumerate(sw, 1):
sickbeard.classes.loading_msg.set_msg_progress(l_msg, '%s/%s' % (i, s_count))
try:
show_obj = sickbeard.helpers.find_show_by_id({s['old_indexer']: s['old_indexer_id']})
except (BaseException, Exception):
if s['new_indexer_id']:
try:
show_obj = sickbeard.helpers.find_show_by_id({s['new_indexer']: s['new_indexer_id']})
except (BaseException, Exception):
continue
if show_obj:
# show id was already switched, but not finished updated, so queue as update
try:
sickbeard.show_queue_scheduler.action.switch_show(
show_obj=show_obj, new_tvid=s['new_indexer'], new_prodid=s['new_indexer_id'],
force_id=bool(s['force_id']), uid=s['uid'],
set_pause=bool(s['set_pause']), mark_wanted=bool(s['mark_wanted']), resume=True,
old_tvid=s['old_indexer'], old_prodid=s['old_indexer_id']
)
except (BaseException, Exception):
continue
continue
if show_obj:
try:
sickbeard.show_queue_scheduler.action.switch_show(
show_obj=show_obj, new_tvid=s['new_indexer'], new_prodid=s['new_indexer_id'],
force_id=bool(s['force_id']), uid=s['uid'],
set_pause=bool(s['set_pause']), mark_wanted=bool(s['mark_wanted'])
)
except (BaseException, Exception):
continue
elif s['new_indexer_id']:
try:
show_obj = sickbeard.helpers.find_show_by_id({s['new_indexer']: s['new_indexer_id']})
except (BaseException, Exception):
continue
if show_obj:
# show id was already switched, but not finished updated, so resume
try:
sickbeard.show_queue_scheduler.action.switch_show(
show_obj=show_obj, new_tvid=s['new_indexer'], new_prodid=s['new_indexer_id'],
force_id=bool(s['force_id']), uid=s['uid'],
set_pause=bool(s['set_pause']), mark_wanted=bool(s['mark_wanted']), resume=True,
old_tvid=s['old_indexer'], old_prodid=s['old_indexer_id']
)
except (BaseException, Exception):
continue
else:
switching = False
# Start an update if we're supposed to
if self.force_update or sickbeard.UPDATE_SHOWS_ON_START:
if not switching and (self.force_update or sickbeard.UPDATE_SHOWS_ON_START):
sickbeard.classes.loading_msg.message = 'Starting a forced show update'
sickbeard.show_update_scheduler.action.run()
@ -575,7 +633,7 @@ class SickGear(object):
self.webserver.switch_handlers()
# main loop
while True:
while 1:
time.sleep(1)
def daemonize(self):
@ -649,13 +707,31 @@ class SickGear(object):
logger.log(u'Loading initial show list')
my_db = db.DBConnection()
sql_result = my_db.select('SELECT indexer AS tv_id, indexer_id AS prod_id, location FROM tv_shows')
sql_result = my_db.select('SELECT indexer AS tv_id, indexer_id AS prod_id, * FROM tv_shows'
' ORDER BY indexer, indexer_id')
imdb_info_sql_result = my_db.select('SELECT * FROM imdb_info'
' ORDER BY indexer, indexer_id')
failed_results = my_db.select('SELECT * FROM tv_shows_not_found ORDER BY indexer, indexer_id')
sickbeard.showList = []
sickbeard.showDict = {}
for cur_result in sql_result:
try:
show_obj = TVShow(int(cur_result['tv_id']), int(cur_result['prod_id']))
tv_id = int(cur_result['tv_id'])
prod_id = int(cur_result['prod_id'])
if imdb_info_sql_result and prod_id == imdb_info_sql_result[0]['indexer_id'] \
and tv_id == imdb_info_sql_result[0]['indexer']:
imdb_info_sql = imdb_info_sql_result.pop(0)
else:
imdb_info_sql = None
show_obj = TVShow(tv_id, prod_id, show_sql=cur_result, imdb_info_sql=imdb_info_sql)
failed_sql = None
for s in failed_results:
if prod_id == s['indexer_id'] and tv_id == s['indexer']:
failed_sql = s
failed_results.remove(s)
break
show_obj._helper_load_failed_db(sql=failed_sql)
sickbeard.showList.append(show_obj)
sickbeard.showDict[show_obj.sid_int] = show_obj
except (BaseException, Exception) as err:

Loading…
Cancel
Save