You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

176 lines
6.8 KiB

Change core system to improve performance and facilitate multi TV info sources. Change migrate core objects TVShow and TVEpisode and everywhere that these objects affect. Add message to logs and disable ui backlog buttons when no media provider has active and/or scheduled searching enabled. Change views for py3 compat. Change set default runtime of 5 mins if none is given for layout Day by Day. Add OpenSubtitles authentication support to config/Subtitles/Subtitles Plugin. Add "Enforce media hash match" to config/Subtitles Plugin/Opensubtitles for accurate subs if enabled, but if disabled, search failures will fallback to use less reliable subtitle results. Add Apprise 0.8.0 (6aa52c3). Add hachoir_py3 3.0a6 (5b9e05a). Add sgmllib3k 1.0.0 Update soupsieve 1.9.1 (24859cc) to soupsieve_py2 1.9.5 (6a38398) Add soupsieve_py3 2.0.0.dev (69194a2). Add Tornado_py3 Web Server 6.0.3 (ff985fe). Add xmlrpclib_to 0.1.1 (c37db9e). Remove ancient Growl lib 0.1 Remove xmltodict library. Change requirements.txt for Cheetah3 to minimum 3.2.4 Change update sabToSickBeard. Change update autoProcessTV. Change remove Twitter notifier. Update NZBGet Process Media extension, SickGear-NG 1.7 → 2.4 Update Kodi addon 1.0.3 → 1.0.4 Update ADBA for py3. Update Beautiful Soup 4.8.0 (r526) to 4.8.1 (r531). Update Send2Trash 1.3.0 (a568370) to 1.5.0 (66afce7). Update soupsieve 1.9.1 (24859cc) to 1.9.5 (6a38398). Change use GNTP (Growl Notification Transport Protocol) from Apprise. Change add multi host support to Growl notifier. Fix Growl notifier when using empty password. Change update links for Growl notifications. Change deprecate confg/Notifications/Growl password field as these are now stored with host setting. Fix prevent infinite memoryError from a particular jpg data structure. Change subliminal for py3. Change enzyme for py3. Change browser_ua for py3. Change feedparser for py3 (sgmlib is no longer available on py3 as standardlib so added ext lib) Fix Guessit. Fix parse_xml for py3. Fix name parser with multi eps for py3. Fix tvdb_api fixes for py3 (search show). Fix config/media process to only display "pattern is invalid" qtip on "Episode naming" tab if the associated field is actually visible. Also, if the field becomes hidden due to a setting change, hide any previously displayed qtip. Note for Javascript::getelementbyid (or $('tag[id="<name>"')) is required when an id is being searched in the dom due to ":" used in a shows id name. Change download anidb xml files to main cache folder and use adba lib folder as a last resort. Change create get anidb show groups as centralised helper func and consolidate dupe code. Change move anidb related functions to newly renamed anime.py (from blacklistandwhitelist.py). Change str encode hex no longer exits in py3, use codecs.encode(...) instead. Change fix b64decode on py3 returns bytestrings. Change use binary read when downloading log file via browser to prevent any encoding issues. Change add case insensitive ordering to anime black/whitelist. Fix anime groups list not excluding whitelisted stuff. Change add Windows utf8 fix ... see: ytdl-org/youtube-dl#820 Change if no qualities are wanted, exit manual search thread. Fix keepalive for py3 process media. Change add a once a month update of tvinfo show mappings to the daily updater. Change autocorrect ids of new shows by updating from -8 to 31 days of the airdate of episode one. Add next run time to Manage/Show Tasks/Daily show update. Change when fetching imdb data, if imdb id is an episode id then try to find and use real show id. Change delete diskcache db in imdbpie when value error (due to change in Python version). Change during startup, cleanup any _cleaner.pyc/o to prevent issues when switching python versions. Add .pyc cleaner if python version is switched. Change replace deprecated gettz_db_metadata() and gettz. Change rebrand "SickGear PostProcessing script" to "SickGear Process Media extension". Change improve setup guide to use the NZBGet version to minimise displayed text based on version. Change NZBGet versions prior to v17 now told to upgrade as those version are no longer supported - code has actually exit on start up for some time but docs were outdated. Change comment out code and unused option sg_base_path. Change supported Python version 2.7.9-2.7.18 inclusive expanded to 3.7.1-3.8.1 inclusive. Change pidfile creation under Linux 0o644. Make logger accept lists to output continuously using the log_lock instead of split up by other processes. Fix long path issues with Windows process media.
6 years ago
"""
ReiserFS file system version 3 parser (other version have not been tested).
Author: Frederic Weisbecker
Creation date: 8 december 2006
Sources:
- http://p-nand-q.com/download/rfstool/reiserfs_docs.html
- http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php
- file://usr/src/linux-2.6.16.19/include/linux/reiserfs_fs.h
NOTES:
The most part of the description of the structures, their fields and their
comments decribed here comes from the file include/linux/reiserfs_fs.h
- written by Hans reiser - located in the Linux kernel 2.6.16.19 and from
the Reiserfs explanations in
http://p-nand-q.com/download/rfstool/reiserfs_docs.html written by Gerson
Kurz.
"""
from hachoir_py3.parser import Parser
from hachoir_py3.field import (FieldSet, Enum,
UInt16, UInt32, String, RawBytes, NullBytes, SeekableFieldSet, Bit)
from hachoir_py3.core.endian import LITTLE_ENDIAN
class BlockState(Bit):
"""The state (used/free) of a ReiserFs Block"""
STATE = {
True: "used",
False: "free"
}
block_nb = 0
def __init__(self, parent, name, nb_block):
"""@param nb_block: Number of the block concerned"""
Bit.__init__(self, parent, name)
self.block_nb = self.__class__.block_nb
self.__class__.block_nb += 1
def createDescription(self):
return "State of the block %d" % self.block_nb
def createDisplay(self):
return self.STATE[Bit.createValue(self)]
class BitmapBlock(SeekableFieldSet):
""" The bitmap blocks are Reiserfs blocks where each byte contains
the state of 8 blocks in the filesystem. So each bit will describe
the state of a block to tell if it is used or not.
"""
def createFields(self):
block_size = self["/superblock/blocksize"].value
for i in range(0, block_size * 8):
yield BlockState(self, "block[]", i)
class BitmapBlockGroup(SeekableFieldSet):
"""The group that manages the Bitmap Blocks"""
def createFields(self):
block_size = self["/superblock/blocksize"].value
nb_bitmap_block = self["/superblock/bmap_nr"].value
# Position of the first bitmap block
self.seekByte(REISER_FS.SUPERBLOCK_OFFSET + block_size, relative=False)
yield BitmapBlock(self, "BitmapBlock[]", "Bitmap blocks tells for each block if it is used")
# The other bitmap blocks
for i in range(1, nb_bitmap_block):
self.seekByte((block_size**2) * 8 * i, relative=False)
yield BitmapBlock(self, "BitmapBlock[]", "Bitmap blocks tells for each block if it is used")
class Journal_params(FieldSet):
static_size = 32 * 8
def createFields(self):
yield UInt32(self, "1st_block", "Journal 1st block number")
yield UInt32(self, "dev", "Journal device number")
yield UInt32(self, "size", "Size of the journal")
yield UInt32(self, "trans_max", "Max number of blocks in a transaction")
# TODO: Must be explained: it was sb_journal_block_count
yield UInt32(self, "magic", "Random value made on fs creation.")
yield UInt32(self, "max_batch", "Max number of blocks to batch into a trans")
yield UInt32(self, "max_commit_age", "In seconds, how old can an async commit be")
yield UInt32(self, "max_trans_age", "In seconds, how old can a transaction be")
def createDescription(self):
return "Parameters of the journal"
class SuperBlock(FieldSet):
# static_size = 204*8
UMOUNT_STATE = {1: "unmounted", 2: "not unmounted"}
HASH_FUNCTIONS = {
0: "UNSET_HASH",
1: "TEA_HASH",
2: "YURA_HASH",
3: "R5_HASH"
}
def createFields(self):
# TODO: This structure is normally divided in two parts:
# _reiserfs_super_block_v1
# _reiserfs_super_block
# It will be divided later to easily support older version of the first
# part
yield UInt32(self, "block_count", "Number of blocks")
yield UInt32(self, "free_blocks", "Number of free blocks")
yield UInt32(self, "root_block", "Root block number")
yield Journal_params(self, "Journal parameters")
yield UInt16(self, "blocksize", "Size of a block")
yield UInt16(self, "oid_maxsize", "Max size of object id array")
yield UInt16(self, "oid_cursize", "Current size of object id array")
yield Enum(UInt16(self, "umount_state", "Filesystem umounted or not"), self.UMOUNT_STATE)
yield String(self, "magic", 10, "Magic string", strip="\0")
# TODO: change the type of s_fs_state in Enum to have more details
# about this fsck state
yield UInt16(self, "fs_state", "Rebuilding phase of fsck ")
yield Enum(UInt32(self, "hash_function", "Hash function to sort names in a directory"), self.HASH_FUNCTIONS)
yield UInt16(self, "tree_height", "Height of disk tree")
yield UInt16(self, "bmap_nr", "Amount of bitmap blocks needed to address each block of file system")
# TODO: find a good description for this field
yield UInt16(self, "version", "Field only reliable on filesystem with non-standard journal")
yield UInt16(self, "reserved_for_journal", "Size in blocks of journal area on main device")
# TODO: same as above
yield UInt32(self, "inode_generation", "No description")
# TODO: same as above and should be an enum field
yield UInt32(self, "flags", "No description")
# TODO: Create a special Type to format this id
yield RawBytes(self, "uuid", 16, "Filesystem unique identifier")
yield String(self, "label", 16, "Filesystem volume label", strip="\0")
yield NullBytes(self, "unused", 88)
yield NullBytes(self, "Bytes before end of the block", self["blocksize"].value - 204)
def createDescription(self):
return "Superblock: ReiserFs Filesystem"
class REISER_FS(Parser):
PARSER_TAGS = {
"id": "reiserfs",
"category": "file_system",
# 130 blocks before the journal +
# Minimal size of journal (513 blocks) +
# 1 block for the rest
# And The Minimal size of a block is 512 bytes
"min_size": (130 + 513 + 1) * (512 * 8),
"description": "ReiserFS file system"
}
endian = LITTLE_ENDIAN
# Offsets (in bytes) of important information
SUPERBLOCK_OFFSET = 64 * 1024
MAGIC_OFFSET = SUPERBLOCK_OFFSET + 52
def validate(self):
# Let's look at the magic field in the superblock
magic = self.stream.readBytes(self.MAGIC_OFFSET * 8, 9).rstrip(b"\0")
if magic in (b"ReIsEr3Fs", b"ReIsErFs", b"ReIsEr2Fs"):
return True
return "Invalid magic string"
def createFields(self):
yield NullBytes(self, "padding[]", self.SUPERBLOCK_OFFSET)
yield SuperBlock(self, "superblock")
yield BitmapBlockGroup(self, "Group of bitmap blocks")