Collisions queue

Incomplete files

The file below is missing the last 20 byte RAR end block. This block contains a CRC of the RAR file and a RAR file volume number.
dupe_queue_incomplete_file.pngdupe_queue_corrupt_rars.png

Repacked releases

The repacked release can be detected by the date of the archived movie file.
repack.png

Release in release directory

The release is a second time in the SRR file.
twice.png

Releases with the same RAR names

Extra RAR file in SRR

dupe_queue_extra_rar_file.png

SFV contains each file twice

dupe_queue_sfv_contains_each_file_twice.png

Empty dupe

dupe_queue_empty_dupe.png

Application name in the middle of the SRR file

When 2 SRR files are concatenated, you still have a valid SRR file, but then there is a second application name in the middle of the file. This caused the SRR hash detection to fail.
dupe_queue_app_name_in_middle2.png
dupe_queue_app_name_in_middle.png

Detecting broken SRRs

dupe_queue_bad_upload_detected.png

Detecting broken SRRs (2)

dupe_queue_bad_upload_detected2a.png
dupe_queue_bad_upload_detected2b.png

The left SRR was created with pyReScene Usenet and was already on the site. The bottom of the details page can show info about possible errors.

Hard to detect repacks

Somewhere in November 2013, the "cleaned up" collisions queue looked a lot like this for almost two months:
collisions_queue_difficult_repack2.png
The reason for this was that there was no obvious indication that the SRR on the site was a repack. After all, it has been there since 2011-07-07, uploaded by SYSTEM. This means that the file came from the now defunct rescene.info database. So the dupes would stay in the queue until I would take the time for a deeper look into the issue, thinking they were probably repacks. The investigation started when another user happened to upload the same SRRs no less than a two months later. Now my interest piqued.
collisions_queue_difficult_repack1.png
You start looking by the obvious indicators first. The file names are exactly the same, so nothing you can search for. The archived file name and dates are the same too. This is what makes it hard: the repack is done by someone who once had the original RAR files. The SFV files also don't have any comments. SFVs from repacks often have comments with a timestamp in them that is after pre date. Checking predbs for these old SFV files didn't help either.

The only thing left to do is comparing bits in the SRR files for differences. Only the first 7 episodes of Star.Trek.DS9.S03 are in doubt, so we can compare with the other SRR files. pysrr -e Star.Trek.DS9.S03E08.Meridian.DVDRip.XviD-VF.srr will show all the details. I did this for one episode and compared the outputs with KDiff3. These are the differences:

File on site:

Block: RAR Archive Header; offset: 0x1D845 (120901 bytes)
|Header bytes: 12e87341010d00000000000000
|HEAD_CRC: 0xE812
|HEAD_TYPE: 0x73 (RAR Archive Header)
|HEAD_FLAGS: 0x0141
| 0x0001 MHD_VOLUME (Volume attribute (RAR is an archive volume))
| 0x0040 MHD_PROTECT (Recovery record present)
| 0x0100 MHD_FIRSTVOLUME (First volume (set only by RAR 3.0 and later))
|HEAD_SIZE: 0xD (13 bytes)
+RESERVED1: 2 bytes: 0
+RESERVED2: 4 bytes: 0

New upload:

Block: RAR Archive Header; offset: 0x1D7EA (120810 bytes)
|Header bytes: 51fc7341000d00000000000000
|HEAD_CRC: 0xFC51
|HEAD_TYPE: 0x73 (RAR Archive Header)
|HEAD_FLAGS: 0x0041
| 0x0001 MHD_VOLUME (Volume attribute (RAR is an archive volume))
| 0x0040 MHD_PROTECT (Recovery record present)
|HEAD_SIZE: 0xD (13 bytes)
+RESERVED1: 2 bytes: 0
+RESERVED2: 4 bytes: 0

File on site:

Block: RAR File; offset: 0x1D852 (120914 bytes)
|Header bytes: [snip]
|HEAD_CRC: 0xA41C
|HEAD_TYPE: 0x74 (RAR File)
|HEAD_FLAGS: 0x80C2
| 0x8000 LONG_BLOCK (ADD_SIZE field present)
| 0x0002 LHD_SPLIT_AFTER (file continued in next volume)
| 0x00C0 LHD_WINDOW (Dictionary size 4096 KiB)
|HEAD_SIZE: 0x47 (71 bytes)
+PACK_SIZE: 14851199 bytes (ADD_SIZE + HIGH_PACK_SIZE field)
+UNP_SIZE: 367433728 bytes
+HOST_OS: Windows used to create this file block.
+FILE_CRC: 1FC106EF
+FTIME: 2003-05-25 12:53:54
+UNP_VER: Version 2.0 is needed to extract.
+METHOD: Storing
+NAME_SIZE: always present
+ATTR: 20
+FILE_NAME: Star.Trek.DS9.S03E07.DVDRip.XviD-VF.avi

New upload:

Block: RAR File; offset: 0x1D7F7 (120823 bytes)
|Header bytes: [snip]
|HEAD_CRC: 0x3322
|HEAD_TYPE: 0x74 (RAR File)
|HEAD_FLAGS: 0x8082
| 0x8000 LONG_BLOCK (ADD_SIZE field present)
| 0x0002 LHD_SPLIT_AFTER (file continued in next volume)
| 0x0080 LHD_WINDOW (Dictionary size 1024 KiB)
|HEAD_SIZE: 0x47 (71 bytes)
+PACK_SIZE: 14851245 bytes (ADD_SIZE + HIGH_PACK_SIZE field)
+UNP_SIZE: 367433728 bytes
+HOST_OS: Windows used to create this file block.
+FILE_CRC: E2992335
+FTIME: 2003-05-25 12:53:54
+UNP_VER: Version 2.0 is needed to extract.
+METHOD: Storing
+NAME_SIZE: always present
+ATTR: 20
+FILE_NAME: Star.Trek.DS9.S03E07.DVDRip.XviD-VF.avi

I did the same for one episode of which there was no doubt:

Block: RAR Archive Header; offset: 0x1E9B3 (125363 bytes)
|Header bytes: 51fc7341000d00000000000000
|HEAD_CRC: 0xFC51
|HEAD_TYPE: 0x73 (RAR Archive Header)
|HEAD_FLAGS: 0x0041
| 0x0001 MHD_VOLUME (Volume attribute (RAR is an archive volume))
| 0x0040 MHD_PROTECT (Recovery record present)
|HEAD_SIZE: 0xD (13 bytes)
+RESERVED1: 2 bytes: 0
+RESERVED2: 4 bytes: 0

Block: RAR File; offset: 0x1E9C0 (125376 bytes)
|Header bytes: [snip]
|HEAD_CRC: 0xBEA9
|HEAD_TYPE: 0x74 (RAR File)
|HEAD_FLAGS: 0x8082
| 0x8000 LONG_BLOCK (ADD_SIZE field present)
| 0x0002 LHD_SPLIT_AFTER (file continued in next volume)
| 0x0080 LHD_WINDOW (Dictionary size 1024 KiB)
|HEAD_SIZE: 0x47 (71 bytes)
+PACK_SIZE: 14851245 bytes (ADD_SIZE + HIGH_PACK_SIZE field)
+UNP_SIZE: 367486976 bytes
+HOST_OS: Windows used to create this file block.
+FILE_CRC: 485D639C
+FTIME: 2003-05-25 13:04:14
+UNP_VER: Version 2.0 is needed to extract.
+METHOD: Storing
+NAME_SIZE: always present
+ATTR: 20
+FILE_NAME: Star.Trek.DS9.S03E08.DVDRip.XviD-VF.avi

The new uploads were the original RAR files all along.

One hour difference

Sometimes there is a collision and the only difference we can see is the the archived file date is exactly 1 hour off. How to handle those? It can be very tricky, but here I'll construct a line of reasoning on how to handle those:

Example

The example where I'll build the theory on: Brothers.and.Sisters.S02E12.720p.HDTV.x264-CTU

Original: ctu-x264-brothers.and.sisters.212.rar 50,000,000 257A9488
brothers.and.sisters.s02e12.720p.hdtv.x264-ctu.mkv archived file date: 2008-02-18 12:00:00

Repack: brothers.and.sisters.s02e12.720p.hdtv.x264-ctu.rar 50,000,000 8C07B5A7
brothers.and.sisters.s02e12.720p.hdtv.x264-ctu.mkv archived file date: 2008-02-18 01:00:00

How do I know which one is the repack? CTU uses a certain pattern to name their archives. Also the comment on the SFV file of the repack gives it away:

; Generated by WIN-SFV32 v1.1a (pure-sfv v0.3) on 2008-08-21 at 03:45.00

Example 2

one_hour_off_repack.png

Daylight saving time

http://en.wikipedia.org/wiki/Daylight_saving_time

GMT +2 Summer time: clocks are advanced by 1 hour
GMT +1 Winter time: an additional hour coming from summer time

From 2002 the rule has been:

  • the summer-time period shall begin, in every Member State, at 1.00 a.m., Greenwich Mean Time, on the last Sunday in March.
  • the summer-time period shall end, in every Member State, at 1.00 a.m., Greenwich Mean Time, on the last Sunday in October.

See https://greenwichmeantime.com/info/bst/

This seems to be the only explanation for the exact one hour difference.

Conclusion

For German releases, it seems safe to assume that:

  • Release pred in winter time? The earliest/oldest time stamp is original. (like our example above)
  • Release pred in summer time? The latest time stamp is original.

For other locations, this might behave differently. Seasons are switched in the southern hemisphere.
http://en.wikipedia.org/wiki/Daylight_saving_time_by_country

RedBlade DEADBABE SFV

redblade.png

RedBlade created RAR volumes that are 14,999,996 bytes large. They padded each volume with 4 bytes to match the predefined CRC32 value (DEADBABE). The process on how to do this is described on https://www.nayuki.io/page/forcing-a-files-crc-to-any-value
ReScene .NET ignores padded bytes at the end of each RAR volume. pyReScene shows the correct size because it keeps track of the padded bytes.

http://www.srrdb.com/release/details/Metal.Hurlant.Chronicles.S01E02.BDRip.XviD-RedBlade
http://www.srrdb.com/release/details/Metal.Hurlant.Chronicles.S01E04.BDRip.XviD-RedBlade

Large differences in NFO file sizes

This can be because all trailing spaces at the end of each line are stripped. We keep the largest (original) file of course!
e.g. http://www.srrdb.com/release/details/Bear_Necessities-Nut_E_1-MA05-1993-OSM

Original

nfo_original.png

Stripped spaces

nfo_stripped_spaces.png

Which file name is original?

XTC_iNT

We had two identical files:

00-various-jungle_hits_vol_3-strcd3-1995--xtc.jpg
00-various-jungle_hits_vol_3-strcd3-1995-xtc.jpg

[00:47] <X> jpg i guess the one which does not have the --
[00:47] <@Gfy> I would guess the other way around
[00:48] <@Gfy> people tend to "fix" things, not break them
[00:48] <@Gfy> that's why I think --

Later I found an example where we only have the -- variant
http://www.srrdb.com/release/details/Adam_Freeland-On_Tour-MAPACDA002-CD-2001-XTC_iNT
And most others were the same or totally different:
http://www.srrdb.com/browse/-cd-/XTC_iNT/1

SRS file name

Named after release name

Due to a script of an early .srr creator, his .srs files were named after the release name. e.g. Rls-GROUP.srs
The proper sample name was stored inside the SRS file.

Appended with -sample

The string -sample is appended to the .srs file name. The file name inside the .srs is correct.
These SRS files need to be replaced on a new upload.

appended-sample.png

Broken images

Images can show errors in various ways.

00-anushka-never_can_decide--(bwood0114dd)-web-cover-wws.-wws.jpg
https://www.srrdb.com/release/details/Anushka-Never_Can_Decide-(BWOOD0114DD)-WEB-2014-wWs

Two different samples

Sometimes there are releases that appear to have two different samples with only a single character difference in the file name.
Look at the mkv meta data to confirm which sample is original. The sample that matches the main mkv is the correct one.
Can it be generalized to say the sample with the dot is created afterwards?

If you do not have access to the main movie file, you can cross check with srs files from the same group around the time of release.

Use mkvinfo.exe (MKVToolNix), MediaInfo or a player that can show the meta data in the container.

Test case: https://www.srrdb.com/release/details/Due.Date.1080p.BluRay.x264-BLOW

Main MKV

+ EBML head
|+ EBML version: 1
|+ EBML read version: 1
|+ Maximum EBML ID length: 4
|+ Maximum EBML size length: 8
|+ Document type: matroska
|+ Document type version: 2
|+ Document type read version: 2
+ Segment: size 7040453633
|+ Seek head (subentries will be skipped)
|+ EBML void: size 4043
|+ Segment information
| + Timestamp scale: 1000000
| + Multiplexing application: libebml v1.0.0 + libmatroska v1.0.0
| + Writing application: mkvmerge v4.0.0 ('The Stars were mine') built on Jun 6 2010 16:18:42
| + Duration: 01:35:16.704000000
| + Date: Tue Feb 01 14:13:25 2011 UTC
| + Segment UID: 0xb7 0x16 0x4a 0xd4 0x1b 0xb9 0xb7 0xb0 0x94 0x90 0x0e 0xf1 0xe7 0xc1 0x96 0x23
|+ Tracks
| + Track
| + Track number: 1 (track ID for mkvmerge & mkvextract: 0)
| + Track UID: 1
| + Track type: video
| + Default track flag: 0
| + Lacing flag: 0
| + Minimum cache: 1
| + Codec ID: V_MPEG4/ISO/AVC
| + Codec's private data: size 40 (h.264 profile: High @L4.1)
| + Default duration: 00:00:00.041708333 (23.976 frames/fields per second for a video track)
| + Video track
| + Pixel width: 1920
| + Pixel height: 800
| + Display width: 1920
| + Display height: 800
| + Track
| + Track number: 2 (track ID for mkvmerge & mkvextract: 1)
| + Track UID: 3047809440
| + Track type: audio
| + Codec ID: A_DTS
| + Language: und
| + Audio track
| + Sampling frequency: 48000
| + Channels: 6
| + Track
| + Track number: 3 (track ID for mkvmerge & mkvextract: 2)
| + Track UID: 1656026785
| + Track type: subtitles
| + Lacing flag: 0
| + Codec ID: S_TEXT/UTF8
| + Language: und
|+ EBML void: size 1118
|+ Cluster

Sample blow-duedate.1080p-sample.mkv

+ EBML head
|+ EBML version: 1
|+ EBML read version: 1
|+ Maximum EBML ID length: 4
|+ Maximum EBML size length: 8
|+ Document type: matroska
|+ Document type version: 2
|+ Document type read version: 2
+ Segment: size 85002559
|+ Seek head (subentries will be skipped)
|+ EBML void: size 4044
|+ Segment information
| + Timestamp scale: 1000000
| + Multiplexing application: libebml v1.0.0 + libmatroska v1.0.0
| + Writing application: mkvmerge v4.0.0 ('The Stars were mine') built on Jun 6 2010 16:18:42
| + Duration: 00:01:02.855000000
| + Date: Tue Feb 01 14:33:14 2011 UTC
| + Segment UID: 0x89 0x4d 0x69 0x53 0xff 0xbe 0x33 0xcd 0xbd 0xaf 0x3a 0x6b 0x9b 0xcc 0xab 0x06
|+ Tracks
| + Track
| + Track number: 1 (track ID for mkvmerge & mkvextract: 0)
| + Track UID: 1
| + Track type: video
| + Default track flag: 0
| + Lacing flag: 0
| + Minimum cache: 1
| + Codec ID: V_MPEG4/ISO/AVC
| + Codec's private data: size 40 (h.264 profile: High @L4.1)
| + Default duration: 00:00:00.041708332 (23.976 frames/fields per second for a video track)
| + Video track
| + Pixel width: 1920
| + Pixel height: 800
| + Display width: 1920
| + Display height: 800
| + Track
| + Track number: 2 (track ID for mkvmerge & mkvextract: 1)
| + Track UID: 3047809440
| + Track type: audio
| + Default track flag: 0
| + Codec ID: A_DTS
| + Language: und
| + Audio track
| + Sampling frequency: 48000
| + Channels: 6
| + Track
| + Track number: 3 (track ID for mkvmerge & mkvextract: 2)
| + Track UID: 1656026785
| + Track type: subtitles
| + Default track flag: 0
| + Lacing flag: 0
| + Codec ID: S_TEXT/UTF8
| + Language: und
|+ EBML void: size 1112
|+ Cluster

Sample blow-duedate.1080p.sample.mkv

+ EBML head
|+ EBML version: 1
|+ EBML read version: 1
|+ Maximum EBML ID length: 4
|+ Maximum EBML size length: 8
|+ Document type: matroska
|+ Document type version: 2
|+ Document type read version: 2
+ Segment: size 55074452
|+ Seek head (subentries will be skipped)
|+ EBML void: size 4044
|+ Segment information
| + Timestamp scale: 1000000
| + Multiplexing application: libebml v1.0.0 + libmatroska v1.0.0
| + Writing application: mkvmerge v4.3.0 ('Escape from the Island') built on Sep 5 2010 10:30:51
| + Duration: 00:00:45.046000000
| + Date: Tue Feb 01 22:32:48 2011 UTC
| + Segment UID: 0x9a 0x55 0x73 0x68 0x4a 0x9e 0x91 0x73 0x87 0xd0 0x4c 0x74 0x45 0xe8 0xd0 0x1d
|+ Tracks
| + Track
| + Track number: 1 (track ID for mkvmerge & mkvextract: 0)
| + Track UID: 1
| + Track type: video
| + Default track flag: 0
| + Lacing flag: 0
| + Minimum cache: 1
| + Codec ID: V_MPEG4/ISO/AVC
| + Codec's private data: size 40 (h.264 profile: High @L4.1)
| + Default duration: 00:00:00.041708332 (23.976 frames/fields per second for a video track)
| + Video track
| + Pixel width: 1920
| + Pixel height: 800
| + Display width: 1920
| + Display height: 800
| + Content encodings
| + Content encoding
| + Content compression
| + Algorithm: 3 (header removal)
| + Settings: length 1, data: 0x00
| + Track
| + Track number: 2 (track ID for mkvmerge & mkvextract: 1)
| + Track UID: 3047809440
| + Track type: audio
| + Codec ID: A_DTS
| + Language: und
| + Audio track
| + Sampling frequency: 48000
| + Channels: 6
| + Content encodings
| + Content encoding
| + Content compression
| + Algorithm: 3 (header removal)
| + Settings: length 4, data: 0x7f 0xfe 0x80 0x01
| + Track
| + Track number: 3 (track ID for mkvmerge & mkvextract: 2)
| + Track UID: 1656026785
| + Track type: subtitles
| + Lacing flag: 0
| + Codec ID: S_TEXT/UTF8
| + Language: und
|+ EBML void: size 1141
|+ Cluster

We don't know

The RAR files have different names, but it seems they are the exact same RARs. (same SFV CRCs and SRRs look the same) However, the RAR hashes don't match and the nfos are different too. Look at the dates. Based on these dates I guess that the one with the NFO closest to the RAR date is the original one.

Undercover.Boss.US.S05E14.HDTV.x264-2HD is an example of a bad SRR file. We don't know what caused this.

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License