Collisions queue

Incomplete files

The file below is missing the last 20 byte RAR end block. This block contains a CRC of the RAR file and a RAR file volume number.
dupe_queue_incomplete_file.pngdupe_queue_corrupt_rars.png

Repacked releases

The repacked release can be detected by the date of the archived movie file.
repack.png

Release in release directory

The release is a second time in the SRR file.
twice.png

Releases with the same RAR names

Extra RAR file in SRR

dupe_queue_extra_rar_file.png

SFV contains each file twice

dupe_queue_sfv_contains_each_file_twice.png

Empty dupe

dupe_queue_empty_dupe.png

Application name in the middle of the SRR file

When 2 SRR files are concatenated, you still have a valid SRR file, but then there is a second application name in the middle of the file. This caused the SRR hash detection to fail.
dupe_queue_app_name_in_middle2.png
dupe_queue_app_name_in_middle.png

Detecting broken SRRs

dupe_queue_bad_upload_detected.png

Detecting broken SRRs (2)

dupe_queue_bad_upload_detected2a.png
dupe_queue_bad_upload_detected2b.png

The left SRR was created with pyReScene Usenet and was already on the site. The bottom of the details page can show info about possible errors.

Hard to detect repacks

Somewhere in November 2013, the "cleaned up" collisions queue looked a lot like this for almost two months:
collisions_queue_difficult_repack2.png
The reason for this was that there was no obvious indication that the SRR on the site was a repack. After all, it has been there since 2011-07-07, uploaded by SYSTEM. This means that the file came from the now defunct rescene.info database. So the dupes would stay in the queue until I would take the time for a deeper look into the issue, thinking they were probably repacks. The investigation started when another user happened to upload the same SRRs no less than a two months later. Now my interest piqued.
collisions_queue_difficult_repack1.png
You start looking by the obvious indicators first. The file names are exactly the same, so nothing you can search for. The archived file name and dates are the same too. This is what makes it hard: the repack is done by someone who once had the original RAR files. The SFV files also don't have any comments. SFVs from repacks often have comments with a timestamp in them that is after pre date. Checking predbs for these old SFV files didn't help either.

The only thing left to do is comparing bits in the SRR files for differences. Only the first 7 episodes of Star.Trek.DS9.S03 are in doubt, so we can compare with the other SRR files. pysrr -e Star.Trek.DS9.S03E08.Meridian.DVDRip.XviD-VF.srr will show all the details. I did this for one episode and compared the outputs with KDiff3. These are the differences:

File on site:

Block: RAR Archive Header; offset: 0x1D845 (120901 bytes)
|Header bytes: 12e87341010d00000000000000
|HEAD_CRC: 0xE812
|HEAD_TYPE: 0x73 (RAR Archive Header)
|HEAD_FLAGS: 0x0141
| 0x0001 MHD_VOLUME (Volume attribute (RAR is an archive volume))
| 0x0040 MHD_PROTECT (Recovery record present)
| 0x0100 MHD_FIRSTVOLUME (First volume (set only by RAR 3.0 and later))
|HEAD_SIZE: 0xD (13 bytes)
+RESERVED1: 2 bytes: 0
+RESERVED2: 4 bytes: 0

New upload:

Block: RAR Archive Header; offset: 0x1D7EA (120810 bytes)
|Header bytes: 51fc7341000d00000000000000
|HEAD_CRC: 0xFC51
|HEAD_TYPE: 0x73 (RAR Archive Header)
|HEAD_FLAGS: 0x0041
| 0x0001 MHD_VOLUME (Volume attribute (RAR is an archive volume))
| 0x0040 MHD_PROTECT (Recovery record present)
|HEAD_SIZE: 0xD (13 bytes)
+RESERVED1: 2 bytes: 0
+RESERVED2: 4 bytes: 0

File on site:

Block: RAR File; offset: 0x1D852 (120914 bytes)
|Header bytes: [snip]
|HEAD_CRC: 0xA41C
|HEAD_TYPE: 0x74 (RAR File)
|HEAD_FLAGS: 0x80C2
| 0x8000 LONG_BLOCK (ADD_SIZE field present)
| 0x0002 LHD_SPLIT_AFTER (file continued in next volume)
| 0x00C0 LHD_WINDOW (Dictionary size 4096 KiB)
|HEAD_SIZE: 0x47 (71 bytes)
+PACK_SIZE: 14851199 bytes (ADD_SIZE + HIGH_PACK_SIZE field)
+UNP_SIZE: 367433728 bytes
+HOST_OS: Windows used to create this file block.
+FILE_CRC: 1FC106EF
+FTIME: 2003-05-25 12:53:54
+UNP_VER: Version 2.0 is needed to extract.
+METHOD: Storing
+NAME_SIZE: always present
+ATTR: 20
+FILE_NAME: Star.Trek.DS9.S03E07.DVDRip.XviD-VF.avi

New upload:

Block: RAR File; offset: 0x1D7F7 (120823 bytes)
|Header bytes: [snip]
|HEAD_CRC: 0x3322
|HEAD_TYPE: 0x74 (RAR File)
|HEAD_FLAGS: 0x8082
| 0x8000 LONG_BLOCK (ADD_SIZE field present)
| 0x0002 LHD_SPLIT_AFTER (file continued in next volume)
| 0x0080 LHD_WINDOW (Dictionary size 1024 KiB)
|HEAD_SIZE: 0x47 (71 bytes)
+PACK_SIZE: 14851245 bytes (ADD_SIZE + HIGH_PACK_SIZE field)
+UNP_SIZE: 367433728 bytes
+HOST_OS: Windows used to create this file block.
+FILE_CRC: E2992335
+FTIME: 2003-05-25 12:53:54
+UNP_VER: Version 2.0 is needed to extract.
+METHOD: Storing
+NAME_SIZE: always present
+ATTR: 20
+FILE_NAME: Star.Trek.DS9.S03E07.DVDRip.XviD-VF.avi

I did the same for one episode of which there was no doubt:

Block: RAR Archive Header; offset: 0x1E9B3 (125363 bytes)
|Header bytes: 51fc7341000d00000000000000
|HEAD_CRC: 0xFC51
|HEAD_TYPE: 0x73 (RAR Archive Header)
|HEAD_FLAGS: 0x0041
| 0x0001 MHD_VOLUME (Volume attribute (RAR is an archive volume))
| 0x0040 MHD_PROTECT (Recovery record present)
|HEAD_SIZE: 0xD (13 bytes)
+RESERVED1: 2 bytes: 0
+RESERVED2: 4 bytes: 0

Block: RAR File; offset: 0x1E9C0 (125376 bytes)
|Header bytes: [snip]
|HEAD_CRC: 0xBEA9
|HEAD_TYPE: 0x74 (RAR File)
|HEAD_FLAGS: 0x8082
| 0x8000 LONG_BLOCK (ADD_SIZE field present)
| 0x0002 LHD_SPLIT_AFTER (file continued in next volume)
| 0x0080 LHD_WINDOW (Dictionary size 1024 KiB)
|HEAD_SIZE: 0x47 (71 bytes)
+PACK_SIZE: 14851245 bytes (ADD_SIZE + HIGH_PACK_SIZE field)
+UNP_SIZE: 367486976 bytes
+HOST_OS: Windows used to create this file block.
+FILE_CRC: 485D639C
+FTIME: 2003-05-25 13:04:14
+UNP_VER: Version 2.0 is needed to extract.
+METHOD: Storing
+NAME_SIZE: always present
+ATTR: 20
+FILE_NAME: Star.Trek.DS9.S03E08.DVDRip.XviD-VF.avi

The new uploads were the original RAR files all along.

One hour difference

Sometimes there is a collision and the only difference we can see is the the archived file date is exactly 1 hour off. How to handle those? It can be very tricky, but here I'll construct a line of reasoning on how to handle those:

Example

The example where I'll build the theory on: Brothers.and.Sisters.S02E12.720p.HDTV.x264-CTU

Original: ctu-x264-brothers.and.sisters.212.rar 50,000,000 257A9488
brothers.and.sisters.s02e12.720p.hdtv.x264-ctu.mkv archived file date: 2008-02-18 12:00:00

Repack: brothers.and.sisters.s02e12.720p.hdtv.x264-ctu.rar 50,000,000 8C07B5A7
brothers.and.sisters.s02e12.720p.hdtv.x264-ctu.mkv archived file date: 2008-02-18 01:00:00

How do I know which one is the repack? CTU uses a certain pattern to name their archives. Also the comment on the SFV file of the repack gives it away:

; Generated by WIN-SFV32 v1.1a (pure-sfv v0.3) on 2008-08-21 at 03:45.00

Daylight saving time

http://en.wikipedia.org/wiki/Daylight_saving_time

GMT +2 Summer time: clocks are advanced by 1 hour
GMT +1 Winter time: an additional hour coming from summer time

From 2002 the rule has been:

  • the summer-time period shall begin, in every Member State, at 1.00 a.m., Greenwich Mean Time, on the last Sunday in March.
  • the summer-time period shall end, in every Member State, at 1.00 a.m., Greenwich Mean Time, on the last Sunday in October.

See https://greenwichmeantime.com/info/bst/

This seems to be the only explanation for the exact one hour difference.

Conclusion

For German releases, it seems safe to assume that:

  • Release pred in winter time? The earliest/oldest time stamp is original. (like our example above)
  • Release pred in summer time? The latest time stamp is original.

For other locations, this might behave differently. Seasons are switched in the southern hemisphere.
http://en.wikipedia.org/wiki/Daylight_saving_time_by_country

RedBlade DEADBABE SFV

redblade.png

RedBlade created RAR volumes that are 14,999,996 bytes large. They padded each volume with 4 bytes to match the predefined CRC32 value (DEADBABE). The process on how to do this is described on https://www.nayuki.io/page/forcing-a-files-crc-to-any-value
ReScene .NET ignores padded bytes at the end of each RAR volume. pyReScene shows the correct size because it keeps track of the padded bytes.

http://www.srrdb.com/release/details/Metal.Hurlant.Chronicles.S01E02.BDRip.XviD-RedBlade
http://www.srrdb.com/release/details/Metal.Hurlant.Chronicles.S01E04.BDRip.XviD-RedBlade

Large differences in NFO file sizes

This can be because all trailing spaces at the end of each line are stripped. We keep the largest (original) file of course!
e.g. http://www.srrdb.com/release/details/Bear_Necessities-Nut_E_1-MA05-1993-OSM

Original

nfo_original.png

Stripped spaces

nfo_stripped_spaces.png

Which file name is original?

XTC_iNT

We had two identical files:

00-various-jungle_hits_vol_3-strcd3-1995--xtc.jpg
00-various-jungle_hits_vol_3-strcd3-1995-xtc.jpg

[00:47] <X> jpg i guess the one which does not have the --
[00:47] <@Gfy> I would guess the other way around
[00:48] <@Gfy> people tend to "fix" things, not break them
[00:48] <@Gfy> that's why I think --

Later I found an example where we only have the -- variant
http://www.srrdb.com/release/details/Adam_Freeland-On_Tour-MAPACDA002-CD-2001-XTC_iNT
And most others were the same or totally different:
http://www.srrdb.com/browse/-cd-/XTC_iNT/1

SRS file name

Named after release name

Due to a script of an early .srr creator, his .srs files were named after the release name. e.g. Rls-GROUP.srs
The proper sample name was stored inside the SRS file.

Appended with -sample

The string -sample is appended to the .srs file name. The file name inside the .srs is correct.
These SRS files need to be replaced on a new upload.

appended-sample.png

Broken images

Images can show errors in various ways.

00-anushka-never_can_decide--(bwood0114dd)-web-cover-wws.-wws.jpg
https://www.srrdb.com/release/details/Anushka-Never_Can_Decide-(BWOOD0114DD)-WEB-2014-wWs

We don't know

The RAR files have different names, but it seems they are the exact same RARs. (same SFV CRCs and SRRs look the same) However, the RAR hashes don't match and the nfos are different too. Look at the dates. Based on these dates I guess that the one with the NFO closest to the RAR date is the original one.

Undercover.Boss.US.S05E14.HDTV.x264-2HD is an example of a bad SRR file. We don't know what caused this.

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License