CRC mismatch in one of the recreated RARs (Dark.Souls.III.The.Ringed.City.Update.v1.14-CODEX)

CRC mismatch in one of the recreated RARs (Dark.Souls.III.The.Ringed.City.Update.v1.14-CODEX)

dark_red 13 May 2017 19:31

I want to recreate "Dark.Souls.III.The.Ringed.City.Update.v1.14-CODEX"
The .srr is from the db: https://www.srrdb.com/release/details/Dark.Souls.III.The.Ringed.City.Update.v1.14-CODEX

Initially I tried pyReScene-0.6.1-w32 in win10 x64
but I got "Unexpected Error: Still not fine :(." (see below)
I've checked issue #46, and based on a comment there,
I tried removing the executables 5.40, 5.31, 5.30… down to 4.20 from inside the '_test' folder (in which I have all RAR executables from v2.0 to v5.40), one at a time,
but without success:

+ show block

D:\2>C:\pyReScene-0.6.1-w32\srr Dark.Souls.III.The.Ringed.City.Update.v1.14-CODEX.srr -z C:\pyReScene-0.6.1-w32\_test -t temp

SRR file created with pyReScene Auto 0.6.
Warning: File D:\2\codex.nfo already exists.
Do you wish to continue? (Y/N): y
Re-creating stored file: codex.nfo
Warning: File D:\2\codex-dark.souls.iii.the.ringed.city.update.v1.14.sfv already exists.
Do you wish to continue? (Y/N): y
Re-creating stored file: codex-dark.souls.iii.the.ringed.city.update.v1.14.sfv
Warning: File D:\2\codex-dark.souls.iii.the.ringed.city.update.v1.14.rar already exists.
Do you wish to continue? (Y/N): y
Re-creating RAR file: codex-dark.souls.iii.the.ringed.city.update.v1.14.rar
Trying to rebuild compressed file CODEX\Game\DarkSoulsIII.cdx.
Grabbing large enough data piece size for testing.
Trying 2016-08-15 5.40.
Good RAR version detected: 2016-08-15 5.40
C:\pyReScene-0.6.1-w32\_test\2016-08-15_rar540.exe a -m3 -mdG -s- -ds -mt1 -vn -o+ -ep -idcd -v27231290b temp\SRR-fbzz4k57\pyReScene_compressed.rar D:\2\CODEX\Game\DarkSoulsIII.cdx
Compressing DarkSoulsIII.cdx...

Creating archive temp\SRR-fbzz4k57\pyReScene_compressed.rar

Adding    D:\2\CODEX\Game\DarkSoulsIII.cdx
Calculating the checksum

Creating archive temp\SRR-fbzz4k57\pyReScene_compressed.r00

...       DarkSoulsIII.cdx                                            OK
Calculating the checksum
37457942
37458108
Still not fine :(.
Traceback (most recent call last):
  File "E:\pyReScene\rescene\srr.py", line 566, in main
  File "E:\pyReScene\rescene\srr.py", line 307, in manage_srr
  File "E:\pyReScene\rescene\main.py", line 1301, in reconstruct
  File "E:\pyReScene\rescene\main.py", line 1690, in get_rar_data_object
  File "E:\pyReScene\rescene\main.py", line 1976, in compressed_rar_file_factory
  File "E:\pyReScene\rescene\main.py", line 2169, in __init__
ValueError: Still not fine :(.
Unexpected Error: Still not fine :(.

Last edited on 15 May 2017 19:53 by dark_red Show more

Re: CRC mismatch in one of the recreated RARs (Dark.Souls.III.The.Ringed.City.Update.v1.14-CODEX)

dark_red 14 May 2017 00:26

Update:

Based on the sticky and this thread
I downloaded the latest version of pyReScene via https://bitbucket.org/Gfy/pyrescene/get/default.zip ,
installed latest Python 3.61 x64
and then tried:

c:\Gfy-pyrescene-cb6f9b1b2d26\bin\windows\pysrr Dark.Souls.III.The.Ringed.City.Update.v1.14-CODEX.srr -z C:\pyReScene-0.6.1-w32\_test -t temp --mt-set 8

i.e. with "—mt-set 8" it completed ok (without any errors, as seen below) -all other values would either lead to crash, or it wouldn't find a matching RAR executable).-
but, unfortunately, when I checked the recreated RAR files against the .sfv file,
the "codex-dark.souls.iii.the.ringed.city.update.v1.14.r03" file has wrong CRC (all others are ok).

+ show block

D:\2>c:\Gfy-pyrescene-cb6f9b1b2d26\bin\windows\pysrr Dark.Souls.III.The.Ringed.City.Update.v1.14-CODEX.srr -z C:\pyReScene-0.6.1-w32\_test -t temp --mt-set 8
SRR file created with pyReScene Auto 0.6.
Warning: File D:\2\codex.nfo already exists.
Do you wish to continue? (Y/N): y
Re-creating stored file: codex.nfo
Warning: File D:\2\codex-dark.souls.iii.the.ringed.city.update.v1.14.sfv already exists.
Do you wish to continue? (Y/N): y
Re-creating stored file: codex-dark.souls.iii.the.ringed.city.update.v1.14.sfv
Warning: File D:\2\codex-dark.souls.iii.the.ringed.city.update.v1.14.rar already exists.
Do you wish to continue? (Y/N): y
Re-creating RAR file: codex-dark.souls.iii.the.ringed.city.update.v1.14.rar
Trying to rebuild compressed file CODEX\Game\DarkSoulsIII.cdx.
Grabbing large enough data piece size for testing.
Trying 2016-08-15 5.40.
Good RAR version detected: 2016-08-15 5.40
C:\pyReScene-0.6.1-w32\_test\2016-08-15_rar540.exe a -m3 -mdG -s- -ds -mt8 -vn -o+ -ep -idcd -v27231290b temp\SRR-qx87mmpr\pyReScene_compressed.rar D:\2\CODEX\Game\DarkSoulsIII.cdx
Compressing DarkSoulsIII.cdx...

Creating archive temp\SRR-qx87mmpr\pyReScene_compressed.rar

Adding    D:\2\CODEX\Game\DarkSoulsIII.cdx
Calculating the checksum

Creating archive temp\SRR-qx87mmpr\pyReScene_compressed.r00

...       DarkSoulsIII.cdx                                            OK
Calculating the checksum
Re-creating RAR file: codex-dark.souls.iii.the.ringed.city.update.v1.14.r00
Re-creating RAR file: codex-dark.souls.iii.the.ringed.city.update.v1.14.r01
Trying to rebuild compressed file CODEX\Game\DarkSoulsIII.exe.
Grabbing large enough data piece size for testing.
Trying 2016-08-15 5.40.
Good RAR version detected: 2016-08-15 5.40
C:\pyReScene-0.6.1-w32\_test\2016-08-15_rar540.exe a -m3 -mdG -s- -ds -mt8 -vn -o+ -ep -idcd -v14942431b temp\SRR-dr3ivek2\pyReScene_compressed.rar D:\2\CODEX\Game\DarkSoulsIII.exe
Compressing DarkSoulsIII.exe...

Creating archive temp\SRR-dr3ivek2\pyReScene_compressed.rar

Adding    D:\2\CODEX\Game\DarkSoulsIII.exe
Calculating the checksum

Creating archive temp\SRR-dr3ivek2\pyReScene_compressed.r00

...       DarkSoulsIII.exe
Calculating the checksum

Creating archive temp\SRR-dr3ivek2\pyReScene_compressed.r01

...       DarkSoulsIII.exe                                            OK
Calculating the checksum
Re-creating RAR file: codex-dark.souls.iii.the.ringed.city.update.v1.14.r02
Re-creating RAR file: codex-dark.souls.iii.the.ringed.city.update.v1.14.r03
Trying to rebuild compressed file CODEX\Game\steam_api64.cdx.
Grabbing large enough data piece size for testing.
Trying 2016-08-15 5.40.
Good RAR version detected: 2016-08-15 5.40
C:\pyReScene-0.6.1-w32\_test\2016-08-15_rar540.exe a -m3 -mdG -s- -ds -mt8 -vn -o+ -ep -idcd temp\SRR-rp4sp21c\pyReScene_compressed.rar D:\2\CODEX\Game\steam_api64.cdx
Compressing steam_api64.cdx...

Creating archive temp\SRR-rp4sp21c\pyReScene_compressed.rar

Adding    D:\2\CODEX\Game\steam_api64.cdx                             OK
Trying to rebuild compressed file CODEX\Game\steam_api64.dll.
Grabbing large enough data piece size for testing.
Trying 2016-08-15 5.40.
Good RAR version detected: 2016-08-15 5.40
C:\pyReScene-0.6.1-w32\_test\2016-08-15_rar540.exe a -m3 -mdG -s- -ds -mt8 -vn -o+ -ep -idcd temp\SRR-3a090rh7\pyReScene_compressed.rar D:\2\CODEX\Game\steam_api64.dll
Compressing steam_api64.dll...

Creating archive temp\SRR-3a090rh7\pyReScene_compressed.rar

Adding    D:\2\CODEX\Game\steam_api64.dll                             OK
Re-creating RAR file: codex-dark.souls.iii.the.ringed.city.update.v1.14.r04
Trying to rebuild compressed file CODEX\Game\steam_emu.ini.
Grabbing large enough data piece size for testing.
Trying 2016-08-15 5.40.
Good RAR version detected: 2016-08-15 5.40
C:\pyReScene-0.6.1-w32\_test\2016-08-15_rar540.exe a -m3 -mdG -s- -ds -mt8 -vn -o+ -ep -idcd temp\SRR-2l_9ug1l\pyReScene_compressed.rar D:\2\CODEX\Game\steam_emu.ini
Compressing steam_emu.ini...

Creating archive temp\SRR-2l_9ug1l\pyReScene_compressed.rar

Adding    D:\2\CODEX\Game\steam_emu.ini                               OK
Trying to rebuild compressed file Update\patchnotes.txt.
Grabbing large enough data piece size for testing.
Trying 2016-08-15 5.40.
Good RAR version detected: 2016-08-15 5.40
C:\pyReScene-0.6.1-w32\_test\2016-08-15_rar540.exe a -m3 -mdG -s- -ds -mt8 -vn -o+ -ep -idcd temp\SRR-2dw_6qyo\pyReScene_compressed.rar D:\2\Update\patchnotes.txt
Compressing patchnotes.txt...

Creating archive temp\SRR-2dw_6qyo\pyReScene_compressed.rar

Adding    D:\2\Update\patchnotes.txt                                  OK
Trying to rebuild compressed file Update\setup-1.bin.
Grabbing large enough data piece size for testing.
Trying 2016-08-15 5.40.
Good RAR version detected: 2016-08-15 5.40
C:\pyReScene-0.6.1-w32\_test\2016-08-15_rar540.exe a -m3 -mdG -s- -ds -mt8 -vn -o+ -ep -idcd -v13056850b temp\SRR-do0hs4x9\pyReScene_compressed.rar D:\2\Update\setup-1.bin
Compressing setup-1.bin...

Creating archive temp\SRR-do0hs4x9\pyReScene_compressed.rar

Adding    D:\2\Update\setup-1.bin
Calculating the checksum

Creating archive temp\SRR-do0hs4x9\pyReScene_compressed.r00

...       setup-1.bin
Calculating the checksum

Creating archive temp\SRR-do0hs4x9\pyReScene_compressed.r01

...       setup-1.bin
Calculating the checksum

Creating archive temp\SRR-do0hs4x9\pyReScene_compressed.r02

...       setup-1.bin
Calculating the checksum

Creating archive temp\SRR-do0hs4x9\pyReScene_compressed.r03

...       setup-1.bin                                                 OK
Calculating the checksum
Re-creating RAR file: codex-dark.souls.iii.the.ringed.city.update.v1.14.r05
Re-creating RAR file: codex-dark.souls.iii.the.ringed.city.update.v1.14.r06
Re-creating RAR file: codex-dark.souls.iii.the.ringed.city.update.v1.14.r07
Re-creating RAR file: codex-dark.souls.iii.the.ringed.city.update.v1.14.r08
Trying to rebuild compressed file Update\Setup.exe.
Grabbing large enough data piece size for testing.
Trying 2016-08-15 5.40.
Good RAR version detected: 2016-08-15 5.40
C:\pyReScene-0.6.1-w32\_test\2016-08-15_rar540.exe a -m3 -mdG -s- -ds -mt8 -vn -o+ -ep -idcd temp\SRR-nv_4cb4d\pyReScene_compressed.rar D:\2\Update\Setup.exe
Compressing Setup.exe...

Creating archive temp\SRR-nv_4cb4d\pyReScene_compressed.rar

Adding    D:\2\Update\Setup.exe                                       OK

D:\2>

1. Could one please check this? Is it a pyReScene issue or the .srr file is wrong?
2. The .srr file page (https://www.srrdb.com/release/details/Dark.Souls.III.The.Ringed.City.Update.v1.14-CODEX)
shows "This file is unconfirmed" (with an "X" button to its right ).
I have registered to srrdb.com.
Which members can "confirm" an .srr file?
How is it done? By clicking the "X" button?

PS. For reference, my initial decompressed files all match with the CRC hashes posted in https://www.srrdb.com/release/details/Dark.Souls.III.The.Ringed.City.Update.v1.14-CODEX

Last edited on 16 May 2017 22:45 by dark_red Show more

Re: CRC mismatch in one of the recreated RARs (Dark.Souls.III.The.Ringed.City.Update.v1.14-CODEX)

Gfy 16 May 2017 20:32

1. Issues with compressed files is always a pyReScene issue.
2. Only moderators can confirm .srr files. It's done in batch mostly by checking the release names, uploader,…

Re: CRC mismatch in one of the recreated RARs (Dark.Souls.III.The.Ringed.City.Update.v1.14-CODEX)

dark_red 16 May 2017 20:55

Thank you for replying.

Someone (guest) 16 May 2017 20:59

try larger mt settings (e.g. 10, 12 or 16), sometimes you get some bad parts if the mt setting is not correct
make sure you only have one rar version in your rar directory
also trying an older rar version this may help sometimes although I didn't see this with rar5xx yet

dark_red 16 May 2017 22:38

(thanks for trying to help and for confirming)
I've posted above that only with "—mt-set 8" it completed.
This must be because my CPU is an i7, 4 cores x 2 (due to hyperthreading).

With 1, 2, 4, 16 and 32 it would crash with "Unexpected Error: Still not fine :(.".
With 6, 10, 12, 24 it wouldn't find a matching RAR executable.

Also, using 5x RAR executables older than 5.40, didn't help.

Last edited on 16 May 2017 22:57 by dark_red Show more

Someone (guest) 17 May 2017 22:17

I don't think your CPU matters here, the mt option tells rar to use N threads which it should regardless of actual cpu cores.
Even a single core cpu can run multiple threads it's just slower but the output should be the same.

Anyway if your bad r03 file is wrong in the same place as the one I got you can just use a hex editor and change the affected bytes as mentioned in my other post.

dark_red 17 May 2017 22:34

I don't think your CPU matters here, the mt option tells rar to use N threads which it should regardless of actual cpu cores.
Even a single core cpu can run multiple threads it's just slower but the output should be the same.

I quote Gfy from this link I posted above:

-mt is a rar.exe switch to force the amount of threads it must use. By default it uses the amount of cores * 2. (due to hyperthreading)
Normally it starts with 1 and goes up automatically, but certain values can provide false positives: while correct early on, it'll fail later.

And about this:

Anyway if your bad r03 file is wrong in the same place as the one I got you can just use a hex editor and change the affected bytes as mentioned in my other post.

Yes it's wrong in the same 5 bytes as yours (I've eventually downloaded the scene release RAR files as well). And, I know I could fix them with a hex editor (thanks anyway).

Last edited on 17 May 2017 22:45 by dark_red Show more

Someone (guest) 16 May 2017 21:25

Just tried it and couldn't rebuild it, I also get a bad r03 and it only differs in five bytes according to fc

fc /B good_DB2FC440.r03 bad_385555B4.r03
Vergleichen der Dateien good_DB2FC440.r03 und BAD_385555B4.R03
00E47A22: 9C 9D
00E47A23: 3E B0
00E47A24: FD 9E
00E47A25: EC ED
00E47A26: 4F 1F