[SOLVED] Python int too large to convert to C long

Problems while recreating compressed releases

Anonymous (guest) 22 Mar 2013 09:38

I can't seem to recreate compressed releases, I tried some from SKIDROW with no success at all then I tried one from RELOADED but srr gave an exception. I was using "srr-0.5.beta1.exe" on windows 7 x64.

************************************************************
WARNING: SRR files for compressed RARs are like SRS files:
         you can never be sure they will reconstruct!
************************************************************
SRR file created with pyReScene Auto 0.4.
Warning: File _ReScene\reloaded.nfo already exists.
Do you wish to continue? (Y/N): y
Recreating stored file: reloaded.nfo
Warning: File _ReScene\rld-an27.sfv already exists.
Do you wish to continue? (Y/N): y
Recreating stored file: rld-an27.sfv
Warning: File _ReScene\rld-an27.rar already exists.
Do you wish to continue? (Y/N): y
Re-creating RAR file: rld-an27.rar
Trying to rebuild compressed file.
Grabbing large enough data piece size for testing.
Trying 2012-06-09 4.20.
Trying 2006-08-05 3.60.
Good RAR version detected: 2006-08-05 3.60
['rar\\2006-08-05_rar360.exe', u'a', '-m1', '-mdA', u'-s-', u'-ds', u'-mt1', u'-
vn', u'-o+', u'-ep', u'tmp\\pyReScene_compressed.rar', u'N:\\srrdb.GAME-PC\\Anno
.2070-RELOADED\\rld-an27.iso']
Compressing rld-an27.iso...

RAR 3.60   Copyright (c) 1993-2006 Alexander Roshal   5 Aug 2006
Shareware version         Type RAR -? for help

Evaluation copy. Please register.

Creating archive tmp\pyReScene_compressed.rar

Adding    N:\srrdb.GAME-PC\Anno.2070-RELOADED\rld-an27.iso            OK
Done
Traceback (most recent call last):
  File "rescene\srr.pyc", line 418, in main
  File "rescene\srr.pyc", line 210, in manage_srr
  File "rescene\main.pyc", line 1133, in reconstruct
  File "rescene\main.pyc", line 1465, in get_rar_data_object
  File "rescene\main.pyc", line 1656, in compressed_rar_file_factory
  File "rescene\main.pyc", line 1766, in __init__
  File "rescene\rarstream.pyc", line 76, in __init__
  File "rescene\rarstream.pyc", line 45, in _check
  File "rescene\rar.pyc", line 1505, in read_all
  File "rescene\rar.pyc", line 1552, in next
  File "rescene\rar.pyc", line 1541, in __next__
  File "rescene\rar.pyc", line 1480, in _read
MemoryError
Unexpected Error:
M:\pyReScene-0.4-win>

Anonymous (guest) 22 Mar 2013 15:08

I have now downloaded Gfy-pyrescene-5257d47ccf28 and still get an error

************************************************************
WARNING: SRR files for compressed RARs are like SRS files:
         you can never be sure they will reconstruct!
************************************************************
SRR file created with pyReScene Usenet.
Recreating stored file: reloaded.nfo
Recreating stored file: rld-arg4.sfv
Re-creating RAR file: rld-arg4.rar
Trying to rebuild compressed file.
Grabbing large enough data piece size for testing.
Trying 2012-06-09 4.20.
Trying 2006-08-05 3.60.
Good RAR version detected: 2006-08-05 3.60
['rar\\2006-08-05_rar360.exe', u'a', '-m3', '-mdA', u'-s-', u'-ds', u'-mt2', u'-
vn', u'-o+', u'-ep', u'tmp\\pyReScene_compressed.rar', u'N:\\srrdb.GAME-PC\\Arca
nia.Gothic.4-RELOADED\\rld-arg4.iso']
Compressing rld-arg4.iso...

RAR 3.60   Copyright (c) 1993-2006 Alexander Roshal   5 Aug 2006
Shareware version         Type RAR -? for help

Evaluation copy. Please register.

Creating archive tmp\pyReScene_compressed.rar

Adding    N:\srrdb.GAME-PC\Arcania.Gothic.4-RELOADED\rld-arg4.iso     OK
Done
Traceback (most recent call last):
  File "M:\pyReScene-GIT\Gfy-pyrescene-5257d47ccf28\rescene\srr.py", line 417, i
n main
    parser.exit(manage_srr(options, infolder, infiles, working_dir))
  File "M:\pyReScene-GIT\Gfy-pyrescene-5257d47ccf28\rescene\srr.py", line 209, i
n manage_srr
    options.rar_executable_dir, options.temp_dir)
  File "M:\pyReScene-GIT\Gfy-pyrescene-5257d47ccf28\rescene\main.py", line 1133,
 in reconstruct
    in_folder, hints, auto_locate_renamed)
  File "M:\pyReScene-GIT\Gfy-pyrescene-5257d47ccf28\rescene\main.py", line 1465,
 in get_rar_data_object
    in_folder, hints, auto_locate_renamed))
  File "M:\pyReScene-GIT\Gfy-pyrescene-5257d47ccf28\rescene\main.py", line 1656,
 in compressed_rar_file_factory
    return CompressedRarFile(block, blocks, src)
  File "M:\pyReScene-GIT\Gfy-pyrescene-5257d47ccf28\rescene\main.py", line 1766,
 in __init__
    self.rarstream = RarStream(out, compressed=True)
  File "M:\pyReScene-GIT\Gfy-pyrescene-5257d47ccf28\rescene\rarstream.py", line
76, in __init__
    if not middle and not _check(first_rar):
  File "M:\pyReScene-GIT\Gfy-pyrescene-5257d47ccf28\rescene\rarstream.py", line
45, in _check
    blocks = rar_reader.read_all()
  File "M:\pyReScene-GIT\Gfy-pyrescene-5257d47ccf28\rescene\rar.py", line 1505,
in read_all
    for block in self:
  File "M:\pyReScene-GIT\Gfy-pyrescene-5257d47ccf28\rescene\rar.py", line 1552,
in next
    return self.__next__()
  File "M:\pyReScene-GIT\Gfy-pyrescene-5257d47ccf28\rescene\rar.py", line 1541,
in __next__
    block = self._read()
  File "M:\pyReScene-GIT\Gfy-pyrescene-5257d47ccf28\rescene\rar.py", line 1480,
in _read
    block_buffer += self._rarstream.read(add_size)
OverflowError: Python int too large to convert to C long
Unexpected Error: Python int too large to convert to C long

the memory usage goes up a few GB and then the script dies after compressing the data, the compressed data stays in the temp folder and extracts without error. My python is 2.7.3 64-bit.

I could successfully rebuild "Anno.2070.Deep.Ocean-RELOADED" but it's only 1.4GB, it seems the script can't handle bigger releases.

Gfy 24 Mar 2013 02:27

Thanks for the bug report. The script could indeed not handle files larger than 4GB. The code is fixed in the Mercurial repository. The srr-0.5.beta2.exe executable has the fixed code.

Last edited on 24 Mar 2013 14:46 by Gfy Show more

Anonymous (guest) 24 Mar 2013 19:18

Great :) now I have successfully recreated some releases bigger than 4GiB.

Tom (guest) 21 Nov 2013 06:16

hi i tried to recreate archives from srr (from srrdb.com) "COMMAND_AND_CONQUER_2_TIBERIAN_SUN-FLT" but no success, no good version of rar found. Do i something wrong? I tried also "COMMAND_AND_CONQUER_2_TIBERIAN_SUN_NT_FIX-FLT" and it was recreated successfully.

E:\tmp\ReScene.NET 1.2\pyReScene-0.5.1-win>srr -z C:\wrar COMMAND_AND_CONQUER_2_TIBERIAN_SUN-FLT.srr
NOTE: install NumPy to speed up the reconstruction of RARs with a Recovery Record.
SRR file created with pyReScene 0.5.
Recreating stored file: flt-ts.nfo
Recreating stored file: CD1/flt-ts1.sfv
Recreating stored file: CD2/flt-ts2.sfv
Re-creating RAR file: flt-ts1.rar
Trying to rebuild compressed file ts1.bin.
Grabbing large enough data piece size for testing.
Trying 1999-10-21 2.60.
Trying 1999-03-24 2.50.
Trying 1999-01-15 2.50.
Trying 1998-12-03 2.06.
Trying 1998-09-21 2.05.
Trying 1998-05-13 2.04.
Trying 1998-03-19 2.03.
Trying 1997-09-16 2.02.
Trying 1997-03-12 2.01.
Trying 1996-09-02 2.00.
Trying 2000-05-11 2.70.
Trying 2000-06-20 2.71.
Trying 2001-03-02 2.80.
Trying 2001-09-07 2.90.
Trying 2002-05-14 3.00.
Trying 2002-12-18 3.10.
Trying 2003-01-15 3.11.
Trying 2003-05-15 3.20.
Trying 2004-01-22 3.30.
Trying 2004-09-08 3.40.
Trying 2004-11-02 3.41.
Trying 2004-12-26 3.42.
Trying 2005-08-03 3.50.
Trying 2005-10-07 3.51.
Trying 2006-08-05 3.60.
Trying 2006-09-14 3.61.
Trying 2006-12-03 3.62.
Trying 2007-05-22 3.70.
Trying 2007-09-20 3.71.
Trying 2008-09-16 3.80.
Trying 2009-08-16 3.90.
Trying 2009-12-12 3.91.
Trying 2010-02-10 3.92.
Trying 2010-03-15 3.93.
Trying 2011-03-02 4.00.
Trying 2011-05-28 4.01.
Trying 2012-01-09 4.10.
Trying 2012-02-17 4.11.
Trying 2012-06-09 4.20.
Grabbing large enough data piece size for testing.
Trying 1999-10-21 2.60.
Trying 1999-03-24 2.50.
Trying 1999-01-15 2.50.
Trying 1998-12-03 2.06.
Trying 1998-09-21 2.05.
Trying 1998-05-13 2.04.
Trying 1998-03-19 2.03.
Trying 1997-09-16 2.02.
Trying 1997-03-12 2.01.
Trying 1996-09-02 2.00.
Trying 2000-05-11 2.70.
Trying 2000-06-20 2.71.
Trying 2001-03-02 2.80.
Trying 2001-09-07 2.90.
Trying 2002-05-14 3.00.
Trying 2002-12-18 3.10.
Trying 2003-01-15 3.11.
Trying 2003-05-15 3.20.
Trying 2004-01-22 3.30.
Trying 2004-09-08 3.40.
Trying 2004-11-02 3.41.
Trying 2004-12-26 3.42.
Trying 2005-08-03 3.50.
Trying 2005-10-07 3.51.
Trying 2006-08-05 3.60.
Trying 2006-09-14 3.61.
Trying 2006-12-03 3.62.
Trying 2007-05-22 3.70.
Trying 2007-09-20 3.71.
Trying 2008-09-16 3.80.
Trying 2009-08-16 3.90.
Trying 2009-12-12 3.91.
Trying 2010-02-10 3.92.
Trying 2010-03-15 3.93.
Trying 2011-03-02 4.00.
Trying 2011-05-28 4.01.
Trying 2012-01-09 4.10.
Trying 2012-02-17 4.11.
Trying 2012-06-09 4.20.
No good RAR version found.
No good RAR version found.

---

E:\tmp\ReScene.NET 1.2\pyReScene-0.5.1-win>srr -z C:\wrar COMMAND_AND_CONQUER_2_TIBERIAN_SUN_NT_FIX-FLT.srr
NOTE: install NumPy to speed up the reconstruction of RARs with a Recovery Record.
SRR file created with pyReScene 0.5.
Trying to rebuild compressed file game.exe.
Grabbing large enough data piece size for testing.
Trying 1999-10-21 2.60.
Trying 1999-03-24 2.50.
Trying 1999-01-15 2.50.
Trying 1998-12-03 2.06.
Good RAR version detected: 1998-12-03 2.06
C:\wrar\1998-12-03_rar206.exe a -m3 -mdC -s- -ds -vn -o+ -ep -idcd -vn c:\users\tom\appdata\local\temp\tmpiu1le8_pyReScene\pyReScene_compressed.rar E:\tmp\ReScene.NET 1.2\pyReScene-0.5.1-win\game.exe
Compressing game.exe…

RAR 2.06 Copyright (c) 1993-98 Eugene Roshal 3 December 1998
Shareware version Type RAR -? for help

Evaluation copy. Please register.

Creating archive c:\users\tom\appdata\local\temp\tmpiu1le8_pyReScene\pyReScene_compressed.rar

Adding game.exe Ok
Done

Gfy 22 Nov 2013 17:36

Hi Tom

That's an old release. It doesn't look like you did something wrong. It just didn't work. Nothing you can do anything about. Maybe I get it to work in the future, but that isn't certain. It's created with an old version of rar.exe and those aren't so complex yet.

I added a bug report: https://bitbucket.org/Gfy/pyrescene/issue/12/srr-for-old-rar-archives-fails-to

Last edited on 22 Nov 2013 17:41 by Gfy Show more