pdSFV is a thorough tool for working with SFV files. If you never have seen files ending in “.sfv” before, this software package probably is completely useless for you. Better luck next time… :)
Basically there are two operating modes in pdSFV: creating SFV files and performing SFV checks. Both modes try to cover a set of features not found in the few other existing SFV tools. While the Windows GUI version tries to maximize the end-user comfort regarding interface friendliness and output information, all of the currently available versions (Windows GUI, Windows Textmode, Windows FTP-Server Plugin, Unix / Linux Textmode, Linux glFTPd Plugin) are optimized for performance at a rational level of resource usage.
- Added option to include files’ dates and sizes as comments to the SFV file (create mode)
- Added option to associate .sfv file extension with pdSFV (Windows GUI version)
- Redesign of the graphical user interface, getting rid of some redundant graphics and enhancing output information (Windows GUI version)
- pdSFV can now be “minimized to tray” (Windows GUI version)
- Added Windows Textmode version
- Added sfv-on-the-fly checking to glFTPd (included in glFTPd 1.18 and higher)
- Added wildcard support to command line utilities
- Added example for recursive / multiple SFV checking to documentation
- Fixed direct MP3 support in UNIX version
- Fixed format of date/time stamp in “Generated by” – comment line to match established standard
- Changed working path setting in Windows FTP server plugin, should run with gay FTP server programs now as well
Of course pdSFV is still distributed including full source code for all versions to encourage interested programmers to enhance it and contribute to this project.
SFV files are used in mainly two areas nowadays: distribution of ISO/VCD releases and distribution of MP3 files via FTP servers by groups tending to refer to themselves as “scene”. While using file formats like ZIP to spread stuff can rely on the good speed of the ZIP algorithm to verify small files’ integrity, neither the format of ISO/VCD releases – RAR archive chunks usually 15 MB in size – nor MP3 files have an built-in, high-performance option to check whether the file uploaded to a FTP site or downloaded to your home is identical with the file on the other side or even the original files made by the packager.
Because of this lack of possibilities SFV files were invented. A SFV file basically is a list of files which belong to a “release” including a checksum for each of these files. Using this checksum it is easy to verify that the files received at location B are identical to those the sender at location A has. The simple algorithm used in SFV files makes sure that two files never (with “never” meaning that it is pretty unprobable) have the same checksum if they are not identical.
In the early days of SFV files there was only one tool used by almost everyone, simply called “SFV” with the Windows version called “WIN-SFV” and the Unix pedant “SFV*NIX”. Everyone was happy with those until SFV files became more and more popular and – especially MP3 groups – started using their own tools to create SFV files. Some MP3 groups even had tools ripping an audio CD, converting it to MP3s, creating a SFV file for them and uploading it to a FTP site fully automated.
Problems emerged when the mass of end-users had to find out that those SFV files could not be processed by WIN-SFV and SFV*NIX. This was because those two SFV file utils added their name as comment into the SFV files they created. The tricky point is that they refused to process any other SFV files, i.e. SFV files not created by WIN-SFV or SFV*NIX. Thus, unless the other SFV creating utilities pretended to be WIN-SFV, the end user had a hard time performing SFV checks.
About the same time some ISO/VCD topsites started adding banners and logos of their own to SFV files – as comments (see the discussion of the SFV file format later). Other topsites’ countermeasures “of course” were to remove those unwanted banners from the SFV files again – mostly done by simply removing all comments from uploaded SFV files. During that process the information that a SFV file was created by WIN-SFV or SFV*NIX got lost. This again led to the problem that end-users had problems with WIN-SFV and SFV*NIX because those tools again refused to process these SFV files.
After some heated discussions the author of WIN-SFV and SFV*NIX was contacted and asked to remove this limitation from his programs and said he would do so – but never did.
For two months the situation remained unsolved – then pdSFV 1.0 was released. Intended as a quick & dirty hack pdSFV’s primary target was to provide end-users with a tool flexible enough for all existing SFV file formats, even those which cannot be processed with WIN-SFV. Because several skilled programmers pointed out their interest in helping to contribute to a flexible SFV utility and their only problem was lacking knowledge of the used CRC/32 checksum algorithm, pdSFV was distributed with its full source to encourage the others to work on the details.
Meanwhile, about a year later, we've decided to release pdSFV v1.2 to the public, having implemented several often-heard user requests and hoping that v1.2 will be as useful to you as v1.0 was.
SFV files are plain ASCII textfiles. Basically there are only two types of lines: comment lines and lines containing a filename with its CRC32 checksum.
Comment lines begin with a semicolor: “;” . For example:
; This is a comment line
Usually comments are seperated from the leading “;” by a single blank character (ascii code 32 (=20hex)), this, however, is a design issue and not a technical necessity.
The very first line of a SFV file usually is a special comment line: the “generated by” message. It basically looks like this:
; Generated by pdSFV v1.2 (WIN) on 2000-03-19 at 15:30.07
The date format is YYYY-MM-DD and the time format is HH:MM.SS . You will figure out that current versions of WIN-SFV and SFV*NIX cannot process SFV files which don’t have
; Generated by WIN-SFV v1
(or the unix version string) as beginning of their “generated by” – line. Thus, what pdSFV is doing in WIN-SFV compatibility is simply pretending that it is WIN-SFV by faking this line.
If files’ date and time stamps are added to the SFV file, there is an established format as well:
; 15000000 10:09.40 2000-03-14 dummy1.r00 1234567890123456789012345678901234567 1 2 3
Especially notice the double-space between file size and time field. Time and date formats are as described above. In other words: file size ends at column 14, time starts at 17, date at 26, filename at 37.
The non-comment lines contain a filename and – seperated by a single blank – the (full 8 digits, with leading zeroes if necessary) hexadecimal value of the file’s CRC32. For example:
If you are interested in how the CRC32 checksum is calculated, you should have a look at pdSFV’s source code. Also feel free to use this (as well as any other) part of pdSFV’s source for projects of your own, that’s what it is made for.
User comments like banner files, group logos etc. are added as regular comments. Usually those banners are preceeded by a “;!SFV_COMMENT_START” line and terminated by a “;!SFV_COMMENT_END” line, however not everyone is doing so yet.
Just mentioned for completeness: a SFV file doesn’t have to contain any comment lines. A simple list of files and their CRCs will do as well. (see next section if you want to go into the details that have caused some trouble on “scene” sites because of that)
Joe Forster/STA,,Hungary,gro.46c|ats#gro.46c|ats,"I'm the author of the site sta.c64.org that you link to on your pdSFV homepage. :-) In the last few years, I've been developing GNU-style programs (like sed, tr, cat, xargs), all of them yet to be published with full source, and got excited about the C source of the Linux version of pdSFV. During this week I've been working on a new release. Modifications so far:
- modified scanning of SFV lines to handle file names with spaces inside
- removed opening file with names in different case in DOS/Windows
- added ""-q"" option for Unix-style quiet mode
- changed messages to those in the Windows/Delphi command line utilities
- replaced end-of-lines in printed strings (in Windows, LF becomes CR+LF)
- changed recognition of archive file extensions to more precise
- simplified check of whether file exists and added skipping directories
- restructured code to collect common functionality into subroutines
- replaced static CRC32 table with a generated (once) one
- replaced file read buffer allocation from implicit stack to explicit heap
- added some #ifdef's for compiling with gcc/MinGW under Windows
- got rid of compilation warnings
- removed ""Oh mama!"" messages :-)
- beautified the source with Uncrustify for consistent style
Some more serious, real life testing, by several people would be nice because me testing it on a few files is not quite enough. (I've restructured the code pretty much so bugs may arise!).
If you're interested, I've uploaded it to http://sta.c64.org/shared/pdsfv.zip (binary, original source, modified source, Uncrustify config file)."
I added a Makefile, too, and put my E-mail address and the download address into the source. It pretty much looks like a release now although, I have to add again, it needs testing. If and when it becomes mature enough for a proper release, this test download will be removed and I will probably add a small homepage, dedicated to this utility, among my pages. Bye,