TargetD64 Changes for V0.2.0

List of Changes

TargetD64 is now "open source" under the terms of the GNU general public license (GPL) Version 2.
All supported archive formats are builtins. No more fiddling with external helpers. No more problems with DOS applications (short filenames). Fallback strategy implemented (-x option).
The Win32 version now can handle wildcards (* and ?), too.
There are configurations for CCS64 2.0 beta and C64S V2.52 available.
New implemented archive formats x64, p00.
New implemented generic archive interface. You can now hook in your favorite archive into TargetD64´s processing with the help of environment variables TD64_GENERIC_EXTn and TD64_GENERIC_CMDn.
TargetD64 is now completely configurable by environment variables. TD64_DEFAULT_OPTIONS can set always needed command line arguments.
The testsuite has been extended and is fully automated for Linux by automatic result verification.
The order of produced D64 images now always reflects the order of the input files (before it depended on the archive class, e.g. [1-4]! were always treated at last).
Error handling is now more fault tolerant (before TargetD64 has been terminated in case of any error now only the erroneous file is skipped).
Improved manual (e.g. added FAQ).
New placeholders for configuring emulator command (basename of first diskimage, temporary directory). Directory of image repository does not have trailing (back)slash anymore.
Shortcut options like "-nd" for "-n -d" are now possible for the command line.
For Win32 additionally to TMPDIR the environment variables TMP and TEMP are evaluated for setting the temporary directory.
Added option -t tracefile to dump the trace output into tracefile. I have included this option, because -d dumps the trace output into stderr and some command.com versions are not able to redirect stderr (would mean 2>tracefile for linux).

Resolved Requests from the Error Tracker

RQID________: 1
PRODUCT_____: TargetD64
COMPONENT___: Executable
VERSION_____: V0.1
FIXED_VERS__: V0.2
OS__________: Win95
ORIGINATOR__: Per Bolmstedt
TYPE________: Bug report
STATE_______: Closed
SEVERITY____: Low
DATE________: 12.02.99
SUBJECT_____: Assertion fails in line 997 of Archive.cpp
DETAILS_____:
This assertion shows up after sample.lnx.gz has been passed to TargetD64
and external helpers are not installed at all or in a wrong version (command
line syntax differs).
ANALYSIS____:
Assertions are there to ensure the integrity of TargetD64 internal states. They are unexpected and should not occur. If they do there has been an internal processing error. This special assertion indicates that there is no emulator image to be passed to an emulator.

System calls using the system() function from <stdlib.h> return a return code of 0 even if the called executable does not exist. TargetD64 assumes successfull processing. This only happens for Win95, not for WinNT (of course not for Linux).

External helpers like starlynx.exe and c1541 often do not indicate processing errors by exit codes. This makes it even difficult to check success for the
other OSs.
CONCLUSION__:
- Explain what assertions are in the manual.
- There has to be another checking for success. At least one file should
be exctracted for each archive processed.
- Change this special assertion to an exception with error text.
REMINDER____:
Get rid of those external helpers sooner or later.

RQID________: 2
PRODUCT_____: TargetD64
COMPONENT___: Executable
VERSION_____: V0.1
FIXED_VERS__: V0.2
OS__________: Win95/NT
ORIGINATOR__: Karlheinz Langguth
TYPE________: Bug report
STATE_______: Closed
SEVERITY____: Medium
DATE________: 14.02.99
SUBJECT_____: Wildcard * only delivers first match for Win32
DETAILS_____:
When using the wildcard * (e.g. *.lnx) only the first file matching the wildcard is processed by TargetD64. It is expected to process all files matching the wildcard.
ANALYSIS____:
For Linux wildcards are replaced by the shell before calling the executable. Nothing like that for Win32.
CONCLUSION__:
Search has to be implemented because nothing can top a "targetd64 -k *" on Linux :-).

RQID________: 3
PRODUCT_____: TargetD64
COMPONENT___: Manual
VERSION_____: V0.1
FIXED_VERS__: V0.2
OS__________: Win95/NT
ORIGINATOR__: Karlheinz Langguth
TYPE________: Change request
STATE_______: Closed
SEVERITY____: Low
DATE________: 14.02.99
SUBJECT_____: Batchfiles to configure TargetD64 are problematic
DETAILS_____:
When using a batchfile to configure TargetD64 the command line arguments have to be passed to TargetD64 via something like "targetd64 %1 %2 %3 ...". This will limit the number of processed files to the number of given parameter placeholders in the batchfile.
ANALYSIS____:
AFAIK nothing like a $* for Win32. Drop the batchfile. Use the properties of the TargetD64 link instead (add the command line options there). To set env variables use startup batchfiles.
CONCLUSION__:
Explain process in detail in the manual. Introduce environment variable to set the default options for TargetD64 (TD64_DEFAULT_OPTIONS).

RQID________: 4
PRODUCT_____: TargetD64
COMPONENT___: Executable
VERSION_____: V0.1
FIXED_VERS__: V0.2
OS__________: All
ORIGINATOR__: Karlheinz Langguth
TYPE________: Change request
STATE_______: Closed
SEVERITY____: Low
DATE________: 14.02.99
SUBJECT_____: Allow concatenated options
DETAILS_____:
Enumerated single options should be equivalent to concatenated options. E.g. "-n -d" should be the same than "-nd" as most Linux executables behave.
ANALYSIS____:
Nothing like getopts() for Win32.
CONCLUSION__:
Implement standard Linux behaviour.

RQID________: 6
PRODUCT_____: TargetD64
COMPONENT___: Executable
VERSION_____: V0.1
FIXED_VERS__: V0.2
OS__________: Win95/NT
ORIGINATOR__: Per Bolmstedt
TYPE________: Change request
STATE_______: Closed
SEVERITY____: Low
DATE________: 17.02.99
SUBJECT_____: Tmp directory is determined by variable TEMP or TMP
DETAILS_____:
TMPDIR is not commonly used for setting a directory for temporary files.
ANALYSIS____:
Even if TMPDIR is the best choice for Linux, TEMP/TMP should be (additionally) used for Win32. To guarantee compatibility with TargetD64 V0.1 go like that (pseudo code):
IF (TMPDIR contains valid directory) THEN
   temporary directory found
ELSE
   IF (TMP contains valid directory) THEN
      temporary directory found
   ELSE
      IF (TEMP contains valid directory) THEN
         temporary directory found
      ELSE
         set temporary directory to c:\TEMP
      ENDIF
   ENDIF
ENDIF
CONCLUSION__:
Implement it.

RQID________: 7
PRODUCT_____: TargetD64
COMPONENT___: Executable
VERSION_____: V0.1
FIXED_VERS__: V0.2
OS__________: Win95/NT
ORIGINATOR__: Per Bolmstedt
TYPE________: Problem report
STATE_______: Closed
SEVERITY____: Medium
DATE________: 18.02.99
SUBJECT_____: TargetD64 does not work with Starlynx V0.8
DETAILS_____:
Processing of lynx files will fail when using Starlynx V0.8 instead of Starlynx V0.81
ANALYSIS____:
Starlynx V0.8 is missing the /Y option which confirms all interactive questions of Starlynx (e.g. about overwriting an existing file). This option has been introduced with Starlynx V0.81.
CONCLUSION__:
Irrelevant, as TargetD64 does not anymore use external helpers for extraction.

RQID________: 8
PRODUCT_____: TargetD64
COMPONENT___: Executable
VERSION_____: V0.1
FIXED_VERS__: V0.2
OS__________: All
ORIGINATOR__: Per Bolmstedt
TYPE________: Remain request
STATE_______: Closed
SEVERITY____: Medium
DATE________: 18.02.99
SUBJECT_____: Building in external helpers will make TargetD64 bloatware.
DETAILS_____:
TargetD64 relying on external helpers is a modular concept and can be considered as "component ware". Do not build in features of external helpers as it will bloat TargetD64 and make it less configurable.
ANALYSIS____:
External helpers are the reason for many problems related to TargetD64. To name a few:
- Insufficient means of diagnosing the success of processing
- Performance decrease (spawn of a process)
- Version problems (e.g. it has to be starlynx V0.81 or beyond)
On the other hand they are easy to replace (with the help of TargetD64 placeholder substitution in command strings), keep TargetD64 lean and less erroneous, decrease development efforts.
CONCLUSION__:
Get rid of the external helpers by default. Implement Option -x as a fallback mode where still external helpers are used. This should only be applied in case TargetD64 runs into errors while extracting (non corrupted) archives.

RQID________: 9
PRODUCT_____: TargetD64
COMPONENT___: Executable
VERSION_____: V0.1
FIXED_VERS__: V0.2
OS__________: Linux
ORIGINATOR__: Andreas Boose
TYPE________: Bug report
STATE_______: Closed
SEVERITY____: Medium
DATE________: 19.02.99
SUBJECT_____: Exception 5 "Stat on file <param1> failed" occurs with kernel 2.1.132
DETAILS_____:
"targetd64 sample.zip" launches the emulator. After exiting the emulator TD64 throws the above stated exception and terminates without cleaning up.
ANALYSIS____:
When cleaning up the temporary directory tree of TD64 is removed from leaves to root. Each entry within a directory is removed. Directories are removed by rmdir(). "." and ".." are found also as entries. ".." can not yet be removed because it is not empty. Sometimes empty directories are created while processing. These could be potentially removed by rmdir("<actdir>/."). It seems that a 2.0.x kernel denies removal of a directory indicated by the absolute dirname followd by "/." with errno = EPERM. A 2.1.132 kernel removes the directory right away. Well may be the reason is somewhere else located than in the kernel but at least on both machines a ext2 filesystem is mounted in which processing happens.
CONCLUSION__:
Exclude "." and ".." from the cleanup procedure. Despite of unsecure reason of error it will be gone in any case.

RQID________: 10
PRODUCT_____: TargetD64
COMPONENT___: Executable
VERSION_____: V0.1
FIXED_VERS__: V0.2
OS__________: All
ORIGINATOR__: Karlheinz Langguth
TYPE________: Change request
STATE_______: Closed
SEVERITY____: Medium
DATE________: 21.02.99
SUBJECT_____: TargetD64 terminates when directory is matched by wildcard
DETAILS_____:
If a directory is passed to TargetD64 all processing is terminated. That means when using wildcards (e.g. *) and it matches a directory (among other files) none of the matched files is processed.
ANALYSIS____:
Each argument passed is checked for type. A directory makes TargetD64 cleaning up and terminating.
CONCLUSION__:
Be error tolerant concerning directories. If at least one regular file is passed to TargetD64 in the same call just ommit the directory also passed.

RQID________: 11
PRODUCT_____: TargetD64
COMPONENT___: Executable
VERSION_____: V0.1
FIXED_VERS__: V0.2
OS__________: Linux
ORIGINATOR__: Karlheinz Langguth
TYPE________: Bug report
STATE_______: Closed
SEVERITY____: Medium
DATE________: 21.02.99
SUBJECT_____: use of TD64_C64ZIP_CMD and TD64_LNX_CMD has restrictions
DETAILS_____:
Any other external helper for [1-4]! than c1541 will not work. Any external helper for .lnx extracting directly raw c64 files into the filesystem will not work.
ANALYSIS_____:
TargetD64 expects the external helper for [1-4]! (TD64_C64ZIP_CMD) to produce always a X64 image (as it was at first designed to work with c1541). Mind that this is only the case for Linux. Any other produced file(s) will not be considered for processing.

TargetD64 expects the external helper for .lnx (TD64_LNX_CMD) to produce resolvable archives (e.g. D64). So if the c64 files are directly extracted into the host filesystem they will be ignored.
CONCLUSION___:
Support X64 images. That means if an external helper produces a X64 image (which will be a rare case), it will be processed "normally". Delete the special treatment built in for [1-4]! files processed by c1541.
After processing [1-4]! and .lnx files also execute the "afterburner" that is used for host archives like .zip. This will ensure that produced final (unresolvable) archives will be processed as well (be put into a D64 image). Fix it even it will be of minor priority, because external helpers will not be used abymore.

RQID________: 12
PRODUCT_____: TargetD64
COMPONENT___: Executable
VERSION_____: V0.1
FIXED_VERS__: V0.2
OS__________: All
ORIGINATOR__: Stefan Langguth
TYPE________: Bug report
STATE_______: Closed
SEVERITY____: Medium
DATE________: 23.02.99
SUBJECT_____: filenames originating from T64 images are padded with spaces
DETAILS_____:
When passing a T64 image to TargetD64 containing unresolvable (C64 raw) files those are put into one or more D64 images. The resulting filenames are unconditionally padded with blanks until maximum length (16 characters).
ANALYSIS____:
The length of filename is not stored within the T64 file record. In contrast to D64 images the filename is not padded with SHIFT_SPACE (0xa0) but with SPACE (0x20). There is no way to differ between files that have really 0x20 characters at the end of the filename and padded filenames.
CONCLUSION__:
As files with real filenames ending with 0x20 characters are rare remove trailing 0x20.

RQID________: 13
PRODUCT_____: TargetD64
COMPONENT___: Testsuite
VERSION_____: V0.1
FIXED_VERS__: V0.2
OS__________: All
ORIGINATOR__: Karlheinz Langguth
TYPE________: Change request
STATE_______: Closed
SEVERITY____: High
DATE________: 23.02.99
SUBJECT_____: assert quality of future releases by automated result verify
DETAILS_____:
There are major changes to come. To ensure the quality of future releases the result verification has to be automated. There is no way to check the results manually as the testsuite will expand also.
CONCLUSION__:
Implement it for Linux only (tools are there). Win32 filesystem is mounted and can be checked by Linux, too.

RQID________: 14
PRODUCT_____: TargetD64
COMPONENT___: Executable
VERSION_____: V0.1
FIXED_VERS__: V0.2
OS__________: All
ORIGINATOR__: Karlheinz Langguth
TYPE________: Change request
STATE_______: Closed
SEVERITY____: Low
DATE________: 06.03.99
SUBJECT_____: support PC64 .P00 files
DETAILS_____:
There are floating around a few .P00 files, so support this format, too. Format description can be found in the online help of PC64.
CONCLUSION__:
Implement it.

RQID________: 15
PRODUCT_____: TargetD64
COMPONENT___: Testsuite
VERSION_____: V0.1
FIXED_VERS__: V0.2
OS__________: All
ORIGINATOR__: Karlheinz Langguth
TYPE________: Bug report
STATE_______: Closed
SEVERITY____: Low
DATE________: 06.03.99
SUBJECT_____: D64 images from [1-4]! are always produced last
DETAILS_____:
When passing more than one archive to TD64 the D64 images related to the [1-4]! files are produced at the end of the processing. This results in a messed up order of D64 images. In example when you are passing a.zip and b.zip to TD64 with a.zip containing [1-4]!a and b.zip containing b.lnx, target001.d64 contains content of b.lnx and target002.d64 contains content of [1-4]!a. Vice versa would have been expected.
ANALYSIS____:
Multipart archives are processed in general after all other archives have been processed. This corrupts the expected order of produced D64 emulator images.
CONCLUSION__:
Change this.

RQID________: 16
PRODUCT_____: TargetD64
COMPONENT___: Executable
VERSION_____: V0.1
FIXED_VERS__: V0.2
OS__________: Win95/NT
ORIGINATOR__: Karlheinz Langguth
TYPE________: Bug report
STATE_______: Closed
SEVERITY____: Low
DATE________: 23.03.99
SUBJECT_____: Cleaning up filesystem fails for write protected files
DETAILS_____:
TargetD64 is supposed to clean up any trace within the filesystem after processing. This will not work for write protected files extracted from archives (whick keep file attributes).
ANALYSIS____:
The remove() system call will not delete write protected files for Win32.
CONCLUSION__:
Use chmod() before removing.

RQID________: 17
PRODUCT_____: TargetD64
COMPONENT___: Executable
VERSION_____: V0.1
FIXED_VERS__: V0.2
OS__________: All
ORIGINATOR__: Karlheinz Langguth
TYPE________: Bug report
STATE_______: Closed
SEVERITY____: Medium
DATE________: 23.03.99
SUBJECT_____: Order of produced D64 image does not reflect input order of files
DETAILS_____:
The order of produced D64 images does not correspond to the order of input files. See related RQID 15. Furthermore, if an unresolvable archive and a resolvable archive are passed to TargetD64 (in this order) the resolvable archive is processed first. The corresponding D64 image is produced first. Expected would be the other way around (according to input files order).
ANALYSIS____:
Code performs multiple loops over archive files contained in an archive. Each loop processes different "classes" of files.
CONCLUSION__:
Only have a single loop in which each archive file is processed in the order in which they have been created. This means huge effort as I have to cope with a design weakness of TargetD64 - should have thought earlier about it.

RQID________: 18
PRODUCT_____: TargetD64
COMPONENT___: Executable
VERSION_____: V0.1
FIXED_VERS__: V0.2
OS__________: All
ORIGINATOR__: Karlheinz Langguth
TYPE________: Bug report
STATE_______: Closed
SEVERITY____: Low
DATE________: 09.04.99
SUBJECT_____: Fatal errors are not indicated by exit code
DETAILS_____:
TargetD64 always exits with exit code 0, even in case of failure.
CONCLUSION__:
Fix it.

RQID________: 19
PRODUCT_____: TargetD64
COMPONENT___: Executable
VERSION_____: V0.1
FIXED_VERS__: V0.2
OS__________: All
ORIGINATOR__: Karlheinz Langguth
TYPE________: Change request
STATE_______: Closed
SEVERITY____: Medium
DATE________: 09.04.99
SUBJECT_____: Make TargetD64 expandable by generic archive interface
DETAILS_____:
TargetD64 nativly supports a variety of archive formats. Still there are a few lacking (e.g. .zoo, .arc). Give the user the opportunity to link in his favorite archive into the TargetD64 processing (external helper application for the archive must exist).
CONCLUSION__:
Introduce environment variables TD64_GENERIC_CMDx and TD64_GENERIC_EXTx (where x is an index starting with 1). In TD64_GENERIC_EXTx the extension of the new archive is stated. In TD64_GENERIC_CMDx the command syntax to extract the new archive is stated. Both are related by index x. The command syntax is the same than for the already used external helper application commands (e.g. TD64_LHA_CMD). Still there is no and never will be any solution for unknown multipart archives like [1-6]!! (sixpack) but TargetD64 will be "open source" in terms of GPL.