Extending compressed file support and storage

[Jari Häkkinen]

It would be nice if BASE core support would include storage of compressed files. Affy files CDFs and CELs shrinks with a factor 5 to 10 when compressed. BASE can store compressed files already but does not know how to handle them. Currently clients and plug-ins must know how to handle files. It is usually assumed that the files are uncompressed.

The core should have support for uncompressing files when clients request them and support client queries like isCompressedFile().

[Jari Häkkinen]

Milestone BASE 2.4 deleted

[Nicklas Nordborg]

(In [3719]) References #411: Most of the basic functionality is now in place and has been documented On major issue is that the Affymetrix Fusion SDK will not work with compressed files since it expects direct access to the java.io.File object. I will try to solve this by modifying the Affymetric Fusion SDK to work with InputStream:s instead. This would be better than to decompress CEL and CDF files everytime they are needed. In most cases we only scan the headers anyway, so it would be good if we can avoid copying the entire 40+ MB file(s) each time.

[Nicklas Nordborg]

(In [3737]) Added modified AffxFusion?.jar that can read from InputStream? for some file formats.

[Nicklas Nordborg]

(In [3750]) References #411. Compiled AffxFusion.jar with target="1.5" to make it possible to compile BASE with Java 1.5

[Nicklas Nordborg]

(In [3751]) References #411. Migration should now automatically compress files based on MIME type.

[Nicklas Nordborg]

(In [3762]) References #554, #746 and fixes #411. File and directories diagram transfered to new MagicDraw and docbook. Added documentation about compression support.

[Nicklas Nordborg]

(In [3889]) References #411: Extending compressed file support and storage

AbstractFlatFileImporter should use File.getSize() instead of File.getBytes() for progress reporting.