Opened 17 years ago

Closed 17 years ago

Last modified 16 years ago

#411 closed enhancement (fixed)

Extending compressed file support and storage

Reported by: Jari Häkkinen Owned by: Nicklas Nordborg
Priority: major Milestone: BASE 2.5
Component: core Version:
Keywords: Cc:

Description (last modified by Nicklas Nordborg)

It would be nice if BASE core support would include storage of compressed files. Affy files CDFs and CELs shrinks with a factor 5 to 10 when compressed. BASE can store compressed files already but does not know how to handle them. Currently clients and plug-ins must know how to handle files. It is usually assumed that the files are uncompressed.

The core should have support for uncompressing files when clients request them and support client queries like isCompressedFile().

Change History (13)

comment:1 by Jari Häkkinen, 17 years ago

Type: defectenhancement

comment:2 by Jari Häkkinen, 17 years ago

Milestone: BASE 2.x+

comment:3 by Jari Häkkinen, 17 years ago

Milestone: BASE 2.x+BASE 2.4

comment:4 by Jari Häkkinen, 17 years ago

Milestone: BASE 2.4BASE 2.3

Milestone BASE 2.4 deleted

comment:5 by Jari Häkkinen, 17 years ago

Milestone: BASE 2.4BASE 2.5

comment:6 by Nicklas Nordborg, 17 years ago

Status: newassigned

comment:7 by Nicklas Nordborg, 17 years ago

(In [3719]) References #411: Most of the basic functionality is now in place and has been documented On major issue is that the Affymetrix Fusion SDK will not work with compressed files since it expects direct access to the java.io.File object. I will try to solve this by modifying the Affymetric Fusion SDK to work with InputStream:s instead. This would be better than to decompress CEL and CDF files everytime they are needed. In most cases we only scan the headers anyway, so it would be good if we can avoid copying the entire 40+ MB file(s) each time.

comment:8 by Nicklas Nordborg, 17 years ago

(In [3737]) Added modified AffxFusion?.jar that can read from InputStream? for some file formats.

comment:9 by Nicklas Nordborg, 17 years ago

(In [3750]) References #411. Compiled AffxFusion.jar with target="1.5" to make it possible to compile BASE with Java 1.5

comment:10 by Nicklas Nordborg, 17 years ago

(In [3751]) References #411. Migration should now automatically compress files based on MIME type.

comment:11 by Nicklas Nordborg, 17 years ago

Description: modified (diff)

comment:12 by Nicklas Nordborg, 17 years ago

Resolution: fixed
Status: assignedclosed

(In [3762]) References #554, #746 and fixes #411. File and directories diagram transfered to new MagicDraw and docbook. Added documentation about compression support.

comment:13 by Nicklas Nordborg, 16 years ago

(In [3889]) References #411: Extending compressed file support and storage

AbstractFlatFileImporter should use File.getSize() instead of File.getBytes() for progress reporting.

Note: See TracTickets for help on using tickets.