Opened 13 years ago

Closed 8 years ago

Last modified 8 years ago

#1135 closed task (wontfix)

Create plug-in for importing raw data to multiple raw bioassays in one go

Reported by: Jari Häkkinen Owned by:
Priority: major Milestone:
Component: coreplugins Version: trunk
Keywords: Cc:

Description

The batch importers now support creation of items down to RawBioAssay level if raw data files are uploaded by other means like ftp or zip-files. Experiments are created through the web interface and raw bioassays are associated to the experiment. However, no data is automatically imported into raw data tables and currently this has to be done manually for each raw bioassay. We should create an import functionality for an experiment that imports data into the raw data tables (if appropriate for the platform and that data was not already imported).

This is needed for Illumina IBS and GenePix and other data. The problem is that there is a need to select plug-in configurations for imports since several configurations may match the file to import (think dye swap). To resolve the issue of different file formats and multiple matching configurations the import should start with a selection of the raw bioassays to import and a configuration should be selected. In cases where no configuration or a single configuration exists maybe this choice can be skipped. Of the selected set of raw bioassays of course data is only imported for those raw bioassays with no data in the raw tables.

The plug-in should check whether it can import each file with the selected import and only perform an import of the formats matching the configuration selected. The plug-in should report the number of successful, skipped and failed imports.

Attachments (1)

multi-rawdata-importer-1.txt (5.1 KB) - added by Nicklas Nordborg 13 years ago.
Implementation specifications v.1

Download all attachments as: .zip

Change History (5)

comment:1 Changed 13 years ago by Nicklas Nordborg

Owner: changed from everyone to Nicklas Nordborg
Status: newassigned
Summary: Add raw bioassay data import functionality in an experimentCreate plug-in for importing raw data to multiple raw bioassays in one go
Type: enhancementtask

Changed 13 years ago by Nicklas Nordborg

Implementation specifications v.1

comment:2 Changed 13 years ago by Nicklas Nordborg

Milestone: BASE 2.9BASE 3.0
Owner: Nicklas Nordborg deleted
Status: assignednew

It doesn't work! The Plugin API has not been design for a plug-in (master) to run other plug-ins (worker). There are several problems:

  1. The parameters needed by the worker plug-in, for example the file to import from, doesn't make sense when used in a multi-import environment. It is hard to design something that can automatically detect the parameters that doesn't make sense and at the same time provide some useful values for those parameters so that the configuration of the worker plug-in doesn't generate an error.
  2. If job agents has been set up it is not sure that the worker plug-in is available on the same job agent as the master plug-in. This is a configuration issue, but it means that it is not possible to be flexible in the setup.

The multi-import feature has to be solved by each import plug-in individually. I'll replace this ticket with new tickets for the generic raw data importer and (on the BASE plug-ins site) the Illumina importer.

comment:3 Changed 8 years ago by Jari Häkkinen

Resolution: wontfix
Status: newclosed

comment:4 Changed 8 years ago by Nicklas Nordborg

Milestone: BASE Future Release
Note: See TracTickets for help on using tickets.