Opened 9 years ago

Closed 4 years ago

Last modified 4 years ago

#1135 closed task (wontfix)

Create plug-in for importing raw data to multiple raw bioassays in one go

Reported by: jari Owned by:
Priority: major Milestone:
Component: coreplugins Version: trunk
Keywords: Cc:

Description

The batch importers now support creation of items down to RawBioAssay level if raw data files are uploaded by other means like ftp or zip-files. Experiments are created through the web interface and raw bioassays are associated to the experiment. However, no data is automatically imported into raw data tables and currently this has to be done manually for each raw bioassay. We should create an import functionality for an experiment that imports data into the raw data tables (if appropriate for the platform and that data was not already imported).

This is needed for Illumina IBS and GenePix and other data. The problem is that there is a need to select plug-in configurations for imports since several configurations may match the file to import (think dye swap). To resolve the issue of different file formats and multiple matching configurations the import should start with a selection of the raw bioassays to import and a configuration should be selected. In cases where no configuration or a single configuration exists maybe this choice can be skipped. Of the selected set of raw bioassays of course data is only imported for those raw bioassays with no data in the raw tables.

The plug-in should check whether it can import each file with the selected import and only perform an import of the formats matching the configuration selected. The plug-in should report the number of successful, skipped and failed imports.

Attachments (1)

multi-rawdata-importer-1.txt (5.1 KB) - added by nicklas 9 years ago.
Implementation specifications v.1

Download all attachments as: .zip

Change History (5)

comment:1 Changed 9 years ago by nicklas

  • Owner changed from everyone to nicklas
  • Status changed from new to assigned
  • Summary changed from Add raw bioassay data import functionality in an experiment to Create plug-in for importing raw data to multiple raw bioassays in one go
  • Type changed from enhancement to task

Changed 9 years ago by nicklas

Implementation specifications v.1

comment:2 Changed 9 years ago by nicklas

  • Milestone changed from BASE 2.9 to BASE 3.0
  • Owner nicklas deleted
  • Status changed from assigned to new

It doesn't work! The Plugin API has not been design for a plug-in (master) to run other plug-ins (worker). There are several problems:

  1. The parameters needed by the worker plug-in, for example the file to import from, doesn't make sense when used in a multi-import environment. It is hard to design something that can automatically detect the parameters that doesn't make sense and at the same time provide some useful values for those parameters so that the configuration of the worker plug-in doesn't generate an error.
  2. If job agents has been set up it is not sure that the worker plug-in is available on the same job agent as the master plug-in. This is a configuration issue, but it means that it is not possible to be flexible in the setup.

The multi-import feature has to be solved by each import plug-in individually. I'll replace this ticket with new tickets for the generic raw data importer and (on the BASE plug-ins site) the Illumina importer.

comment:3 Changed 4 years ago by jari

  • Resolution set to wontfix
  • Status changed from new to closed

comment:4 Changed 4 years ago by nicklas

  • Milestone BASE Future Release deleted
Note: See TracTickets for help on using tickets.