Query used in exportPlainMatrix method in BioAssaySetExporter must be usable outside the method ...

[Jari Häkkinen]

... I think.

My problem is that I export a plain matrix, make manipulations to the matrix, and want to store the new matrix back. I need to track spot information to accomplish this, and to do this I need to know how the plainmatrix was queried.

What are the side effects of creating a method like

DynamicSpotQuery plainMatrixQuery(BioAssaySet bioassayset)
{
  DynamicSpotQuery query=bioassayset.getSpotData();
  query.select(Dynamic.select(VirtualColumn.POSITION));
  query.select(Dynamic.select(VirtualColumn.COLUMN));
  for (int i=1; i<=nof_channels; ++i)
  {
    query.select(Dynamic.select(VirtualColumn.channel(i)));
  }
  query.order(Orders.asc(Dynamic.column(VirtualColumn.POSITION)));
  query.order(Orders.asc(Dynamic.column(VirtualColumn.COLUMN)));
  return query;
}

The obvious cost is that the query will be made twice in my use case. Once for the export and the once more when I want to store the child bioassayset.

Another solution could be that the plain matrix exporter generates two files ... the plain matrix and the spot data in another file. This would open up for exporting other information in the second file.

Of course, all of the above can be accomplished outside exportPlainMatrix. The question is really, should this be a core plug-in feature or not? I'd like to see exportPlainMatrix replaced (or call it enhanced) with a two file export. This would not break the API, the download immediately feature for plaimatrix cannot be supported. Of course, the web GUI does not support plainmatrix export at all currently. In my world, the second file should contain data required to reload a child back to base and annotations but the content should of course be specified in more detail.

The plainMatrix export is more or less useless today, isn't? If nothing else we should through away the export ... breaking the API, but does anyone really use the export more than maybe to play with?

Thoughts?

[Nicklas Nordborg]

My recommendation is to avoid queries that are not absolutely neccessary. Running the same query twice is not a good idea. I have not included the plain matrix export in #1238, but a similar query for BASEFile export can take several minutes with large bioassay sets. I think it is better to make a single query that return all data that is needed for both the export and the import.

I would like to move the plain matrix export to it's own class to avoid making the BioAssaySetExporter bloated. It doesn't have to be a full-blown plug-in class. If we want the plain matrix to be a plug-in (accessible from the gui) we should just create a wrapper around the implementation.

We should really do the same with the MeV and BASEfile exports (ticket #742). The current design makes it hard to use the export functionality by itself without going through the plug-in API. Another problem is that it is hard to extend the functionality. Just have a look at the current implementation that has a lot of overloaded and deprecated methods.

[Jari Häkkinen]

Replying to nicklas:

My recommendation is to avoid queries that are not absolutely neccessary. Running the same query twice is not a good idea. I have not included the plain matrix export in #1238, but a similar query for BASEFile export can take several minutes with large bioassay sets. I think it is better to make a single query that return all data that is needed for both the export and the import.

I agree. For now I have to do it in my sQuantilerNormalizer plug-in but I'll add a ticket to replace the export/import with whatever BASE core decides.

I would like to move the plain matrix export to it's own class to avoid making the BioAssaySetExporter bloated. It doesn't have to be a full-blown plug-in class. If we want the plain matrix to be a plug-in (accessible from the gui) we should just create a wrapper around the implementation.

Fine with me.

We should really do the same with the MeV and BASEfile exports (ticket #742). The current design makes it hard to use the export functionality by itself without going through the plug-in API. Another problem is that it is hard to extend the functionality. Just have a look at the current implementation that has a lot of overloaded and deprecated methods.

Would we schedule the re-factoring to the next (2.11) milestone or later? For me it is not crucial since I am going to work against 2.10 anyway, and use two queries (sorry).

[Nicklas Nordborg]

Would we schedule the re-factoring to the next (2.11) milestone or later?

Depends on contributions. It is not high-priority.

[Nicklas Nordborg]

Since it is a bad idea to repeat the query I think the core should not expose it.