Package net.sf.basedb.util.export.spotdata

Class BfsExporter

java.lang.Object

net.sf.basedb.util.export.spotdata.AbstractBioAssaySetExporter

net.sf.basedb.util.export.spotdata.BfsExporter

Direct Known Subclasses:

MatrixBfsExporter, SerialBfsExporter

public abstract class BfsExporter
extends AbstractBioAssaySetExporter

Base class for exporting spotdata to BFS format. This class will do most of the hard work. A subclass is required to:

• create a list of data writers that will output the actual spot data (see createDataWriters()).
• rearrange the data that is loaded by the query to the correct data writer (see arrangeData(int, short, Object[][], Object[])).

Before the exporter can be used it must be configured. The most important options are:

• AbstractBioAssaySetExporter.setDbControl(DbControl) (required): Sets the DbControl that is used for database access.
• AbstractBioAssaySetExporter.setSource(net.sf.basedb.core.BioAssaySet) (required): Sets the bioassay set that data should be exported from.
• setMetadataWriter(MetadataWriter) (required): Sets the writer stream were BFS metadata is written.
• setReporterAnnotationsWriter(AnnotationWriter) and addReporterField(DynamicField) (optional): Sets the writer stream were reporter annoatations are written and selects which reporter annotations to export. If no writer is specified the reporter annotations file is not created.
• setAssayAnnotationsWriter(AnnotationWriter) and addAssayField(AssayField) (optional): Sets the writer stream were assay annotations is written and selects which assay annotations to export. If no writer is specified the assay annotations file is not created.
• setDataWriterFactory(DataWriterFactory) and addSpotField(DynamicField) (optional): Sets the factory that should be used to create output files and selects which spot data to export. The number of files and the data arrangement depends on the subtype.
• setParameter(String, String...) (optional): Additional parameters that are needed by, for example, a plug-in that is about to process the exported data files.

Version:

2.15

Author:

Nicklas

Last modified

$Date: 2024-10-30 13:32:46 +0100 (Wed, 30 Oct 2024) $

Field Summary

Fields

Modifier and Type	Field	Description
private AnnotationWriter	assayWriter
private boolean	autoCloseWriters
private ChainedProgressReporter	chainedProgress
private DataWriterFactory	dataWriterFactory
private List<DataWriter>	dataWriters
private Set<Integer>	exportedPositions
private List<MetadataModel.SectionEntry>	extraFiles
private MetadataWriter	metadataWriter
private Map<String,String[]>	parameters
private AnnotationWriter	reporterWriter
private long	spotCount

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	BfsExporter()

Method Summary

Modifier and Type	Method	Description
void	addAssayField(AssayField field)	Add information about an assay field that should be exported.
void	addAssayFields(Collection<? extends AssayField> fields)	Add multiple assay fields in one go.
void	addExtraFile(String key, String filename)	Add information about an extra data file that is included as part of the BFS.
void	addReporterField(DynamicField field)	Add information about a reporter field that should be exported.
void	addReporterFields(Collection<? extends DynamicField> fields)	Add multiple reporter fields in one go.
void	addSpotField(DynamicField spotField)	Add information about a spot field that should be exported.
void	addSpotFields(Collection<? extends DynamicField> fields)	Add multiple spot fields in one go.
protected abstract void	arrangeData(int position, short column, Object[][] data, Object[] spotData)	Arrange the data that was returned by the query into the data writers array.
protected void	beginExport()	Validates that all required properties has been set and initialises other things that are needed.
protected DataWriter	createDataWriter(Object owner, String suggestedFilename)	Utility method for calling creating a data writer.
protected abstract List<DataWriter>	createDataWriters()	Create the exact number of data writers that is needed to export the desired data.
protected void	endExport(RuntimeException e)	Clean up.
protected void	exportAssayAnnotations()	Export the assay annotations.
protected void	exportGlobalFooter()	Exports reporter annotations.
protected boolean	exportGlobalHeader()	Export the metadata and assay annotations.
protected void	exportMetadata()	Export the metadata file which contains information about the other files (reporter annotations, assay annotations and spot data), plug-in parameters, etc.
protected void	exportReporterAnnotations(ProgressReporter progress, long count)	Export the reporter annotations.
protected void	exportSectionData()	Export all spot data.
protected long	exportSpotData(ProgressReporter progress, long count)	Export spot data.
protected AnnotationWriter	getAssayAnnotationsWriter()	Get the writer were assay annotations should be printed.
List<AssayField>	getAssayFields()	Get the list of registered assay fields, in the order they were registered.
String	getBFSSubtype()	Get the BFS subtype.
protected DataWriterFactory	getDataWriterFactory()	Get the factory that creates data writers.
protected MetadataWriter	getMetadataWriter()	Get the writer were metadata should be printed.
String[]	getParameter(String key)	Get the current value of a parameter.
protected AnnotationWriter	getReporterAnnotationsWriter()	Get the writer were reporter annotations should be printed.
List<DynamicField>	getReporterFields()	Get the list of registered reporter fields, in the order they were registered.
List<DynamicField>	getSpotFields()	Get the list of registered spot fields, in the order they were registered.
void	setAssayAnnotationsWriter(AnnotationWriter assayWriter)	Set the stream were assay annotations should be written.
void	setAutoCloseWriters(boolean autoClose)	If this option is set then all writers are automatically closed when all data has been writted to them.
void	setDataWriterFactory(DataWriterFactory<?> dataWriterFactory)	Set the data writer factory to use for generating spot data files.
void	setMetadataWriter(MetadataWriter metadataWriter)	Set the stream were the metadata should be written.
void	setParameter(String key, String... values)	Add a parameter that is exported in the [parameters] section.
void	setReporterAnnotationsWriter(AnnotationWriter reporterWriter)	Set the stream were reporter annotations should be written.
protected void	validate()	Validate that all required options has been set and that other options have sensible values.

Methods inherited from class net.sf.basedb.util.export.spotdata.AbstractBioAssaySetExporter

addExperimentalFactorsAsAssayFields, cacheReporterData, copyReporterFields, doExport, exportSectionFooter, exportSectionHeader, getAssayIndex, getAverageOnReporter, getBioAssays, getDbControl, getProgressReporter, getReporterQuery, getSnapshotManager, getSource, getSpotQuery, prepareAssayIndexMap, setAverageOnReporter, setDbControl, setProgress, setProgressReporter, setSnapshotManager, setSource

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

autoCloseWriters

private boolean autoCloseWriters

dataWriterFactory

private DataWriterFactory dataWriterFactory

parameters

private Map<String,String[]> parameters

metadataWriter

private MetadataWriter metadataWriter

reporterWriter

private AnnotationWriter reporterWriter

assayWriter

private AnnotationWriter assayWriter

dataWriters

private List<DataWriter> dataWriters

extraFiles

private List<MetadataModel.SectionEntry> extraFiles

chainedProgress

private ChainedProgressReporter chainedProgress

spotCount

private long spotCount

exportedPositions

private Set<Integer> exportedPositions

Constructor Details

BfsExporter

protected BfsExporter()

Method Details

setAutoCloseWriters

public void setAutoCloseWriters(boolean autoClose)

If this option is set then all writers are automatically closed when all data has been writted to them. This setting is enabled by default.

setMetadataWriter

public void setMetadataWriter(MetadataWriter metadataWriter)

Set the stream were the metadata should be written. This stream is required before it is possible to start the export. It is expected that the given writer is a fresh writer and that no data has been written to it yet.

setReporterAnnotationsWriter

public void setReporterAnnotationsWriter(AnnotationWriter reporterWriter)

Set the stream were reporter annotations should be written. This is optional and if not given, no reporter annotations file is generated. The writer must return a filename from AnnotationWriter.getFilename() since this is required in the metadata file.

Use addReporterField(DynamicField) to register reporter fields to export. The position number is always exported.

Parameters:

reporterWriter - An annotation writer, or null if no reporter annotation file should be created

setAssayAnnotationsWriter

public void setAssayAnnotationsWriter(AnnotationWriter assayWriter)

Set the stream were assay annotations should be written. This is optional and if not given, no assay annotations file is generated. The writer must return a filename from AnnotationWriter.getFilename() since this is required in the metadata file.

Use addAssayField(AssayField) to register fields to export. The assay id is always exported.

Parameters:

assayWriter - An annotation writer, or null if no assay annotations file should be created

setDataWriterFactory

public void setDataWriterFactory(DataWriterFactory<?> dataWriterFactory)

Set the data writer factory to use for generating spot data files. What files that are required depends on the BFS subtype. Eg. the serial format uses a file per assay and the matrix format uses a file per spot field. A data writer factory is optional and if not given, no spot data is exported.

Parameters:

dataWriterFactory - A data writer factory, or null if no spot data files should be created

setParameter

public void setParameter(String key,
 String... values)

Add a parameter that is exported in the [parameters] section. There can be one or more values for a given key, which are then separated by tab characters. The parameters are written to the file in the order they are registered by this method.

NOTE! Calling this method twice with the same key, overwrites the existing parameters, but doesn't change the order they are written to the file.

Parameters:

key - The parameter key (if null this method call is ignored)

values - The parameter value (may be null)

getParameter

public String[] getParameter(String key)

Get the current value of a parameter.

Parameters:

key - The parameter key

Returns:

The value, or null if no values has been registered for the key (or if the registered value is null)

addReporterField

public void addReporterField(DynamicField field)

Description copied from class: AbstractBioAssaySetExporter

Add information about a reporter field that should be exported.

Overrides:

addReporterField in class AbstractBioAssaySetExporter

Parameters:

field - The reporter field (null is ignored)

addReporterFields

public void addReporterFields(Collection<? extends DynamicField> fields)

Add multiple reporter fields in one go. A null collection is ignored and so are null values in the collection.

Parameters:

fields - A collection with the fields that should be added

getReporterFields

public List<DynamicField> getReporterFields()

Description copied from class: AbstractBioAssaySetExporter

Get the list of registered reporter fields, in the order they were registered.

Overrides:

getReporterFields in class AbstractBioAssaySetExporter

addAssayField

public void addAssayField(AssayField field)

Description copied from class: AbstractBioAssaySetExporter

Add information about an assay field that should be exported.

Overrides:

addAssayField in class AbstractBioAssaySetExporter

Parameters:

field - The assay field (null is ignored)

addAssayFields

public void addAssayFields(Collection<? extends AssayField> fields)

Add multiple assay fields in one go. A null collection is ignored and so are null values in the collection.

Parameters:

fields - A collection with the fields that should be added

getAssayFields

public List<AssayField> getAssayFields()

Description copied from class: AbstractBioAssaySetExporter

Get the list of registered assay fields, in the order they were registered.

Overrides:

getAssayFields in class AbstractBioAssaySetExporter

addSpotField

public void addSpotField(DynamicField spotField)

Add information about a spot field that should be exported.

Overrides:

addSpotField in class AbstractBioAssaySetExporter

Parameters:

spotField - The spot field information

addSpotFields

public void addSpotFields(Collection<? extends DynamicField> fields)

Add multiple spot fields in one go. A null collection is ignored and so are null values in the collection.

Parameters:

fields - A collection with the fields that should be added

getSpotFields

public List<DynamicField> getSpotFields()

Description copied from class: AbstractBioAssaySetExporter

Get the list of registered spot fields, in the order they were registered.

Overrides:

getSpotFields in class AbstractBioAssaySetExporter

addExtraFile

public void addExtraFile(String key,
 String filename)

Add information about an extra data file that is included as part of the BFS. This exporter will register this information as an entry in the [files] section in the metadata file. It will not check that the file actually exists or do any other action with the file.

File entries are written in the order they are registered by this method. Multiple entries with the same key are allowed (eg. the previous entry is not replaced).

Parameters:

key - The key for the file

filename - The name of the file

validate

protected void validate()

Description copied from class: AbstractBioAssaySetExporter

Validate that all required options has been set and that other options have sensible values. Subclasses that override this method should call super.validate(). The default implementation will at least validate that a DbControl and a source bioassay set has been set.

Overrides:

validate in class AbstractBioAssaySetExporter

beginExport

protected void beginExport()

Validates that all required properties has been set and initialises other things that are needed.

Overrides:

beginExport in class AbstractBioAssaySetExporter

exportGlobalHeader

protected boolean exportGlobalHeader()

Export the metadata and assay annotations.

Overrides:

exportGlobalHeader in class AbstractBioAssaySetExporter

Returns:

TRUE to continue with a section AbstractBioAssaySetExporter.exportSectionHeader(), FALSE to continue with global footers AbstractBioAssaySetExporter.exportGlobalFooter()

exportSectionData

protected void exportSectionData()

Export all spot data.

Overrides:

exportSectionData in class AbstractBioAssaySetExporter

exportGlobalFooter

protected void exportGlobalFooter()

Exports reporter annotations.

Overrides:

exportGlobalFooter in class AbstractBioAssaySetExporter

endExport

protected void endExport(RuntimeException e)

Clean up.

Overrides:

endExport in class AbstractBioAssaySetExporter

getBFSSubtype

public String getBFSSubtype()

Get the BFS subtype. This method may be overridden by subclasses. The default implementation returns null. The subtype is written to the metadata file: MetadataWriter.setSubtype(String).

getMetadataWriter

protected MetadataWriter getMetadataWriter()

Get the writer were metadata should be printed.

getReporterAnnotationsWriter

protected AnnotationWriter getReporterAnnotationsWriter()

Get the writer were reporter annotations should be printed. Can be null.

getAssayAnnotationsWriter

protected AnnotationWriter getAssayAnnotationsWriter()

Get the writer were assay annotations should be printed. Can be null.

getDataWriterFactory

protected DataWriterFactory getDataWriterFactory()

Get the factory that creates data writers. Can be null.

exportMetadata

protected void exportMetadata()

Export the metadata file which contains information about the other files (reporter annotations, assay annotations and spot data), plug-in parameters, etc. Plug-in parameters are registered with setParameter(String, String[]) and are written to the [parameters] section. If no parameters has been registered this section is skipped.

exportReporterAnnotations

protected void exportReporterAnnotations(ProgressReporter progress,
 long count)

Export the reporter annotations. This method is ignored if getReporterAnnotationsWriter() returns null. The position is used as the value for the id column. Other columns are exported as given by getReporterFields().

If a progress reporter is given, progress is updated between 0 and 100%.

This method needs to be executed after exportSpotData(ProgressReporter, long) since we need to know which positions that was exported.

Parameters:

progress - An optional progress reporter

count - The total number of reporters, or 0 if not known

exportAssayAnnotations

protected void exportAssayAnnotations()

Export the assay annotations. This method is ignored if getAssayAnnotationsWriter() returns null. The assay ID is used as the value for the id column. Other columns are exported as given by getAssayFields().

exportSpotData

protected long exportSpotData(ProgressReporter progress,
 long count)

Export spot data. This method needs to be executed even if no spot fields has been selected. The reason is that we need to find out the positions that are used in the bioassay set when exporting reporter annotations. The only exception is if also no reporter annotations are exported.

Parameters:

progress - An optional progress reporter

count - The expected number of spots or 0 if not known

Returns:

The number of spots processed

createDataWriter

protected DataWriter createDataWriter(Object owner,
 String suggestedFilename)
                               throws IOException

Utility method for calling creating a data writer. Calls DataWriterFactory.createDataWriter(Object, String).

Returns:

A data writer, or null if no factory has been specified

Throws:

IOException

createDataWriters

protected abstract List<DataWriter> createDataWriters()
                                               throws IOException

Create the exact number of data writers that is needed to export the desired data. Each data writer needs to specify a filename (DataWriter.getFilename() and the number of data columns (DataWriter.getColumnCount()) it is going to write.

This method must be implemented by a subclass and is called once from the beginExport() method unless no data writer factory has been set up.

Returns:

A list with data writers

Throws:

IOException - If there is a problem with creating the writers

arrangeData

protected abstract void arrangeData(int position,
 short column,
 Object[][] data,
 Object[] spotData)

Arrange the data that was returned by the query into the data writers array. The data array is a two-demensional array. The first index represents a datawriter and goes from 0 to the one below the number of data writers returned by createDataWriters(). The size of the second index is determined by the number of columns for the data writer (eg. DataWriter.getColumnCount().

This method must be implemented by a subclass which should copy data from the spotdata array into the data array. The data array is flushed to the data writers every time the position coordinate changes. Thus, this method can be called multiple times with different column (=assay) coordinates.

Parameters:

position - The current position of the spot data

column - The current column of the spot data

data - The data array to copy the data to

spotData - The current row of spot data, in the same order as getSpotFields()