net.sf.basedb.plugins.batchimport
Class AbstractItemImporter<I extends BasicItem>

java.lang.Object
  net.sf.basedb.core.plugin.AbstractPlugin
      net.sf.basedb.plugins.AbstractFlatFileImporter
          net.sf.basedb.plugins.batchimport.AbstractItemImporter<I>

All Implemented Interfaces:

AutoDetectingImporter, InteractivePlugin, Plugin, SignalTarget, ConfigureByExample

Direct Known Subclasses:

ArrayBatchImporter, ArrayDesignImporter, ArraySlideImporter, BioPlateImporter, BioSourceImporter, ExtractImporter, HybridizationImporter, LabeledExtractImporter, RawBioAssayImporter, SampleImporter, ScanImporter

public abstract class AbstractItemImporter<I extends BasicItem>
extends AbstractFlatFileImporter
implements InteractivePlugin, ConfigureByExample

An abstract base class for plug-ins that import items in batch. This class implements job configuration, file parsing, logging, error handling, etc. Actual implementations have to provide item-specific parts, such as:

• The context the plug-in can be used = the type of item it can import. Eg, implement the InteractivePlugin.getGuiContexts() method.
• The methods that can be used for item identification: getIdMethods()
• Job configuration parameters for column mappings to item-specific properties: addMoreColumnMappingParameters(List)
• Query used to find items of the specific type: createItemQuery()
• Create actual column mappers used to extract data from the file when running the plug-in: createColumnMappers(FlatFileParser, boolean)
• Create a new item and populate it with data from the file: createItem(DbControl, FlatFileParser.Data)
• Update an item with data from the file: updateItem(DbControl, BasicItem, FlatFileParser.Data) and updateMultiLineItem(DbControl, BasicItem, FlatFileParser.Data, int)

Version:

2.8

Author:

nicklas

Last modified

$Date: 2011-05-19 12:55:36 +0200 (Thu, 19 May 2011) $

Nested Class Summary

Nested classes/interfaces inherited from interface net.sf.basedb.core.plugin.Plugin

Plugin.MainType

Field Summary

private ItemQuery<ArraySlide>	arraySlideQuery
private ItemQuery<ArrayBatch>	batchQuery
private ItemQuery<BioPlate>	bioPlateQuery
private ItemQuery<BioPlateType>	bioPlateTypeQuery
private ItemQuery<BioSource>	bioSourceQuery
private static String	CONFIGURE_OPTIONS
private RequestInformation	configureOptions
private RequestInformation	configureParser
private boolean	createNotFoundItems
private boolean	cropStrings
private DateFormatter	dateFormatter
private DbControl	dc
protected static PluginParameter<String>	defaultsSection
protected static PluginParameter<String>	descriptionColumnMapping
private ItemQuery<ArrayDesign>	designQuery
private boolean	dryRun
private ClassMapErrorHandler	errorHandler
protected static PluginParameter<String>	externalIdColumnMapping
private ItemQuery<Extract>	extractQuery
private boolean	failIfItemExists
private boolean	failIfMultipleFoundItems
private boolean	failIfMultipleFoundReferences
private boolean	failIfNotFoundItems
private boolean	failIfNotFoundReference
private boolean	failIfNoUsePermissionToReference
private boolean	failIfNoWritePermissionToItem
private ItemQuery<Software>	featureQuery
private FlatFileParser	ffp
private ItemQuery<DataFileType>	fileTypeQuery
private ItemQuery<Hardware>	freezerQuery
private ItemQuery<PlateGeometry>	geometryQuery
private ItemQuery<Hybridization>	hybQuery
private ItemQuery<Hardware>	hybStationQuery
protected static PluginParameter<String>	identificationSection
private Mapper	idMapper
private String	idMapping
private IdMethod	idMethod
protected static PluginParameter<String>	internalIdColumnMapping
private Map<String,BasicItem>	itemCache Cache that holds loaded/created items.
private static PluginParameter<String>	itemExistsErrorParameter
private static PluginParameter<String>	itemNotFoundErrorParameter
private ItemQuery<I>	itemQuery
private ItemQuery<LabeledExtract>	labeledExtractQuery
private ItemQuery<Label>	labelQuery
private String	lastIdentifier
private I	lastItem
private static PluginParameter<String>	multipleItemsFoundErrorParameter
private static PluginParameter<String>	multipleReferencesFoundErrorParameter
protected static PluginParameter<String>	nameColumnMapping
private static PluginParameter<String>	noUsePermissionToReferenceErrorParameter
private static PluginParameter<String>	noWritePermissionToItemErrorParameter
private int	numCreated
private int	numError
private int	numExisted
private int	numMultiLines
private int	numNotFound
private int	numUpdated
protected static StringParameterType	optionalColumnMapping
protected static PluginParameter<String>	permissionTemplateColumnMapping
private Mapper	permissionTemplateMapper
private ItemQuery<PermissionTemplate>	permissionTemplateQuery
private ItemQuery<Platform>	platformQuery
protected static PluginParameter<String>	protocolColumnMapping
private ItemQuery<Protocol>	protocolQuery
private static PluginParameter<String>	referenceNotFoundErrorParameter
protected static StringParameterType	requiredColumnMapping
private ItemQuery<Hardware>	robotQuery
private ItemQuery<Sample>	sampleQuery
private ItemQuery<Hardware>	scannerQuery
private ItemQuery<Scan>	scanQuery
private boolean	skippedLast
private boolean	updateExistingItems
private ItemQuery<PlatformVariant>	variantQuery

Fields inherited from class net.sf.basedb.plugins.AbstractFlatFileImporter

CHARSET, charsetType, complexMappings, dataFooterRegexpParameter, dataHeaderRegexpParameter, dataSplitterRegexpParameter, DECIMAL_SEPARATOR, decimalSeparatorType, defaultErrorParameter, errorSection, fileParameter, fileType, headerRegexpParameter, ignoreRegexpParameter, invalidUseOfNullErrorParameter, mappingSection, maxDataColumnsParameter, minDataColumnsParameter, numberFormatErrorParameter, numberOutOfRangeErrorParameter, numDataColumnsType, optionalRegexpType, parserSection, requiredRegexpType, sectionRegexpParameter, stringTooLongErrorParameter, trimQuotesParameter

Fields inherited from class net.sf.basedb.core.plugin.AbstractPlugin

annotationSection, configuration, COPY_ANNOTATIONS, job, OVERWRITE_ANNOTATIONS, sc

Constructor Summary

AbstractItemImporter()

Method Summary

protected void	addMoreColumnMappingParameters(List<PluginParameter<?>> parameters) Add column mapping parameters for item-specific properties.
protected void	addMoreErrorParameters(List<PluginParameter<?>> parameters) Add extra parameters to the "Error handling" section that the subclass needs.
protected void	addMoreJobParameters(List<PluginParameter<?>> parameters) Add extra job parameters that the subclass needs.
protected void	addMoreParsingParameters(List<PluginParameter<?>> parameters) Add extra parameters to the "Parser settings" section that the subclass needs.
protected void	begin(FlatFileParser ffp) Setup error handling and pre-load some of the configuration options.
protected void	beginData() Setup column mapping.
void	configure(GuiContext context, Request request, Response response) Configure the plugin.
protected void	createColumnMappers(FlatFileParser ffp, boolean cropStrings) Create mappers that can be used to extract data values from the file that is currently being parsed by the ffp FlatFileParser.
protected abstract I	createItem(DbControl dc, FlatFileParser.Data data) Create a new item and populate it with data from the current line.
protected abstract ItemQuery<I>	createItemQuery() Create a new query that returns items of the importable type.
protected void	doneWithItem(DbControl dc, I item, int numMultiLines) Called when the importer is finished with an item.
protected void	end(boolean success) Called just after the last line of the file has been parsed, or immediately after an error has ocurred.
protected ArrayBatch	findArrayBatch(DbControl dc, IdMethod idMethod, String identifier) Find an array batch with a given identifier.
protected ArrayDesign	findArrayDesign(DbControl dc, IdMethod idMethod, String identifier) Find an array design with a given identifier.
protected ArraySlide	findArraySlide(DbControl dc, IdMethod idMethod, String identifier) Find an array slide with a given identifier.
protected BioPlate	findBioPlate(DbControl dc, IdMethod idMethod, String identifier) Find a bioplate with a given identifier.
protected BioPlateType	findBioPlateType(DbControl dc, IdMethod idMethod, String identifier) Find a bioplate type with a given identifier.
protected BioSource	findBioSource(DbControl dc, IdMethod idMethod, String identifier) Find a biosource with a given identifier.
protected BioWell	findBioWell(DbControl dc, BioPlate plate, String row, String column) Find a well on a bioplate.
protected DataFileType	findDataFileType(DbControl dc, IdMethod idMethod, String identifier) Find a data file type with a given identifier.
protected Extract	findExtract(DbControl dc, IdMethod idMethod, String identifier) Find an extract with a given identifier.
protected Software	findFeatureExtractionSoftware(DbControl dc, IdMethod idMethod, String identifier) Find a feature extraction software with a given identifier.
protected Hardware	findFreezer(DbControl dc, IdMethod idMethod, String identifier) Find a freezer with a given identifier.
protected Hybridization	findHybridization(DbControl dc, IdMethod idMethod, String identifier) Find a hybridization with a given identifier.
protected Hardware	findHybridizationStation(DbControl dc, IdMethod idMethod, String identifier) Find a hybridization station with a given identifier.
protected Image	findImage(DbControl dc, Scan scan, File imageFile) Find an image with given scan and file.
protected Label	findLabel(DbControl dc, IdMethod idMethod, String identifier) Find a label with a given identifier.
protected LabeledExtract	findLabeledExtract(DbControl dc, IdMethod idMethod, String identifier) Find a labled extract with a given identifier.
protected PermissionTemplate	findPermissionTemplate(DbControl dc, IdMethod idMethod, String identifier) Find a permission template with a given identifier.
protected PlateGeometry	findPlateGeometry(DbControl dc, IdMethod idMethod, String identifier) Find a plate geometry with a given identifier.
protected Platform	findPlatform(DbControl dc, IdMethod idMethod, String identifier) Find a platform with a given identifier.
protected Hardware	findPrintRobot(DbControl dc, IdMethod idMethod, String identifier) Find a print robot with a given identifier.
protected Protocol	findProtocol(DbControl dc, IdMethod idMethod, String identifier) Same as findProtocol(DbControl, IdMethod, String, ProtocolType) with a null protocol type.
protected Protocol	findProtocol(DbControl dc, IdMethod idMethod, String identifier, ProtocolType type) Find a protocol with a given identifier.
protected RawDataType	findRawDataType(DbControl dc, Platform platform, PlatformVariant variant, String identifier) Find a matching raw data type.
protected <T extends BasicItem> T	findReferencedItem(DbControl dc, IdMethod idMethod, ItemQuery<T> query, String identifier, boolean ignoreNotFound) Find a referenced item.
protected Sample	findSample(DbControl dc, IdMethod idMethod, String identifier) Find a sample with a given identifier.
protected Scan	findScan(DbControl dc, IdMethod idMethod, String identifier) Find a scan with a given identifier.
protected Hardware	findScanner(DbControl dc, IdMethod idMethod, String identifier) Find a scanner with a given identifier.
protected PlatformVariant	findVariant(DbControl dc, Platform platform, IdMethod idMethod, String identifier) Find a platform variant with a given identifier.
protected RequestInformation	getConfigureOptionsParameters(GuiContext context)
protected RequestInformation	getConfigureParserParameters(GuiContext context, boolean forJob)
protected IdMethod	getIdMethod(String method) Get the id method to use for finding items.
protected IdMethod[]	getIdMethods() Get an array with all identification methods that can be used to find items.
protected FlatFileParser	getInitializedFlatFileParser() Create and initialise a flat file parser by setting all regular expressions and other options.
protected FlatFileParser	getInitializedFlatFileParser(ParameterValues parameters)
private Pattern	getPattern(ParameterValues parameters, String name)
protected Platform	getProjectDefaultPlatform(DbControl dc) Get the default platform of the currently active project.
protected RawDataType	getProjectDefaultRawDataType(DbControl dc) Get the default raw data type of the currently active project.
protected PlatformVariant	getProjectDefaultVariant(DbControl dc) Get the default platform variant of the currently active project.
RequestInformation	getRequestInformation(GuiContext context, String command) This method will return the RequestInformation for a given command, i.e.
protected String	getSuccessMessage(int skippedLines) Called if the parsing was successful to let the subclass generate a simple message that is sent back to the core and user interface.
protected void	handleData(FlatFileParser.Data data) Called by the parser for every line in the file that is a data line.
protected <T extends BasicItem> ItemQuery<T>	initReferenceQuery(DbControl dc, IdMethod idMethod, ItemQuery<T> query) Initialise a query that is used to find referenced items.
String	isInContext(GuiContext context, Object item) Check if the plugin can be used on the specified object.
protected Date	parseDate(String date) Parse a string to a date using the date format specified by the 'Date format' parameter.
boolean	requiresConfiguration() Return TRUE, since the implementation requires it for finding the regular expressions used by the FlatFileParser.
protected void	setUpErrorHandling() Don't use AbstractFlatFileImporter to handle errors.
boolean	supportsConfigurations() Returns TRUE, since that is how the plugins used to work before this method was introduced.
protected abstract void	updateItem(DbControl dc, I item, FlatFileParser.Data data) Update an item with data from the current line.
protected void	updateMultiLineItem(DbControl dc, I item, FlatFileParser.Data data, int multiLineNum) Update an item with data from a multi-line entry.
protected void	updatePermissions(DbControl dc, Shareable item, FlatFileParser.Data data, boolean merge) If a mapping for a permission template has been specified for the job, this method is automatically called.

Methods inherited from class net.sf.basedb.plugins.AbstractFlatFileImporter

addErrorHandler, checkColumnMapping, checkColumnMapping, continueWithNextFileAfterError, doImport, finish, getCharset, getCharset, getCharsetParameter, getDecimalSeparator, getDecimalSeparatorParameter, getErrorHandler, getErrorOption, getFileIterator, getMainType, getMapper, getNumberFormat, getNumBytes, getPrimaryLocationFilter, getProgress, getSignalHandler, getTotalFileSize, handleHeader, handleSection, isImportable, isImportable, log, log, log, log, run, start, wrapInputStream

Methods inherited from class net.sf.basedb.core.plugin.AbstractPlugin

checkInterrupted, cloneParameterWithDefaultValue, closeLogFile, createLogFile, done, getCopyAnnotationsParmeter, getCurrentConfiguration, getCurrentJob, getJobOrConfigurationValue, getOverwriteAnnotationsParameters, getPermissions, init, log, log, storeValue, storeValue, storeValues, validateRequestParameters

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface net.sf.basedb.core.plugin.InteractivePlugin

getGuiContexts

Methods inherited from interface net.sf.basedb.core.plugin.Plugin

done, getAbout, getMainType, getPermissions, init, run

Field Detail

CONFIGURE_OPTIONS

private static final String CONFIGURE_OPTIONS

See Also:

Constant Field Values

optionalColumnMapping

protected static final StringParameterType optionalColumnMapping

requiredColumnMapping

protected static final StringParameterType requiredColumnMapping

defaultsSection

protected static final PluginParameter<String> defaultsSection

identificationSection

protected static final PluginParameter<String> identificationSection

internalIdColumnMapping

protected static final PluginParameter<String> internalIdColumnMapping

nameColumnMapping

protected static final PluginParameter<String> nameColumnMapping

descriptionColumnMapping

protected static final PluginParameter<String> descriptionColumnMapping

externalIdColumnMapping

protected static final PluginParameter<String> externalIdColumnMapping

protocolColumnMapping

protected static final PluginParameter<String> protocolColumnMapping

permissionTemplateColumnMapping

protected static final PluginParameter<String> permissionTemplateColumnMapping

itemNotFoundErrorParameter

private static final PluginParameter<String> itemNotFoundErrorParameter

itemExistsErrorParameter

private static final PluginParameter<String> itemExistsErrorParameter

multipleItemsFoundErrorParameter

private static final PluginParameter<String> multipleItemsFoundErrorParameter

referenceNotFoundErrorParameter

private static final PluginParameter<String> referenceNotFoundErrorParameter

multipleReferencesFoundErrorParameter

private static final PluginParameter<String> multipleReferencesFoundErrorParameter

noWritePermissionToItemErrorParameter

private static final PluginParameter<String> noWritePermissionToItemErrorParameter

noUsePermissionToReferenceErrorParameter

private static final PluginParameter<String> noUsePermissionToReferenceErrorParameter

dc

private DbControl dc

ffp

private FlatFileParser ffp

dateFormatter

private DateFormatter dateFormatter

idMethod

private IdMethod idMethod

idMapping

private String idMapping

idMapper

private Mapper idMapper

permissionTemplateMapper

private Mapper permissionTemplateMapper

itemQuery

private ItemQuery<I extends BasicItem> itemQuery

lastIdentifier

private String lastIdentifier

lastItem

private I extends BasicItem lastItem

numMultiLines

private int numMultiLines

skippedLast

private boolean skippedLast

dryRun

private boolean dryRun

errorHandler

private ClassMapErrorHandler errorHandler

cropStrings

private boolean cropStrings

createNotFoundItems

private boolean createNotFoundItems

failIfNotFoundItems

private boolean failIfNotFoundItems

updateExistingItems

private boolean updateExistingItems

failIfItemExists

private boolean failIfItemExists

failIfMultipleFoundItems

private boolean failIfMultipleFoundItems

failIfNotFoundReference

private boolean failIfNotFoundReference

failIfMultipleFoundReferences

private boolean failIfMultipleFoundReferences

failIfNoWritePermissionToItem

private boolean failIfNoWritePermissionToItem

failIfNoUsePermissionToReference

private boolean failIfNoUsePermissionToReference

numUpdated

private int numUpdated

numCreated

private int numCreated

numError

private int numError

numNotFound

private int numNotFound

numExisted

private int numExisted

protocolQuery

private ItemQuery<Protocol> protocolQuery

labelQuery

private ItemQuery<Label> labelQuery

hybStationQuery

private ItemQuery<Hardware> hybStationQuery

scannerQuery

private ItemQuery<Hardware> scannerQuery

robotQuery

private ItemQuery<Hardware> robotQuery

freezerQuery

private ItemQuery<Hardware> freezerQuery

featureQuery

private ItemQuery<Software> featureQuery

platformQuery

private ItemQuery<Platform> platformQuery

variantQuery

private ItemQuery<PlatformVariant> variantQuery

fileTypeQuery

private ItemQuery<DataFileType> fileTypeQuery

permissionTemplateQuery

private ItemQuery<PermissionTemplate> permissionTemplateQuery

bioSourceQuery

private ItemQuery<BioSource> bioSourceQuery

sampleQuery

private ItemQuery<Sample> sampleQuery

extractQuery

private ItemQuery<Extract> extractQuery

labeledExtractQuery

private ItemQuery<LabeledExtract> labeledExtractQuery

bioPlateQuery

private ItemQuery<BioPlate> bioPlateQuery

arraySlideQuery

private ItemQuery<ArraySlide> arraySlideQuery

hybQuery

private ItemQuery<Hybridization> hybQuery

scanQuery

private ItemQuery<Scan> scanQuery

designQuery

private ItemQuery<ArrayDesign> designQuery

batchQuery

private ItemQuery<ArrayBatch> batchQuery

geometryQuery

private ItemQuery<PlateGeometry> geometryQuery

bioPlateTypeQuery

private ItemQuery<BioPlateType> bioPlateTypeQuery

itemCache

private Map<String,BasicItem> itemCache

Cache that holds loaded/created items. The cache key is item type + identifier, eq. SAMPLE:My sample, EXTRACT:An extract. The same item may be cached under more than one key depending on which identifier has been used to look it up, eg name, id, etc.: SAMPLE:My sample, SAMPLE:10.

configureParser

private RequestInformation configureParser

configureOptions

private RequestInformation configureOptions

Constructor Detail

AbstractItemImporter

public AbstractItemImporter()

Method Detail

requiresConfiguration

public boolean requiresConfiguration()

Description copied from class: AbstractFlatFileImporter

Return TRUE, since the implementation requires it for finding the regular expressions used by the FlatFileParser. If this method is overridden and returns FALSE, the subclass must also override the AbstractFlatFileImporter.getInitializedFlatFileParser() method and provide a parser with all regular expressions and other options set.

Specified by:

requiresConfiguration in interface Plugin

Overrides:

requiresConfiguration in class AbstractFlatFileImporter

Returns:

TRUE or FALSE

supportsConfigurations

public boolean supportsConfigurations()

Description copied from class: AbstractPlugin

Returns TRUE, since that is how the plugins used to work before this method was introduced.

Specified by:

supportsConfigurations in interface Plugin

Overrides:

supportsConfigurations in class AbstractPlugin

Returns:

TRUE or FALSE

isInContext

public String isInContext(GuiContext context,
                          Object item)

Description copied from interface: InteractivePlugin

Check if the plugin can be used on the specified object. The item is either a BasicItem or BasicData object, or null if called from a list context. A client application should only call this method for contexts matching one of the contexts in the set returned by the InteractivePlugin.getGuiContexts() method. For GuiContext.Type.ITEM contexts an item of the correct type should be passed as a parameter.

The plug-in can response with a null value indicating success, a warning-level message or throw an exception to indicate an error-level message. Client application should normally only show error-level messages to users, unless the user has requested to see warning messages.

NOTE! Before BASE 2.4 all messages were treated as error-level messages.

Specified by:

isInContext in interface InteractivePlugin

Parameters:

context - The current context of the client application, it is one of the values found in set returned by InteractivePlugin.getGuiContexts()

item - The currently active item, it's type should match the GuiContext.getItem() type, or null if the context is a list context

Returns:

Null if the plugin can use that item, or a warning-level message explaining why the plugin can't be used

configure

public void configure(GuiContext context,
                      Request request,
                      Response response)

Description copied from interface: InteractivePlugin

Configure the plugin. Hopefully the client is supplying values for the parameters specified by InteractivePlugin.getRequestInformation(GuiContext, String).

Specified by:

configure in interface InteractivePlugin

Parameters:

context - The current context of the client application, it is one of the values found in set returned by InteractivePlugin.getGuiContexts()

request - Request object with the command and parameters

response - Response object in for the plugin to response through

getRequestInformation

public RequestInformation getRequestInformation(GuiContext context,
                                                String command)
                                         throws BaseException

Description copied from interface: InteractivePlugin

This method will return the RequestInformation for a given command, i.e. the list of parameters and some nice help text.

Specified by:

getRequestInformation in interface InteractivePlugin

Parameters:

context - The current context of the client application, it is one of the values found in set returned by InteractivePlugin.getGuiContexts()

command - The command

Returns:

The RequestInformation for the command

Throws:

BaseException - if there is an error

getInitializedFlatFileParser

protected FlatFileParser getInitializedFlatFileParser()
                                               throws BaseException

Description copied from class: AbstractFlatFileImporter

Create and initialise a flat file parser by setting all regular expressions and other options. This implementation gets all parameters from the AbstractPlugin.job or AbstractPlugin.configuration settings. If a subclass doesn't store the parameters there it must override this method and initialise the parser. Note that this method is called once for each file returned by the AbstractFlatFileImporter.getFileIterator() and that a new parser is needed for each file.

Overrides:

getInitializedFlatFileParser in class AbstractFlatFileImporter

Returns:

An intialised flat file parser

Throws:

BaseException

setUpErrorHandling

protected void setUpErrorHandling()

Don't use AbstractFlatFileImporter to handle errors.

Overrides:

setUpErrorHandling in class AbstractFlatFileImporter

begin

protected void begin(FlatFileParser ffp)

Setup error handling and pre-load some of the configuration options.

Overrides:

begin in class AbstractFlatFileImporter

See Also:

AbstractFlatFileImporter.end(boolean)

beginData

protected void beginData()

Setup column mapping. Creates DbControl and query to find items.

Overrides:

beginData in class AbstractFlatFileImporter

handleData

protected void handleData(FlatFileParser.Data data)
                   throws BaseException

Description copied from class: AbstractFlatFileImporter

Called by the parser for every line in the file that is a data line.

Specified by:

handleData in class AbstractFlatFileImporter

Throws:

BaseException

end

protected void end(boolean success)

Description copied from class: AbstractFlatFileImporter

Called just after the last line of the file has been parsed, or immediately after an error has ocurred. A subclass should clean up any resources aquired in the AbstractFlatFileImporter.begin(FlatFileParser) method here. Note that this metod is called once for every file returned by the AbstractFlatFileImporter.getFileIterator() iterator.

Overrides:

end in class AbstractFlatFileImporter

Parameters:

success - TRUE if the file was parsed successfully, FALSE otherwise

See Also:

AbstractFlatFileImporter.begin(FlatFileParser)

getSuccessMessage

protected String getSuccessMessage(int skippedLines)

Description copied from class: AbstractFlatFileImporter

Called if the parsing was successful to let the subclass generate a simple message that is sent back to the core and user interface. An example message might by: 178 reporters imported successfully. The default implementation always return null. Note that this method is called once for every file returned by AbstractFlatFileImporter.getFileIterator().

Overrides:

getSuccessMessage in class AbstractFlatFileImporter

Parameters:

skippedLines - The number of data lines that were skipped due to errors

getIdMethods

protected IdMethod[] getIdMethods()

Get an array with all identification methods that can be used to find items. The default implementation returns an array with the PropertyIdMethod.NAME, InternalIdMethod.INTERNAL_ID and FallbackIdMethod.NAME_OR_ID ID methods. Subclasses that import items that can use other ID methods, should override this method.

addMoreJobParameters

protected void addMoreJobParameters(List<PluginParameter<?>> parameters)

Add extra job parameters that the subclass needs. This method is called during configuration of the plug-in and the default implementation does nothing.

Parameters:

parameters - The list of parameters to add the column mapping parameters to

addMoreParsingParameters

protected void addMoreParsingParameters(List<PluginParameter<?>> parameters)

Add extra parameters to the "Parser settings" section that the subclass needs. This method is called during configuration of the plug-in and the default implementation does nothing.

Parameters:

parameters - The list of parameters to add the column mapping parameters to

addMoreColumnMappingParameters

protected void addMoreColumnMappingParameters(List<PluginParameter<?>> parameters)

Add column mapping parameters for item-specific properties. A subclass must add parameters for all properties, including name and description. This class has already created some useful parameters for this purpose, for example: nameColumnMapping and descriptionColumnMapping. This method is called during configuration of the plug-in.

Parameters:

parameters - The list of parameters to add the column mapping parameters to

addMoreErrorParameters

protected void addMoreErrorParameters(List<PluginParameter<?>> parameters)

Add extra parameters to the "Error handling" section that the subclass needs. This method is called during configuration of the plug-in and the default implementation does nothing.

Parameters:

parameters - The list of parameters to add the column mapping parameters to

createItemQuery

protected abstract ItemQuery<I> createItemQuery()

Create a new query that returns items of the importable type. The query should be a plain query not including any restrictions or other elements. This will automatically be added as needed by the core implementation. This method is called during execution of the plug-in. Typically, the result from getQuery() should be returned, eg. Sample.getQuery().

Returns:

A query

createColumnMappers

protected void createColumnMappers(FlatFileParser ffp,
                                   boolean cropStrings)

Create mappers that can be used to extract data values from the file that is currently being parsed by the ffp FlatFileParser. This method is called during initialisation of the plug-in when the FlatFileParser.parseHeaders() has successfully found the start of data. Use the AbstractFlatFileImporter.getMapper(FlatFileParser, String, Integer, Mapper) to create column mappers. Example:

nameMapper = getMapper(ffp, job.getValue("nameColumnMapping"), 
   cropStrings ? Nameable.MAX_NAME_LENGTH : null, null);

Parameters:

ffp - The parser that is parsing the data file

cropStrings - If strings that are too long to fit in the database should be cropped or not (=generate an error)

createItem

protected abstract I createItem(DbControl dc,
                                FlatFileParser.Data data)

Create a new item and populate it with data from the current line. Do not call DbControl.saveItem(BasicItem). This is done automatically by the core implementation.

BioSource bs = BioSource.getNew(dc);
// Set properties; see example on updateItem method
return bs;

If the current data line doesn't have data for all required properties, this method should throw an InvalidUseOfNullException exception.

This method is called during import when current item can be found and if the user has selected to create missing item.

Parameters:

dc - The DbControl to use for database access

data - The current data line

Returns:

A new item

updateItem

protected abstract void updateItem(DbControl dc,
                                   I item,
                                   FlatFileParser.Data data)

Update an item with data from the current line. Example:

if (nameMapper != null) item.setName(nameMapper.getValue(data));
if (descriptionMapper != null) item.setDescription(descriptionMapper.getValue(data));
...

This method is called during import to update an existing item.

Parameters:

dc - The DbControl to use for database access

item - The item to update

data - The current data line

updateMultiLineItem

protected void updateMultiLineItem(DbControl dc,
                                   I item,
                                   FlatFileParser.Data data,
                                   int multiLineNum)

Update an item with data from a multi-line entry. Items which can have multiple parents should have data for the other parents on lines following the first line.

Parameters:

dc - The DbControl to use for database access

item - The item to update

data - The current data line

multiLineNum - The number of the multi-line entry, starting with 1 for the first multi-line

doneWithItem

protected void doneWithItem(DbControl dc,
                            I item,
                            int numMultiLines)

Called when the importer is finished with an item. This method is useful when working with multi-line entries, when the subclass needs to do some additional work after all data has been set. NOTE! This assumes that the lines in the data file are sorted in the correct order and the no lines for the same item will follow later on.

Parameters:

dc - The DbControl to use for database access

item - The item to update

numMultiLines - The number of the multi-line entries, starting with 1 for the first multi-line (0 for single-line entries)

updatePermissions

protected void updatePermissions(DbControl dc,
                                 Shareable item,
                                 FlatFileParser.Data data,
                                 boolean merge)

If a mapping for a permission template has been specified for the job, this method is automatically called. It will merge or replace the permissions on the item with the permissions from the template.

Parameters:

dc - A DbControl used to lookup the permission template

item - The item to update

data - Data from the file

merge - TRUE to merge the permissions, FALSE to replace the permissions

Since:

2.16

getIdMethod

protected IdMethod getIdMethod(String method)

Get the id method to use for finding items. This implementation searches the methods returned by getIdMethods() for a method with the given name. If no method can be found PropertyIdMethod.NAME is returned.

parseDate

protected Date parseDate(String date)

Parse a string to a date using the date format specified by the 'Date format' parameter. This is a utility method that subclasses can use to extract date values, since the Mapper interface doesn't supports it.

findProtocol

protected Protocol findProtocol(DbControl dc,
                                IdMethod idMethod,
                                String identifier)

Same as findProtocol(DbControl, IdMethod, String, ProtocolType) with a null protocol type.

findProtocol

protected Protocol findProtocol(DbControl dc,
                                IdMethod idMethod,
                                String identifier,
                                ProtocolType type)

Find a protocol with a given identifier. This is a utility method that subclasses can use when creating or updating items.

NOTE! The first time this method is called a query object is initialised using the IdMethod.prepareQuery(DbControl, ItemQuery) method. Subsequent calls uses the same query. Thus, this method should always be called with the same id method object, otherwise the result is undefined.

NOTE! If this method is called with a non-null protocol type parameter and no protocol is found the query will be retried without any protocol type.

Parameters:

dc - The DbControl to use for database access

identifier - The identifier protocol

type - The protocol type of the protocol, or null if the type doesn't matter

Returns:

A protocol, or null if no item could be found

Since:

2.9

findLabel

protected Label findLabel(DbControl dc,
                          IdMethod idMethod,
                          String identifier)

Find a label with a given identifier. This is a utility method that subclasses can use when creating or updating items.

NOTE! The first time this method is called a query object is initialised using the IdMethod.prepareQuery(DbControl, ItemQuery) method. Subsequent calls uses the same query. Thus, this method should always be called with the same id method object, otherwise the result is undefined.

Parameters:

dc - The DbControl to use for database access

identifier - The identifier protocol

Returns:

A label, or null if no item could be found

findHybridizationStation

protected Hardware findHybridizationStation(DbControl dc,
                                            IdMethod idMethod,
                                            String identifier)

Find a hybridization station with a given identifier. This is a utility method that subclasses can use when creating or updating items.

NOTE! A scanner is a Hardware item with it type set to HardwareType.HYBRIDIZATION_STATION.

NOTE! The first time this method is called a query object is initialised using the IdMethod.prepareQuery(DbControl, ItemQuery) method. Subsequent calls uses the same query. Thus, this method should always be called with the same id method object, otherwise the result is undefined.

Parameters:

dc - The DbControl to use for database access

identifier - The identifier protocol

Returns:

A scanner, or null if no item could be found

findScanner

protected Hardware findScanner(DbControl dc,
                               IdMethod idMethod,
                               String identifier)

Find a scanner with a given identifier. This is a utility method that subclasses can use when creating or updating items.

NOTE! A scanner is a Hardware item with it type set to HardwareType.SCANNER.

NOTE! The first time this method is called a query object is initialised using the IdMethod.prepareQuery(DbControl, ItemQuery) method. Subsequent calls uses the same query. Thus, this method should always be called with the same id method object, otherwise the result is undefined.

Parameters:

dc - The DbControl to use for database access

identifier - The identifier protocol

Returns:

A scanner, or null if no item could be found

findPrintRobot

protected Hardware findPrintRobot(DbControl dc,
                                  IdMethod idMethod,
                                  String identifier)

Find a print robot with a given identifier. This is a utility method that subclasses can use when creating or updating items.

NOTE! A print robot is a Hardware item with it type set to HardwareType.PRINT_ROBOT.

NOTE! The first time this method is called a query object is initialised using the IdMethod.prepareQuery(DbControl, ItemQuery) method. Subsequent calls uses the same query. Thus, this method should always be called with the same id method object, otherwise the result is undefined.

Parameters:

dc - The DbControl to use for database access

identifier - The identifier protocol

Returns:

A print robot, or null if no item could be found

findFreezer

protected Hardware findFreezer(DbControl dc,
                               IdMethod idMethod,
                               String identifier)

Find a freezer with a given identifier. This is a utility method that subclasses can use when creating or updating items.

NOTE! A freezer is a Hardware item with it type set to HardwareType.FREEZER.

NOTE! The first time this method is called a query object is initialised using the IdMethod.prepareQuery(DbControl, ItemQuery) method. Subsequent calls uses the same query. Thus, this method should always be called with the same id method object, otherwise the result is undefined.

Parameters:

dc - The DbControl to use for database access

identifier - The identifier protocol

Returns:

A freezer, or null if no item could be found

Since:

2.16

findFeatureExtractionSoftware

protected Software findFeatureExtractionSoftware(DbControl dc,
                                                 IdMethod idMethod,
                                                 String identifier)

Find a feature extraction software with a given identifier. This is a utility method that subclasses can use when creating or updating items.

NOTE! A feature extraction software is a Software item with it's type set to SoftwareType.FEATURE_EXTRACTION.

NOTE! The first time this method is called a query object is initialised using the IdMethod.prepareQuery(DbControl, ItemQuery) method. Subsequent calls uses the same query. Thus, this method should always be called with the same id method object, otherwise the result is undefined.

Parameters:

dc - The DbControl to use for database access

identifier - The identifier protocol

Returns:

A software item, or null if no item could be found

findPlatform

protected Platform findPlatform(DbControl dc,
                                IdMethod idMethod,
                                String identifier)

Find a platform with a given identifier. This is a utility method that subclasses can use when creating or updating items.

NOTE! The first time this method is called a query object is initialised using the IdMethod.prepareQuery(DbControl, ItemQuery) method. Subsequent calls uses the same query. Thus, this method should always be called with the same id method object, otherwise the result is undefined.

Parameters:

dc - The DbControl to use for database access

identifier - The identifier protocol

Returns:

A platform, or null if no item could be found

getProjectDefaultPlatform

protected Platform getProjectDefaultPlatform(DbControl dc)

Get the default platform of the currently active project.

Returns:

The active platform, or null if no project is active or the project doesn't specify a platform

Since:

2.9

getProjectDefaultVariant

protected PlatformVariant getProjectDefaultVariant(DbControl dc)

Get the default platform variant of the currently active project.

Returns:

The active platform variant, or null if no project is active or the project doesn't specify a variant

Since:

2.9

findVariant

protected PlatformVariant findVariant(DbControl dc,
                                      Platform platform,
                                      IdMethod idMethod,
                                      String identifier)

Find a platform variant with a given identifier. This is a utility method that subclasses can use when creating or updating items.

NOTE! The first time this method is called a query object is initialised using the IdMethod.prepareQuery(DbControl, ItemQuery) method. Subsequent calls uses the same query. Thus, this method should always be called with the same id method object, otherwise the result is undefined.

Parameters:

dc - The DbControl to use for database access

platform - If given, searched only variants of this platform

identifier - The identifier protocol

Returns:

A platform, or null if no item could be found

getProjectDefaultRawDataType

protected RawDataType getProjectDefaultRawDataType(DbControl dc)

Get the default raw data type of the currently active project.

Returns:

The active raw data type, or null if no project is active or the project doesn't specify raw data type

Since:

2.9

findDataFileType

protected DataFileType findDataFileType(DbControl dc,
                                        IdMethod idMethod,
                                        String identifier)

Find a data file type with a given identifier. This is a utility method that subclasses can use when creating or updating items.

NOTE! The first time this method is called a query object is initialised using the IdMethod.prepareQuery(DbControl, ItemQuery) method. Subsequent calls uses the same query. Thus, this method should always be called with the same id method object, otherwise the result is undefined.

Parameters:

dc - The DbControl to use for database access

identifier - The identifier protocol

Returns:

A data file, or null if no item could be found

findPermissionTemplate

protected PermissionTemplate findPermissionTemplate(DbControl dc,
                                                    IdMethod idMethod,
                                                    String identifier)

Find a permission template with a given identifier. This is a utility method that subclasses can use when creating or updating items.

NOTE! The first time this method is called a query object is initialised using the IdMethod.prepareQuery(DbControl, ItemQuery) method. Subsequent calls uses the same query. Thus, this method should always be called with the same id method object, otherwise the result is undefined.

Parameters:

dc - The DbControl to use for database access

identifier - The identifier for the template

Returns:

A permission template, or null if no item could be found

findBioSource

protected BioSource findBioSource(DbControl dc,
                                  IdMethod idMethod,
                                  String identifier)

Find a biosource with a given identifier. This is a utility method that subclasses can use when creating or updating items.

NOTE! The first time this method is called a query object is initialised using the IdMethod.prepareQuery(DbControl, ItemQuery) method. Subsequent calls uses the same query. Thus, this method should always be called with the same id method object, otherwise the result is undefined.

Parameters:

dc - The DbControl to use for database access

identifier - The identifier protocol

Returns:

A biosource, or null if no item could be found

findSample

protected Sample findSample(DbControl dc,
                            IdMethod idMethod,
                            String identifier)

Find a sample with a given identifier. This is a utility method that subclasses can use when creating or updating items.

NOTE! The first time this method is called a query object is initialised using the IdMethod.prepareQuery(DbControl, ItemQuery) method. Subsequent calls uses the same query. Thus, this method should always be called with the same id method object, otherwise the result is undefined.

Parameters:

dc - The DbControl to use for database access

identifier - The identifier protocol

Returns:

A sample, or null if no item could be found

findExtract

protected Extract findExtract(DbControl dc,
                              IdMethod idMethod,
                              String identifier)

Find an extract with a given identifier. This is a utility method that subclasses can use when creating or updating items.

NOTE! The first time this method is called a query object is initialised using the IdMethod.prepareQuery(DbControl, ItemQuery) method. Subsequent calls uses the same query. Thus, this method should always be called with the same id method object, otherwise the result is undefined.

Parameters:

dc - The DbControl to use for database access

identifier - The identifier protocol

Returns:

An extract, or null if no item could be found

findLabeledExtract

protected LabeledExtract findLabeledExtract(DbControl dc,
                                            IdMethod idMethod,
                                            String identifier)

Find a labled extract with a given identifier. This is a utility method that subclasses can use when creating or updating items.

NOTE! The first time this method is called a query object is initialised using the IdMethod.prepareQuery(DbControl, ItemQuery) method. Subsequent calls uses the same query. Thus, this method should always be called with the same id method object, otherwise the result is undefined.

Parameters:

dc - The DbControl to use for database access

identifier - The identifier protocol

Returns:

A labeled extract, or null if no item could be found

findBioPlate

protected BioPlate findBioPlate(DbControl dc,
                                IdMethod idMethod,
                                String identifier)

Find a bioplate with a given identifier. This is a utility method that subclasses can use when creating or updating items.

NOTE! The first time this method is called a query object is initialised using the IdMethod.prepareQuery(DbControl, ItemQuery) method. Subsequent calls uses the same query. Thus, this method should always be called with the same id method object, otherwise the result is undefined.

Parameters:

dc - The DbControl to use for database access

identifier - The identifier protocol

Returns:

A bioplate, or null if no item could be found

findBioWell

protected BioWell findBioWell(DbControl dc,
                              BioPlate plate,
                              String row,
                              String column)

Find a well on a bioplate. Null is allowed for all parameters but will always result in a null return value.

Parameters:

dc - The DbControl to use for database access

plate - The plate to look on

row - The row coordinate in alphabetic or 1-based values

column - The column coordinate in alphabetic or 1-based values

Returns:

A biowell, or null if no item could be found

Since:

2.16

findArraySlide

protected ArraySlide findArraySlide(DbControl dc,
                                    IdMethod idMethod,
                                    String identifier)

Find an array slide with a given identifier. This is a utility method that subclasses can use when creating or updating items.

NOTE! The first time this method is called a query object is initialised using the IdMethod.prepareQuery(DbControl, ItemQuery) method. Subsequent calls uses the same query. Thus, this method should always be called with the same id method object, otherwise the result is undefined.

Parameters:

dc - The DbControl to use for database access

identifier - The identifier protocol

Returns:

An array slide, or null if no item could be found

findHybridization

protected Hybridization findHybridization(DbControl dc,
                                          IdMethod idMethod,
                                          String identifier)

Find a hybridization with a given identifier. This is a utility method that subclasses can use when creating or updating items.

NOTE! The first time this method is called a query object is initialised using the IdMethod.prepareQuery(DbControl, ItemQuery) method. Subsequent calls uses the same query. Thus, this method should always be called with the same id method object, otherwise the result is undefined.

Parameters:

dc - The DbControl to use for database access

identifier - The identifier protocol

Returns:

A hybridization, or null if no item could be found

findScan

protected Scan findScan(DbControl dc,
                        IdMethod idMethod,
                        String identifier)

Find a scan with a given identifier. This is a utility method that subclasses can use when creating or updating items.

NOTE! The first time this method is called a query object is initialised using the IdMethod.prepareQuery(DbControl, ItemQuery) method. Subsequent calls uses the same query. Thus, this method should always be called with the same id method object, otherwise the result is undefined.

Parameters:

dc - The DbControl to use for database access

identifier - The identifier protocol

Returns:

A scan, or null if no item could be found

findImage

protected Image findImage(DbControl dc,
                          Scan scan,
                          File imageFile)

Find an image with given scan and file. This is a utility method that subclasses can use when creating or updating items. New image will be created if given scan hasn't been saved to the database or if no image could be found for the given file. Null-value is returned if more then one image were found and failIfMultipleFoundReferences is set to FALSE.

Parameters:

dc - The DbControl to use for database access

scan - The Scan for which the image should belong to.

imageFile - The File, which the image is connected to.

Returns:

An image, or null.

Since:

2.12

findArrayDesign

protected ArrayDesign findArrayDesign(DbControl dc,
                                      IdMethod idMethod,
                                      String identifier)

Find an array design with a given identifier. This is a utility method that subclasses can use when creating or updating items.

NOTE! The first time this method is called a query object is initialised using the IdMethod.prepareQuery(DbControl, ItemQuery) method. Subsequent calls uses the same query. Thus, this method should always be called with the same id method object, otherwise the result is undefined.

Parameters:

dc - The DbControl to use for database access

identifier - The identifier protocol

Returns:

An array design, or null if no item could be found

findArrayBatch

protected ArrayBatch findArrayBatch(DbControl dc,
                                    IdMethod idMethod,
                                    String identifier)

Find an array batch with a given identifier. This is a utility method that subclasses can use when creating or updating items.

NOTE! The first time this method is called a query object is initialised using the IdMethod.prepareQuery(DbControl, ItemQuery) method. Subsequent calls uses the same query. Thus, this method should always be called with the same id method object, otherwise the result is undefined.

Parameters:

dc - The DbControl to use for database access

identifier - The identifier protocol

Returns:

An array batch, or null if no item could be found

findPlateGeometry

protected PlateGeometry findPlateGeometry(DbControl dc,
                                          IdMethod idMethod,
                                          String identifier)

Find a plate geometry with a given identifier. This is a utility method that subclasses can use when creating or updating items.

NOTE! The first time this method is called a query object is initialised using the IdMethod.prepareQuery(DbControl, ItemQuery) method. Subsequent calls uses the same query. Thus, this method should always be called with the same id method object, otherwise the result is undefined.

Parameters:

dc - The DbControl to use for database access

identifier - The identifier protocol

Returns:

A plate geometry, or null if no item could be found

Since:

2.16

findBioPlateType

protected BioPlateType findBioPlateType(DbControl dc,
                                        IdMethod idMethod,
                                        String identifier)

Find a bioplate type with a given identifier. This is a utility method that subclasses can use when creating or updating items.

NOTE! The first time this method is called a query object is initialised using the IdMethod.prepareQuery(DbControl, ItemQuery) method. Subsequent calls uses the same query. Thus, this method should always be called with the same id method object, otherwise the result is undefined.

Parameters:

dc - The DbControl to use for database access

identifier - The identifier protocol

Returns:

A bioplate type, or null if no item could be found

Since:

2.16

findRawDataType

protected RawDataType findRawDataType(DbControl dc,
                                      Platform platform,
                                      PlatformVariant variant,
                                      String identifier)

Find a matching raw data type. This method will look in the following order:

1. The raw data type locked to the platform variant: PlatformVariant.getRawDataType()
2. The raw data type locked to the platform: Platform.getRawDataType()
3. Any registered raw data type with the given 'id': RawDataTypes.getRawDataType(String)
4. Any registered raw data type with the given 'name': RawDataTypes.getRawDataTypes()
5. The project default raw data type: getProjectDefaultRawDataType(DbControl)

If a raw data type is found by a check, the remaining checks are ignored. If no item is found or if multiple items are found (in the 'name' lookup) the settings (failIfMultipleFoundReferences and failIfNotFoundReference) determine if an exception is thrown or not.

Parameters:

platform - The current platform or null to not use a platform for lookup

variant - The current platform variant or null to not use a variant for lookup

identifier - The 'id' or 'name' of a raw data type, or null to not use this for lookup

Returns:

A raw data type or null if not found

Since:

2.15

initReferenceQuery

protected <T extends BasicItem> ItemQuery<T> initReferenceQuery(DbControl dc,
                                                                IdMethod idMethod,
                                                                ItemQuery<T> query)

Initialise a query that is used to find referenced items. This method delegates to IdMethod.prepareQuery(DbControl, ItemQuery) and then adds Include options: MINE, IN_PROJECT, SHARED and OTHERS.

Parameters:

dc - A DbControl used for database access

idMethod - The identification method to use

query - The query to initialised

Returns:

The initialised query

findReferencedItem

protected <T extends BasicItem> T findReferencedItem(DbControl dc,
                                                     IdMethod idMethod,
                                                     ItemQuery<T> query,
                                                     String identifier,
                                                     boolean ignoreNotFound)

Find a referenced item. This method will first look in the itemCache. If no item is found in the cache it uses the query and id method (IdMethod.find(DbControl, ItemQuery, String)) to look for the item. The result of the lookup is stored in the cache (even if no item is found).

Parameters:

dc - A DbControl used for database access

idMethod - The identification method

query -

identifier -

ignoreNotFound - TRUE to ignore not found errors

Returns:

The item, or null if none is found

Throws:

ItemNotFoundException - If no item is found and the error handling is set to fail for this condition

BaseException - If multiple items are found and the error handling is set to fail for this condition

getPattern

private Pattern getPattern(ParameterValues parameters,
                           String name)
                    throws BaseException

Throws:

BaseException

getInitializedFlatFileParser

protected FlatFileParser getInitializedFlatFileParser(ParameterValues parameters)
                                               throws BaseException

Throws:

BaseException

getConfigureParserParameters

protected RequestInformation getConfigureParserParameters(GuiContext context,
                                                          boolean forJob)

getConfigureOptionsParameters

protected RequestInformation getConfigureOptionsParameters(GuiContext context)