Package net.sf.basedb.plugins.batchimport

Class AnyToAnyImporter

java.lang.Object

net.sf.basedb.core.plugin.AbstractPlugin

net.sf.basedb.plugins.AbstractFlatFileImporter

net.sf.basedb.plugins.batchimport.AnyToAnyImporter

All Implemented Interfaces:

AutoDetectingImporter, InteractivePlugin, Plugin, SignalTarget, ConfigureByExample

public class AnyToAnyImporter
extends AbstractFlatFileImporter
implements InteractivePlugin, ConfigureByExample

Plug-in for importing any-to-any links from simple text files. The plug-in supports all files that can be parsed with the FlatFileParser class. This plug-in works without a configuration but can use a configuration to store regular expressions and other settings for the flat file parser. In both cases, the job configuration is a two-step process:

1. Setup regular expressions and other options for the flat file parser. If a configuration is used all values should already be filled in. In this step a file to import from must also be selected.
2. Setup error handling options and other settings for the plug-in.

Version:

3.17

Author:

nicklas

Nested Class Summary

Nested classes/interfaces inherited from interface net.sf.basedb.core.plugin.Plugin

Plugin.MainType

Field Summary

Fields

Modifier and Type	Field	Description
private Project	activeProject
private static final String	CONFIGURE_OPTIONS
private RequestInformation	configureOptions
private RequestInformation	configureParser
private boolean	createMode
private DbControl	dc
private Mapper	descriptionMapper
private boolean	dryRun
private boolean	failIfMultipleFoundTargets
private boolean	failIfMultipleSourceItems
private boolean	failIfNotFoundSource
private boolean	failIfNotFoundTarget
private boolean	failIfNoUsePermissionToTarget
private boolean	failIfNoWritePermissionToSource
private FlatFileParser	ffp
private static Set<GuiContext>	guiContexts
private static final PluginParameter<String>	idColumnMapping
private static final PluginParameter<String>	identificationSection
private Mapper	idMapper
private IdMethod	idMethod
private Set<Include>	includes
private ItemList	itemList
private static final PluginParameter<String>	itemNotFoundErrorParameter
private ItemQuery<?>	itemQuery
private Item	itemType
protected static final PluginParameter<String>	linkDescriptionColumnMapping
private static final PluginParameter<String>	linkNameColumnMapping
private Mapper	linkNameMapper
private static final PluginParameter<String>	multipleItemsFoundErrorParameter
private static final PluginParameter<String>	multipleTargetsFoundErrorParameter
private static final PluginParameter<String>	noUsePermissionToTargetErrorParameter
private static final PluginParameter<String>	noWritePermissionToItemErrorParameter
private int	numCreated
private int	numRemoved
private int	numSkipped
private int	numUpdated
private static final BooleanParameterType	optionalBooleanType
private static final StringParameterType	optionalColumnMapping
private boolean	removeMode
private static final StringParameterType	requiredColumnMapping
private Mapper	targetIdMapper
private static final PluginParameter<String>	targetItemIdColumnMapping
private Map<Item,ItemQuery<?>>	targetItemQueries
private static final PluginParameter<String>	targetItemTypeColumnMapping
private static final PluginParameter<String>	targetNotFoundErrorParameter
private Mapper	targetTypeMapper
private boolean	updateMode

Fields inherited from class net.sf.basedb.plugins.AbstractFlatFileImporter

complexMappings, dataFooterRegexpParameter, dataHeaderRegexpParameter, dataSplitterRegexpParameter, defaultErrorParameter, errorSection, excelSheetParameter, fileParameter, fileType, headerRegexpParameter, ignoreRegexpParameter, invalidUseOfNullErrorParameter, mappingSection, maxDataColumnsParameter, minDataColumnsParameter, numberFormatErrorParameter, numberOutOfRangeErrorParameter, numDataColumnsType, optionalRegexpType, parserSection, requiredRegexpType, sectionRegexpParameter, stringTooLongErrorParameter, trimQuotesParameter

Fields inherited from class net.sf.basedb.core.plugin.AbstractPlugin

annotationSection, configuration, COPY_ANNOTATIONS, job, OVERWRITE_ANNOTATIONS, sc

Constructor Summary

Constructors

Constructor	Description
AnyToAnyImporter()

Method Summary

Modifier and Type	Method	Description
protected void	begin(FlatFileParser ffp)	Setup error handling and pre-load some of the configuration options.
protected void	beginData()	Setup column mapping.
void	configure(GuiContext context, Request request, Response response)	Configure the plugin.
private ItemQuery<?>	createQuery(Item itemType, ItemList list)	Create a query that return items of the specified type.
protected void	end(boolean success)	Called just after the last line of the file has been parsed, or immediately after an error has ocurred.
private BasicItem	findTargetItem(String targetId, String targetType, FlatFileParser.Data data)
protected RequestInformation	getConfigureOptionsParameters(GuiContext context)
private RequestInformation	getConfigureParserParameters(GuiContext context, boolean forJob)
Set<GuiContext>	getGuiContexts()	This plug-in works in list context of all Nameable items, except bioassay sets, bioassays and wells because they are not standalone items.
protected IdMethod	getIdMethod(DbControl dc, Item item, String method)
private List<IdMethod>	getIdMethods(DbControl dc, Item item)
private ItemList	getItemListFromContext(DbControl dc, GuiContext context)	If the current context is the "Members" tab for an item list we load the item list and will later require that all items to be linked are members of that list.
RequestInformation	getRequestInformation(GuiContext context, String command)	This method will return the RequestInformation for a given command, i.e.
protected String	getSuccessMessage(int skippedLines)	Called if the parsing was successful to let the subclass generate a simple message that is sent back to the core and user interface.
protected void	handleData(FlatFileParser.Data data)	Called by the parser for every line in the file that is a data line.
private boolean	hasExternalId(Item item)	Check if the current item has an 'externalId' property.
String	isInContext(GuiContext context, Object item)	Check if the plugin can be used on the specified object.
boolean	requiresConfiguration()	Return TRUE, since the implementation requires it for finding the regular expressions used by the FlatFileParser.
boolean	supportsConfigurations()	Returns TRUE, since that is how the plugins used to work before this method was introduced.

Methods inherited from class net.sf.basedb.plugins.AbstractFlatFileImporter

addErrorHandler, checkColumnMapping, checkColumnMapping, continueWithNextFileAfterError, doImport, finish, getCharset, getCharset, getDateFormatter, getDecimalSeparator, getErrorHandler, getErrorOption, getFileIterator, getInitializedFlatFileParser, getInitializedFlatFileParser, getMainType, getMapper, getMapper, getNumberFormat, getNumBytes, getPattern, getPattern, getProgress, getSignalHandler, getTimestampFormatter, getTotalFileSize, handleHeader, handleSection, isImportable, isImportable, log, log, log, log, run, setUpErrorHandling, start, wrapInputStream

Methods inherited from class net.sf.basedb.core.plugin.AbstractPlugin

cloneParameterWithDefaultValue, closeLogFile, createLogFile, done, getCopyAnnotationsParmeter, getCurrentConfiguration, getCurrentJob, getJobOrConfigurationValue, getOverwriteAnnotationsParameters, getPermissions, init, isLogging, log, log, storeValue, storeValue, storeValues, validateRequestParameters

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface net.sf.basedb.core.plugin.Plugin

done, getMainType, getPermissions, init, run

Field Details

guiContexts

private static volatile Set<GuiContext> guiContexts

CONFIGURE_OPTIONS

private static final String CONFIGURE_OPTIONS

See Also:

• Constant Field Values

optionalColumnMapping

private static final StringParameterType optionalColumnMapping

requiredColumnMapping

private static final StringParameterType requiredColumnMapping

optionalBooleanType

private static final BooleanParameterType optionalBooleanType

identificationSection

private static final PluginParameter<String> identificationSection

idColumnMapping

private static final PluginParameter<String> idColumnMapping

linkNameColumnMapping

private static final PluginParameter<String> linkNameColumnMapping

targetItemTypeColumnMapping

private static final PluginParameter<String> targetItemTypeColumnMapping

targetItemIdColumnMapping

private static final PluginParameter<String> targetItemIdColumnMapping

linkDescriptionColumnMapping

protected static final PluginParameter<String> linkDescriptionColumnMapping

itemNotFoundErrorParameter

private static final PluginParameter<String> itemNotFoundErrorParameter

multipleItemsFoundErrorParameter

private static final PluginParameter<String> multipleItemsFoundErrorParameter

targetNotFoundErrorParameter

private static final PluginParameter<String> targetNotFoundErrorParameter

multipleTargetsFoundErrorParameter

private static final PluginParameter<String> multipleTargetsFoundErrorParameter

noWritePermissionToItemErrorParameter

private static final PluginParameter<String> noWritePermissionToItemErrorParameter

noUsePermissionToTargetErrorParameter

private static final PluginParameter<String> noUsePermissionToTargetErrorParameter

dc

private DbControl dc

activeProject

private Project activeProject

ffp

private FlatFileParser ffp

itemType

private Item itemType

idMethod

private IdMethod idMethod

createMode

private boolean createMode

updateMode

private boolean updateMode

removeMode

private boolean removeMode

includes

private Set<Include> includes

itemQuery

private ItemQuery<?> itemQuery

itemList

private ItemList itemList

targetItemQueries

private Map<Item,ItemQuery<?>> targetItemQueries

idMapper

private Mapper idMapper

linkNameMapper

private Mapper linkNameMapper

targetTypeMapper

private Mapper targetTypeMapper

targetIdMapper

private Mapper targetIdMapper

descriptionMapper

private Mapper descriptionMapper

dryRun

private boolean dryRun

failIfNotFoundSource

private boolean failIfNotFoundSource

failIfMultipleSourceItems

private boolean failIfMultipleSourceItems

failIfNotFoundTarget

private boolean failIfNotFoundTarget

failIfMultipleFoundTargets

private boolean failIfMultipleFoundTargets

failIfNoWritePermissionToSource

private boolean failIfNoWritePermissionToSource

failIfNoUsePermissionToTarget

private boolean failIfNoUsePermissionToTarget

numCreated

private int numCreated

numUpdated

private int numUpdated

numRemoved

private int numRemoved

numSkipped

private int numSkipped

configureParser

private RequestInformation configureParser

configureOptions

private RequestInformation configureOptions

Constructor Details

AnyToAnyImporter

public AnyToAnyImporter()

Method Details

requiresConfiguration

public boolean requiresConfiguration()

Description copied from class: AbstractFlatFileImporter

Return TRUE, since the implementation requires it for finding the regular expressions used by the FlatFileParser. If this method is overridden and returns FALSE, the subclass must also override the AbstractFlatFileImporter.getInitializedFlatFileParser() method and provide a parser with all regular expressions and other options set.

Specified by:

requiresConfiguration in interface Plugin

Overrides:

requiresConfiguration in class AbstractFlatFileImporter

Returns:

TRUE or FALSE

supportsConfigurations

public boolean supportsConfigurations()

Description copied from class: AbstractPlugin

Returns TRUE, since that is how the plugins used to work before this method was introduced.

Specified by:

supportsConfigurations in interface Plugin

Overrides:

supportsConfigurations in class AbstractPlugin

Returns:

TRUE or FALSE

getGuiContexts

public Set<GuiContext> getGuiContexts()

This plug-in works in list context of all Nameable items, except bioassay sets, bioassays and wells because they are not standalone items. We use Metadata.getAnnotatableItems() to create the contexts.

Specified by:

getGuiContexts in interface InteractivePlugin

Returns:

A Set containing Item:s, or null if the plugin is not concerned about items

isInContext

public String isInContext(GuiContext context,
 Object item)

Description copied from interface: InteractivePlugin

Check if the plugin can be used on the specified object. The item is either a BasicItem or BasicData object, or null if called from a list context. A client application should only call this method for contexts matching one of the contexts in the set returned by the InteractivePlugin.getGuiContexts() method. For GuiContext.Type.ITEM contexts an item of the correct type should be passed as a parameter.

The plug-in can response with a null value indicating success, a warning-level message or throw an exception to indicate an error-level message. Client application should normally only show error-level messages to users, unless the user has requested to see warning messages.

NOTE! Before BASE 2.4 all messages were treated as error-level messages.

Specified by:

isInContext in interface InteractivePlugin

Parameters:

context - The current context of the client application, it is one of the values found in set returned by InteractivePlugin.getGuiContexts()

item - The currently active item, it's type should match the GuiContext.getItem() type, or null if the context is a list context

Returns:

Null if the plugin can use that item, or a warning-level message explaining why the plugin can't be used

getRequestInformation

public RequestInformation getRequestInformation(GuiContext context,
 String command)
                                         throws BaseException

Description copied from interface: InteractivePlugin

This method will return the RequestInformation for a given command, i.e. the list of parameters and some nice help text.

Specified by:

getRequestInformation in interface InteractivePlugin

Parameters:

context - The current context of the client application, it is one of the values found in set returned by InteractivePlugin.getGuiContexts()

command - The command

Returns:

The RequestInformation for the command

Throws:

BaseException - if there is an error

configure

public void configure(GuiContext context,
 Request request,
 Response response)

Description copied from interface: InteractivePlugin

Configure the plugin. Hopefully the client is supplying values for the parameters specified by InteractivePlugin.getRequestInformation(GuiContext, String).

Specified by:

configure in interface InteractivePlugin

Parameters:

context - The current context of the client application, it is one of the values found in set returned by InteractivePlugin.getGuiContexts()

request - Request object with the command and parameters

response - Response object in for the plugin to response through

begin

protected void begin(FlatFileParser ffp)

Setup error handling and pre-load some of the configuration options.

Overrides:

begin in class AbstractFlatFileImporter

See Also:

• AbstractFlatFileImporter.end(boolean)

beginData

protected void beginData()

Setup column mapping. Creates DbControl and query to find items.

Overrides:

beginData in class AbstractFlatFileImporter

handleData

protected void handleData(FlatFileParser.Data data)

Description copied from class: AbstractFlatFileImporter

Called by the parser for every line in the file that is a data line.

Specified by:

handleData in class AbstractFlatFileImporter

findTargetItem

private BasicItem findTargetItem(String targetId,
 String targetType,
 FlatFileParser.Data data)

end

protected void end(boolean success)

Description copied from class: AbstractFlatFileImporter

Called just after the last line of the file has been parsed, or immediately after an error has ocurred. A subclass should clean up any resources aquired in the AbstractFlatFileImporter.begin(FlatFileParser) method here. Note that this metod is called once for every file returned by the AbstractFlatFileImporter.getFileIterator() iterator.

Overrides:

end in class AbstractFlatFileImporter

Parameters:

success - TRUE if the file was parsed successfully, FALSE otherwise

See Also:

• AbstractFlatFileImporter.begin(FlatFileParser)

getSuccessMessage

protected String getSuccessMessage(int skippedLines)

Description copied from class: AbstractFlatFileImporter

Called if the parsing was successful to let the subclass generate a simple message that is sent back to the core and user interface. An example message might by: 178 reporters imported successfully. The default implementation always return null. Note that this method is called once for every file returned by AbstractFlatFileImporter.getFileIterator().

Overrides:

getSuccessMessage in class AbstractFlatFileImporter

Parameters:

skippedLines - The number of data lines that were skipped due to errors

getConfigureParserParameters

private RequestInformation getConfigureParserParameters(GuiContext context,
 boolean forJob)

getConfigureOptionsParameters

protected RequestInformation getConfigureOptionsParameters(GuiContext context)

getIdMethods

private List<IdMethod> getIdMethods(DbControl dc,
 Item item)

hasExternalId

private boolean hasExternalId(Item item)

Check if the current item has an 'externalId' property. We use reflection to look for the 'getExternalId' method in the item's data class.

getIdMethod

protected IdMethod getIdMethod(DbControl dc,
 Item item,
 String method)

createQuery

private ItemQuery<?> createQuery(Item itemType,
 ItemList list)

Create a query that return items of the specified type. We use reflection to call the static method 'getQuery' on the item's item class, for example Sample.getQuery().

Parameters:

itemType - The type of items to search

Returns:

A query that searches items of the specified type.

getItemListFromContext

private ItemList getItemListFromContext(DbControl dc,
 GuiContext context)

If the current context is the "Members" tab for an item list we load the item list and will later require that all items to be linked are members of that list.