Class FileImporter
- java.lang.Object
-
- net.sf.basedb.core.plugin.AbstractPlugin
-
- net.sf.basedb.plugins.AbstractFlatFileImporter
-
- net.sf.basedb.plugins.batchimport.AbstractItemImporter<File>
-
- net.sf.basedb.plugins.batchimport.FileImporter
-
- All Implemented Interfaces:
AutoDetectingImporter
,InteractivePlugin
,Plugin
,SignalTarget
,ConfigureByExample
public class FileImporter extends AbstractItemImporter<File>
Plug-in for importing file items in a batch. The plug-in can create new items (offline or external) and updated existing items (all).- Since:
- 3.3
- Author:
- nicklas
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
FileImporter.PathIdMethod
-
Nested classes/interfaces inherited from interface net.sf.basedb.core.plugin.Plugin
Plugin.MainType
-
-
Field Summary
Fields Modifier and Type Field Description protected static PluginParameter<String>
characterSetColumnMapping
private Mapper
characterSetMapper
private Mapper
descriptionMapper
protected static PluginParameter<String>
fileServerColumnMapping
private Mapper
fileServerMapper
private static Set<GuiContext>
guiContexts
protected static PluginParameter<String>
mimeTypeColumnMapping
private Mapper
mimeTypeMapper
protected static PluginParameter<String>
pathColumnMapping
private FileImporter.PathIdMethod
pathIdMethod
private Mapper
pathMapper
private Map<String,Directory>
pathToDirectory
private String
rootPath
protected static PluginParameter<String>
urlColumnMapping
private Mapper
urlMapper
-
Fields inherited from class net.sf.basedb.plugins.batchimport.AbstractItemImporter
addMembersMode, createMode, defaultsSection, descriptionColumnMapping, externalIdColumnMapping, idAnnotationColumnMapping, identificationSection, internalIdColumnMapping, kitColumnMapping, nameColumnMapping, optionalColumnMapping, permissionTemplateColumnMapping, protocolColumnMapping, registeredColumnMapping, removeMembersMode, requiredColumnMapping, subtypeColumnMapping, updateMode
-
Fields inherited from class net.sf.basedb.plugins.AbstractFlatFileImporter
complexMappings, dataFooterRegexpParameter, dataHeaderRegexpParameter, dataSplitterRegexpParameter, defaultErrorParameter, errorSection, excelSheetParameter, fileParameter, fileType, headerRegexpParameter, ignoreRegexpParameter, invalidUseOfNullErrorParameter, mappingSection, maxDataColumnsParameter, minDataColumnsParameter, numberFormatErrorParameter, numberOutOfRangeErrorParameter, numDataColumnsType, optionalRegexpType, parserSection, requiredRegexpType, sectionRegexpParameter, stringTooLongErrorParameter, trimQuotesParameter
-
Fields inherited from class net.sf.basedb.core.plugin.AbstractPlugin
annotationSection, configuration, COPY_ANNOTATIONS, job, OVERWRITE_ANNOTATIONS, sc
-
-
Constructor Summary
Constructors Constructor Description FileImporter()
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description protected void
addMoreColumnMappingParameters(List<PluginParameter<?>> parameters)
Adds column mappings for name, externalId and description.protected void
addMoreJobParameters(List<PluginParameter<?>> parameters)
Add extra job parameters that the subclass needs.protected void
createColumnMappers(FlatFileParser ffp, boolean cropStrings)
Create mappers that can be used to extract data values from the file that is currently being parsed by theffp
FlatFileParser
.protected File
createItem(DbControl dc, FlatFileParser.Data data)
Create a new item and populate it with data from the current line.protected ItemQuery<File>
createItemQuery()
CallsFile.getQuery()
.private Directory
getDirectory(DbControl dc, Path path)
Set<GuiContext>
getGuiContexts()
Get a set containing all items that the plugin handles.protected IdMethod
getIdMethod(DbControl dc, String method)
We need to initialize the PathIdMethod when this is called.protected IdMethod[]
getIdMethods()
Useable methods are:FileImporter.PathIdMethod
InternalIdMethod.INTERNAL_ID
protected Item
getItemForSubtypes()
Does the importer want to use item subtypes when importing?private String
getRootPath(DbControl dc)
private static String
resolvePath(String path, String rootPath)
protected void
updateItem(DbControl dc, File file, FlatFileParser.Data data)
Update an item with data from the current line.-
Methods inherited from class net.sf.basedb.plugins.batchimport.AbstractItemImporter
addMoreErrorParameters, addMoreParsingParameters, begin, beginData, configure, createItemQuery, doneWithItem, end, findAnnotationTypeCategory, findArrayBatch, findArrayDesign, findArraySlide, findBioPlate, findBioPlateType, findBioSource, findBioSource, findBioWell, findDataFileType, findDerivedBioAssay, findExtract, findExtract, findFileServer, findHardware, findHardware, findHybridization, findItemSubtype, findItemSubtype, findKit, findLabeledExtract, findPermissionTemplate, findPhysicalBioAssay, findPlateGeometry, findPlatform, findProtocol, findProtocol, findRawDataType, findReferencedItem, findReferencedItemWithSubtype, findSample, findSample, findSoftware, findSoftware, findTag, findTag, findUnit, findVariant, getAnnotationIdMethods, getConfigureOptionsParameters, getConfigureParserParameters, getDefaultItemFunction, getItemForAnnotationTypes, getProjectDefaultPlatform, getProjectDefaultRawDataType, getProjectDefaultVariant, getRequestInformation, getSuccessMessage, handleData, initReferenceQuery, isInContext, parseDate, requiresConfiguration, setOrAddFile, setUpErrorHandling, supportsConfigurations, updateItemSubtype, updateMultiLineItem, updatePermissions
-
Methods inherited from class net.sf.basedb.plugins.AbstractFlatFileImporter
addErrorHandler, checkColumnMapping, checkColumnMapping, continueWithNextFileAfterError, doImport, finish, getCharset, getCharset, getDateFormatter, getDecimalSeparator, getErrorHandler, getErrorOption, getFileIterator, getInitializedFlatFileParser, getInitializedFlatFileParser, getMainType, getMapper, getMapper, getNumberFormat, getNumBytes, getPattern, getPattern, getProgress, getSignalHandler, getTimestampFormatter, getTotalFileSize, handleHeader, handleSection, isImportable, isImportable, log, log, log, log, run, start, wrapInputStream
-
Methods inherited from class net.sf.basedb.core.plugin.AbstractPlugin
cloneParameterWithDefaultValue, closeLogFile, createLogFile, done, getCopyAnnotationsParmeter, getCurrentConfiguration, getCurrentJob, getJobOrConfigurationValue, getOverwriteAnnotationsParameters, getPermissions, init, log, log, storeValue, storeValue, storeValues, validateRequestParameters
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface net.sf.basedb.core.plugin.Plugin
done, getMainType, getPermissions, init, run
-
-
-
-
Field Detail
-
guiContexts
private static final Set<GuiContext> guiContexts
-
pathColumnMapping
protected static final PluginParameter<String> pathColumnMapping
-
fileServerColumnMapping
protected static final PluginParameter<String> fileServerColumnMapping
-
urlColumnMapping
protected static final PluginParameter<String> urlColumnMapping
-
mimeTypeColumnMapping
protected static final PluginParameter<String> mimeTypeColumnMapping
-
characterSetColumnMapping
protected static final PluginParameter<String> characterSetColumnMapping
-
rootPath
private String rootPath
-
pathIdMethod
private FileImporter.PathIdMethod pathIdMethod
-
pathMapper
private Mapper pathMapper
-
fileServerMapper
private Mapper fileServerMapper
-
urlMapper
private Mapper urlMapper
-
mimeTypeMapper
private Mapper mimeTypeMapper
-
characterSetMapper
private Mapper characterSetMapper
-
descriptionMapper
private Mapper descriptionMapper
-
-
Method Detail
-
getGuiContexts
public Set<GuiContext> getGuiContexts()
Description copied from interface:InteractivePlugin
Get a set containing all items that the plugin handles. Ie. if the plugin imports reporters, return a set containingItem.REPORTER
. This information is used by client applications to put the plugin in the proper place in the user interface.- Returns:
- A
Set
containingItem
:s, or null if the plugin is not concerned about items
-
getIdMethods
protected IdMethod[] getIdMethods()
Useable methods are:- Overrides:
getIdMethods
in classAbstractItemImporter<File>
-
getIdMethod
protected IdMethod getIdMethod(DbControl dc, String method)
We need to initialize the PathIdMethod when this is called.- Overrides:
getIdMethod
in classAbstractItemImporter<File>
-
getItemForSubtypes
protected Item getItemForSubtypes()
Description copied from class:AbstractItemImporter
Does the importer want to use item subtypes when importing? If this method returns a non-null, this class will add a parameter in the "Item identification" section that makes it possible to select which subtypes the importer should use to limit the search for existing items. The default implementation returns null.- Overrides:
getItemForSubtypes
in classAbstractItemImporter<File>
- Returns:
- An Item or null
-
addMoreJobParameters
protected void addMoreJobParameters(List<PluginParameter<?>> parameters)
Description copied from class:AbstractItemImporter
Add extra job parameters that the subclass needs. This method is called during configuration of the plug-in and the default implementation does nothing.- Overrides:
addMoreJobParameters
in classAbstractItemImporter<File>
- Parameters:
parameters
- The list of parameters to add the column mapping parameters to
-
addMoreColumnMappingParameters
protected void addMoreColumnMappingParameters(List<PluginParameter<?>> parameters)
Adds column mappings for name, externalId and description.- Overrides:
addMoreColumnMappingParameters
in classAbstractItemImporter<File>
- Parameters:
parameters
- The list of parameters to add the column mapping parameters to
-
createItemQuery
protected ItemQuery<File> createItemQuery()
CallsFile.getQuery()
.- Specified by:
createItemQuery
in classAbstractItemImporter<File>
- Returns:
- A query
-
createColumnMappers
protected void createColumnMappers(FlatFileParser ffp, boolean cropStrings)
Description copied from class:AbstractItemImporter
Create mappers that can be used to extract data values from the file that is currently being parsed by theffp
FlatFileParser
. This method is called during initialisation of the plug-in when theFlatFileParser.parseHeaders()
has successfully found the start of data. Use theAbstractFlatFileImporter.getMapper(FlatFileParser, String, Integer, Mapper)
to create column mappers. Example:nameMapper = getMapper(ffp, job.getValue("nameColumnMapping"), cropStrings ? Nameable.MAX_NAME_LENGTH : null, null);
- Overrides:
createColumnMappers
in classAbstractItemImporter<File>
- Parameters:
ffp
- The parser that is parsing the data filecropStrings
- If strings that are too long to fit in the database should be cropped or not (=generate an error)
-
createItem
protected File createItem(DbControl dc, FlatFileParser.Data data)
Description copied from class:AbstractItemImporter
Create a new item and populate it with data from the current line. Do not callDbControl.saveItem(BasicItem)
. This is done automatically by the core implementation.BioSource bs = BioSource.getNew(dc); // Set properties. See example on updateItem method return bs;
If the current data line doesn't have data for all required properties, this method should throw an
InvalidUseOfNullException
exception.This method is called during import when current item can be found and if the user has selected to create missing item.
- Specified by:
createItem
in classAbstractItemImporter<File>
- Parameters:
dc
- The DbControl to use for database accessdata
- The current data line- Returns:
- A new item
-
updateItem
protected void updateItem(DbControl dc, File file, FlatFileParser.Data data)
Description copied from class:AbstractItemImporter
Update an item with data from the current line. Example:if (nameMapper != null) item.setName(nameMapper.getValue(data)); if (descriptionMapper != null) item.setDescription(descriptionMapper.getValue(data)); ...
This method is called during import to update an existing item.- Specified by:
updateItem
in classAbstractItemImporter<File>
- Parameters:
dc
- The DbControl to use for database accessfile
- The item to updatedata
- The current data line
-
-