The method is called to create a FlatFileParser and set the regular expressions that should be used for parsing the file. The default implementation assumes that your plug-in has used the built-in PluginParameter objects and has stored the values at the configuration level. You should override this method if you need to initialise the parser in a different way. See for example the code for the PrintMapFlatFileImporter plug-in which has a fixed format and doesn't use configurations.

@Override
protected FlatFileParser getInitializedFlatFileParser()
   throws BaseException
{
   FlatFileParser ffp = new FlatFileParser();
   ffp.setSectionRegexp(Pattern.compile("\\[(.+)\\]"));
   ffp.setHeaderRegexp(Pattern.compile("(.+)=,(.*)"));
   ffp.setDataSplitterRegexp(Pattern.compile(","));
   ffp.setDataFooterRegexp(Pattern.compile(""));
   ffp.setMinDataColumns(12);
   return ffp;
}

This method is called from the isImportable(InputStream) method, AFTER FlatFileParser.nextSection() and FlatFileParser.parseHeaders() has been called a single time and if the parseHeaders method didn't stop on an unknown line. The default implementation of this method always returns TRUE, since obviously some data has been found. A subclass may override this method if it wants to do more checks, for example, make that a certain header is present with a certain value. It may also continue parsing the file. Here is a code example from the PrintMapFlatFileImporter which checks if a FormatName header is present and contains either TAM or MwBr.

/**
   Check that the file is a TAM or MwBr file.
   @return TRUE if a FormatName header is present and contains "TAM" or "MwBr", FALSE
      otherwise
*/
@Override
protected boolean isImportable(FlatFileParser ffp)
{
   String formatName = ffp.getHeader("FormatName");
   return formatName != null && 
      (formatName.contains("TAM") || formatName.contains("MwBr"));
}

This method is called just before the parsing of the file begins. Override this method if you need to initialise some internal state. This is, for example, a good place to open a DbControl object, read parameters from the job and configuration and put them into more useful variables. The default implementation does nothing, but we recommend that super.begin() is always called.

// Snippets from the RawDataFlatFileImporter class
private DbControl dc;
private RawDataBatcher batcher;
private RawBioAssay rawBioAssay;
private Map<String, String> columnMappings;
private int numInserted;

@Override
protected void begin()
   throws BaseException
{
   super.begin();

   // Get DbControl
   dc = sc.newDbControl();
   rawBioAssay = (RawBioAssay)job.getValue(rawBioAssayParameter.getName());

   // Reload raw bioassay using current DbControl
   rawBioAssay = RawBioAssay.getById(dc, rawBioAssay.getId());
   
   // Create a batcher for inserting spots
   batcher = rawBioAssay.getRawDataBatcher();

   // For progress reporting
   numInserted = 0;
}					

This method is called once for every header line that is found in the file. The line parameter contains information about the header. The default implementation of this method does nothing.

@Override
protected void handleHeader(Line line) 
   throws BaseException
{
   super.handleHeader(line);
   if (line.name() != null && line.value() != null)
   {
      rawBioAssay.setHeader(line.name(), line.value());
   }
}

This method is called once for each section that is found in the file. The line parameter contains information about the section. The default implementation of this method does nothing.

This method is called after the headers has been parsed, but before the first line of data. This is a good place to add code that depends on information in the headers, for example, put together column mappings.

private Mapper reporterMapper;
private Mapper blockMapper;
private Mapper columnMapper;
private Mapper rowMapper;
// ... more mappers

@Override
protected void beginData()
{
   boolean cropStrings = ("crop".equals(job.getValue("stringTooLongError")));

   // Mapper that always return null; used if no mapping expression has been entered
   Mapper nullMapper = new ConstantMapper((String)null);
   
   // Column mappers
   reporterMapper = getMapper(ffp, (String)configuration.getValue("reporterIdColumnMapping"), 
      cropStrings ? ReporterData.MAX_EXTERNAL_ID_LENGTH : null, nullMapper);
   blockMapper = getMapper(ffp, (String)configuration.getValue("blockColumnMapping"), 
      null, nullMapper);
   columnMapper = getMapper(ffp, (String)configuration.getValue("columnColumnMapping"), 
      null, nullMapper);
   rowMapper = getMapper(ffp, (String)configuration.getValue("rowColumnMapping"), 
      null, nullMapper);
   // ... more mappers: metaGrid coordinate, X-Y coordinate, extended properties
   // ...
}

This method is abstract and must be implemented by all subclasses. It is called once for every data line in the the file.

// Snippets from the RawDataFlatFileImporter class
@Override
protected void handleData(Data data)
   throws BaseException
{
   // Create new RawData object
   RawData raw = batcher.newRawData();

   // External ID for the reporter
   String externalId = reporterMapper.getValue(data);
   
   // Block, row and column numbers
   raw.setBlock(blockMapper.getInt(data));
   raw.setColumn(columnMapper.getInt(data));
   raw.setRow(rowMapper.getInt(data));
   // ... more: metaGrid coordinate, X-Y coordinate, extended properties
   
   // Insert raw data to the database
   batcher.insert(raw, externalId);
   numInserted++;
}

Called when the parsing has ended, either because the end of file was reached or because an error has occurred. The subclass should close any open resources, ie. the DbControl object. The success parameter is true if the parsing was successful, false otherwise. The default implementation does nothing.

@Override
protected void end(boolean success)
   throws BaseException
{
   try
   {
      // Commit if the parsing was successful
      if (success)
      {
         batcher.close();
         dc.commit();
      }
   }
   catch (BaseException ex)
   {
      // Well, now we got an exception
      success = false;
      throw ex;
   }
   finally
   {
      // Always close... and call super.end()
      if (dc != null) dc.close();
      super.end(success);
   }
}			

This is the last method that is called, and it is only called if everything went suceessfully. This method allows a subclass to generate a short message that is sent back to the database as a final progress report. The default implementation returns null, which means that no message will be generated.

@Override
protected String getSuccessMessage()
{
   return numInserted + " spots inserted";
}