26.6. Other plug-ins

26.6.1. Secondary file storage plugins
Primary vs. secondary storage
The SecondaryStorageController interface
Configuration settings
26.6.2. File unpacker plug-ins
26.6.3. File packer plug-ins

26.6.1. Secondary file storage plugins

[Note] Note
This feature has been deprecated in BASE 3.14 and will be removed in a future release. The recommendation is to replace this with an external files solution instead. See Section 27.8.8, “Connection managers” for more information.

Primary vs. secondary storage

BASE has support for storing files in two locations, the primary storage and the secondary storage. The primary storage is always disk-based and must be accessible by the BASE server as a path on the file system. The path to the primary storage is configured by the userfiles setting in the base.config file. The primary storage is internal to the core. Client applications don't get access to read or manipulate the files directly from the file system.

The secondary storage can be anything that can store files. It could, for example, be another directory, a remote FTP server, or a tape based archiving system. A file located in the secondary storage is not accessible by the core, client applications or plug-ins. The secondary storage can only be accessed by the secondary storage controller. The core (and client) applications uses flags on the file items to handle the interaction with the secondary storage.

Each file has an action attribute which default's to File.Action.NOTHING. It can take two other values:

  1. File.Action.MOVE_TO_SECONDARY

  2. File.Action.MOVE_TO_PRIMARY

All files with the action attribute set to MOVE_TO_SECONDARY should be moved to the secondary storage by the controller, and all files with the action attribute set to MOVE_TO_PRIMARY should be brought back to primary storage.

The moving of files between primary and secondary storage doesn't happen immediately. It is up to the server administrator to configure how often and at what times the controller should check for files that should be moved. This is configured by the secondary.storage.interval and secondary.storage.time settings in the base.config file.

The SecondaryStorageController interface

All you have to do to create a secondary storage controller is to create a class that implements the net.sf.basedb.core.SecondaryStorageController interface. In your base.config file you then specify the class name in the secondary.storage.driver setting and its initialisation parameters in the secondary.storage.init setting.

Your class must have a public no-argument constructor. The BASE application will create only one instance of the class for lifetime of the BASE server. Here are the methods that you must implement:

public void init(String settings);

This method is called just after the object has been created with its argument taken from the secondary.storage.init setting in your base.config file. This method is only called once for an object.

public void run();

This method is called whenever the core thinks it is time to do some management of the secondary storage. How often the run() method is called is controlled by the secondary.storage.interval and secondary.storage.time settings in the base.config file. When this method is called the controller should:

  • Move all files which has action=MOVE_TO_SECONDARY to the secondary storage. When the file has been moved call File.setLocation(Location.SECONDARY) to tell the core that the file is now in the secondary storage. You should also call File.setAction(File.Action.NOTHING) to reset the action attribute.

  • Restore all files which has action=MOVE_TO_PRIMARY. The core will set the location attribute automatically, but you should call File.setAction(File.Action.NOTHING) to reset the action attribute.

  • Delete all files from the secondary storage that are not present in the database with location=Location.SECONDARY. This includes files which has been deleted and files that have been moved offline or re-uploaded.

As a final act the method should send a message to each user owning files that has been moved from one location to the other. The message should include a list of files that has been moved to the secondary storage and a list of files moved from the secondary storage and a list of files that has been deleted due to some of the reasons above.

public void close()();

This method is called when the server is closing down. After this the object is never used again.

Configuration settings

The configuration settings for the secondary storage controller is located in the base.config file. Here is an overview of the settings. For more information read Appendix B, base.config reference.


The class name of the secondary storage plug-in.


Initialisation parameters sent to the plug-in by calling the init() method.


Interval in seconds between each execution of the secondary storage controller plug-in.


Time points during the day when the secondary storage controller plugin should be executed.

26.6.2. File unpacker plug-ins

The BASE web client has integrated support for unpacking of compressed files. See Section 7.2.1, “Upload a new file”. Behind the scenes, this support is provided by plug-ins. The standard BASE distribution comes with support for ZIP files (net.sf.basedb.plugins.ZipFileUnpacker) and TAR files (net.sf.basedb.plugins.TarFileUnpacker).

To add support for additional compressed formats you have to create a plug-in that implements the net.sf.basedb.util.zip.FileUnpacker interface. The best way to do this is to extend the net.sf.basedb.util.zip.AbstractFileUnpacker which implements all methods in the Plugin and InteractivePlugin interfaces. This leaves you with the actual unpacking of the files as the only thing to implement.

[Note] No support for configurations
The integrated upload in the web interface only works with plug-ins that does not require a configuration to run.

Methods in the FileUnpacker interface

public String getFormatName();

Return a short string naming the file format. For example: ZIP files or TAR files.

public Set<String> getExtensions();

Return a set of strings with the file extensions that are most commonly used with the compressed file format. For example: [zip, jar]. Do not include the dot in the extensions. The web client and the AbstractFlatFileUnpacker.isInContext() method will use this information to automatically guess which plug-in to use for unpacking the files.

public Set<String> getMimeTypes();

Return a set of string with the MIME types that commonly used with the compressed file format. For example: [application/zip, application/java-archive]. This information is used by the AbstractFlatFileUnpacker.isInContext() method to automatically guess which plug-in to use for unpacking the files.

public int unpack(DbControl dc,
                  Directory dir,
                  InputStream in,
                  File sourceFile,
                  boolean overwrite,
                  AbsoluteProgressReporter progress)
    throws IOException, BaseException;

Unpack the files and store them in the BASE file system.

  • Do not close() or commit() the DbControl passed to this method. This is done automatically by the AbstractFileUnpacker or by the web client.

  • The dir parameter is the root directory where the unpacked files should be placed. If the compressed file contains subdirectories the plug-in must create those subdirectories unless they already exists.

  • If the overwrite parameter is FALSE no existing file should be overwritten unless the file is OFFLINE or marked as removed (do not forget to clear the removed attribute).

  • The in parameter is the stream containing the compressed data. The stream may come directly from the web upload or from an existing file in the BASE file system.

  • The sourceFile parameter is the file item representing the compressed file. This item may already be in the database, or a new item that may or may not be saved in the database at the end of the transaction. The information in this parameter can be used to discover the options for file type, character set, MIME type, etc. that was selected by the user in the upload dialog. The PackUtil has a useful method that can be used for copying information.

  • The progress parameter, if not null, should be used to report the progress back to the calling code. The plug-in should count the number of bytes read from the in stream. If it is not possible by other means the stream can be wrapped by a net.sf.basedb.util.InputStreamTracker object which has a getNumRead() method.

When the compressed file is uncompressed during the file upload from the web interface, the call sequence to the plug-in is slightly altered from the standard call sequence described in the section called “Executing a job”.

  • After the plug-in instance has been created, the Plugin.init() method is called with null values for both the configuration and job parameters.

  • Then, the unpack() method is called. The Plugin.run() method is never called in this case.

26.6.3. File packer plug-ins

BASE has support for compressing and downloading a set of selected files and/or directories. This functionality is provided by a plug-in, the PackedFileExporter. This plug-in doesn't do the actual packing itself. This is delegated to classes implementing the net.sf.basedb.util.zip.FilePacker interface.

BASE ships with a number of packing methods, including ZIP and TAR. To add support for other methods you have to provide an implementation of the FilePacker interface. Then, create a new configuration for the PackedFileExporter and enter the name of your class in the configuration wizard.

The FilePacker interface is not a regular plug-in interface (ie. it is not a subinterface to Plugin). This means that you don't have to mess with configuration or job parameters. Another difference is that your class must be installed in Tomcat's classpath (ie. in one of the WEB-INF/classes or WEB-INF/lib folders).

[Important] This may be converted to an extension point in the future

There are certain plans to convert the packing mechanism to an extension point in the future. The main reason is easier installation since code doesn't have to be installed in the WEB-INF/lib or WEB-INF/classes directory. See ticket #1600: Convert file packing plug-in system to an extension point for more information.

Methods in the FilePacker interface

public String getDescription();

Return a short description the file format that is suitable for use in dropdown lists in client applications. For example: Zip-archive (.zip) or TAR-archive (.tar).

public String getFileExtension();

Return the default file extension of the packed format. The returned value should not include the dot. For example: zip or tar.

public String getMimeType();

Return the standard MIME type of the packed file format. For example: application/zip or application/x-tar.

public void setOutputStream(OutputStream out)
    throws IOException;

Sets the outputstream that the packer should write the packed files to.

public void pack(String entryName,
                 InputStream in,
                 long size,
                 long lastModified)
    throws IOException;

Add another file or directory to the packed file. The entryName is the name of the new entry, including path information. The in is the stream to read the file data from. If in is null then the entry denotes a directory. The size parameter gives the size in bytes of the file (zero for empty files or directories). The lastModified is that time the file was last modified or 0 if not known.

public void close()
    throws IOException;

Finish the packing. The packer should release any resources, flush all data and close all output streams, including the out stream set in the setOutputStream method.