File items should be able to reference external files

[Nicklas Nordborg]

If we add a URL field to a file item, it should be possible to link a file item in BASE with a file on the internet. It should be "invisible" to users in the sense that File.getDownloadStream() should act as a proxy for the file. To begin with we should support at least http and https URLs.

We need to investigate how some of the other file properties should be interpreted. For example:

• Location: can be PRIMARY, SECONDARY and OFFLINE. Many places will only work when location=PRIMARY since that is the only setting were getDownloadStream() returns any data, but PRIMARY also means that the file should be located on the BASE server.... Hmmm... maybe we should add a fourth option (EXTERNAL?) and add a notice about a possible incompatible change.
• Size: The file size is usually stored automatically when the file is upload and is retained if the file is taken offline. Some code may require file size > 0. Since we don't know the size of external files (or can we issue a HEAD request to find out?) we may have to update some code that makes decisions based on the size.
• External files should not contribute to quota as far as BASE is concerned.

---

For instructions on how to setup a test environment with a https server that only accepts clients with a trusted certificate see HttpsFiles.

[Nicklas Nordborg]

(In [5325]) References #1485: File items should be able to reference external files

Added URL field to File item and Location.EXTERNAL. Updated test code with some more test cases and file download. This change can be a major issue for code that rely on File.getSize() or File.getLocation().

Plug-ins that use the file size to report progress most likely needs to be updated to cover the case were the file size is not known.

[Nicklas Nordborg]

(In [5326]) References #1485: File items should be able to reference external files

It is now possible to add external files using the GUI. Fixed the list and view pages so that they behave correctly for external files.

[Nicklas Nordborg]

(In [5327]) References #1485: File items should be able to reference external files

Fixed code that use Location.PRIMARY to determine if a file is usable or not.

• Code that throws error if location != PRIMARY
• Code that adds a filter location==PRIMARY to list pages when selecting files

[Nicklas Nordborg]

(In [5328]) References #1485: File items should be able to reference external files

Redirect file links in the GUI for EXTERNAL files to the external URL.

[Nicklas Nordborg]

(In [5329]) References #1485: File items should be able to reference external files

Fixed code that use File.getSize() for stuff like progress reporting, etc.

Fixed progress reporter implementations so that they can handle -1 to indicate unknown progress.

[Nicklas Nordborg]

(In [5330]) References #1485: File items should be able to reference external files

Added data for external files in quota listings.

[Nicklas Nordborg]

(In [5331]) References #1485: File items should be able to reference external files

Fixes some issues with links to the external file in the view and list pages.

[Nicklas Nordborg]

(In [5332]) References #1485: File items should be able to reference external files

Implemented option to issue a HEAD request to a given URL to get metadata about the target. Most importantly we are interested in the size, but we can also get last modification time, MIME type and character set.

[Nicklas Nordborg]

(In [5333]) References #1485: File items should be able to reference external files

The change in TestFile should have been part of the last commit...

[Nicklas Nordborg]

(In [5334]) References #1485: File items should be able to reference external files

File download test failed on windows because subversion modified line breaks in the reference file. Do not change the svn:eol-style of this file!

[Nicklas Nordborg]

(In [5335]) References #1485: File items should be able to reference external files

Improved error handling with non-existing url:s.

[Jari Häkkinen]

Replying to nicklas:

(In [5334]) References #1485: File items should be able to reference external files

File download test failed on windows because subversion modified line breaks in the reference file. Do not change the svn:eol-style of this file!

For future reference: Another option could be to set the file as binary, subversion doesn't tamper with binary files.

[Nicklas Nordborg]

So far it seems to work fine with http and also with https if the server has a valid certificate and doesn't require any authentication. It doesn't work with self-signed, expired or other invalid certificates or if the server requires authentication.

The Proteios project has solved some parts of this by creating a specific SSLSocketFactory that reads information about trusted servers and authentication information from certificate files stored on the server.

See ​Application.java, line 923-1044

[Nicklas Nordborg]

(In [5336]) Fixes #1485: File items should be able to reference external files

Added support for using key-store and trust-store where authentication certificates and trusted self-signed certificates can be added.