Opened 17 years ago

Closed 17 years ago

Last modified 17 years ago

#451 closed task (fixed)

Zipped download

Reported by: Nicklas Nordborg Owned by: Nicklas Nordborg
Priority: minor Milestone: BASE 2.4
Component: coreplugins Version:
Keywords: Cc:

Description

Create a plugin that allows a user to select one or more files/directories and download everything as a zip file.

Change History (19)

comment:1 by Jari Häkkinen, 17 years ago

I'd like to see support for the formats that are supported by the zip uploader (cf. ticket:366 and ticket:470), i.e., allow user to select format.

comment:2 by Jari Häkkinen, 17 years ago

Milestone: BASE 2.x+BASE 2.4

comment:3 by Martin Svensson, 17 years ago

Owner: changed from Johan Enell to Martin Svensson
Status: newassigned

comment:4 by Martin Svensson, 17 years ago

(In [3568]) References #451 Packer plugin for zip, tar.gz and tar.bz2 format Should be tested more.

comment:5 by Martin Svensson, 17 years ago

(In [3571]) References #451 Some minor bugs are fixed.

comment:6 by Martin Svensson, 17 years ago

Resolution: fixed
Status: assignedclosed

(In [3572]) Fixes #451 This plugin is now installed with BASE as a core plugin and the html-page with core plugins have been updated.

comment:7 by Martin Svensson, 17 years ago

(In [3576]) References #451 Avoid null pointer exception

comment:8 by Nicklas Nordborg, 17 years ago

Resolution: fixed
Status: closedreopened

I have made some tests and there are some remaining issues:

  1. If I don't select a directory I get a NullPointerException when the job is executing:
    java.lang.NullPointerException
    at net.sf.basedb.plugins.PackedFileExporter.performExport(PackedFileExporter.java:166)
    at net.sf.basedb.core.plugin.AbstractExporterPlugin.run(AbstractExporterPlugin.java:143)
    at net.sf.basedb.core.PluginExecutionRequest.invoke(PluginExecutionRequest.java:88)
    at net.sf.basedb.core.InternalJobQueue$JobRunner.run(InternalJobQueue.java:420)
    at java.lang.Thread.run(Thread.java:619)
    
  1. If I select a directory and some files I get a PermissionDeniedException when the job is executing:
    net.sf.basedb.core.PermissionDeniedException: Permission denied: Not allowed to write Directory[id=26; name=images]
    at net.sf.basedb.core.BasicItem.checkPermission(BasicItem.java:109)
    at net.sf.basedb.core.Directory.setRemoved(Directory.java:415)
    at net.sf.basedb.plugins.ZipFilePacker.pack(ZipFilePacker.java:110)
    at net.sf.basedb.plugins.PackedFileExporter.performExport(PackedFileExporter.java:170)
    at net.sf.basedb.core.plugin.AbstractExporterPlugin.run(AbstractExporterPlugin.java:143)
    at net.sf.basedb.core.PluginExecutionRequest.invoke(PluginExecutionRequest.java:88)
    at net.sf.basedb.core.InternalJobQueue$JobRunner.run(InternalJobQueue.java:420)
    at java.lang.Thread.run(Thread.java:619)
    

This is fixed by giving the plug-in WRITE permission to DIRECTORY. But I suppose it will need DELETE permission if the Remove files/directories options is going to work.

  1. If I select just a directory and no files I immediately get the error message: null
    The Tomcat log file has more the following stack trace:
    java.lang.NullPointerException
    	at net.sf.basedb.plugins.PackedFileExporter.configure(PackedFileExporter.java:208)
    	at net.sf.basedb.core.PluginConfigurationRequest.invoke(PluginConfigurationRequest.java:113)
    	at org.apache.jsp.common.plugin.index_jsp._jspService(index_jsp.java:457)
    	at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
    	at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
    	at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:334)
    	at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:314)
    	at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:264)
    	at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
    	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
    	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
    	at net.sf.basedb.clients.web.servlet.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:71)
    	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
    	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
    	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
    	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
    	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
    	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
    	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
    	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
    	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
    	at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
    	at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
    	at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
    	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
    	at java.lang.Thread.run(Thread.java:619)
    

As a last point: I don't see a reason for using a two-step wizard. Can't the options in the second step be included in the first step?

comment:9 by Nicklas Nordborg, 17 years ago

There is also a problem if a user hasn't delete permission or if a file is write protected. Selecting the 'Remove files/directories' will generate a PermissionDeniedException and no files will be packed. I think the plug-in should check for DELETE permission before trying to delete items.

comment:10 by Nicklas Nordborg, 17 years ago

Clearing the 'Save as' to use immediate download displays the following stacktrace in the download window:

java.lang.NullPointerException at net.sf.basedb.plugins.PackedFileExporter.performExport(PackedFileExporter.java:158) at net.sf.basedb.core.plugin.AbstractExporterPlugin.doExport(AbstractExporterPlugin.java:185) at net.sf.basedb.core.PluginResponse.downloadImmediately(PluginResponse.java:156) at org.apache.jsp.common.plugin.index_jsp._jspService(index_jsp.java:551) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97) at javax.servlet.http.HttpServlet.service(HttpServlet.java:802) at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:334) at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:314) at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:264) at javax.servlet.http.HttpServlet.service(HttpServlet.java:802) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) at net.sf.basedb.clients.web.servlet.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:71) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869) at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664) at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527) at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684) at java.lang.Thread.run(Thread.java:619)

The Tomcat error log has another stacktrace:

13:24:41,211 ERROR [jsp]:253 - Servlet.service() for servlet jsp threw exception
java.lang.IllegalStateException: getOutputStream() has already been called for this response
	at org.apache.catalina.connector.Response.getWriter(Response.java:599)
	at org.apache.catalina.connector.ResponseFacade.getWriter(ResponseFacade.java:195)
	at org.apache.jasper.runtime.JspWriterImpl.initOut(JspWriterImpl.java:124)
	at org.apache.jasper.runtime.JspWriterImpl.flushBuffer(JspWriterImpl.java:117)
	at org.apache.jasper.runtime.PageContextImpl.release(PageContextImpl.java:182)
	at org.apache.jasper.runtime.JspFactoryImpl.internalReleasePageContext(JspFactoryImpl.java:115)
	at org.apache.jasper.runtime.JspFactoryImpl.releasePageContext(JspFactoryImpl.java:75)
	at org.apache.jsp.common.plugin.index_jsp._jspService(index_jsp.java:621)
	at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
	at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:334)
	at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:314)
	at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:264)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
	at net.sf.basedb.clients.web.servlet.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:71)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
	at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
	at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
	at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
	at java.lang.Thread.run(Thread.java:619)

comment:11 by Nicklas Nordborg, 17 years ago

Here is the first stacktrace of the previous comment with line breaks:

java.lang.NullPointerException 
at net.sf.basedb.plugins.PackedFileExporter.performExport(PackedFileExporter.java:158) 
at net.sf.basedb.core.plugin.AbstractExporterPlugin.doExport(AbstractExporterPlugin.java:185) 
at net.sf.basedb.core.PluginResponse.downloadImmediately(PluginResponse.java:156) 
at org.apache.jsp.common.plugin.index_jsp._jspService(index_jsp.java:551) 
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97) 
at javax.servlet.http.HttpServlet.service(HttpServlet.java:802) 
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:334) 
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:314) 
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:264) 
at javax.servlet.http.HttpServlet.service(HttpServlet.java:802) 
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252) 
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) 
at net.sf.basedb.clients.web.servlet.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:71) 
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202) 
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) 
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) 
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178) 
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126) 
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105) 
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107) 
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148) 
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869) 
at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664) 
at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527) 
at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80) 
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684) 
at java.lang.Thread.run(Thread.java:619)

comment:12 by Nicklas Nordborg, 17 years ago

Using a hardcoded name ("temp.tar") for a temporarary file in TarFilePacker will make the plug-in crash if more than one user uses it at the same time. Why not just connect the TarOutputStream to the 'out' parameter just as the ZipFilePacker does?

comment:13 by Nicklas Nordborg, 17 years ago

I tested the plug-in when running BASE on Windows and it doesn't handle subdirectories correctly. My layout is:

index.html
dir/pict.gif
dir/subdir/pict2.gif

The packed zip file comes as:

index.html
\dir/pict.gif
\subdir/pict2.gif

There are two problems:

  1. A \ is prepended to the directory name
  2. Subdirectories are moved up to the root level

comment:14 by Nicklas Nordborg, 17 years ago

2) in the previous comment happens on Linux as well.

I think maybe a slight redesign is needed. The file/directory handling should not be done for each compressed format. It would be better if that was done at a central place and only the actual compression of a file needed to be implemented by the FilePacker classes. The same argument goes for the FileUnpacker plug-ins which suffer from the same thing (but this should be added as another ticket).

comment:15 by Nicklas Nordborg, 17 years ago

Owner: changed from Martin Svensson to Nicklas Nordborg
Status: reopenednew

comment:16 by Nicklas Nordborg, 17 years ago

Status: newassigned

comment:17 by Nicklas Nordborg, 17 years ago

(In [3641]) References #451: Zipped download

Major redesign of the packing plug-in structure.

comment:18 by Nicklas Nordborg, 17 years ago

Resolution: fixed
Status: assignedclosed

(In [3643]) Fixes #451: Zipped download

Added user and plug-in developer documentation

comment:19 by Nicklas Nordborg, 17 years ago

(In [3646]) References #451: Zipped download

Fixed test program and added it to TestAll. Fixed some bugs resulting from problems with closing the TAR stream properly.

Note: See TracTickets for help on using tickets.