Opened 16 years ago

Last modified 15 years ago

#294 closed enhancement

Performance testing: Plugin-in execution time — at Version 14

Reported by: Johan Enell Owned by: Nicklas Nordborg
Priority: blocker Milestone: BASE 2.5
Component: coreplugins Version: trunk
Keywords: Cc:

Description (last modified by Nicklas Nordborg)

The core plug-ins are slow. Just by running plug-ins on the demo server I've noticed that they are alot slower then the plug-ins from base1.

I've tested the intenisty calculator and Lowess but I think this affects every plug-in. Maybe some more tests should be done to locate if there is a bottleneck and it would be nice if we could create a best practice document.

Change History (16)

comment:1 Changed 16 years ago by Jari Häkkinen

Milestone: BASE 2.x+BASE 2.1

comment:2 Changed 16 years ago by Martin Svensson

Owner: changed from base to Johan Enell

comment:3 Changed 16 years ago by Jari Häkkinen

Milestone: BASE 2.1BASE 2.2
Summary: Plugin-in execution timePerformance testing: Plugin-in execution time

comment:4 Changed 16 years ago by Jari Häkkinen

Milestone: BASE 2.2BASE 2.x+

comment:5 Changed 16 years ago by Jari Häkkinen

Milestone: BASE 2.x+BASE 2.4

comment:6 Changed 16 years ago by Jari Häkkinen

Performance tuning for the whole application is also needed.

comment:7 Changed 15 years ago by Jari Häkkinen

Milestone: BASE 2.4BASE 2.3

Milestone BASE 2.4 deleted

comment:8 Changed 15 years ago by Nicklas Nordborg

Priority: majorcritical

comment:9 Changed 15 years ago by Nicklas Nordborg

Owner: changed from Johan Enell to Nicklas Nordborg
Status: newassigned

I will start to create a BASE 2 test framework that can be used to:

  • Import reporters
  • Create one array design
  • Import a given number of raw bioassays from one or two data files
  • Create an experiment and add the raw bioassays to it
  • Create a root bioassayset from all raw bioassays
  • Create a filtered bioassayset from the root bioassayset
  • Normalize the filtered bioassayset with Lowess

Some parameters may be given on the command line that starts the test (for example the number of raw bioassays to create and which subtests to run). Some parameters may be given in a configuration file (for example parameters to the filter and lowess plug-ins).

I expect someone with more knowledge about BASE 1 and microarray analysis to help me with selecing proper parameter values, setting up a BASE 1 server for comparison, etc.

comment:10 Changed 15 years ago by Nicklas Nordborg

(In [3659]) References #294: Performance testing

Added framework for testing BASE 2 plug-ins.

comment:11 Changed 15 years ago by Nicklas Nordborg

Milestone: BASE 2.4BASE 2.5

comment:12 Changed 15 years ago by Nicklas Nordborg

Owner: Nicklas Nordborg deleted
Status: assignednew

comment:13 Changed 15 years ago by Jari Häkkinen

Milestone: BASE 2.5BASE 2.x+

comment:14 Changed 15 years ago by Nicklas Nordborg

Description: modified (diff)
Milestone: BASE 2.x+BASE 2.5
Owner: set to Nicklas Nordborg
Priority: criticalblocker
Status: newassigned

Johan V-C has done some interesting test with the following plug-ins:

  • Create root bioassayset. The same 55K raw data set has been used for 10-40 bioassays. Median FG - Median BG was used for intensity values
  • Jep filter plugin: ch(1) > 0 && ch(2) > 0 && raw('flag') == 0
  • Lowess plugin: step=0.1, window size=0.33, iterations=4, blockgroup size=1

Corresponding tests were done on the BASE 1 server. The results show that BASE 1 execution time is linear with respect to the number of bioassays. With BASE 2 only the filter plug-in is linear but takes about 3 times longer. The other plug-ins seems to have a random execution time that is 20-50 times longer than in BASE 1.

Changed 15 years ago by Nicklas Nordborg

BASE 1 performance with respect to number of bioassays

Changed 15 years ago by Nicklas Nordborg

BASE 2 performance with respect to number of bioassays

Note: See TracTickets for help on using tickets.