Test procedures for predefined roles

This document defines a procedure for testing that the predefined roles can perform their work as intended. The main purpose is to weed out permission problems resulting from:

The test procedure also tests that the basic functionality is working:

Contents
  1. Summary of the test procedure
  2. Root user tests
  3. Administrator tests
  4. Power user tests
  5. User tests
  6. Analysis tests

Last updated: $Date: 2008-09-05 17:15:25 +0200 (Fri, 05 Sep 2008) $

1. Summary of the test procedure

Here is a summary of the test procedure:

  1. Always start with a fresh installation.
  2. The root user creates an administrator.
  3. The administrator creates more users and some global resources:
  4. The power user creates item related to the project management:
  5. The user creates items related to an actual experiment:
  6. Both the user and the guest then do some analysis:

These tests can also be run in automated mode by test programs. This will of course not test the web client, but are useful if one quickly needs to do parts of the test and then continue with, for example, the user or analysis tests on the web.

The data files needed by the tests are NOT included in the subversion repository. The main reason is that they are too large, and that we don't have permission to make them publicly available for download. To get the test file you need to be a core developer. Read the instructions on the DeveloperInformation page, Test data section on the Base 2 web site. The automated test programs require that file are placed (checked out) in the 'testdata' directory located in the BASE root directory. NOTE! Some test data files are bzip-compressed. These need to be uncompressed before they are uploaded to BASE.

To run the tests do the following:

  1. Compile the core and the test programs: ant main test.
  2. Change to the build/test/ directory.
  3. Run test programs: ./test.sh roles <cmds> where <cmds> is one or more of the following:

2. Root user tests

The root user creates an administrator which is a server-wide admin.

  1. Create a new user and set the following properties. All other properties may remain unchanged.
    Name Login/Password Quota Quota group Membership
    Admin admin/admin Unlimited - Roles: Administrator

3. Administrator tests

The administrator creates users for a project and gives them permissions that are suitable for their role in the project. The administrator also sets up quota and group membership.

  1. Create a new group and set the following properties:
    Name Quota
    Group A 1GB

  2. Create the following users:
    Name Login/Password Quota Quota group Membership Other
    Power user power/power 1GB Group A Roles: Power user -
    User user/user 1GB Group A Roles: User -
    Guest guest/guest 10MB Group A Roles: Guest Multi-user account checked

  3. Give USE permission for the listed users to the following plugins:
    User Plugins
    Power user
    • Plate importer
    • Reporter map importer
    • Print map importer

  4. Create file formats (i.e., plug-in configurations) for importing reporters. The formats marked as optional are not used in the test procedure, but may be useful to weed out import problems, since they allow importing all info from the raw data files. You may either enter the regular expressions as specified or use the "Test with file" feature.
    Name Plugin Configuration values
    Reporters for project A Reporter importer Test with file: plates_and_reporters.mouse.v4.37k.txt
    Data header 384_number\t384_column\t384_row\t384_position\toligo_id.*
    Data splitter \t
    Min data columns 5
    Name \oligo_id\
    Reporter ID \oligo_id\
    Description \description_Ensembl*\
    Gene symbol \gene_symbol_Ensembl*\
    Sequence \oligo_sequence\
    GenePix reporter importer
    (optional)
    Reporter importer Test with file: genepix.mouse.v4.37k.00h.gpr
    Data header "Block"\t"Column"\t"Row"\t"Name"\t"ID"\t.*
    Data splitter \t
    Min data columns 48
    Max data columns 48
    Name \Name\
    Reporter ID \ID\
    Reporters from Affymetrix annotations file Reporter importer Test with file: MG_U74Av2_annot.csv.bz2
    Data header "Probe Set ID","GeneChip Array".*
    Data splitter (?!"),(?=")
    Name \Probe Set ID\
    Reporter ID \Probe Set ID\
    Description \Target Description\
    Gene symbol \Gene Symbol\

  5. Import reporters:
    1. Go to the View -> Reporters page.
    2. Click on the Import button.
    3. Choose auto-detect and then upload the file plates_and_reporters.mouse.v4.37k.txt.
    4. The Reporters for project A format should be found.
    5. Select the skip option for the "Missing a required" value since the file contains rows with empty reporter ID:s.
    6. Continue and wait for the import to finish. It should create 35,912 new reporters.
    7. Repeat the procedure with the MG_U74Av2_annot.csv file. This time also select crop for the "String too long" setting since the file contains data that is too large for the datbase. 12,488 new reporters should be created.

4. Power user tests

The power user is the typical owner/administrator of a project. The power user sets up common resources used in the project, such as hardware, software, protocols, file formats and annotation types. In this case the power user is also responsible for managing the LIMS.

  1. Create a project:
    Name Members
    Project A Groups: Group A (Use permission)

  2. Activate the project.

  3. Create protocols:
    Name Type
    Sampling A Sampling
    Extraction A Extraction
    Labeling A Labeling
    Hybridization A Hybridization
    Scanning A Scanning
    Feature extraction A Feature extraction
    Printing A Printing

  4. Create hardware:
    Name Type
    Hybridization station A Hybridization station
    Scanner A Scanner
    Print robot A Print robot

  5. Create software:
    Name Type
    Software A Feature extraction

  6. Create annotation types:
    Name Type Interface Values Item types
    Drug resistance String radiobuttons high, medium, low Biosource
    Time (hours) Integer text box - Sample
    Dye swap Boolean - - Raw bioassay

  7. Create file formats (i.e., plug-in configurations). The formats marked as optional are not used in the test procedure, but may be useful to weed out import problems, since they allow importing all info from the raw data files. You may either enter the regular expressions as specified or use the "Test with file" feature.
    Name Plugin Configuration values
    Plates for project A Plate importer Test with file: plates_and_reporters.mouse.v4.37k.txt
    Data header 384_number\t384_column\t384_row\t384_position\toligo_id.*
    Data splitter \t
    Min data columns 5
    Plate number/name \384_number\
    Row \384_row\
    Column \384_column\
    Reporter ID \oligo_id\
    GenePix feature importer
    (optional)
    Reporter map importer Test with file: genepix.mouse.v4.37k.00h.gpr
    Data header "Block"\t"Column"\t"Row"\t"Name"\t"ID"\t.*
    Data splitter \t
    Min data columns 48
    Max data columns 48
    Reporter ID \ID\
    Block \Block\
    Column \Column\
    Row \Row\
    Raw data for project A Raw data importer Test with file: genepix.mouse.v4.37k.00h.gpr
    Raw data type Genepix
    Header "(.+)=(.*)"
    Data header "Block"\t"Column"\t"Row"\t"Name"\t"ID"\t.*"Ratio of Medians \(532\/635\)".*
    Data splitter \t
    Min data columns 48
    Max data columns 48
    Block \Block\
    Column \Column\
    Row \Row\
    X \X\
    Y \Y\
    Reporter ID \ID\
    Spot diameter \Dia.\
    Channel 1 foreground median \F635 Median\
    Channel 1 foreground mean \F635 Mean\
    Channel 1 foreground standard deviation \F635 SD\
    Channel 1 background median \B635 Median\
    Channel 1 background mean \B635 Mean\
    Channel 1 background standard deviation \B635 SD\
    Percent pixels within 1 standard deviation \% > B635+1SD\
    Percent pixels within 2 standard deviations \% > B635+2SD\
    Percent saturated pixels \F635 % Sat.\
    Channel 2 foreground median \F532 Median\
    Channel 2 foreground mean \F532 Mean\
    Channel 2 foreground standard deviation \F532 SD\
    Channel 2 background median \B532 Median\
    Channel 2 background mean \B532 Mean\
    Channel 2 background standard deviation \B532 SD\
    Percent pixels within 1 standard deviation \% > B532+1SD\
    Percent pixels within 2 standard deviations \% > B532+2SD\
    Percent saturated pixels \F532 % Sat.\
    Foreground pixels \F Pixels\
    Background pixels \B Pixels\
    Flags \Flags\
    Raw data for project A (dye-swap) Raw data importer Test with file: genepix.mouse.v4.37k.00h.dyeswap.gpr
    Raw data type Genepix
    Header "(.+)=(.*)"
    Data header "Block"\t"Column"\t"Row"\t"Name"\t"ID"\t.*"Ratio of Medians \(635\/532\)".*
    Data splitter \t
    Min data columns 48
    Max data columns 48
    Block \Block\
    Column \Column\
    Row \Row\
    X \X\
    Y \Y\
    Reporter ID \ID\
    Spot diameter \Dia.\
    Channel 1 foreground median \F532 Median\
    Channel 1 foreground mean \F532 Mean\
    Channel 1 foreground standard deviation \F532 SD\
    Channel 1 background median \B532 Median\
    Channel 1 background mean \B532 Mean\
    Channel 1 background standard deviation \B532 SD\
    Percent pixels within 1 standard deviation \% > B532+1SD\
    Percent pixels within 2 standard deviations \% > B532+2SD\
    Percent saturated pixels \F532 % Sat.\
    Channel 2 foreground median \F635 Median\
    Channel 2 foreground mean \F635 Mean\
    Channel 2 foreground standard deviation \F635 SD\
    Channel 2 background median \B635 Median\
    Channel 2 background mean \B635 Mean\
    Channel 2 background standard deviation \B635 SD\
    Percent pixels within 1 standard deviation \% > B635+1SD\
    Percent pixels within 2 standard deviations \% > B635+2SD\
    Percent saturated pixels \F635 % Sat.\
    Foreground pixels \F Pixels\
    Background pixels \B Pixels\
    Flags \Flags\

  8. Annotate the file formats:
    File format Annotation Value
    Raw data for project A Dye swap false
    Raw data for project A (dye swap) Dye swap true
    This will make the raw data importer automatically annotate the raw bioassays with the specified annotations.

  9. Create plate type:
    Name Geometry
    Plate type A 384-well (16 x 24)

  10. Import plates:
    1. Go to the Array LIMS -> Plates page.
    2. Click on the Import button.
    3. Choose auto-detect and select the file plates_and_reporters.mouse.v4.37k.txt.
    4. The Plates for project A format should be found.
    5. Specify the following parameters:
      Plate type Plate type A
      Plate name prefix Plate A
      Plate name padding 4
    6. Continue and wait for the import to finish. It should create 96 plates.

  11. Create array designs and upload data files to them:
    Name Platform File(s)
    Array design A Generic Print map: printmap.mouse.v4.37k.tam
    Affymetrix A Affymetrix CDF file: MG_U74Av2.cdf.bz2

  12. Connect Array design A with plates. Select the imported plates (plate names starting with Plate A) and sort them in the correct order (as indicated by their names).

  13. Import features:
    1. Click on the Import button when viewing properties for the array design.
    2. Choose auto-detect and upload the file printmap.mouse.v4.37k.tam.
    3. The Print map importer plug-in should be found.
    4. Continue and wait for the import to finish. It should create 36,864 features and 48 blocks.

  14. Set project defaults. Go to the projects page and edit the Project A project. On the Defaults tab, set the following defaults:
    Setting Value
    Platform / raw data type Generic / Genepix
    Software Software A
    Array design Array design A
    Hybridization station Hybridization station A
    Scanner Scanner A
    Print robot Print robot A
    Scanning protocol Scanning A
    Sampling protocol Sampling A
    Extraction protocol Extraction A
    Labeling protocol Labeling A
    Hybridization protocol Hybridization A
    Feat. extraction protocol Feature extraction A
    Printing protocol Printing A

  15. Create array batches:
    Name Array design Print robot Protocol
    Array batch A Array design A Robot A Printing A
    Affymetrix batch A Affymetrix A

  16. Create array slides with the Create slides wizard.
    Name Array batch Quantity
    Array slide A. Array batch A 4
    Affymetrix slide A. Affymetrix batch A 3

5. User tests

The user is a typical worker in the project. The user does the actual experimentation in the lab, which includes collecting samples, doing extraction, labeling and hybridizations. The user also scans and analyses the raw data resulting from the images.

  1. Activate the Project A project

  2. Create a biosource:
    Name Annotations
    Biosource A
    Drug resistance: medium

  3. Create samples:
    Name Protocol Biosource Annotations
    Sample A.00h Sampling A Biosource A
    Time (hours): 0
    Sample A.24h Sampling A Biosource A
    Time (hours): 24
    Sample A.ref Sampling A - -

  4. Create extracts:
    Name Protocol Sample
    Extract A.00h Extraction A Sample A.00h
    Extract A.24h Extraction A Sample A.24h
    Extract A.ref Extraction A Sample A.ref

  5. Create labeled extracts:
    Name Label Protocol Extract
    Labeled extract A.00h cy3 Labeling A Extract A.00h
    Labeled extract A.24h cy3 Labeling A Extract A.24h
    Labeled extract A.ref cy5 Labeling A Extract A.ref
    Labeled extract A.00h (dye-swap) cy5 Labeling A Extract A.00h
    Labeled extract A.24h (dye-swap) cy5 Labeling A Extract A.24h
    Labeled extract A.ref (dye-swap) cy3 Labeling A Extract A.ref

  6. Create hybridizations:
    Name Protocol Hardware Array slide Labeled extracts
    Hybridization A.00h Hybridization A Hybridization station A Array slide A.1 Labeled extract A.00h,
    Labeled extract A.ref
    Hybridization A.24h Hybridization A Hybridization station A Array slide A.2 Labeled extract A.24h,
    Labeled extract A.ref
    Hybridization A.00h (dye-swap) Hybridization A Hybridization station A Array slide A.3 Labeled extract A.00h (dye-swap),
    Labeled extract A.ref (dye-swap)
    Hybridization A.24h (dye-swap) Hybridization A Hybridization station A Array slide A.4 Labeled extract A.24h (dye-swap),
    Labeled extract A.ref (dye-swap)
    Affymetrix hyb A.1 Hybridization A Hybridization station A Affymetrix slide A.1 Labeled extract A.00h
    Affymetrix hyb A.2 Hybridization A Hybridization station A Affymetrix slide A.2 Labeled extract A.24h
    Affymetrix hyb A.3 Hybridization A Hybridization station A Affymetrix slide A.3 Labeled extract A.ref

  7. Create scans:
    Name Hybridization Scanner Protocol
    Scan A.00h Hybridization A.00h Scanner A Scanning A
    Scan A.24h Hybridization A.24h Scanner A Scanning A
    Scan A.00h (dye-swap) Hybridization A.00h (dye-swap) Scanner A Scanning A
    Scan A.24h (dye-swap) Hybridization A.24h (dye-swap) Scanner A Scanning A
    Affymetrix scan A.1 Affymetrix hyb A.1 Scanner A Scanning A
    Affymetrix scan A.2 Affymetrix hyb A.2 Scanner A Scanning A
    Affymetrix scan A.3 Affymetrix hyb A.3 Scanner A Scanning A

  8. Add images to the scans (TODO - we don't have any images yet):
    Scan Image files
    Scan A.00h -
    Scan A.24h -
    Scan A.00h (dye-swap) -
    Scan A.24h (dye-swap) -

  9. Create raw bioassays:
    Name Platform/Raw data type Array design Protocol Scan Software File(s)
    Raw bioassay A.00h Generic/GenePix Array design A Feature extraction A Scan A.00h Software A Raw data: genepix.mouse.v4.37k.00h.gpr
    Raw bioassay A.24h Generic/GenePix Array design A Feature extraction A Scan A.24h Software A Raw data: genepix.mouse.v4.37k.24h.gpr
    Raw bioassay A.00h (dye-swap) Generic/GenePix Array design A Feature extraction A Scan A.00h (dye-swap) Software A Raw data: genepix.mouse.v4.37k.00h.dyeswap.gpr
    Raw bioassay A.24h (dye-swap) Generic/GenePix Array design A Feature extraction A Scan A.24h (dye-swap) Software A Raw data: genepix.mouse.v4.37k.24h.dyeswap.gpr
    Affymetrix raw A.1 Affymetrix Affymetrix A Feature extraction A Affymetrix scan A.1 Software A CEL file: jos1761.cel.bz2
    Affymetrix raw A.2 Affymetrix Affymetrix A Feature extraction A Affymetrix scan A.2 Software A CEL file: jos1762.cel.bz2
    Affymetrix raw A.3 Affymetrix Affymetrix A Feature extraction A Affymetrix scan A.3 Software A CEL file: jos1763.cel.bz2

  10. Inherit the annotations from the samples and biosource for each raw bioassay.

  11. Use the auto-detect feature to upload and import raw data for each Genepix raw bioassay. Select the smart option for the "Feature mismatch" parameter and null for "Invalid numeric value".
    Raw bioassay Raw data file Spots inserted/with null reporter/skipped) Annotations created
    Raw bioassay A.00h genepix.mouse.v4.37k.00h.gpr 36,864/632/768 Dye swap: false
    Raw bioassay A.24h genepix.mouse.v4.37k.24h.gpr 36,864/632/768 Dye swap: false
    Raw bioassay A.00h (dye-swap) genepix.mouse.v4.37k.00h.dyeswap.gpr 36,864/632/768 Dye swap: true
    Raw bioassay A.24h (dye-swap) genepix.mouse.v4.37k.24h.dyeswap.gpr 36,864/632/768 Dye swap: true

  12. Create experiments:
    Name Raw data type Raw bioassays Experimental factors
    Experiment A GenePix Raw bioassay A.00h,
    Raw bioassay A.24h,
    Raw bioassay A.00h (dye-swap),
    Raw bioassay A.24h (dye-swap)
    Drug resistance, Time (hours), Dye swap
    Affymetrix A Affymetrix Affymetrix raw A.1,
    Affymetrix raw A.2,
    Affymetrix raw A.3
    Drug resistance, Time (hours)

  13. Check the experiment overview page. It should display one warnings for the Genepix experiment. The warning is related to a missing biosurce on the reference sample.

    The Affymetrix experiment gives some more warnings. Most of them are related to not using the project default items, missing protocols and missing hardware. There should also be an error about missing experimental factor values for the Affymetrix.3 raw bioassay. It is expected since this comes from the reference sample which doesn't have values for those annotations.

    Change validation options to reduce the number of warnings:

    Validation option Setting
    Project defaults Set all to Ignore
    Missing items Set all to Ignore
    Annotations - Missing factor value Warning
    After the changes there should now only be two warnings about the missing factor values.

6. Analysis tests

Now it is time to analyse the data. The analysis test should be done by both a regular user and a guest.

  1. Activate the Project A project

  2. Go to the Experiment A experiment and select the Bioassay sets tab.

  3. Create a new root bioassay set:
    Bioassay set name Raw bioassays Formula
    Root bioassay set all Mean FG - Mean BG
    Wait for the plug-in to finish.

  4. Select the created bioassay set and create a filtered bioassayset:
    Child name Filter preset Expression
    Filtered bioassay set - ch(1) > 0 && ch(2) > 0 && rep('id') != null
    Wait for the plug-in to finish. It should report that 136,498 spots remain and that 10,958 spots has been removed.

  5. Select the filtered bioassay set and run a normalization plug-in:
    Plugin Parameters
    Normalization: Lowess Accept the most of default parameters, but change the "Block group size" to 12. Otherwise it will take too long time.
    Wait for the plug in to finish. It should report that 136,498 spots has been normalized and 0 spots has been removed.

  6. Select the normalized bioassay set and check the overview plots and the correction factor plots. Here are four examples:
    Overview plots Correction factor plots


  7. Try the plot tool with the following plots. Use the Save function to save one them as a file in the BASE file system, and the Download function to download a plot to your computer.
    Plot type Y-axis preset X-axis preset Other options
    Scatter plot M, log2(ch1 / ch2) A, log10(ch1 * ch2) / 2 -
    Histogram plot Count Ratio, ch1 / ch2
    Log scale checked
    Bin size 0.1
    Annotation Time (hours)

    Here are two examples:

    Scatter plot

    Histogram plot