Hybridizations and raw data

This document covers the details of how hybridizations and raw data is handled by BASE.

  1. Hybridizations
  2. Scans
  3. Raw data sets
  4. Raw data
  5. Spot images
See also

Last updated: $Date: 2009-04-06 14:52:39 +0200 (må, 06 apr 2009) $

1. Hybridizations

  1. A hybridization is attached to a list set of labeled extracts. The same labeled extract may be used several times. The position in the list does not mean anything to BASE, but may be used by plugins subsequently used to create derived data from raw data.
  2. A hybridization may be attached to an array slide.
  3. The hybridization may be dissociated from the array slide and the labeled extracts at any time.
  4. A hybridization protocol must may be picked.
  5. A hybridiation may be annotated, but annotations on its labeled extracts are not transferred to it.

2. Scans

  1. A scan (or image acquisition) represents the scanning of a slide.
  2. A hybridization may have any number of scans.
  3. A scan is associated with a scanner as well as with a scanning protocol.
  4. Images may be attached to a scan.
  5. An image consists of a pointer to an uploaded file, information about what channel(s) it has to do with, what format it's in (TIFF or JPEG), whether it's a preview or the full image, and whether it should be used for generating spot images.

3. Raw data sets

  1. A raw data set describes the result of applying some software to a set of images (obtained from scanning a microarray) in order to quantify the spots and identify them with features or reporters. This includes the generated spot quantifications, which we refer to as raw data.
  2. A raw data set normally belongs to a scan, but it should also be possible to create raw data sets with no connection to a scan.
  3. It should be possible to attach a scan-less raw data set to a scan at a later stage.
  4. A raw data set is can be associated with an software item.
  5. The file(s) generated by the software should be attached to the raw data set. Typically this is one file, which we refer to as a raw result file. [NOTE] As it is implemented, only one file can be attached.
  6. A raw data set may point to the array design it has to do with, if any, but only if the array design has features.
  7. The array design of a raw data set is typically that of its hybridization's array slide, but it doesn't have to be. A raw data set created without connection to a scan may still point to an array design.

4. Raw data

  1. Because different software produces different sets of spot measurements, it should be possible to define new types of raw data.
  2. There is a single table which is used for all raw data types, in which information common to all types is stored.
  3. The columns common to all types of raw data are at least:
  4. For each type of raw data, there is one table with type-specific data. This table is described in detailed in the database. For each column, the following is recorded:
    1. Column name and type
    2. Whether the column holds an intensity, a standard deviation, or neither
    3. Whether the column holds a foreground value, a background value, or neither
    4. Whether the column holds a mean or median or neither
    5. An optional label id, in the case that the raw data type concerns itself with labels.
  5. In the table with raw data type specific columns, the spots should be identified by raw data set and position. We use the id of the rawdata entry.
  6. The raw data set should store not only what type of raw data it contains, but also which of the type-specific and non-type-specific columns it uses.
  7. Raw spots may be flagged/commented by users. There should be a table with possible comments (modifiable by some users only), and each spot may point to such a comment. This is the only property of a raw spot that may change after the raw data set is added. The raw data set should know the datetime of the last change to one of its spots.

5. Spot images

  1. By spot images we mean small images of the individual spots of a raw data set, meant to convey information about the morphology of spots. These images are meant to be shown to users, often many at a time from an arbitrary set of spots.
  2. It should be possible to generate spot images from a raw data set whose spots have physical coordinates specified, if it is connected to a scan which has sufficient images attached, and if it has no more than three channels. If it has more than three channels a user may select up to three images for the spot image generation.
  3. By sufficient images we mean one high-resolution TIFF image per channel, possibly stored in a single file.
  4. The user may need to enter information about how to scale and offset the physical spot coordinates to get the corresponding image coordinates. This information might be extracted from the raw result file (or from the images).
  5. The size of the area to cut out for each spot image needs to be given by the user.
  6. The input images cannot be visualized without modification, as the dynamic range of the scanner far exceeds that of the user's screen, and most spots would be completely black if rescaled to 8 bits per gun. Therefore, the colors of each spot should be rescaled to use the full intensity range, with the same rescaling done on all 1-3 channels. Gamma correction may also be applied before going to 8 bpg.
  7. The spot images should be saved as JPEG or some other format with good compression. To avoid an excessive number of small space-wasting files, they should be lumped together in reasonable numbers before compression. With JPEG, this means that the each spot image should be a square with side divisible by 8 pixels (to avoid interference between spots).
  8. The scales, offsets, spotsize, gamma correction and JPEG quality value should be stored in the database, along with the identities of the images used to create spot images.
  9. It should be possible to remove and re-generate spot images. When no spot images exist for a raw data set, the parameters for generating them may be altered, but when spot images exist they may not.