Your submission: H1_mix_PacBioOnly on PacBio data

Background

Challenge 1 is evaluated according to four criteria:

  1. Broad GENCODE Annotation
  2. Subset of manually curated loci selected by GENCODE
  3. SIRV Lexogen Set 4
  4. Simulated data.

The LRGASP uses SQANTI categories to define evaluating features and metrics for Challenge 1.

LRGASP Challenge 1 Definitions:

This document shows the performance of your pipeline for criteria 1. and 3. Critical data for evaluation according to 2. and 4. will be made available after the closure of the challenge, and therefore pre-evaluation reports cannot be provided. Note you’re your criterion 1 metrics reported here have been calculated using GENCODE human v38 and mouse M27 releases while the final evaluation will use human v39 and mouse M28 to be released after completing of the challenge.

Evaluation of detected transcripts for Challenge 1

Here, SIRVs are not taken into account for none of the metrics.

Global overview

Value
Number of genes detected 18593
Number of known genes detected 13742
Number of transcripts detected 54854
Number of transcripts associated to a known gene 49145
Number of unique SJ detected 140061
Absolute value Relative value (%)
Novel SJ 4912 0.04
Non-canonical SJ 437 0.00

Evaluation of FSM

Absolute value Relative value (%)
Number of isoforms 31200 -
Reference Match 12688 40.67
5’ reference supported (transcript) 22514 72.16
3’ reference supported (transcript) 17545 56.23
5’ reference supported (gene) 27399 87.82
3’ reference supported (gene) 24743 79.3
5’ CAGE supported 26334 84.4
3’ polyA supported 30714 98.44
Supported Reference Transcript Model (SRTM) 28835 92.42
Intra-priming 824 2.64
Reference redundancy Level 1.51 -

Evaluation of ISM

Absolute value Relative value (%)
Number of isoforms 2700 -
5’ reference supported (transcript) 1124 41.63
3’ reference supported (transcript) 1264 46.81
5’ and 3’ reference supported (gene) 967 35.81
5’ reference supported (gene) 1635 1635
3’ reference supported (gene) 1850 68.52
5’ CAGE supported 1621 60.04
3’ polyA supported 2518 93.26
Supported Reference Transcript Model (SRTM) 1683 62.33
Intra-priming 246 9.11
Reference redundancy Level 1.3 -

Evaluation NIC

Absolute value Relative value (%)
Number of isoforms 12464 -
5’ and 3’ reference supported (gene) 9878 79.25
5’ reference supported (gene) 11424 91.66
3’ reference supported (gene) 10678 85.67
5’ CAGE supported 10917 87.59
3’ polyA supported 12165 97.6
Supported Novel Transcript Model (SNTM) 10675 85.65
Intra-priming 449 3.6
Intron retention incidence 3687 29.58

Evaluation NNC

Absolute value Relative value (%)
Number of isoforms 2781 -
5’ and 3’ reference supported (gene) 1429 51.38
5’ reference supported (gene) 1947 70.01
3’ reference supported (gene) 2000 71.92
5’ CAGE supported 2057 73.97
3’ polyA supported 2725 97.99
Supported Novel Transcript Model (SNTM) 1910 68.68
Non-canonical SJ incidence 346 12.44
Full Illumina SJ support 2354 84.65
Intra-priming 86 3.09
RT-switching incidence 210 7.55

Evaluation of Spike-Ins (SIRVs)

The following metrics and definitions apply to SIRV transcripts:

ATENTION If in this chunk of the evaluation all the results are 0, please, check if the reference genome and/or transcriptome used for building your transcript-model contain information about spike-ins.

Value
SIRV transcripts 82
True Positive detections (TP) 81
SIRV transcripts associated to TP (Reference Match) 81
Partial True Positive detections (PTP) 1
SIRV transcripts associated to PTP 1
False Negative (FN) 2
False Positive (FP) 0
Sensitivity 0.96
Precision 0.99
Non Redundant Precision 0.99
Positive Detection Rate 0.98
False Discovery Rate 0.01
False Detection Rate 0
Redundancy 1