Your submission: human_WTC11_cDNA_pb on PacBio data

Background

Challenge 1 is evaluated according to four criteria:

  1. Broad GENCODE Annotation
  2. Subset of manually curated loci selected by GENCODE
  3. SIRV Lexogen Set 4
  4. Simulated data.

The LRGASP uses SQANTI categories to define evaluating features and metrics for Challenge 1.

LRGASP Challenge 1 Definitions:

This document shows the performance of your pipeline for criteria 1. and 3. Critical data for evaluation according to 2. and 4. will be made available after the closure of the challenge, and therefore pre-evaluation reports cannot be provided. Note you’re your criterion 1 metrics reported here have been calculated using GENCODE human v38 and mouse M27 releases while the final evaluation will use human v39 and mouse M28 to be released after completing of the challenge.

Evaluation of detected transcripts for Challenge 1

Here, SIRVs are not taken into account for none of the metrics.

Global overview

Value
Number of genes detected 61733
Number of known genes detected 22758
Number of transcripts detected 328749
Number of transcripts associated to a known gene 271046
Number of unique SJ detected 352678
Absolute value Relative value (%)
Novel SJ 155301 0.44
Non-canonical SJ 41771 0.12

Evaluation of FSM

Absolute value Relative value (%)
Number of isoforms 55889 -
Reference Match 10361 18.54
5’ reference supported (transcript) 34098 61.01
3’ reference supported (transcript) 17991 32.19
5’ reference supported (gene) 43383 77.62
3’ reference supported (gene) 29784 53.29
5’ CAGE supported 42148 75.41
3’ polyA supported 52106 93.23
Supported Reference Transcript Model (SRTM) 44208 79.1
Intra-priming 1678 3
Reference redundancy Level 1.82 -

Evaluation of ISM

Absolute value Relative value (%)
Number of isoforms 20140 -
5’ reference supported (transcript) 6198 30.77
3’ reference supported (transcript) 5578 27.7
5’ and 3’ reference supported (gene) 3145 15.62
5’ reference supported (gene) 8859 8859
3’ reference supported (gene) 9213 45.74
5’ CAGE supported 9115 45.26
3’ polyA supported 17535 87.07
Supported Reference Transcript Model (SRTM) 8723 43.31
Intra-priming 1274 6.33
Reference redundancy Level 1.49 -

Evaluation NIC

Absolute value Relative value (%)
Number of isoforms 85510 -
5’ and 3’ reference supported (gene) 48167 56.33
5’ reference supported (gene) 64643 75.6
3’ reference supported (gene) 61642 72.09
5’ CAGE supported 62496 73.09
3’ polyA supported 81066 94.8
Supported Novel Transcript Model (SNTM) 50833 59.45
Intra-priming 3022 3.53
Intron retention incidence 37856 44.27

Evaluation NNC

Absolute value Relative value (%)
Number of isoforms 109507 -
5’ and 3’ reference supported (gene) 52595 48.03
5’ reference supported (gene) 76683 70.03
3’ reference supported (gene) 72910 66.58
5’ CAGE supported 79575 72.67
3’ polyA supported 104876 95.77
Supported Novel Transcript Model (SNTM) 27424 25.04
Non-canonical SJ incidence 29021 26.5
Full Illumina SJ support 34800 31.78
Intra-priming 4303 3.93
RT-switching incidence 5353 4.89

Evaluation of Spike-Ins (SIRVs)

The following metrics and definitions apply to SIRV transcripts:

ATENTION If in this chunk of the evaluation all the results are 0, please, check if the reference genome and/or transcriptome used for building your transcript-model contain information about spike-ins.

Value
SIRV transcripts 324
True Positive detections (TP) 79
SIRV transcripts associated to TP (Reference Match) 79
Partial True Positive detections (PTP) 2
SIRV transcripts associated to PTP 118
False Negative (FN) 3
False Positive (FP) 127
Sensitivity 0.94
Precision 0.24
Non Redundant Precision 0.24
Positive Detection Rate 0.96
False Discovery Rate 0.4
False Detection Rate 0.39
Redundancy 2.43