Your submission: WTC11_cDNA_PacBio on PacBio data

Background

Challenge 1 is evaluated according to four criteria:

  1. Broad GENCODE Annotation
  2. Subset of manually curated loci selected by GENCODE
  3. SIRV Lexogen Set 4
  4. Simulated data.

The LRGASP uses SQANTI categories to define evaluating features and metrics for Challenge 1.

LRGASP Challenge 1 Definitions:

This document shows the performance of your pipeline for criteria 1. and 3. Critical data for evaluation according to 2. and 4. will be made available after the closure of the challenge, and therefore pre-evaluation reports cannot be provided. Note you’re your criterion 1 metrics reported here have been calculated using GENCODE human v38 and mouse M27 releases while the final evaluation will use human v39 and mouse M28 to be released after completing of the challenge.

Evaluation of detected transcripts for Challenge 1

Here, SIRVs are not taken into account for none of the metrics.

Global overview

Value
Number of genes detected 6530
Number of known genes detected 6198
Number of transcripts detected 9895
Number of transcripts associated to a known gene 9468
Number of unique SJ detected 57148
Absolute value Relative value (%)
Novel SJ 999 0.02
Non-canonical SJ 0 0.00

Evaluation of FSM

Absolute value Relative value (%)
Number of isoforms 7014 -
Reference Match 3976 56.69
5’ reference supported (transcript) 5567 79.37
3’ reference supported (transcript) 5014 71.49
5’ reference supported (gene) 6473 92.29
3’ reference supported (gene) 6290 89.68
5’ CAGE supported 6250 89.11
3’ polyA supported 6979 99.5
Supported Reference Transcript Model (SRTM) 6825 97.31
Intra-priming 1 0.01
Reference redundancy Level 1 -

Evaluation of ISM

Absolute value Relative value (%)
Number of isoforms 382 -
5’ reference supported (transcript) 125 32.72
3’ reference supported (transcript) 191 50
5’ and 3’ reference supported (gene) 139 36.39
5’ reference supported (gene) 192 192
3’ reference supported (gene) 274 71.73
5’ CAGE supported 198 51.83
3’ polyA supported 352 92.15
Supported Reference Transcript Model (SRTM) 222 58.12
Intra-priming 0 0
Reference redundancy Level 1.03 -

Evaluation NIC

Absolute value Relative value (%)
Number of isoforms 1501 -
5’ and 3’ reference supported (gene) 1256 83.68
5’ reference supported (gene) 1380 91.94
3’ reference supported (gene) 1359 90.54
5’ CAGE supported 1327 88.41
3’ polyA supported 1486 99
Supported Novel Transcript Model (SNTM) 1304 86.88
Intra-priming 1 0.07
Intron retention incidence 431 28.71

Evaluation NNC

Absolute value Relative value (%)
Number of isoforms 571 -
5’ and 3’ reference supported (gene) 402 70.4
5’ reference supported (gene) 473 82.84
3’ reference supported (gene) 462 80.91
5’ CAGE supported 508 88.97
3’ polyA supported 567 99.3
Supported Novel Transcript Model (SNTM) 464 81.26
Non-canonical SJ incidence 0 0
Full Illumina SJ support 503 88.09
Intra-priming 0 0
RT-switching incidence 37 6.48

Evaluation of Spike-Ins (SIRVs)

The following metrics and definitions apply to SIRV transcripts:

ATENTION If in this chunk of the evaluation all the results are 0, please, check if the reference genome and/or transcriptome used for building your transcript-model contain information about spike-ins.

Value
SIRV transcripts 84
True Positive detections (TP) 74
SIRV transcripts associated to TP (Reference Match) 74
Partial True Positive detections (PTP) 2
SIRV transcripts associated to PTP 2
False Negative (FN) 8
False Positive (FP) 8
Sensitivity 0.88
Precision 0.88
Non Redundant Precision 0.88
Positive Detection Rate 0.9
False Discovery Rate 0.12
False Detection Rate 0.1
Redundancy 1