Your submission: ES_CapTrap_ONT on ONT data

Background

Challenge 1 is evaluated according to four criteria:

  1. Broad GENCODE Annotation
  2. Subset of manually curated loci selected by GENCODE
  3. SIRV Lexogen Set 4
  4. Simulated data.

The LRGASP uses SQANTI categories to define evaluating features and metrics for Challenge 1.

LRGASP Challenge 1 Definitions:

This document shows the performance of your pipeline for criteria 1. and 3. Critical data for evaluation according to 2. and 4. will be made available after the closure of the challenge, and therefore pre-evaluation reports cannot be provided. Note you’re your criterion 1 metrics reported here have been calculated using GENCODE human v38 and mouse M27 releases while the final evaluation will use human v39 and mouse M28 to be released after completing of the challenge.

Evaluation of detected transcripts for Challenge 1

Here, SIRVs are not taken into account for none of the metrics.

Global overview

Value
Number of genes detected 4855
Number of known genes detected 4236
Number of transcripts detected 7903
Number of transcripts associated to a known gene 6673
Number of unique SJ detected 18363
Absolute value Relative value (%)
Novel SJ 3622 0.2
Non-canonical SJ 0 0.0

Evaluation of FSM

Absolute value Relative value (%)
Number of isoforms 1689 -
Reference Match 710 42.04
5’ reference supported (transcript) 1138 67.38
3’ reference supported (transcript) 994 58.85
5’ reference supported (gene) 1347 79.75
3’ reference supported (gene) 1265 74.9
5’ CAGE supported 27 1.6
3’ polyA supported 1544 91.42
Supported Reference Transcript Model (SRTM) 1274 75.43
Intra-priming 13 0.77
Reference redundancy Level 1 -

Evaluation of ISM

Absolute value Relative value (%)
Number of isoforms 1345 -
5’ reference supported (transcript) 133 9.89
3’ reference supported (transcript) 793 58.96
5’ and 3’ reference supported (gene) 100 7.43
5’ reference supported (gene) 194 194
3’ reference supported (gene) 1004 74.65
5’ CAGE supported 4 0.3
3’ polyA supported 1264 93.98
Supported Reference Transcript Model (SRTM) 158 11.75
Intra-priming 2 0.15
Reference redundancy Level 1.03 -

Evaluation NIC

Absolute value Relative value (%)
Number of isoforms 972 -
5’ and 3’ reference supported (gene) 514 52.88
5’ reference supported (gene) 603 62.04
3’ reference supported (gene) 798 82.1
5’ CAGE supported 6 0.62
3’ polyA supported 939 96.6
Supported Novel Transcript Model (SNTM) 526 54.12
Intra-priming 0 0
Intron retention incidence 183 18.83

Evaluation NNC

Absolute value Relative value (%)
Number of isoforms 2667 -
5’ and 3’ reference supported (gene) 1686 63.22
5’ reference supported (gene) 2059 77.2
3’ reference supported (gene) 2098 78.67
5’ CAGE supported 44 1.65
3’ polyA supported 2523 94.6
Supported Novel Transcript Model (SNTM) 570 21.37
Non-canonical SJ incidence 0 0
Full Illumina SJ support 903 33.86
Intra-priming 1 0.04
RT-switching incidence 120 4.5

Evaluation of Spike-Ins (SIRVs)

The following metrics and definitions apply to SIRV transcripts:

ATENTION If in this chunk of the evaluation all the results are 0, please, check if the reference genome and/or transcriptome used for building your transcript-model contain information about spike-ins.

Value
SIRV transcripts 287
True Positive detections (TP) 17
SIRV transcripts associated to TP (Reference Match) 17
Partial True Positive detections (PTP) 5
SIRV transcripts associated to PTP 8
False Negative (FN) 62
False Positive (FP) 262
Sensitivity 0.2
Precision 0.06
Non Redundant Precision 0.06
Positive Detection Rate 0.26
False Discovery Rate 0.93
False Detection Rate 0.91
Redundancy 1.14