Your submission: ES_cdna_ont_ls on ONT+Illumina data

Background

Challenge 1 is evaluated according to four criteria:

  1. Broad GENCODE Annotation
  2. Subset of manually curated loci selected by GENCODE
  3. SIRV Lexogen Set 4
  4. Simulated data.

The LRGASP uses SQANTI categories to define evaluating features and metrics for Challenge 1.

LRGASP Challenge 1 Definitions:

This document shows the performance of your pipeline for criteria 1. and 3. Critical data for evaluation according to 2. and 4. will be made available after the closure of the challenge, and therefore pre-evaluation reports cannot be provided. Note you’re your criterion 1 metrics reported here have been calculated using GENCODE human v38 and mouse M27 releases while the final evaluation will use human v39 and mouse M28 to be released after completing of the challenge.

Evaluation of detected transcripts for Challenge 1

Here, SIRVs are not taken into account for none of the metrics.

Global overview

Value
Number of genes detected 30099
Number of known genes detected 11703
Number of transcripts detected 44754
Number of transcripts associated to a known gene 25987
Number of unique SJ detected 81857
Absolute value Relative value (%)
Novel SJ 4447 0.05
Non-canonical SJ 221 0.00

Evaluation of FSM

Absolute value Relative value (%)
Number of isoforms 19611 -
Reference Match 15865 80.9
5’ reference supported (transcript) 17786 90.69
3’ reference supported (transcript) 16272 82.97
5’ reference supported (gene) 18717 95.44
3’ reference supported (gene) 17600 89.75
5’ CAGE supported 303 1.55
3’ polyA supported 15744 80.28
Supported Reference Transcript Model (SRTM) 18478 94.22
Intra-priming 599 3.05
Reference redundancy Level 1.1 -

Evaluation of ISM

Absolute value Relative value (%)
Number of isoforms 808 -
5’ reference supported (transcript) 99 12.25
3’ reference supported (transcript) 407 50.37
5’ and 3’ reference supported (gene) 81 10.02
5’ reference supported (gene) 181 181
3’ reference supported (gene) 489 60.52
5’ CAGE supported 4 0.5
3’ polyA supported 652 80.69
Supported Reference Transcript Model (SRTM) 151 18.69
Intra-priming 22 2.72
Reference redundancy Level 1.04 -

Evaluation NIC

Absolute value Relative value (%)
Number of isoforms 3443 -
5’ and 3’ reference supported (gene) 2544 73.89
5’ reference supported (gene) 3113 90.42
3’ reference supported (gene) 2784 80.86
5’ CAGE supported 64 1.86
3’ polyA supported 3285 95.41
Supported Novel Transcript Model (SNTM) 2561 74.38
Intra-priming 27 0.78
Intron retention incidence 561 16.29

Evaluation NNC

Absolute value Relative value (%)
Number of isoforms 2125 -
5’ and 3’ reference supported (gene) 975 45.88
5’ reference supported (gene) 1401 65.93
3’ reference supported (gene) 1440 67.76
5’ CAGE supported 21 0.99
3’ polyA supported 1938 91.2
Supported Novel Transcript Model (SNTM) 1162 54.68
Non-canonical SJ incidence 21 0.99
Full Illumina SJ support 1964 92.42
Intra-priming 15 0.71
RT-switching incidence 94 4.42

Evaluation of Spike-Ins (SIRVs)

The following metrics and definitions apply to SIRV transcripts:

ATENTION If in this chunk of the evaluation all the results are 0, please, check if the reference genome and/or transcriptome used for building your transcript-model contain information about spike-ins.

Value
SIRV transcripts 92
True Positive detections (TP) 75
SIRV transcripts associated to TP (Reference Match) 75
Partial True Positive detections (PTP) 4
SIRV transcripts associated to PTP 4
False Negative (FN) 5
False Positive (FP) 13
Sensitivity 0.89
Precision 0.82
Non Redundant Precision 0.82
Positive Detection Rate 0.94
False Discovery Rate 0.18
False Detection Rate 0.14
Redundancy 1