Your submission: H1_mix_cdna_ont on ONT data

Background

Challenge 1 is evaluated according to four criteria:

  1. Broad GENCODE Annotation
  2. Subset of manually curated loci selected by GENCODE
  3. SIRV Lexogen Set 4
  4. Simulated data.

The LRGASP uses SQANTI categories to define evaluating features and metrics for Challenge 1.

LRGASP Challenge 1 Definitions:

This document shows the performance of your pipeline for criteria 1. and 3. Critical data for evaluation according to 2. and 4. will be made available after the closure of the challenge, and therefore pre-evaluation reports cannot be provided. Note you’re your criterion 1 metrics reported here have been calculated using GENCODE human v38 and mouse M27 releases while the final evaluation will use human v39 and mouse M28 to be released after completing of the challenge.

Evaluation of detected transcripts for Challenge 1

Here, SIRVs are not taken into account for none of the metrics.

Global overview

Value
Number of genes detected 10310
Number of known genes detected 10242
Number of transcripts detected 36577
Number of transcripts associated to a known gene 35171
Number of unique SJ detected 100815
Absolute value Relative value (%)
Novel SJ 7072 0.07
Non-canonical SJ 16702 0.17

Evaluation of FSM

Absolute value Relative value (%)
Number of isoforms 23469 -
Reference Match 16667 71.02
5’ reference supported (transcript) 21437 91.34
3’ reference supported (transcript) 17587 74.94
5’ reference supported (gene) 23025 98.11
3’ reference supported (gene) 20733 88.34
5’ CAGE supported 17667 75.28
3’ polyA supported 21988 93.69
Supported Reference Transcript Model (SRTM) 22704 96.74
Intra-priming 268 1.14
Reference redundancy Level 1.22 -

Evaluation of ISM

Absolute value Relative value (%)
Number of isoforms 1279 -
5’ reference supported (transcript) 641 50.12
3’ reference supported (transcript) 358 27.99
5’ and 3’ reference supported (gene) 338 26.43
5’ reference supported (gene) 903 903
3’ reference supported (gene) 632 49.41
5’ CAGE supported 883 69.04
3’ polyA supported 1000 78.19
Supported Reference Transcript Model (SRTM) 743 58.09
Intra-priming 18 1.41
Reference redundancy Level 1.14 -

Evaluation NIC

Absolute value Relative value (%)
Number of isoforms 4465 -
5’ and 3’ reference supported (gene) 2321 51.98
5’ reference supported (gene) 2738 61.32
3’ reference supported (gene) 2425 54.31
5’ CAGE supported 2396 53.66
3’ polyA supported 2781 62.28
Supported Novel Transcript Model (SNTM) 2348 52.59
Intra-priming 18 0.4
Intron retention incidence 656 14.69

Evaluation NNC

Absolute value Relative value (%)
Number of isoforms 5958 -
5’ and 3’ reference supported (gene) 3751 62.96
5’ reference supported (gene) 4629 77.69
3’ reference supported (gene) 4168 69.96
5’ CAGE supported 4330 72.68
3’ polyA supported 5069 85.08
Supported Novel Transcript Model (SNTM) 1171 19.65
Non-canonical SJ incidence 2035 34.16
Full Illumina SJ support 1374 23.06
Intra-priming 29 0.49
RT-switching incidence 310 5.2

Evaluation of Spike-Ins (SIRVs)

The following metrics and definitions apply to SIRV transcripts:

ATENTION If in this chunk of the evaluation all the results are 0, please, check if the reference genome and/or transcriptome used for building your transcript-model contain information about spike-ins.

Value
SIRV transcripts 115
True Positive detections (TP) 65
SIRV transcripts associated to TP (Reference Match) 65
Partial True Positive detections (PTP) 0
SIRV transcripts associated to PTP 2
False Negative (FN) 19
False Positive (FP) 48
Sensitivity 0.77
Precision 0.57
Non Redundant Precision 0.57
Positive Detection Rate 0.77
False Discovery Rate 0.42
False Detection Rate 0.42
Redundancy 1.03