Your submission: ES_PacBioOnly on PacBio data

Background

Challenge 1 is evaluated according to four criteria:

  1. Broad GENCODE Annotation
  2. Subset of manually curated loci selected by GENCODE
  3. SIRV Lexogen Set 4
  4. Simulated data.

The LRGASP uses SQANTI categories to define evaluating features and metrics for Challenge 1.

LRGASP Challenge 1 Definitions:

This document shows the performance of your pipeline for criteria 1. and 3. Critical data for evaluation according to 2. and 4. will be made available after the closure of the challenge, and therefore pre-evaluation reports cannot be provided. Note you’re your criterion 1 metrics reported here have been calculated using GENCODE human v38 and mouse M27 releases while the final evaluation will use human v39 and mouse M28 to be released after completing of the challenge.

Evaluation of detected transcripts for Challenge 1

Here, SIRVs are not taken into account for none of the metrics.

Global overview

Value
Number of genes detected 24612
Number of known genes detected 12336
Number of transcripts detected 50351
Number of transcripts associated to a known gene 37652
Number of unique SJ detected 126853
Absolute value Relative value (%)
Novel SJ 5094 0.04
Non-canonical SJ 932 0.01

Evaluation of FSM

Absolute value Relative value (%)
Number of isoforms 23639 -
Reference Match 8766 37.08
5’ reference supported (transcript) 14893 63
3’ reference supported (transcript) 13873 58.69
5’ reference supported (gene) 19486 82.43
3’ reference supported (gene) 18002 76.15
5’ CAGE supported 574 2.43
3’ polyA supported 22182 93.84
Supported Reference Transcript Model (SRTM) 18684 79.04
Intra-priming 746 3.16
Reference redundancy Level 1.54 -

Evaluation of ISM

Absolute value Relative value (%)
Number of isoforms 2146 -
5’ reference supported (transcript) 671 31.27
3’ reference supported (transcript) 1088 50.7
5’ and 3’ reference supported (gene) 608 28.33
5’ reference supported (gene) 1058 1058
3’ reference supported (gene) 1460 68.03
5’ CAGE supported 24 1.12
3’ polyA supported 1849 86.16
Supported Reference Transcript Model (SRTM) 914 42.59
Intra-priming 154 7.18
Reference redundancy Level 1.25 -

Evaluation NIC

Absolute value Relative value (%)
Number of isoforms 8214 -
5’ and 3’ reference supported (gene) 6194 75.41
5’ reference supported (gene) 7278 88.6
3’ reference supported (gene) 6951 84.62
5’ CAGE supported 179 2.18
3’ polyA supported 7832 95.35
Supported Novel Transcript Model (SNTM) 6508 79.23
Intra-priming 196 2.39
Intron retention incidence 2247 27.36

Evaluation NNC

Absolute value Relative value (%)
Number of isoforms 3653 -
5’ and 3’ reference supported (gene) 1642 44.95
5’ reference supported (gene) 2222 60.83
3’ reference supported (gene) 2699 73.88
5’ CAGE supported 53 1.45
3’ polyA supported 3383 92.61
Supported Novel Transcript Model (SNTM) 1446 39.58
Non-canonical SJ incidence 875 23.95
Full Illumina SJ support 2674 73.2
Intra-priming 97 2.66
RT-switching incidence 399 10.92

Evaluation of Spike-Ins (SIRVs)

The following metrics and definitions apply to SIRV transcripts:

ATENTION If in this chunk of the evaluation all the results are 0, please, check if the reference genome and/or transcriptome used for building your transcript-model contain information about spike-ins.

Value
SIRV transcripts 83
True Positive detections (TP) 81
SIRV transcripts associated to TP (Reference Match) 81
Partial True Positive detections (PTP) 1
SIRV transcripts associated to PTP 2
False Negative (FN) 2
False Positive (FP) 0
Sensitivity 0.96
Precision 0.98
Non Redundant Precision 0.98
Positive Detection Rate 0.98
False Discovery Rate 0.01
False Detection Rate 0
Redundancy 1.01