Your submission: ES_cdna_pacbio_ls on PacBio+Illumina data

Background

Challenge 1 is evaluated according to four criteria:

  1. Broad GENCODE Annotation
  2. Subset of manually curated loci selected by GENCODE
  3. SIRV Lexogen Set 4
  4. Simulated data.

The LRGASP uses SQANTI categories to define evaluating features and metrics for Challenge 1.

LRGASP Challenge 1 Definitions:

This document shows the performance of your pipeline for criteria 1. and 3. Critical data for evaluation according to 2. and 4. will be made available after the closure of the challenge, and therefore pre-evaluation reports cannot be provided. Note you’re your criterion 1 metrics reported here have been calculated using GENCODE human v38 and mouse M27 releases while the final evaluation will use human v39 and mouse M28 to be released after completing of the challenge.

Evaluation of detected transcripts for Challenge 1

Here, SIRVs are not taken into account for none of the metrics.

Global overview

Value
Number of genes detected 23280
Number of known genes detected 13091
Number of transcripts detected 46958
Number of transcripts associated to a known gene 36445
Number of unique SJ detected 133568
Absolute value Relative value (%)
Novel SJ 5947 0.04
Non-canonical SJ 248 0.00

Evaluation of FSM

Absolute value Relative value (%)
Number of isoforms 23636 -
Reference Match 19207 81.26
5’ reference supported (transcript) 21709 91.85
3’ reference supported (transcript) 19558 82.75
5’ reference supported (gene) 22835 96.61
3’ reference supported (gene) 21395 90.52
5’ CAGE supported 421 1.78
3’ polyA supported 19973 84.5
Supported Reference Transcript Model (SRTM) 22555 95.43
Intra-priming 1068 4.52
Reference redundancy Level 1.11 -

Evaluation of ISM

Absolute value Relative value (%)
Number of isoforms 500 -
5’ reference supported (transcript) 86 17.2
3’ reference supported (transcript) 226 45.2
5’ and 3’ reference supported (gene) 114 22.8
5’ reference supported (gene) 176 176
3’ reference supported (gene) 331 66.2
5’ CAGE supported 5 1
3’ polyA supported 410 82
Supported Reference Transcript Model (SRTM) 168 33.6
Intra-priming 26 5.2
Reference redundancy Level 1.04 -

Evaluation NIC

Absolute value Relative value (%)
Number of isoforms 8427 -
5’ and 3’ reference supported (gene) 6212 73.72
5’ reference supported (gene) 7535 89.41
3’ reference supported (gene) 6864 81.45
5’ CAGE supported 173 2.05
3’ polyA supported 8047 95.49
Supported Novel Transcript Model (SNTM) 6627 78.64
Intra-priming 181 2.15
Intron retention incidence 2695 31.98

Evaluation NNC

Absolute value Relative value (%)
Number of isoforms 3882 -
5’ and 3’ reference supported (gene) 2129 54.84
5’ reference supported (gene) 2774 71.46
3’ reference supported (gene) 2929 75.45
5’ CAGE supported 69 1.78
3’ polyA supported 3718 95.78
Supported Novel Transcript Model (SNTM) 2526 65.07
Non-canonical SJ incidence 24 0.62
Full Illumina SJ support 3678 94.74
Intra-priming 60 1.55
RT-switching incidence 150 3.86

Evaluation of Spike-Ins (SIRVs)

The following metrics and definitions apply to SIRV transcripts:

ATENTION If in this chunk of the evaluation all the results are 0, please, check if the reference genome and/or transcriptome used for building your transcript-model contain information about spike-ins.

Value
SIRV transcripts 84
True Positive detections (TP) 84
SIRV transcripts associated to TP (Reference Match) 84
Partial True Positive detections (PTP) 0
SIRV transcripts associated to PTP 0
False Negative (FN) 0
False Positive (FP) 0
Sensitivity 1
Precision 1
Non Redundant Precision 1
Positive Detection Rate 1
False Discovery Rate 0
False Detection Rate 0
Redundancy 1