Your submission: WTC11_CapTrap_PacBio on PacBio data

Background

Challenge 1 is evaluated according to four criteria:

  1. Broad GENCODE Annotation
  2. Subset of manually curated loci selected by GENCODE
  3. SIRV Lexogen Set 4
  4. Simulated data.

The LRGASP uses SQANTI categories to define evaluating features and metrics for Challenge 1.

LRGASP Challenge 1 Definitions:

This document shows the performance of your pipeline for criteria 1. and 3. Critical data for evaluation according to 2. and 4. will be made available after the closure of the challenge, and therefore pre-evaluation reports cannot be provided. Note you’re your criterion 1 metrics reported here have been calculated using GENCODE human v38 and mouse M27 releases while the final evaluation will use human v39 and mouse M28 to be released after completing of the challenge.

Evaluation of detected transcripts for Challenge 1

Here, SIRVs are not taken into account for none of the metrics.

Global overview

Value
Number of genes detected 12271
Number of known genes detected 11962
Number of transcripts detected 29234
Number of transcripts associated to a known gene 28853
Number of unique SJ detected 88349
Absolute value Relative value (%)
Novel SJ 1010 0.01
Non-canonical SJ 72 0.00

Evaluation of FSM

Absolute value Relative value (%)
Number of isoforms 27114 -
Reference Match 26682 98.41
5’ reference supported (transcript) 26847 99.02
3’ reference supported (transcript) 26863 99.07
5’ reference supported (gene) 26998 99.57
3’ reference supported (gene) 26946 99.38
5’ CAGE supported 18995 70.06
3’ polyA supported 22202 81.88
Supported Reference Transcript Model (SRTM) 26934 99.34
Intra-priming 663 2.45
Reference redundancy Level 1 -

Evaluation of ISM

Absolute value Relative value (%)
Number of isoforms 924 -
5’ reference supported (transcript) 244 26.41
3’ reference supported (transcript) 419 45.35
5’ and 3’ reference supported (gene) 203 21.97
5’ reference supported (gene) 421 421
3’ reference supported (gene) 609 65.91
5’ CAGE supported 533 57.68
3’ polyA supported 860 93.07
Supported Reference Transcript Model (SRTM) 561 60.71
Intra-priming 42 4.55
Reference redundancy Level 1.03 -

Evaluation NIC

Absolute value Relative value (%)
Number of isoforms 467 -
5’ and 3’ reference supported (gene) 340 72.81
5’ reference supported (gene) 422 90.36
3’ reference supported (gene) 373 79.87
5’ CAGE supported 414 88.65
3’ polyA supported 459 98.29
Supported Novel Transcript Model (SNTM) 410 87.79
Intra-priming 4 0.86
Intron retention incidence 50 10.71

Evaluation NNC

Absolute value Relative value (%)
Number of isoforms 348 -
5’ and 3’ reference supported (gene) 111 31.9
5’ reference supported (gene) 201 57.76
3’ reference supported (gene) 188 54.02
5’ CAGE supported 275 79.02
3’ polyA supported 338 97.13
Supported Novel Transcript Model (SNTM) 242 69.54
Non-canonical SJ incidence 19 5.46
Full Illumina SJ support 296 85.06
Intra-priming 3 0.86
RT-switching incidence 18 5.17

Evaluation of Spike-Ins (SIRVs)

The following metrics and definitions apply to SIRV transcripts:

ATENTION If in this chunk of the evaluation all the results are 0, please, check if the reference genome and/or transcriptome used for building your transcript-model contain information about spike-ins.

Value
SIRV transcripts 82
True Positive detections (TP) 81
SIRV transcripts associated to TP (Reference Match) 81
Partial True Positive detections (PTP) 1
SIRV transcripts associated to PTP 1
False Negative (FN) 2
False Positive (FP) 0
Sensitivity 0.96
Precision 0.99
Non Redundant Precision 0.99
Positive Detection Rate 0.98
False Discovery Rate 0.01
False Detection Rate 0
Redundancy 1