A molecular-annotation-spec for SAM/BAM/CRAM with new reserved tags

[mrvollger]

Hello!

I would like to propose a new set of reserved tags for SAM/BAM/CRAM that can describe molecular annotation along individual sequencing reads. With the advent of functional single-molecule sequencing, there is an emerging need to annotate non-genetic elements on individual DNA molecules (e.g., nucleosome positions, accessible chromatin, CTCF binding events, or TSSs). This is handled for base modifications using the MM and ML tags. However, we (the functional long-read community) have a need for a more extensible format beyond base modifications that allows for generic annotation of any segment of a molecule. To accomplish this, I am proposing the MA tag.

Integration of this spec or a similar one into the SAM/BAM/CRAM is needed to standardize the representation of molecular annotations across different sequencing platforms and analysis tools. Importantly, a standardized format will allow for integration in a wide range of genomic tools, e.g., visualization in genome browsers, and allow developers to be justified in spending time integrating this format into their tools.

I have a more detailed spec outline here:
https://github.com/fiberseq/Molecular-annotation-spec

But in summary, I am proposing that the molecular annotation format uses four related tags:

MA:Z: - Annotation start positions (required, u32 arrays separated by annotation type prefixes)
AL:B:I - Annotation Lengths (required, u32 array)
AQ:B:C - Annotation Quality scores (required, u8 array)
AN:Z: - Annotation Names (optional labels for individual annotations)

Example:

MA:Z:annotation_type1+P:start1,start2;annotation_type2-:start1
AL:B:I,len1,len2,len3
AQ:B:C,qual1,qual2,qual3
AN:Z:name1,name2,name3

Regex for MA tag:

^(([a-zA-Z0-9_]+)[+-.][PQ]?:((\d+)(,\d+)*);?)+$

This style is meant to mimic many of the conventions used by the MM/ML tags while being more flexible to new annotation types of varying lengths.

I'd really appreciate any thoughts and next steps I can/should take to work towards the formal incorporation of these tags.

Thanks,
Mitchell

I have already gotten feedback from the following individuals invested in making long-read annoations:
"Andrew Stergachis" absterga@uw.edu; "Nick Altemose" altemose@stanford.edu; "tommytullius@mac.com" tommytullius@mac.com; "Danilo Dubocanin" dubocd@stanford.edu; "wtimp@jhu.edu" wtimp@jhu.edu; "lmorina2@jhu.edu" lmorina2@jhu.edu; "Stephanie Bohaczuk" bohaczuk@uw.edu; "Gregory Young" gyoung@pacificbiosciences.com; "Art Rand" art.rand@nanoporetech.com; "Aaron Wenger" awenger@pacificbiosciences.com; "minkinaa@uw.edu" minkinaa@uw.edu; "sjn@uw.edu" sjn@uw.edu

[jkbonfield]

How much traction has this gained so far with the other interested parties? Are there implementations of it already?
Were there points of discussion or contention that got ironed out? Some background would be good. (We can discuss all this at the meeting tomorrow too.)

On a very quick glance through just now:

• Zero based positions. I don't mind either way really for read coordinates, but note the old PT tag for arbitrary read annotations used 1-base positions so if there's a choice then it may be good to keep consistency here.
• Original orientation - yes! That's needed to ensure we don't invalidate the tag when aligners process the data without understanding what the tag means.
• Consider an MN equivalent tag for this. It was added to cope with tools that modify the sequence length by, for example, trimming off adapters. It's a simple error check basically so downstream tools can detect when the data has been processed by something that's not MA-aware and will have broken the reported coordinates. This is something we didn't think about with base-modifications, and it bit us later.
• What's the rationale for splitting start and lengths up into two tags? Eg consider PT again which uses start and end. I don't really like that if there's a lot of data as start/length is smaller to encode, but that depends on how many items you'll be encoding.
• Is quality really mandatory? It feels like it runs the risk of people making it up when they don't have an easy way to assign quality.

[sathish-t]

Hello,

I am a researcher working with a DNA replication lab (https://www.earlham.ac.uk/profile/conrad-nieduszynski). We would like to support this tag proposal as well. A large part of our work involves parsing BrdU DNA modifications from BAM records (stored in the MM, ML tags) and marking single-molecule features associated with DNA replication such as initiation sites, replisome movement, replisome arrests and so on. This would fall under the "non-genetic-element" category that Mitchell mentioned. I can imagine us doing MA:Z:left_fork+.. for example. I think if this became a standard, this would help us develop standard pipelines and visualization tools.

Regards
Sathish

[mrvollger]

Thanks for the feedback! I have met with the hts-spec team and I have some updates to make and benchmarks to run but I think we have a clear path forward with a good chance for formal incorporation in the future.

[mrvollger]

@jkbonfield I did a test moving the length field into the MA tag directly instead of putting it into AL (start-length, instead of just start,). I find that this encoding (M2) is actually a tiny bit smaller than MA + AL on a test fiber-seq CRAM using samtools cram-size.

#   Content_ID  Uncomp.size    Comp.size   Ratio Method  Data_series
BLOCK  4279362    163951675     57691838  35.19% r       ALB
BLOCK  4280642     14109439      1448937  10.27% r_      AQB
BLOCK  5059162    356990775    145824692  40.85% R       M2Z
BLOCK  5063002    220790284     88570332  40.12% R       MAZ

I think I should probably eliminate the length tag and just include it directly in MA. Do you have any thoughts?