Small mutation report upload has gene finding failure if hgvsGenomic line is present

[matthewstuartedwards]

When uploading a Small Mutation using the command line ipr command, I get an 'ERROR gene finding failures for' message for records that contain all required fields plus hgvsProtein, hgvsCds and hgvsGenomic optional fields. If I remove the hgvsGenomic field only, I no longer get the gene finding failure error when uploading the report. I have reduced my dataset to a single record that can be used to reproduce the error. I also confirmed that the gene symbol is in the GraphKB database I am using.

I have a second case that does exhibits the same gene finding failure, but the removal of the hgvsGenomic line was not enough to have the gene be successfully found. Only the removal of all hgvs lines caused the gene to be found.

To Reproduce
I am running this from the pori_python repository at commit SHA: 3a7392b and have created a venv based on the instructions in the repository.
Steps to reproduce the behaviour:

1. Run command 'ipr -c ~/testJsonImportingSmall.txt --ipr_url 'http://ipr/api' --password iprdemo --username iprdemo --graphkb_password secret --graphkb_username graphkb_importer --graphkb_url 'http://graphkb/api''
2. with input 'testJsonImportingSmall.txt' as attached.
3. See error:
   Deprecated function 'create_report' called - use ipr_report instead
05-07-25 09:54:58 ipr INFO connecting to graphkb: http://graphkb/api
05-07-25 09:54:58 ipr INFO annotating 1 small mutations
05-07-25 09:54:59 ipr ERROR gene finding failures for ['TAS1R3']
05-07-25 09:54:59 ipr ERROR 1 gene finding failures for positional variants
05-07-25 09:54:59 ipr ERROR skipped 1 positional variants due to errors
05-07-25 09:54:59 ipr INFO matched 1 positional variants to 0 graphkb annotations
05-07-25 09:54:59 ipr INFO annotating 0 structural variants
05-07-25 09:54:59 ipr INFO matched 0 positional variants to 0 graphkb annotations
05-07-25 09:54:59 ipr INFO annotating 0 copy variants
05-07-25 09:54:59 ipr INFO Starting annotation of 0 copy category_variants
05-07-25 09:54:59 ipr INFO matched 0 copy category variants to 0 graphkb annotations
05-07-25 09:54:59 ipr INFO annotating 0 expression variants
05-07-25 09:54:59 ipr INFO Starting annotation of 0 expression category_variants
05-07-25 09:54:59 ipr INFO matched 0 expression variants to 0 graphkb annotations
05-07-25 09:54:59 ipr INFO fetching gene annotations
05-07-25 09:54:59 graphkb INFO fetching variant related genes list
05-07-25 09:54:59 graphkb INFO fetching oncogenes list
05-07-25 09:54:59 graphkb INFO fetching tumour supressors list
05-07-25 09:54:59 graphkb INFO fetching cancerGeneListMatch list
05-07-25 09:54:59 graphkb INFO fetching therapeutic associated genes lists
05-07-25 09:55:00 graphkb INFO Setting gene_info flags on 1 genes
05-07-25 09:55:00 ipr INFO generating analyst comments
05-07-25 09:55:01 ipr INFO Uploading to IPR http://ipr/api
Request payload size: 606 bytes
05-07-25 09:55:01 ipr INFO {'message': 'Report upload was successful', 'ident': 'd064abad-691d-4f79-814f-3a0990f6991f'}
05-07-25 09:55:01 ipr INFO made 109 requests to graphkb
05-07-25 09:55:01 ipr INFO average load 37 req/s

Expected behaviour
By removing the hgvsGenomic line, the gene finding failure is not longer present. I would not expect these optional input lines to have an effect on the gene finding. See attached testJsonImportingSmallNohgvsGenomic.txt and this is the log message:
ipr -c ~/testJsonImportingSmallNohgvsGenomic.txt --ipr_url 'http://ipr/api' --password iprdemo --username iprdemo --graphkb_password secret --graphkb_username graphkb_importer --graphkb_url 'http://graphkb/api'
Deprecated function 'create_report' called - use ipr_report instead
05-07-25 09:55:08 ipr INFO connecting to graphkb: http://graphkb/api
05-07-25 09:55:09 ipr INFO annotating 1 small mutations
05-07-25 09:55:09 ipr INFO matched 1 positional variants to 0 graphkb annotations
05-07-25 09:55:09 ipr INFO annotating 0 structural variants
05-07-25 09:55:09 ipr INFO matched 0 positional variants to 0 graphkb annotations
05-07-25 09:55:09 ipr INFO annotating 0 copy variants
05-07-25 09:55:09 ipr INFO Starting annotation of 0 copy category_variants
05-07-25 09:55:09 ipr INFO matched 0 copy category variants to 0 graphkb annotations
05-07-25 09:55:09 ipr INFO annotating 0 expression variants
05-07-25 09:55:09 ipr INFO Starting annotation of 0 expression category_variants
05-07-25 09:55:09 ipr INFO matched 0 expression variants to 0 graphkb annotations
05-07-25 09:55:09 ipr INFO fetching gene annotations
05-07-25 09:55:09 graphkb INFO fetching variant related genes list
05-07-25 09:55:09 graphkb INFO fetching oncogenes list
05-07-25 09:55:09 graphkb INFO fetching tumour supressors list
05-07-25 09:55:09 graphkb INFO fetching cancerGeneListMatch list
05-07-25 09:55:09 graphkb INFO fetching therapeutic associated genes lists
05-07-25 09:55:10 graphkb INFO Setting gene_info flags on 1 genes
05-07-25 09:55:10 ipr INFO generating analyst comments
05-07-25 09:55:12 ipr INFO Uploading to IPR http://ipr/api
Request payload size: 586 bytes
05-07-25 09:55:12 ipr INFO {'message': 'Report upload was successful', 'ident': 'aef54213-56c3-44c7-a4dc-85196ce8cc85'}
05-07-25 09:55:12 ipr INFO made 103 requests to graphkb
05-07-25 09:55:12 ipr INFO average load 33 req/s

Versions (please complete the following information):

• OS: Ubuntu 22.04

• Python: 3.12.4

• Versions

• • pori_python v0.1.2
• • pori_ipr_api release/v8.0.1
• • graphkb_api container: bcgsc/pori-graphkb-api:latest

• GraphKB and IPR URL are a local kubernetes cluster at UofC.

Additional context
Similar behaviour exhibited for even smaller case. See testJsonImportingCdsSmall.txt and testJsonImportingCdsSmallNoHGVSLines.txt. I had to remove all hvgs lines from the input for gene finding to succeed.

testJsonImportingSmall.txt
testJsonImportingSmallNohgvsGenomic.txt
testJsonImportingCdsSmallNoHGVSLines.txt
testJsonImportingCdsSmall.txt

[mathieulemieux]

Hi @matthewstuartedwards,
First, thank you for this superbly well-documented description!

The error you're getting is normal, but the message describing it is cryptic if not misleading, I'm giving you that!
What is happening is that GraphKB is trying to match every given represenation of a variant (hgvsGenomic, hgvsProtein & hgvsCds),
and while doing so is failing at matching 'NC_000001.11:g.1333917G>C' because the reference 'NC_000001.11' is not an existing Feature record in the database.

The error message "ipr ERROR gene finding failures for ['TAS1R3']" is meant to be interpreted as "failing at finding some of the references given to describe variation on gene TAS1R3".

At the GSC, we're using chromosome names for such cases, e.g. 'chr1:g.1333917G>C', where 'chr1' is actually an existing Feature record.

Adding the '--log_level debug' flag to the command line will give you more info, i.e.:

...
graphkb DEBUG /query - 200 - 22 ms
ipr DEBUG failed to match positional variants (NC_000001.11:g.1333917G>C): unable to find the gene (NC_000001.11) or any equivalent representations
ipr ERROR gene finding failures for ['TAS1R3']
ipr ERROR 1 gene finding failures for positional variants
ipr ERROR skipped 1 positional variants due to errors
...

I'll see what could be a better way to log that error without confusing existing users and I might end up making a PR for it.
Thank you for brigning that to our attention!

[mathieulemieux]

@elewis2 & @dustinbleile, what do you thing about replacing:

ipr DEBUG failed to match positional variants (NC_000001.11:g.1333917G>C): unable to find the gene (NC_000001.11) or any equivalent representations
ipr ERROR gene finding failures for ['TAS1R3']

by:

ipr DEBUG failed to match positional variants (NC_000001.11:g.1333917G>C): unable to find the reference (NC_000001.11) or any equivalent representations
ipr ERROR failed at finding references ['NC_000001.11']
ipr ERROR reference finding failures for genes ['TAS1R3']

[dustinbleile]

ipr ERROR failed at finding references ['NC_000001.11']
Is the error specifically about a chromosome reference?

NC_000001.11:g.1333917G>C

This is a position GRCh38 chromosome 1, right?
https://www.ncbi.nlm.nih.gov/nuccore/NC_000001.11

I think the GRCh37 chromosome definition would be:
https://www.ncbi.nlm.nih.gov/nuccore/NC_000001.10

@mathieulemieux - might just want to add those chromosomes to graphkb ?

[mathieulemieux]

@dustinbleile,

Is the error specifically about a chromosome reference?

not specifically. it's about the confusion in the log between not finding the reference of a variant vs. not finding the corresponding gene.

This is a position GRCh38 chromosome 1, right?
https://www.ncbi.nlm.nih.gov/nuccore/NC_000001.11
I think the GRCh37 chromosome definition would be:
https://www.ncbi.nlm.nih.gov/nuccore/NC_000001.10

yes; yes

might just want to add those chromosomes to graphkb ?

Sure, users (GSC, UofCalgary, etc) can add these references manually. We might also just update the pori_graphkb_loader/data/chromosomes.json file accordingly.

In any cases, I suggest updating the logging messages since it's for all reference errors.