[Multi-Modal Image+Text] Explore available software packages to pre-process the text reports.

[sumedhasingla]

Location: /pghbio/dbmi/batmanlab/Data/radiologyTextDataset2/singla/RAD-ALL.deid

Keyword or concept tagging

• Noble Coder Named Entity Recognition (NER) engine for biomedical text. DBMI tool . Can be used with TIES

• TIES inbuild annotation tool

• Work with Mike to get the annotations which TIES stores for each report.

• Apache cTAKES

Negation identifier

• DEEPEN - https://github.com/jessica-glover/deepen
• NegEx - https://github.com/mongoose54/negex/tree/master/negex.python
  - https://github.com/chapmanbe/negex
• pyConTextNLP - https://github.com/chapmanbe/pyConTextNLP

Pre-processing for VQA

• Converting the report to question

Pre-processing for Image Captioning

• Converting the report to a template

[kayhan-batmanghelich]

@pyadolla the template is explained in this issue (one of the papers there):
#35

[kayhan-batmanghelich]

Here is a link to the standford software:
https://nlp.stanford.edu/software/lex-parser.shtml

it is based on this link:
https://stackoverflow.com/questions/19145948/converting-an-english-statement-into-a-questi0n

[sumedhasingla]

Relevant MICCAI 2018 paper: TextRay: Mining Clinical Reports to Gain a
Broad Understanding of Chest X-rays

[kayhan-batmanghelich]

In this paper, they used a tool from NIH called Medical Text Indexer. Here is what they did:

This might be helpful for tagging. Please take a look.

[kayhan-batmanghelich]

@pyadolla would you please add the results of the CliNER here for the record.

[sumedhasingla]

@Sumedha
Run TIES, Medical Text Indexer (MTI), CliNER on Finding and Impression sections of the report.

[kayhan-batmanghelich]

@sumedhasingla if you got some preliminary results from TIES, paste an example here.

[sumedhasingla]

NOBLE Tool, extensively tags the reports with the concepts + semantic type with a chosen thesaurus. I am using "NCI_Metathesaurus". The concepts found by NOBLe are used as input for pyContext to find the negations.
The result of NOBLE Tool tag on about 8k reports is at location: '/pghbio/dbmi/batmanlab/Data/radiologyTextDataset2/singla/RAD-ALL-NOBLE-ContextPY-ImageFileName.csv'

[sumedhasingla]

TIES annotation tool, cannot run the reports we have in RAD-ALL.deid as these reports were directly extracted from MARS and there is no way to query them or find them through TIES interface.

To process and get tags using TIES tool, we again have to extract reports from the TIES (500k) and save annotation information with the report. We ran this process through a small sample of about 5k reports. The results are at location: /pghbio/dbmi/batmanlab/Data/radiologyTextDataset2/Reports/test-concepts

The problem with this approach is, TIES can handle these annotation for only 5k files at a time. The process have to re-run after every 5K reports. Also, while building query in TIES to extract reports, the queries should be such that the number of reports , resulted from the query is atmost 5K.

As, TIES uses NOBLE Tool under the hood. So may be we can skip TIES annotation.

[sumedhasingla]

An analysis of the unique word in these 8k reports.
Vocabulary size: 7,495

Top-20 semantic type

Top-20 concept words