Hello~ I'm considering implementing llm assisted features

[sickerin]

Hi~

I'm considering maybe adding an LLM assisted OCR extraction for the ones that might not be supported. I'm assuming that the unsupported statements are because it's too hard to extract using hand-coded logic? If not, would you be able to share the reasons why the debit statements can't be supported (or just that you haven't gotten around to like implementing it)?

My goal is to create something where each month I download all my statements from my banks (like DBS, OCBC) / credit card (DBS, Citi, etc) -> get csv/ structured data -> and upload to a self-hosted app, where I'm able to have one platform where I can have a complete overview of my finances. And the key things I want to be able to add comments/ tag them (like I forget what the transactions are for if I look at the statements like a few months later), ignore / resolve transactions when calculating my spend (i.e I pay 1234 to my credit card, so it should be balanced out, because the credit card statement contains the breakdown of where I made each spend). So, I'm hoping that monopoly would be able to help create the structured data necessary for that.

Initially I was using DBS Nav Planner to add comments, manually add transactions/ spends, but they lost my data.... (also they only retain 6 months history). So, my goal is to be able to add transaction/ transfer statements from banks, credit or debit transactions. And be able to generate budget and spend reports, that take into account transactions between my own accounts, i.e payments to credit-card, transfers between my accounts shouldn't contribute to my budget and spend reports.

Also, do you use any self hosted app to manage your budgets? I'm considering https://github.com/maybe-finance/maybe and https://github.com/actualbudget/actual?

[benjamin-awd]

Hey @sickerin,

would you be able to share the reasons why the debit statements can't be supported

Generally if I don't support the debit statement for a bank, it's because I don't have a statement from that particular bank and therefore I'm unable to add any support. I'm happy to work on it otherwise.

The LLM OCR is a nice idea, but it's a pretty tricky one since most users are somewhat reluctant to send their statements out to a remote server, and so far I've not been too impressed by the accuracy of the various LLM agents. It's also hard to validate transactions are correct, or whether transactions are missing. I'd still be open to a PR that enables this behind a feature flag though, if you'd like to give it a shot.

Also, do you use any self hosted app to manage your budgets? I'm considering https://github.com/maybe-finance/maybe and https://github.com/actualbudget/actual?

Personally I just upload my parsed statements to a cloud bucket, then load them into Gsheets via Google Bigquery. It's then possible to do some basic operations like tagging or filtering using SQL. I've not tried those particular apps, but Maybe does look quite nice. I'd be interested to see if you manage to do something with it.

[sickerin]

Ah I see, I saw that in the code that you had StatementConfig and transaction_pattern regex for DBS Debit and DBS Consolidated statement, so I was wondering whether it's because it was too inconsistent between pages/ statements or hard to solve.
(Edit on above: oops I misread the README, I thought that DBS wasn't ✅, because I got ValueError: No transactions found - statement extraction failed, I'll try to run the debugger to figure out what's wrong). I have Trust and SC Statement, so I might try adding them then~

Yes, for LLM OCR, I think it's kind of hard to achieve high accuracy without the training data. I do document OCR/ VLM data extraction, but even with the data, it's still not as good as having access to the raw pdf text data and writing hand-written rules. In the PDFs, I found that some Models are able to turn the semi structured text data to csv quite decently. But the challenge would be to find a model that can be small enough that might be able to run on a mac mini and be decent.

For traditional OCR from images, I would say PaddleOCR is the best I've found, we fine tune based on that. For open source VLMs, Qwen2.5VL-7B does quite well (I think the 3B model might be able to run on a mac mini), but they do tend to hallucinate.

[sickerin]

02/12/2024     Advice FAST Payment / Receipt                                                         1.00        1,234.56
INCOMING PAYNOW REF 12345
FROM: ABC HELLO HI
PAYNOW TRANSFER
OTHER
22/12/2024     Advice FAST Payment / Receipt                                                      1,00        1,234.56
<next transaction info>

This is what my DBS consolidated looks like. All matches seem to be skipping at

monopoly/src/monopoly/statements/base.py

Line 188 in d5b5be6

if self._check_bound(match):

The bound value for transactions lines after adding line.strip()

(99, 89, 84, 86, 86, 80, 80, 96, 78, 78, 78, 81, 80, 83, 83, 83, 93, 84, 84, 99)
(207, 197, 192, 194, 194, 192, 192, 208, 190, 190, 190, ...) # And without adding line.strip()

Other info, the raw lines themselves seem to have a lot of padded white space before and after, so perhaps that's why it exceeds the transaction bound. After doing line.strip it's able to be within bounds? And collect transactions, and the Balance amount will be in the line. Also, I have multiple accounts in the statement (I think it's not implemented).

The polarity info seems to not work quite well, perhaps because I added the line.strip(). This is what my statement looks like.

[benjamin-awd]

Hey @sickerin, apologies for the delayed response as I've been on holiday for the last week.

For the DBS consolidated statement, that is indeed a tricky one. To determine whether a transaction is negative or positive, the parser looks at the position of the amount and whether it is closer to the "withdrawal" or "deposit" column header. Stripping the line will therefore unfortunately break the polarity of a transaction. I've only had one statement to work with, so it could be that there's differences between statements that's not accounted for.

Also, I have multiple accounts in the statement (I think it's not implemented).

And yup, sadly this isn't implemented yet. If you want to give it a try, please feel free (but my hunch is that this might not be straightforward). Otherwise happy to take a look if you want to send the statement over.