Package only built for specific input csv formats

[jrwrigh]

Column contents and names are hard coded. They currently follow the Mint CSV export format.

There needs to be more flexibility to what CSV files can be imported. Suggest it be combined into the translation.json in someway.

[gorf]

I want to join the job. I lived in China. I want to import bank ledger to gnucash. Different bank has different csv files. I want to use an ini file to config them.

[jrwrigh]

I'm happy to hear you want to help out! Feel free to fork the repository, fix the changes, and submit a pull request. I'll try and compile a quick list of things that need to be changed.

I've done a good bit of commenting on the code, so hopefully you can follow along with what everything does there. Otherwise, feel free to ask any questions you have and I'll try and answer them (it's been awhile since I've looked at the project).

[jrwrigh]

List of things to make module data source agnostic. Some of these can be fixed by simply changing the hard-coded strings to dictionary lookups. Others (such as the debit/credit distinction, see below) will require more of an architecture switch. I've seperated them as such below.

Also, we'd probably want to have a separate .json to hold the settings required for different CSV formats. This so we can store a folder of different csv formats and users can simply drop them in separate from the translations.json file. I'll just call this csvoptions.json for now.

We'd also need to factor in whether or not certain CSVs have certain information to begin with (such as Notes, descriptions, etc.).

Simple Dictionary Changes (ie. different CSV headers):

1. "Amount" on line 180 of main.py

2. "Description", "Date", "Notes", and "Original Description" headers on lines 246-250 in main.py and on lines 295-298 in main.py

3. "Category" on lines 150-151 in main.py

4. "Account Name" on lines 144-145 in main.py

Architecture Switch:

1. Debit/Credit distinction on lines 137-141 of main.py.
   Mint doesn't export a negative or positive amount, so that sequence of statements will turn the "Amount" column negative or positive depending on whether the transaction is a debit or credit.

This will probably need to be split into different sets of logic statements. For example, in the csvoptions.json, there could be a switch for whether the credit/debit logic sequence is needed or if the amount is given as negative or positive transfers.

2. "Notes" and "Original Description" headers on lines 249-250 in main.py and lines 299-300 in main.py

These lines combines two separate strings, so this may have issues is users don't have anything to combine. Maybe something like:

listofstringheaders = ["Notes", "Original Description"]

if len(listofstringheaders) > 0:
   notestring = ''
   for header in listofstringheaders:
       notestring += ' ' + current_transaction[header]
   temp['note'] = notestring
else:
    temp['note'] = ''

where listofstringheaders would be defined in csvoptions.json.

I think that's it. Feel free to add as you find other things.

[gorf]

Thank you. I should read your code carefully first.

[EntPyle]

I'd also like to make this package more versatile. Some kind of translation builder tool would be key, I think, rather than trying to make the code smart enough to handle any csv thrown at it. I personally have 5 different transaction sources and would need at least 4 configurations to import everything smoothly. I think we can create an automatic translation builder than have a set of translation review and editing tools. A simple gui comes to mind for the last part. Thoughts?

[jrwrigh]

I think starting more minimal would be my preference. I'm a bit wary of trying to develop anything more automated than what I already have built in (namely the internal-transaction detection), as I'm still not completely trusting of it (hence the instructions on the README for how to double-check it). Since this tools is primary made for migration (ie. one-use), having to do a little bit of upfront work that "guarantees" (depending on the user) the system will work seems more preferable than purely relying on an automated system.

I'm honestly not really sure how an automated system would work as the inputs aren't guaranteed to be in any specific format (heading names, content, etc.). That said, I'm willing to hear ideas.

[gorf]

I worked on it recently. I found that I can not keep compatibility. So I make another package based on this project. Speaking of translation, I can translate from English to Chinese.

[EntPyle]

What I had in mind was a split process between expense accounts (categories on bank statements) and asset/liability accounts (bank accounts/credit cards) using an exact match search, regex match search for the remaining, followed by manual review and completion. It'd get the user well along the translation path.

…

On Mon, Dec 14, 2020, 4:06 AM gorf ***@***.***> wrote: I worked on it recently. I found that I can not keep compatibility. So I make another package based on this project. Speaking of translation, I can translate from English to Chinese. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#13 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIQ7MPHC32HRC3I2VHTUVFDSUXPSFANCNFSM4FO4CPGQ> .