johannesgerer / buchhaltung Goto Github PK

Smart tools for the ledger universe

Home Page: http://johannesgerer.com/buchhaltung

License: MIT License

Haskell 99.27% Shell 0.73%

buchhaltung's Introduction

Buchhaltung

What advantages does he derive from the system of book-keeping by double entry! It is among the finest inventions of the human mind; every prudent master of a house should introduce it into his economy. -- Johann Wolfgang von Goethe

Buchhaltung (['buːχˌhaltʊŋ], German book keeping), written in Haskell, helps you keep track of your finances on the command line with minimal effort. It provides tools that help you in creating a complete ledger of all your bank and savings accounts', credit cards', and other transactions, in a text-based ledger format, that is readable by the ledger CLI tool and its many derivatives.

Fetch your bank transaction directly via FinTS/HBCI/OFXDirectConnect
Import transactions from
- PayPal
- Barclaycard Visa
- comdirect Visa
- Revolut
- Barclays UK CSV
- NatWest International CSV
- and Monefy expense tracker app
- ... or define your own formats
Semi-automatically match transactions to accounts using Bayesian classification
Semi-automatic transaction entry with meaningful suggestions in keyboard-based speed mode
It is couples/room-mates aware: Create several transaction simultaneously (see Multi-user add)

Status & aim

I am actively and successfully using this software since 2010 and my ledger now contains more than 12,000 transactions accurately and continuously tracking the finances of my spouse and me including four checking and two savings accounts, one credit card, two PayPal accounts, two cash wallets in EUR, bitcoin trading (both physical and on exchanges) and other currencies like USD, GPB used on trips.

The software is in alpha phase and I am looking for early adopters and their use cases. The aim of this stage is to agree about the functionality and customizability and produce a first shippable version, that can be used without tinkering with the source.

Right now, I am using it on Linux but it should also run wherever GHC runs.

Installation

Prerequisites

Haskell Stack, more specifically the Glasgow Haskell Compiler and Stack

Required to compile the software.
AqBanking Command Line Tool (optional)

This is required for direct retrieval of bank transactions via FinTS/HBCI/EBICS (Germany) or OFXDirectConnect (USA, Canada, UK). Packages available e.g. on Ubuntu (aqbanking-tools) and ArchLinux (aqbanking). (AqBanking is also the used by GnuCash for this purpose.)
dbacl (optional, needed to run match)

Bayesian classifier used to match transaction to accounts. Packages available e.g. on Ubuntu and ArchLinux (AUR).
ledger CLI tool or a compatible derivative (optional)

... to query the ledger, create balance and report statements, web interface, etc.

Under the hood

AqBanking or any other supported CSV source provides the raw transaction data. The buchhaltung binary converts this data into the ledger format and puts the original source in the comments; thereby producing selfcontained transactions without the need for an external database or log. These comments are used later for duplicates detection and account classification.

The buchhaltung binary internally uses the great hledger library. The resulting ledger files can be processed/queried with one of the hledger tools or any other compatible tool.

Download, compile & install

# download
git clone https://github.com/johannesgerer/buchhaltung.git
cd buchhaltung


# compile and install (usually in ~/.local/bin)
stack install

Configure

Create a folder that will hold all your config and possibly ledger files:
```
mkdir ~/.buchhaltung
cp /path/to/buchhaltung/config.yml ~/.buchhaltung/config.yml
```
If you want a folder under a different location, either create a symlink or set the BUCHHALTUNG environment variable to that location.
Edit the config.yml.
Make sure the configured ledger files exist.

Getting help

The config.yml file provides excessive comments.
This readme documents most functionality.
Every command and subcommand shows a help message when invoked with -h.
Read the haddock documentation and source code on Hackage.
Open an issue.
Write an email.

Usage

First usage / clean

To initialize AqBanking after you edited the config file, you need to run:

buchhaltung setup

To clean everything aqbanking related remove the configured aqBanking.configDir and rerun the setup command.

Manual AqBanking setup

Currently only the HBCI PinTan method is supported by the setup command (pull requests welcome). For other methods or if the AqBanking setup fails due to other reasons, you can configure AqBanking manually into the configured aqBanking.configDir (see for help here or here, usually via aqhbci-tool4 -C <aqBanking.configDir>).

So far, I have not tested the OFXDirectConnect capabilities of AqBanking (please share your experiences), but buchhaltung should transparently support any method offered by AqBanking.

Importing transactions

There various ways (including from PayPal CSV files) to import transactions into your configured ledgers.imported file. They are presented in the following, but consult

buchhaltung import -h

and the -h calls to its subcommands to see the currently available functionality.

The accounts of the imported transactions will be taken from the configured bankAccounts and the offsetting balance will be posted to an account named TODO, and will be replaced by match.

The original source information will be included in the second posting's comment and used for learning the account mappings and to find and handle duplicates.

AqBanking

buchhaltung update

This command fetches and imports all available transactions from all configured AqBanking connections.

In order to import an existing file in AqBanking format, use:

buchhaltung import <file.csv> aqbanking

Other formats

For other CSV formats you have two options:

Use the importers for PayPal, Barclaycard Visa, comdirect Visa, Revolut, and Monefy expense tracker app.
Bring your CSV into a currently supported format. The AqBanking format works especially well
Hack the source, which is fun and should be easy since you only need to copy and adapt from the other importers.

You can look at aqbankingImporter, comdirectVisaImporter and paypalImporter in Buchhaltung/Importers.hs (or on GitHub).

Resolve duplicates

Banks often minimally change the way they report transactions which leads to unwanted duplicates.

When importing, Buchhaltung will identify duplicates based on ([(Commodity,Quantity)], AccountName, Day) and interactively resolve them by showing the user what fields have changed. If there are several candidates, it sorts the candidates according to similarity using levenshteinDistance from edit-distance. (See Buchhaltung.Uniques.addNew)

Match accounts

buchhaltung match

This command asks the user for the offsetting accounts of imported transactions, or more specifically, transaction whose second posting's account begins with TODO and does not match any regex in ignoredAccountsOnMatch.

Have a look at the example output here.

The significantly speed up this process, it learns the account mapping from existing transactions in the configured ledgers.imported file using the original source of the imported transaction.

Please note that you will have to enter account information in reverse order: for example Expenses:Food has to be entered as Food:Expenses. See this information about the account input field.

Best practices

The Bayesian classifier can only work if similar transactions are always matched with the same account.

Consider frequent credit card payments to Starbucks:

Match them with Expenses:Food:Starbucks if you know that these should always be booked to that account.
Match them with Accounts receivable:Starbucks and manually enter your paper receipts if
- you want to make sure they charge you the correct amounts.
- you sometimes order for friends and get reimbursed later.
- ...

See also this issues on how you could use this.

Enter transactions

buchhaltung add

This command opens a transaction editor. Here is an example of the output of this command.

The amount of manual typing is kept to a minimum by two clever suggestion mechanisms and TAB completion.

Input and TAB completion

All input fields save their history in the current directory. It can be browsed using up and down arrow keys.

The account input fields support TAB completion. To make this even more useful, the account hierarchy is read in reverse order. For example Expenses:Food has to be entered as Food:Expenses. This behaviour can be deactivated by setting reverseAccountInput to False.

Suggested transactions

After the amount is entered, the user can select a transaction whose title, date, amount and second posting's account will used to pre fill an offsetting transaction. Suggestions will consist of all transactions

whose second posting has not been cleared (i.e. marked with an asterisk in front of the account name, and
whose first posting's amount has the absolute value as the entered amount
whose first posting's account is contained in the configured bankAccounts and does not match any of regexes in ignoredAccountsOnAdd.

Suggested accounts

Once the first posting's account has been entered, the editor suggests accounts for the second posting based on the frequency of the resulting transaction's accounts in the configured ledgers.addedByThisUser file.

Assertions & assignments

Amounts can be entered with assertions or can be assigned.

Default currency

In order to be able to enter naked amounts and have the currency added automatically, add a default currency to the configured addedByThisUser ledger file. Example:

D 1,000.000 EUR

Multi-user add

buchhaltung add -w alice

If there is more than one user configured — possibly each with their own ledger, they can be included/activated via the command-line argument -w. This enables you to enter a transaction where postings belong to different users. When done, a transaction for each user will be generated containing their respective postings and a balancing posting to an account prefixed with the configured accountPrefixOthers.

Example taken from the output of the above command:

#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

2016/12/19 Dinner

        Account          |  Amount  | Assertion
-------------------------+----------+-----------+----------
0,jo    Wallet:Assets    | $ -100.0 |           |
1,jo    Food:Expenses    | $ 50     |           |
2,alice Food:Expenses    | $ 50     |           |
-------------------------+----------+-----------+----------
Open Balance             | 0        |

generates the following transactions

#######  jo:  Balanced Transaction   #######

2016/12/19 Dinner    ; Entered on "2016-12-19T19:01:00Z" by 'buchhaltung' user jo
    Wallet:Assets                             $ -100.0
    Food:Expenses                                 $ 50
    Accounts receivable:Friends:alice:jo          $ 50



#######  alice:  Balanced Transaction   #######

2016/12/19 Dinner    ; Entered on "2016-12-19T19:01:00Z" by 'buchhaltung' user jo
    Accounts receivable:Friends:jo:jo         $ -50
    Food:Expenses                              $ 50

Getting results

Get current AqBanking account balances

buchhaltung lb

Call `ledger` or `hledger`

buchhaltung ledger

buchhaltung hledger

This calls the respective program with the LEDGER environment variable set to the configured mainLedger or mainHledger.

Commit the changes

buchhaltung commit -a -m'checking account ok'

buchhaltung commitHledger -a -m'checking account ok'

this commits all changes to the git repository that contains the mainLedger file. The commit message will also contain the output of buchhaltung lb and buchhaltung ledger balance --end tomorrow.

Todo

Try to get Barclays UK direct feed working? https://stackoverflow.com/questions/3469628/banking-api-protocol

buchhaltung's People

Contributors

Stargazers

Watchers

Forkers

hvr ohbadiah jottr atgreen silky gppassos dpaetzel

buchhaltung's Issues

Edit transaction purpose during match

Often I'd like to add a comment on a transaction to the purpose field during the match process.
It would be great if one could edit/add to the purpose on the fly during match (by pressing p i.e.).

Import of paypal transaction fails

I'm not sure if this is an issue with my shell configuration or rather an issue with how buchhaltung parses Umlauts.

$ buchhaltung import /tmp/paypal.csv paypal jottr
buchhaltung: Headers do not match. Expected by format 'paypal' version '2016':
["Datum"," Zeit"," Zeitzone"," Name"," Typ"," Status"," W\228hrung"," Brutto"," Geb\252hr"," Netto"," Von E-Mail-Adresse"," An E-Mail-Adresse"," Transactionscode"," Status der Gegenpartei","
Adressstatus"," Artikelbezeichnung"," Artikelnummer"," Betrag f\252r Versandkosten"," Versicherungsbetrag"," Umsatzsteuer"," Option 1 - Name"," Option 1 - Wert"," Option 2 - Name"," Option 2 - Wert"," Auktions-Site"," K\228ufer-ID"," Artikel-URL"," Angebotsende"," Vorgangs-Nr."," Rechnungs-Nr."," Txn-Referenzkennung"," Rechnungsnummer"," Individuelle Nummer"," Belegnummer"," Guthaben"," Adresszeile 1"," Zus\228tzliche Angaben"," Ort"," Staat/Provinz/Region/Landkreis/Territorium/Pr\228fektur/Republik"," PLZ"," Land"," Telefonnummer"," "]
Given:
["Datum"," Zeit"," Zeitzone"," Name"," Typ"," Status"," W\195\164hrung"," Brutto"," Geb\195\188hr"," Netto"," Von E-Mail-Adresse"," An E-Mail-Adresse"," Transactionscode"," Zahlungsart","Status der Gegenpartei"," Adressstatus"," Artikelbezeichnung"," Artikelnummer"," Betrag f\195\188r Versandkosten"," Versicherungsbetrag"," Umsatzsteuer"," Option 1 - Name"," Option 1 - Wert"," Option 2 - Name"," Option 2 - Wert"," Auktions-Site"," K\195\164ufer-ID"," Artikel-URL"," Angebotsende"," Txn-Referenzkennung"," Rechnungsnummer"," Individuelle Nummer"," Belegnummer"," Guthaben"," Adresszeile 1"," Zus\195\164tzliche Angaben"," Ort"," Staat/Provinz/Region/Landkreis/Territorium/Pr\195\164fektur/Republik"," PLZ"," Land"," Telefonnummer"," "]

CallStack (from HasCallStack):
  error, called at src/Buchhaltung/Commandline.hs:37:11 in main:Buchhaltung.Commandline

provide example config

from the README:

cp config-sample ~/.buchhaltung/config

however, the config-sample file is missing, unfortunately.

Find a better name for the project

Please leave suggestions and your votes in this thread.

Consider expanding duplicate detection

I import transactions from both a checking account and a credit card. When I pay the credit card from the checking account, the transaction is imported twice, once for each account:

2017/12/04 ACH WEBSINGLE  XXXXX1234 BARCLAYCARD US   CREDITCARD
    Assets:Checking:PNC                     -1,234.56 USD
    Liabilities:Credit Cards:Barclaycard     1,234.56 USD    ; 

2017/12/02 Payment Received    
    Liabilities:Credit Cards:Barclaycard     1,234.56 USD
    Assets:Checking:PNC                     -1,234.56 USD    ;

I'm unsure if this is best handled through some accounting practice I'm unaware of, or if we might adjust duplicate detection to catch this case.

State of this project

Hello @johannesgerer!

Are there any plans to advance this project? Or fix some of the more urgent bugs such as #40 (buchhaltung is incompatible with more recent versions of aqbanking but since PSD2 is in effect, old aqbanking releases cannot connect to any bank anymore unless you use one of the latest beta releases)?

This is not meant to blame or anything; I just really loved the simplicity of doing buchhaltung update—which is not possible anymore for me due to PSD2 and #40.

If you'd like to continue and need help: I'd be willing to! 🙂

Documentation: Explain relationship between different ledger files

I've tried setting up buchhaltung with the mainLedger and imported keys set to distinctive ledger files.
After reading the documentation and the comments in config.yml, I still can't quite grasp the relationship between the two files.
I first understood it like so: imported stores all the transactions recently imported via the update command.
After they were matched via the match command, the newly matched transaction are written to the mainLedger-file.

This seems to not be the case, since only the imported-ledger is edited in place, i.e. the information is only stored in the imported ledger, and not transferred into mainLedger.

Feature Request: date range as argument for update and/or match command

Upon first import of my bank statements I get several thousand statements.
I don't have the need to run the matcher on the full transaction history.
Instead it would be sufficient (less cumbersome) to be able to pass the update/match-commands either a date range, i.e. somedate-today or a certain number of n transactions which should be run through the matcher.

Use it only for AqBanking auto-import?

Hi Johannes,

I stumbled over this project when I searched for an importer for banking transactions over HBCI.
I've already setup AqBanking, I even have written a rules file to import the downloaded CSV.

But I wonder what's the best way to import it into the ledger journal without getting duplicates.

My best idea is to let AqBanking download transactions

from the last ledger transaction + 1 day
to yesterday.

(My bank does not provide transactionIds with the transactions.)

Then I would add it to the journal and call hledger check-dups.

But how do you solve this problem?

Using buchhaltung and gnucash at the same time

I've been using gnucash and i recently started trying to use aqbanking. It seems to me that they will both use the same accounts in aqbanking which in turns means whenever I use one it will override the state of the other. So buchhaltung on it's first use will add the transactions since the last gnucash fetch. Maybe I'm wrong and something else happens, but at the very least aqbanking seems to remember when the last transaction was fetched.

Is there any way i can make buchhaltung fetch all transactions?

Unable to get credit card transactions

I have a bank account that is tied to a credit card. When I log into the account (German bank) I can access both the checking account and the credit card.

When running buchhaltung setup, aqBanking correctly finds two accounts (Account 0 and Account 1) with account 0 being the checking and 1 being the credit card. It only creates one folder, for Account 0, but not for the credit card account.

When running buchhaltung update, only the transactions from the checking account at imported, but not the credit card.

In my config.yml, I have both the numbers of the account and the credit card listed under the same bank ID.

What am I doing wrong?

Seed autocompletion from "account <x>" lines or a bogus transaction

I created a accountnames.ledger containing a list of statements like:

account Expenses:Cats
account Expenses:Dogs
account Income:Sausages

and so on. then i include accountnames.ledger in my main ledger file. The hope was that buchhaltung would use these to seed TAB-autocompletion in match or add. However, it seems it does not.

Then, I tried to workaround this by generating a bogus transaction with each of these accounts involved at 0 EUR for each account, and put this on top of the import-ledger. Again, buchhaltung did not pick it up as autocompletion targets ….

I believe it’d be way more convenient for users who prefer a fixed list of pre-defined accounts to seed auto-completion like this. It also makes accidentally entering invalid or typo-ed accounts less likely.

Nevertheless, thanks for your great tool ^_^

Invalid flag name: \"bytestring--lt-0_10_4\"" during install

Using debian's haskell-stack 1.1.2-7+b1 I get the following install error:

$ stack install
Downloaded lts-12.11 build plan.    
AesonException "Error in $.packages.cassava.constraints.flags: failed to parse field packages: failed to parse field constraints: failed to parse field flags: Invalid flag name: \"bytestring--lt-0_10_4\""

Do I need more up to date dependencies, or is something else wrong?

buchhaltung match needs multiple iterations for all open transactions

buchhaltung update found 12 new transactions:

found 12 new of 143 total transactions

First round of buchhaltung match:

» buchhaltung match
...
Current Transaction: 1, Remaining: 11
..
Current Transaction: 3, Remaining: 9
...
Current Transaction: 5, Remaining: 7
...
Current Transaction: 7, Remaining: 5
...
Current Transaction: 9, Remaining: 3
...
Current Transaction: 11, Remaining: 1
...
Current Transaction: 12, Remaining: 0
...
Current Transaction: 12, Remaining: 0
<< DONE! Use 'save' to exit >>
[<, >, save, RET]:	save

3 Transactions were changed

But we need more rounds to match all transactions.

Second round:

» buchhaltung match
...
Current Transaction: 1, Remaining: 4
...
Current Transaction: 3, Remaining: 2
...
Current Transaction: 5, Remaining: 0
...
Current Transaction: 5, Remaining: 0
<< DONE! Use 'save' to exit >>
...
3 Transactions were changed

Third round
...

Better documentation for "buchhaltung update"

I'm trying buchhaltung for the first time. Setting it up, I failed after buchhaltung update because the ledgers.imported file is not present, and I have no idea how to add it properly:

The hledger journal file "/home/varac/hledger/buchhaltung-import.ledger" was not found.
Please create it first, eg with "hledger add" or a text editor.
Or, specify an existing journal file with -f or LEDGER_FILE.

Is this file mandatory ? When I don't want to import already existing .csv files but start fresh with buchhaltung ?

cabal install errors

src/Buchhaltung/Add.hs:23:1: error:
    Failed to load interface for ‘Hledger.Cli.Options’
    Perhaps you meant
      Hledger.Cli.CliOptions (from hledger-0.27.1)
      Hledger.Cli.Utils (from hledger-0.27.1)
      Hledger.Cli.Main (from hledger-0.27.1)

After fixing this (using the suggested Hledger.Cli.CliOptions):

src/Buchhaltung/Add.hs:190:41: error:
    • Couldn't match expected type ‘ClearedStatus’
                  with actual type ‘Bool’
    • In the ‘pstatus’ field of a record
      In the expression: x {pstatus = True}
      In the first argument of ‘changeNth’, namely
        ‘(\ x -> x {pstatus = True})’

src/Buchhaltung/Add.hs:210:19: error:
    • Couldn't match expected type ‘Bool’
                  with actual type ‘ClearedStatus’
    • In the first argument of ‘not’, namely ‘(pstatus p2)’
      In the first argument of ‘(&&)’, namely ‘not (pstatus p2)’
      In the second argument of ‘(&&)’, namely
        ‘not (pstatus p2) && not (or $ ignoredAccounts <*> [paccount p2])’

src/Buchhaltung/Add.hs:235:27: error:
    • Couldn't match expected type ‘ClearedStatus’
                  with actual type ‘Bool’
    • In the pattern: False
      In the pattern: Posting {pstatus = False}
      In an equation for ‘toMyP’:
          toMyP t p@(Posting {pstatus = False})
            = MyP (tdate t) (paccount p) <$> listToMaybe (amounts $ pamount p)

buchhaltung match changes non-imported transactions in import file.

I have both my main hledger as well as the imported transactions from buchhaltung in version control.
I noticed that during a buchhaltung match, transactions in the imported transactions file are changed by removing some whitespaces, even those that I didn't import:

-2017/08/14 Mr. Foo, Barbar                       ; generated by 'buchhaltung' 2017-08-16 01:19:48.256125 CEST
+2017/08/14 Mr. Foo, Barbar    ; generated by 'buchhaltung' 2017-08-16 01:19:48.256125 CEST

I.e. I run buchhaltung match, import one single transaction, that save and exit, and 5 others have changed in the way described above.

This messes up proper version control so a fix is appreciated.

Make reverse order of accounts optional

I find the reverse order of accounts during the matching process confusing and would like to use the original order instead. Please add a command-line/config file parameter to make this optional, thx!

Consider splitting `add` or adding flag for truly manual transactions

Despite the Starbucks example in the documentation, issues #13 and #16 indicate some confusion about the intended purposes of match and add. Here's what I've teased out:

match is meant to complement import and update. Accounts chosen during match should be deterministic so that:
1. The classifier can consistently choose the right account for everything, making the process mostly a rubber-stamp.
2. The import ledger effectively reproduces the record from the real accounts transactions are being imported from.
add is used to create transactions offsetting those from the import:
1. By entering the amount, the desired source transaction can be automatically selected.
2. Once a destination account is chosen and the newly entered transaction is finished, the source transaction is marked so it doesn't come up in a subsequent add.
But add is also used to manually enter transactions that will not appear on any bank or credit account statement. Cash transactions are the most notable example. No one is tracking them but oneself, so there's no electronic record to pull them from. But they constitute a nontrivial part of one's finances.

Both of these uses of add are similar in that they won't match a bank statement and they are ad-hoc and per-transaction, but they are nontrivially different as well. The second use is for real transactions, not just for bookkeeping and classification purposes, and should never offset imported transactions.

At the least, the documentation of this process and these distinctions could be improved. But perhaps these two senses of add should be split into two separate subtly-different commands. The second sense seems to me to reflect the word add better; the first sense might be called offset or whatever the accounting term for this process is called. If two separate commands aren't called for, perhaps a flag to add could distinguish between them.

buchhaltung on stackage

Please add your package to Stackage as described here.

`add` again suggests tranactions that have been offset

Generally, when you have buchhaltung added an offsetting transaction to an imported one, it will no longer be suggested in a subsequent run of buchhaltung add. However, this seems to break down when you change the title of the offsetting transaction (using the t command during add).

How is buchhaltung supposed to match offsetting transactions to the corresponding imported one? Is the title used for this?

From a quick reading of the code, the "status" (cleared/unmarked) of transactions is used when deciding which ones to suggest for add. Is add supposed to add a "cleared" flag (*) to the imported transaction when it creates a corresponding offsetting one? Apparently it doesn't do that at the moment.

A better way to handle Starbucks

I'm referring to the Starbucks example from the documentation. I'm still new to this, but I find it difficult to manually add corresponding and offsetting transactions to the more general categories I choose for the sake of consistency during match. What if it were possible to add offsetting transactions during the match process itself? Or perhaps a posting or transaction could be marked to be ignored by the classifier.

Please follow the XDG Base Dir Spec

Although buchhaltung allows specifying a custom location for configuration, I'm tired of having almost 50 env vars at this point to accomodate for apps that don't conform to how application should store data on Linux and related environments. Instead, it would be nice to have sane defaults that follow the spec and falls back to the legacy default locations.

There is a nice haskell library that implements this.

buchhaltung commit should work with hledger

I'm using hledger and started using buchhaltung - first of all thank you for this project !!!
It would be great if buchhaltung commit would would with hledger, currently it doesn't:

--- ~/hledger ‹master* AM⁇› » buchhaltung commit -m'First steps with buchhaltung match'
buchhaltung: ledger: readCreateProcess: runInteractiveProcess: exec: does not exist (No such file or directory)

`match` does not go through transactions sequentially

buchhaltung match seems to only go through the odd numbered transactions in its queue. Here's a snippet:

Current Transaction: 27, Remaining: 57
uncertainty: 4.88	Groceries:Food:Expenses
[<, >, save, RET]:
Learning: Expenses:Food:Groceries
Done:     Expenses:Food:Groceries
Field            | Value
---------------- | --------------------
Amount           | -2.50
Category         | DEBIT
Description      | PPA ON STREET METERS
Transaction Date | 04/09/2017
formatName       | barclaycard
formatVersion    | May 2017

Current Transaction: 29, Remaining: 55
uncertainty: 4.96	Parking:Car:Transportation:Expenses
[<, >, save, RET]:
Learning: Expenses:Transportation:Car:Parking
Done:     Expenses:Transportation:Car:Parking
Field         | Value
------------- | ----------------------------------
Date          | 04/11/2017
Description   | ACH WEBRECUR XXXXX4679PA AES STDNT
              | LOAN
Withdrawals   | $123.45
formatName    | pncbank
formatVersion | May 2017

Current Transaction: 31, Remaining: 53

Navigation using < and > is still possible.

First time AqBanking update complains about unfound AccountId

Hey,
so I've successfully run buchhaltung setup. The problems lies in buchhaltung update. I get (data truncated and anonymized):

[...]
3:2018/02/11 20-56-14:aqbanking(26960):banking.c:  672: Bad IBAN (country code not in upper case)
AqHBCI finished.
Executing Jobs: 3 of 3
Postprocessing jobs
Job Get Transactions: error
Job Get Transactions: error
Job Get Transactions: finished
Resetting provider queues
Executing Jobs: Finished.
buchhaltung: lookupErr: AccountId {aBank = "43060967", aAccount = "MY_ACCOUNT_NUMBER"} not found: Account not configured
CallStack (from HasCallStack):
  error, called at src/Buchhaltung/Commandline.hs:38:11 in main:Buchhaltung.Commandline

The relevant part of my config.yml:

     aqBanking:                                                                                        
       configDir: aqbanking                                                                            
       connections:                                                                                    
         - user: 'MY_LOGIN_NAME'                                                                 
           blz: '43060967'                                                                         
           url: "https://hbci-pintan.gad.de/cgi-bin/hbciservlet"                                       
           name: 'GLS'                                                                                 
           hbciv: HBCI300                         
           type: PinTan

I've never used ledger before, so my ledger files are totally empty by now. What do I need to enter into them so that the update works?

buchhaltung update duplicate transaction

Hi, after using buchhaltung for a few weeks now I came across one credit card payment that results in 4 transactions like this:

» hledger reg amt:262.78
2017/06/29 ABC BANK  ... USA ... USD 290,00 K..  Aktiva:ABC:Kreditkarte                                   -262,78 EUR   -262,78 EUR
                                                                   TODO:Positive                                             262,78 EUR             0
2017/07/27 ABC Bank eG  Kreditkarte Abrechnung         Aktiva:ABC:Giro                                          -262,78 EUR   -262,78 EUR
                                                                   TODO:Positive                                             262,78 EUR             0
2017/07/28 ABC Bank  Abrechnung vom 18/07/17 Kreditkarte            Aktiva:ABC:Kreditkarte                                   -262,78 EUR   -262,78 EUR
                                                                   TODO:Positive                                             262,78 EUR             0
2017/07/28 ABC Bank  Abrechnung vom 18/07/17 Kreditkarte            Aktiva:ABC:Kreditkarte                                    262,78 EUR    262,78 EUR
                                                                   TODO:Negative                                            -262,78 EUR             0

When I remove these transactions from the buchhaltung import file, and run buchhaltung update again, they appear in exact the same manner.

How could I supply more details in a safe way ? GPG/Mail ?

Support aqbanking v6

I'd like to use the new (yet unreleased) aqbanking v6. Some cli arguments changed, so some buchhaltung cmds don't work anymore, i.e.:

varac@aqbanking_container:~/.aqbanking$ buchhaltung setup
3:2019/04/15 13-04-51:gwen(39):args.c:  124: Unknown short option "C"
ERROR: Could not parse arguments main
buchhaltung: callProcess: aqhbci-tool4 "-C" "/home/varac/.aqbanking/xxx-yyy" "adduser" "-t" "pintan" "--context=1" "-b" "xxx" "-u" "zzz" "-s" "..." "-N" "..." "--hbciversion=300" (exit 1): failed

varac@aqbanking_container:~/.aqbanking$ aqhbci-tool4 --help
This is version 5.99.23
Usage: aqhbci-tool4 [GLOBAL OPTIONS] COMMAND [LOCAL OPTIONS]

Global Options:

 [-D PARAM]
 [--cfgdir=PARAM]
   Specify the configuration folder

 [-n]
 [--noninteractive]
   Select non-interactive mode.
   This automatically returns a confirmative answer to any non-critical
   message.

 [-A]
 [--acceptvalidcerts]
   Automatically accept all valid TLS certificate

 [--charset=PARAM]
   Specify the output character set

 [-P PARAM]
 [--pinfile=PARAM]
   Specify the PIN file

 [-h]
 [--help]
   Show this help screen. For help on commands, run aqbanking-cli <COMMAND> --help.

Will the transaction download via aqbanking continue to work as PSD2 will be rolled out?

Thanks for the nice project!! Very appreciated!

I'm just trying to understand the current situation around the ongoing regulatory changes. Will the transaction download via aqbanking continue to work as PSD2 compliance will be implemented out across the German banks? I failed to find a good resource on this.

Better documentation for the match process

I still haven't figuered out what the '>' or '<' symbols mean in the match process prompt:

`[<, >, save, RET]:`

Please update the documentation so all options are described.
Futher more, how do I skip one transaction that I don't want to match right now and carry on matching ?

Commit command broken in latest master

Using latest master (291ce75) it looks like the commit command no longer forwards the command line arguments to git:

$ buchhaltung commitHledger -a -m'checking account ok'
Invalid option `-a'

Usage: buchhaltung [-u|--user USER] [-p|--profile FOLDER] COMMAND
[…]

It'll work if I split the arguments like $ buchhaltung commitHledger -- -a -m'checking account ok'.

Split import.hledger into multiple files

After importing a few years worth of transactions from multiple accounts, the import.hledger file grows to ~2MB and becomes no longer navigable with proper syntax highlighting / etc.

Is there a way to split the import file into multiple (perhaps year-based) files and have the buchhaltung update command just file everything into the appropriate smaller files?

Implement version command

to find out which version of buchhaltung is installed (or how could I find out otherwise ?).

Support for custom aqbanking settings (TLS prematurely terminated)

Hi @johannesgerer ,

I came across your project and like it very much, thank you.

One of my banks however gave me a bit of trouble when setting up buchhaltung and aqbanking: their server seems to prematurely terminate the TLS connection failing all aqbanking requests unless a specific flag (tlsIgnPrematureClose) is set.

Initially I tried to manually set-up aqbanking but then buchhaltung complained about a missing .context which seems to be created during buchhaltung setup.

Currently my solution is to hardcode the setting of the flag for all accounts (and temporarily commenting out the other accounts for the setup):

diff --git a/src/Buchhaltung/AQBanking.hs b/src/Buchhaltung/AQBanking.hs
index 8b0841a..4999954 100644
--- a/src/Buchhaltung/AQBanking.hs
+++ b/src/Buchhaltung/AQBanking.hs
@@ -92,6 +92,7 @@ aqbankingSetup = do
             , "-s", aqUrl conn
             , "-N", aqName conn
             , "--hbciversion=" <> toArg (aqHbciv conn)]
+  callAqhbci [ "adduserflags", "-b", aqBlz conn, "-f", "tlsIgnPrematureClose" ]
   callAqhbci [ "getsysid" ]
   callAqhbci [ "getaccounts" ]
   callAqhbci [ "listaccounts" ]

If I could do more in Haskel then blindly copying lines of code I could image to add a userflags parameter to the aqBanking/connections section of the config.yml

Other than that maybe this issue can help others when they face issues with the HBCI setup part of buchhaltung to find a suitable workaround.

Newer hledger version

Why is the hledger dependency specified as hledger >= 1.1 && < 1.2 and hledger-lib >= 1.1 && < 1.2?
For me (nixos-unstable), compilation with stack works fine with hledger >= 1.1 and hledger-lib >= 1.1, which results in using hledger 1.2.

Additional means for update beyond AqBanking

It seems that plaid may be an affordable API for connecting to banks. This is necessary in the USA because we lack HBCI, and what we do have (OFX) is unworkable.

I'd love to try out connecting to plaid via buchhaltung, but I think that means I'd need to integrate plaid with AqBanking. AqBanking is open source, but I'm not proficient enough in C++ to give it a whirl, and it seems undesirable that we should rely on a C++ project for all such functionality.

A similar use case: I track my work time in a Trello board. I've considered connecting to the Trello API and generating transactions from my work log entries, which will tell me how much I can expect to be paid based on an hourly wage.

Does anyone have any ideas what the best way would be to integrate another http client?

Failed to learn account: dbacl:warning: dirichlet measure did not converge

I'm not even sure if this is the correct place to report it, but during buchhaltung match I repeatedly stumble upon errors when learning from my accounts:

Done:     Assets:Receivable:Misc
Failed:   Expenses:Personal:Household
with Code ExitSuccess

And error:
dbacl:warning: dirichlet measure did not converge.

Do you have any hints what what could cause this / what I should look for in my imported ledger file?

Allow absolute path in config.yml

I've tried to properly set up the path to my journal files in the relevant config.yml but found it difficult to do so.
Neither the documentation nor the config example make it obvious whether the mainLedger and imported keys expect absolute or relative paths.
I.e. I would like to point them to ~/jottr/.ledger/, but when using the k/v pair of mainLedger: ~/.ledger/, buchhaltung setup creates a subdirectory in ~/.buchhaltung/~/jottr/.ledger instead.

Is there a means to config buchhaltung with absolute paths?

Build error on Debian

Using cabal:

Building buchhaltung-0.0.7...
Failed to install buchhaltung-0.0.7
Build log ( /home/richih/.cabal/logs/ghc-8.4.4/buchhaltung-0.0.7-5R7USOtuokwHjZGxbAxU.log ):
cabal: Entering directory '/tmp/cabal-tmp-7060/buchhaltung-0.0.7'
Configuring buchhaltung-0.0.7...
Preprocessing executable 'buchhaltung' for buchhaltung-0.0.7..
Building executable 'buchhaltung' for buchhaltung-0.0.7..
[ 1 of 15] Compiling Buchhaltung.Utils ( src/Buchhaltung/Utils.hs, dist/build/buchhaltung/buchhaltung-tmp/Buchhaltung/Utils.o )
[ 2 of 15] Compiling Buchhaltung.Types ( src/Buchhaltung/Types.hs, dist/build/buchhaltung/buchhaltung-tmp/Buchhaltung/Types.o )
[ 3 of 15] Compiling Buchhaltung.Ask  ( src/Buchhaltung/Ask.hs, dist/build/buchhaltung/buchhaltung-tmp/Buchhaltung/Ask.o )
[ 4 of 15] Compiling Buchhaltung.Common ( src/Buchhaltung/Common.hs, dist/build/buchhaltung/buchhaltung-tmp/Buchhaltung/Common.o )

src/Buchhaltung/Common.hs:154:11: error:
    • Couldn't match expected type ‘Bool
                                    -> FilePath -> IO (Either [Char] Journal)’
                  with actual type ‘IO (Either String Journal)’
    • The function ‘readJournalFile’ is applied to four arguments,
      but its type ‘InputOpts
                    -> PrefixedFilePath -> IO (Either String Journal)’
      has only two
      In a stmt of a 'do' block:
        ej <- readJournalFile Nothing Nothing False journalPath
      In the second argument of ‘($)’, namely
        ‘do ej <- readJournalFile Nothing Nothing False journalPath
            let (j, n) = either error change ej
            if n == 0 then
                putStrLn "\nNo transactions were changed!\n"
            else
                do let ...
                   ....
            return
              $ maybe
                  j
                  (\ j
                     -> let ...
                        in
                          if (n == m) then
                              j2
                          else
                              error
                                $ printf
                                    "Error 123, see source code. Solution: Use a proper database instead of a file. read: %d passed: %d"
                                    n
                                    m)
                  journal’
    |
154 |     ej <- readJournalFile Nothing Nothing False -- ignore balance assertions
    |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^...

src/Buchhaltung/Common.hs:154:27: error:
    • Couldn't match expected type ‘InputOpts’
                  with actual type ‘Maybe a2’
    • In the first argument of ‘readJournalFile’, namely ‘Nothing’
      In a stmt of a 'do' block:
        ej <- readJournalFile Nothing Nothing False journalPath
      In the second argument of ‘($)’, namely
        ‘do ej <- readJournalFile Nothing Nothing False journalPath
            let (j, n) = either error change ej
            if n == 0 then
                putStrLn "\nNo transactions were changed!\n"
            else
                do let ...
                   ....
            return
              $ maybe
                  j
                  (\ j
                     -> let ...
                        in
                          if (n == m) then
                              j2
                          else
                              error
                                $ printf
                                    "Error 123, see source code. Solution: Use a proper database instead of a file. read: %d passed: %d"
                                    n
                                    m)
                  journal’
    |
154 |     ej <- readJournalFile Nothing Nothing False -- ignore balance assertions
    |                           ^^^^^^^

src/Buchhaltung/Common.hs:154:35: error:
    • Couldn't match type ‘Maybe a3’ with ‘[Char]’
      Expected type: PrefixedFilePath
        Actual type: Maybe a3
    • In the second argument of ‘readJournalFile’, namely ‘Nothing’
      In a stmt of a 'do' block:
        ej <- readJournalFile Nothing Nothing False journalPath
      In the second argument of ‘($)’, namely
        ‘do ej <- readJournalFile Nothing Nothing False journalPath
            let (j, n) = either error change ej
            if n == 0 then
                putStrLn "\nNo transactions were changed!\n"
            else
                do let ...
                   ....
            return
              $ maybe
                  j
                  (\ j
                     -> let ...
                        in
                          if (n == m) then
                              j2
                          else
                              error
                                $ printf
                                    "Error 123, see source code. Solution: Use a proper database instead of a file. read: %d passed: %d"
                                    n
                                    m)
                  journal’
    |
154 |     ej <- readJournalFile Nothing Nothing False -- ignore balance assertions
    |                                   ^^^^^^^

src/Buchhaltung/Common.hs:464:42: error:
    • Couldn't match expected type ‘Bool
                                    -> FilePath -> IO (Either String Journal)’
                  with actual type ‘IO (Either String Journal)’
    • The function ‘readJournalFile’ is applied to three arguments,
      but its type ‘InputOpts
                    -> PrefixedFilePath -> IO (Either String Journal)’
      has only two
      In the first argument of ‘mapM’, namely
        ‘(readJournalFile Nothing Nothing False)’
      In the second argument of ‘(<$>)’, namely
        ‘mapM (readJournalFile Nothing Nothing False) jfiles’
    |
464 |      right mconcat' . sequence <$> mapM (readJournalFile Nothing Nothing False) jfiles
    |                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

src/Buchhaltung/Common.hs:464:58: error:
    • Couldn't match expected type ‘InputOpts’
                  with actual type ‘Maybe a0’
    • In the first argument of ‘readJournalFile’, namely ‘Nothing’
      In the first argument of ‘mapM’, namely
        ‘(readJournalFile Nothing Nothing False)’
      In the second argument of ‘(<$>)’, namely
        ‘mapM (readJournalFile Nothing Nothing False) jfiles’
    |
464 |      right mconcat' . sequence <$> mapM (readJournalFile Nothing Nothing False) jfiles
    |                                                          ^^^^^^^

src/Buchhaltung/Common.hs:464:66: error:
    • Couldn't match type ‘Maybe a1’ with ‘[Char]’
      Expected type: PrefixedFilePath
        Actual type: Maybe a1
    • In the second argument of ‘readJournalFile’, namely ‘Nothing’
      In the first argument of ‘mapM’, namely
        ‘(readJournalFile Nothing Nothing False)’
      In the second argument of ‘(<$>)’, namely
        ‘mapM (readJournalFile Nothing Nothing False) jfiles’
    |
464 |      right mconcat' . sequence <$> mapM (readJournalFile Nothing Nothing False) jfiles
    |                                                                  ^^^^^^^
cabal: Leaving directory '/tmp/cabal-tmp-7060/buchhaltung-0.0.7'
Warning: could not create symlinks in /home/richih/.cabal/bin for hledger,
ppsh because the files exist there already and are not managed by cabal. You
can create symlinks for these executables manually if you wish. The executable
files have been installed at /home/richih/.cabal/bin/hledger,
/home/richih/.cabal/bin/ppsh
cabal: Error: some packages failed to install:
buchhaltung-0.0.7-5R7USOtuokwHjZGxbAxU failed during the building phase. The
exception was:
ExitFailure 1

Interactive `match` and `add`

I find I prefer to book transactions as I match them, rather than going through an entire match and then adding each one. That seems to require the following workflow:

Come upon a transaction during match that needs to be booked
Match it, then save and exit
Run add, typing in the amount from the transaction and selecting the transaction from the menu
Book the transaction
C-c C-c in order to exit, ignoring warning message
Start up match again

This workflow is okay, but it would be really nice if it were streamlined, even to the point of booking and matching in the same interactive buchhaltung process.