30

Initial import. · beancount/beancount@4c45733 · GitHub

 3 years ago
source link: https://github.com/beancount/beancount/commit/4c45733b4454b739759944410043d4be787c1001
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

Initial import. · beancount/beancount@4c45733 · GitHubPermalink

Browse files

Initial import.

blais

committed on 24 Apr 2008

0 parents commit 4c45733b4454b739759944410043d4be787c1001
Showing with 13,903 additions and 0 deletions.

Show comments

@@ -0,0 +1,6 @@
syntax: glob
core
*.pyc
*.swp
*~
TAGS

Show comments

@@ -0,0 +1,4 @@
========================
beancount: CHANGES
========================

Show comments

@@ -0,0 +1,8 @@
#!/usr/bin/env make
demo:
python bin/bean-serve --debug examples/demo.ledger
rebuild:
make -C doc beancount.create

50

README

Show comments

@@ -0,0 +1,50 @@
==================================================
BeanCount: Command-line Accounting in Python
==================================================
.. contents::
..
1 Description
2 Running the Demo
3 Documentation
4 Copyright and License
5 Author
Description
===========
A powerful accounting system that uses only a simple text file format
and a few Python scripts.
Running the Demo
================
There is a demonstration ledger input file under the examples
directory. To run a local web server on the example file, type the
following command in the root directory::
demo.sh
Documentation
=============
- `CHANGES <CHANGES>`_
- `TODO <TODO>`_
- `Notes on Ledger in Python </beancount/doc/pyledger.html>`_
- `Notes on acccounting </beancount/doc/accounting.html>`_
- `SQL data model for simple accounting </beancount/doc/beancount.html>`_
Copyright and License
=====================
Copyright (C) 2007-2008 Martin Blais. All Rights Reserved.
Author
======
Martin Blais <[email protected]>

309

TODO

Show comments

@@ -0,0 +1,309 @@
=====================
beancount: TODO
=====================
.. contents::
..
1 Beancount (Accounting)
2 Ideas
2.1 Links
3 Reporting
Beancount (Accounting)
======================
main focus:
* support for import scripts
* check capital gains, write tests
* write tests for date inequalities
* add page and note filtering
parser:
- Bug: A transaction without postings should barf.
- Review the dates inequalities, incl + excl, like compsci
* Clarify this for @check as well, it should probably be at the end
of the day.
- make De and Cr just D and C
- Allow commas in amounts. I like commas sometimes.
- Validate commodities using the defaccount declaration.
- Add a @defcomm directive, that checks validity for the commodities
that are seen.
- Implement the {{ amount }} syntax.
- Implement the @@ amount syntax.
- Finish implementing the (Account) and [Account] syntaxes.
- Display option: add an option to render only up to a specific level
of the tree of accounts.
- Capital gains should not count commissions nor on the buy nor on the
sell side. How do we book them like this? Create a small example.
- IMPORTANT: Using the pickle, the errors only get reported when you
parse initially. We must make the balance checks in a separate
stage!!! Fix that, save the errors in the Ledger object and report
them every time.
- Make my Capital-Gains use the most appropriate syntax and make sure
that the commissions aren't counted in (add a test).
You need to do a test for capital gains.
checks:
- Detect and find potential duplicates.
import:
- Modify all import scripts so that they avoid reimporting already
imported stuff. All the import scripts should take an existing
Ledger file as input.
- The OFX importer needs to unescape & and others.
- Write a generic import routine that will try to heuristically match
partially completed transactions from an existing Ledger.
scripts:
- Write a script to laod the data into an SQL database.
- Implement a web server and all pages being served from memory.
- Figure out how to show balance to market value.
- Make this available publicly (segregate the flair code more
clearly).
- Figure out how to do stock splits properly.
- Output reports using the debits and credits format.
- Check time ranges should also output the ranges of transactions
present in the file.
- Check balances for transactions that are in [].
- Add a command to print the parsed transactions register in the order
they were read in.
- Add declarations for important dates:
* @date 2007-01-01 Start of year 2007
- Add directives specific to conversion scripts as well::
@defvar ofx accid 000016726282 Assets:Current:RBC:Checking
@defvar paypal acc_sales Income:Book-Sales
@defvar paypal acc_deposit Assets:Current:PayPal
@defvar paypal acc_fee Expenses:Financial:Commissions:PayPal
tests:
- Test Wallet += None
- Write functional tests
- Write automated tests for everything.
doc/examples/demos:
- Make a presentation
- Write simple documentation, maybe tests should be part of the
documentation?
- Create an 'examples' subdirectory, with typical use cases.
- Examples:
- mortgage, buying a home
- capital gains (with commissions correctly)
- cie expenses, the way I'm doing it.
- misc, e.g. credit card
- Include examples
- This is causing me a problem::
2008-02-14 * FUNDS TRANSFER
Assets:Current:RBC:Checking-US -89050.66 USD @ 0.9901 CAD
Assets:Current:RBC:Savings 88169.06 CAD
The costs are:
2008-02-14 * FUNDS TRANSFER
Assets:Current:RBC:Checking-US -88169.06 CAD
Assets:Current:RBC:Savings 88169.06 CAD
So when I show the balance sheet "at cost", it shows the account
Assets:Current:RBC:Checking-US as having had some CAD debited from
it.
emacs:
- Make ledger-expand-account rotate between the various choices.
- Make it possible to select an account with partial completion.
serve:
- Modify htmlout to cache the result of rendering nodes, and reuse
that code.
- Make it possible to upload a new file to the server to be parsed.
This way, I wouldn't even have to log in ssh in order to update the
in-memory database...
- When you render an account, you should be able to click on any of
the components of the account name.
reporting:
- Implement `--code-as-payee' combined with `--by-payee', this
provides an interesting view.
Ideas
=====
- Create a script to support generating lists of file locations for
navigating the input file in a certain order.
Let's say that I would want to inspect the input for some
arbitraty list of filtered transactions that relate to
postings: all I have to do is write a script that outputs
"errors" in a way that Emacs knows to parse, and then
'next-error and 'previous-error takes my cursor there with
a single keystroke!
- Not sure if we need this with the @imported directive, but how about
a special field in the transaction's posting::
Assets:Investments:HSBC-Broker -100 IVV @@ 136.2901 USD {HD7egE62}
Income:Investment:Capital-Gains
This special kind of id would get computed in a uniform way from the
date and the account being imported, so that we could check if this
posting or transaction had already been imported before.
- Add directives to support import:
@imported <FROMDATE> <TODATE> <ACCOUNT>
Using the intersection of these date intervals and the account name,
you can determine what has already been imported and avoid importing
twice.
* We need central support for these tasks as well.
* The conversion scripts should always parse a ledger file.
- You should be able to click on dates and see all postings by date
too.
- You should be able to click on a payee to view its transactions.
Links
-----
Description of a data model very similar to my idea.
http://homepages.tcp.co.uk/~m-wigley/gc_wp_ded.html
Reporting
=========
- trial balance (view all accounts)
- total
- at intervals: per-week, per-month, etc.
- register view
- total
- at intervals (?)
- balance sheet (A = L+E)
- total
- at intervals: per-week, per-month, etc.
- pnl view (R-I)
- total
- at intervals
- errors / check
- ranges of checks
- ranges of transactions
- info page
- list of uncleared transactions
- custom reports:
* current cie expenses list and amount that needs be paid.
* stuff that I lent and that I'm waiting for.
* currency exposure.
* list how much liquid assets are available
- Three levels of views for register:
* Basic:
date, payee, description, amount, balance
* Matching:
date, payee, narration
date, posting-other amount
* Full:
date, payee, narration
date, posting amount
date, posting-other amount
date check amount <------------- green

Show comments

@@ -0,0 +1,39 @@
#!/usr/bin/env python
"""
Compute and print the balance of each selected account.
"""
# stdlib imports
import re
from os.path import *
# beancount imports
from beancount import cmdline
from beancount.utils import render_tree
from beancount.ledger import compute_balsheet
def main():
import optparse
parser = optparse.OptionParser(__doc__.strip())
parser.add_option('-l', '--local', action='store_true',
help="Display the local account's balance only.")
parser.add_option('-B', '--at-cost', action='store_true',
help="Compute amounts at cost units instead of in amount units.")
opts, ledger, args = cmdline.main(parser)
compute_balsheet(ledger, 'local_balance', 'balance', opts.at_cost)
aname = 'local_balance' if opts.local else 'balance'
for acc, branch, line in render_tree(ledger.get_root_account()):
aline = branch + line
print ' %-40s %s' % (aline, getattr(acc, aname).round())
if __name__ == '__main__':
main()

Show comments

@@ -0,0 +1,45 @@
#!/usr/bin/env python
"""
Run balance checks and list the validity ranges of all accounts.
"""
# stdlib imports
from os.path import *
from datetime import date
from operator import attrgetter
# beancount imports
from beancount.utils import render_tree
from beancount import cmdline
def main():
import optparse
parser = optparse.OptionParser(__doc__.strip())
parser.add_option('-v', '--verbose', '--ranges', action='store_true',
help="Print out the date ranges as well.")
parser.add_option('--all', action='store_true',
help="Show all accounts.")
opts, ledger, args = cmdline.main(parser)
# Display the check ranges.
today = date.today()
if opts.verbose:
pred = attrgetter('checked') if not opts.all else None
for acc, branch, line in render_tree(ledger.get_root_account(), pred):
aline = branch + line
if acc.checked:
elapsed = today - acc.check_max
print '%-50s %s -> %s (%s days)' % (aline, acc.check_min, acc.check_max, elapsed.days)
else:
print aline
if __name__ == '__main__':
main()

Show comments

@@ -0,0 +1,174 @@
#!/usr/bin/env python
"""
Read an OFX file and output its entries as Ledger syntax.
This program also reads a mapping from account id to account name, from the
ledger file itself, which should include lines of this form::
@accid 000067632326 Assets:Current:RBC:Checking
@accid 000023245336 Assets:Current:RBC:US-Checking
@accid 000018783275 Assets:Current:RBC:Savings
@accid 3233762676464639 Liabilities:Credit-Card:RBC-VISA
(Note that this is not part of the normal Ledger format, which is why the
directives are in comments.)
"""
# stdlib imports
import os, re
from datetime import datetime
from decimal import Decimal
# other imports
from BeautifulSoup import BeautifulStoneSoup
class Account(object):
"An OFX account object."
accname = None
currency = None
transactions = None
start, end = None, None
bal_amount = None
bal_time = None
def __init__(self):
self.transactions = []
class Transaction(object):
"An OFX transaction object."
trntype = None
dtposted = None
trnamt = None
fitid = None
name = None
memo = None
checknum = None
def initialize(self):
if self.dtposted:
self.date = ofx_parse_time(self.dtposted).date()
self.description = ' -- '.join(filter(None, [self.name, self.memo]))
self.amount = Decimal(self.trnamt)
def __str__(self):
date_s = self.date.strftime('%Y-%m-%d')
if self.checknum:
return '%s ! (%s) %s' % (date_s, self.checknum,
self.description)
else:
return '%s ! %s' % (date_s, self.description)
def process_ofx(fn, accounts_map):
"Read an OFX file and process it. Return a list of account objects."
all = []
soup = BeautifulStoneSoup(open(fn))
for st in soup.findAll(['stmttrnrs', 'ccstmttrnrs']):
acc = Account()
accid, acc.currency = find_account(st)
try:
acc.accname = accounts_map[accid]
except KeyError:
raise KeyError("Missing account name in map for %s" % accid)
# Find transactions list.
tranlist = st.find('banktranlist')
# Find start and end times.
acc.start = ofx_parse_time(tranlist.dtstart.contents[0])
acc.end = ofx_parse_time(tranlist.dtend.contents[0])
# Process transactions themselves.
for trn in tranlist.findAll('stmttrn'):
t = Transaction()
acc.transactions.append(t)
for a in 'trntype dtposted trnamt fitid name memo checknum'.split():
n = trn.find(a)
if n:
setattr(t, a, n.contents[0].strip())
t.initialize()
# Find expected balance.
acc.bal_amount = Decimal(st.ledgerbal.balamt.contents[0].strip())
acc.bal_time = ofx_parse_time(st.ledgerbal.dtasof.contents[0].strip())
all.append(acc)
return all
def ofx_parse_time(s):
"Parse an OFX time string and return a datetime object.."
if len(s) < 14:
return datetime.strptime(s[:8], '%Y%m%d')
else:
return datetime.strptime(s[:14], '%Y%m%d%H%M%S')
def find_account(st):
"Get and return the account information and currency."
s = st.find(['stmtrs', 'ccstmtrs'])
assert s is not None
acct = s.find(['bankacctfrom', 'ccacctfrom'])
# Note: RBC offers a malformed XML.
acctid = acct.acctid.contents[0].strip()
currency = s.curdef.contents[0].strip()
return acctid, currency
def read_accounts_map(fn):
"Extract the account number mappings from a ledger file."
accmap = {}
for line in open(fn):
mo = re.match('^\s*@accid[ \t]+(.*)[ \t]+(.*)$', line)
if not mo:
continue
x = [x.strip() for x in mo.group(1, 2)]
if len(x) != 2:
raise SystemExit("Error reading map file: %s" % line)
accid, name = x
accmap[accid] = name
return accmap
def main():
import optparse
parser = optparse.OptionParser(__doc__.strip())
opts, args = parser.parse_args()
if args < 2:
parser.error("You must provide a ledger file with account mappings "
"and a list of OFX files to convert.")
ledger_fn = args[0]
ofx_files = args[1:]
accounts_map = read_accounts_map(ledger_fn)
if not accounts_map:
parser.error("The ledger file does not contain account mappings.")
for arg in ofx_files:
print '\n'*3
print ';;;;; Import of %s' % arg
for acc in process_ofx(arg, accounts_map):
print '\n'*3
print '; Import account: %s ' % acc.accname
print '; Start import: %s ' % acc.start
print
for t in acc.transactions:
print str(t)
print ' %-60s %10.2f %s' % (acc.accname, t.amount, acc.currency)
print
print '; End import: %s ' % acc.end
print '@check %s %s %s %s' % (
acc.bal_time.date(), acc.accname, acc.bal_amount, acc.currency)
print
print '\n'
if __name__ == '__main__':
main()

Show comments

@@ -0,0 +1,121 @@
#!/usr/bin/env python
"""
Interpret the PayPal CSV file and output transactions suitable for Ledger.
"""
# stdlib imports
import sys, re, cgi, logging, time
from datetime import date
from decimal import Decimal
from xml.sax.saxutils import unescape
from pprint import pprint, pformat
from namedtuple import namedtuple
from itertools import imap, count, starmap
# other imports
from BeautifulSoup import BeautifulSoup
# My default accounts.
acc_sales = 'Income:Book-Sales'
acc_deposit = 'Assets:Current:PayPal'
acc_fee = 'Expenses:Financial:Commissions:PayPal'
def parse_date(s):
return date.fromtimestamp(time.mktime(time.strptime(s, '%m/%d/%Y')))
def main():
import optparse
parser = optparse.OptionParser(__doc__.strip())
opts, args = parser.parse_args()
if not args:
parser.error("You must specify an CSV filenames to parse.")
## parser.add_option('-f', '--ledger-file', action='store',
## help="Use the given ledger file in order to infer the Paypal account name.")
##
## if opts.ledger_file:
## ledger = Ledger()
## ledger.parse_file(opts.ledger_file)
## else:
## ledger = None
## accname = infer_account_name(ledger)
for fn in args:
rows = [(parse_date(x.date), x) for x in parse_csv_file(fn)]
rows = reversed(list(parse_csv_file(fn)))
## rows.sort(key=lambda x: (x[0], x.time))
i = 0
for x in rows:
i += 1
date_ = parse_date(x.date)
isreceived = re.search('payment.*received', x.type, re.I)
isupdate = re.search('update', x.type, re.I)
email = x.from_email_address if isreceived else x.to_email_address
description = ', '.join(filter(None, (x.type, x.item_title, x.transaction_id, x.name, email)))
gross, net, fee = map(tonum, (x.gross, x.net, x.fee))
if isreceived:
# Note: auto-clear those transactions.
print '%s * %s' % (date_, description)
print ' %-50s %s %s' % (acc_sales, -gross, x.currency)
print ' %-50s %s %s' % (acc_fee, -fee, x.currency)
print ' %-50s %s %s' % (acc_deposit, net, x.currency)
print
elif isupdate:
# Note: auto-clear those transactions.
print ';;%s * %s' % (date_, description)
print ';; %-50s %s %s' % (acc_sales, -gross, x.currency)
print ';; %-50s %s %s' % (acc_fee, -fee, x.currency)
print ';; %-50s %s %s' % (acc_deposit, net, x.currency)
print
else:
assert fee == 0
print '%s ! %s' % (date_, description)
print ' %-50s %s %s' % (acc_deposit, net, x.currency)
print
## if i % 5 == 0:
## balance = tonum(x.balance)
## print '@check %s %-50s %s %s' % (date_, acc_deposit, balance, x.currency)
## print
balance = tonum(x.balance)
print '@check %s %-50s %s %s' % (date_, acc_deposit, balance, x.currency)
def tonum(x):
return Decimal(x.strip().replace(',', ''))
def parse_csv_file(fn):
"""
Parse a CSV file and return a list of rows as named_tuple objects.
We assume that the first row is a title row.
"""
import csv
reader = csv.reader(open(fn, "rb"))
ireader = iter(reader)
cols = []
dummycount = count().next
for x in ireader.next():
cx = x.strip().lower().replace(' ', '_')
if not cx:
cx = 'dummy_%d' % dummycount()
cols.append(cx)
Row = namedtuple('Row', cols)
return (Row(*x) for x in ireader)
if __name__ == '__main__':
main()

Show comments

@@ -0,0 +1,200 @@
#!/usr/bin/env python
"""
Try your best to parse the crappy HTML coming out of RBC Direct Investing.
Go to the RBC Direct Investing account and use the Web Developer extension to
save the Generate Source (from the Source menu) for each month of activity. This
script can extract the table and produce valid Ledger format text.
This has got to be the absolute worst HTML I have ever seen. Also, note that you
need to save the generated source, not the source you see directly.
"""
# stdlib imports
import sys, re, cgi, logging
from datetime import date
from decimal import Decimal
from xml.sax.saxutils import unescape
from pprint import pprint, pformat
from namedtuple import namedtuple
# other imports
from BeautifulSoup import BeautifulSoup
def main():
import optparse
parser = optparse.OptionParser(__doc__.strip())
parser.add_option('-d', '--debug', action='store_true',
help="Generate debug output.")
global opts
opts, args = parser.parse_args()
if len(args) < 2:
parser.error("You must specify an account and a filename to parse.")
acct, filenames = args[0], args[1:]
all = []
for fn in filenames:
print >> sys.stderr, "\n\nParsing file %s\n" % fn
all.extend(parse_file(fn))
## sortfun = lambda x: (x.currency, x.settlement, x.priority)
sortfun = lambda x: (x.settlement, x.priority)
all.sort(key=sortfun)
acctmap = infer_account_names(acct, all)
for p in all:
print_as_ledger(p, acctmap[p.currency])
def infer_account_names(acct, postings):
"""
Given the name of an account and a list of postings, figure out if the
account name contains one of the currencies we saw, and if so, infer
alternate account names for it.
This is to handle the case of two accounts::
Assets:Investments:RBC-Broker:Account-USD
Assets:Investments:RBC-Broker:Account-CAD
"""
currencies = frozenset(x.currency for x in postings)
for cur in currencies:
acct = acct.replace(cur, '%(currency)s')
return dict( (cur, acct % {'currency': cur})
for cur in currencies )
def print_as_ledger(p, acct):
write = sys.stdout.write
write(fmtdate(p.settlement))
if p.date != p.settlement:
write('=')
write(fmtdate(p.date))
desc = ' -- '.join(filter(None, (p.action, p.symbol, p.description)))
# Note: we used to dispatch the output format here to effect appropriate
# rounding here, on the 1000THS. We now use the Decimal object, which takes
# care of this automatically.
thou = re.match('\\b1000THS\\b', p.description)
ffmt = 's' if thou else 's'
vdict = p._asdict().copy()
vdict['quantity'] = (p.quantity / 1000) if thou else p.quantity
vdict['account'] = acct
vdict['symbol'] = '"%s"' % p.symbol if re.search('[0-9]', p.symbol) else p.symbol
if opts.debug:
for k, v in vdict.iteritems():
write(';; %s : %s\n' % (k, repr(v)))
write(' ! %s\n' % desc)
assert p.currency, p
if p.quantity:
if p.price:
assert p.action in ('Buy', 'Sell', 'DIV F6')
write(' %(account)-70s %(quantity)s %(symbol)s @ %(price)F %(currency)s\n'.replace('F', ffmt) % vdict)
else:
write(' %(account)-70s %(quantity)s %(symbol)s\n'.replace('F', ffmt) % vdict)
if p.amount:
write(' %(account)-70s %(amount)s %(currency)s\n'.replace('F', ffmt) % vdict)
write('\n')
Entry = namedtuple('Entry',
'date symbol description action quantity price amount currency settlement priority')
def parse_file(fn):
soup = BeautifulSoup(open(fn),
convertEntities=BeautifulSoup.HTML_ENTITIES)
soup.prettify()
# Find the header row (it's the only one).
tr = soup.find('tr', 'dataTableHeaderRow')
postings = []
priority = 1
tr = tr.findNextSibling('tr')
while 1:
# Work your way down the table, two rows at a time.
if tr is None:
break
tr1 = tr
tr2 = tr.findNextSibling('tr')
tr = tr2.findNextSibling('tr')
# Parse the first row.
tdlist = tr1.findAll('td')
assert len(tdlist) == 2
date, symbol_desc = [clean_str(x.contents[0]) for x in tdlist]
# Parse the second row.
tdlist = tr2.findAll('td')
assert len(tdlist) == 8
e1, e2, action, quantity, price, amount, currency, settlement = [
clean_str(x.contents[0] if x.contents else '') for x in tdlist]
assert not e1
assert not e2
# Convert to appropriate data types.
date = parse_date(date)
settlement = parse_date(settlement)
if amount:
amount = tonum(amount)
if price:
price = tonum(price)
if quantity:
quantity = tonum(quantity)
symbol, description = [x.strip() for x in symbol_desc.split('-', 1)]
mo = re.search('REINVEST @ \$(.*)', description)
if mo:
assert not price
price = tonum(mo.group(1))
trace(price)
p = Entry(date, symbol, description, action, quantity, price, amount,
currency, settlement, priority)
postings.append(p)
priority += 1
return postings
## FIXME: What is the appropriate way to do this for all entities?
## other_entities = {' ': ''}
def clean_str(s):
return s.strip()
## return unescape(s, other_entities).strip()
_months = 'jan feb mar apr may jun jul aug sep oct nov dec'.split()
def parse_date(dstr):
mo = re.match('(\d\d) (.*) (\d\d\d\d)', dstr)
mth = _months.index(mo.group(2).lower()) + 1
d = date(int(mo.group(3)), mth, int(mo.group(1)))
return d
def fmtdate(d):
return d.strftime('%Y-%m-%d')
def tonum(s):
return Decimal(s.replace(',', ''))
if __name__ == '__main__':
main()

Show comments

@@ -0,0 +1,26 @@
#!/usr/bin/env python
"""
Print some information on a ledger.
"""
# beancount imports
from beancount.utils import render_tree
from beancount import cmdline
def main():
import optparse
parser = optparse.OptionParser(__doc__.strip())
opts, ledger, args = cmdline.main(parser)
print
for line in ledger.dump_info():
print line
print
for acc, branch, line in render_tree(ledger.get_root_account()):
print branch + line
print
if __name__ == '__main__':
main()

Show comments

@@ -0,0 +1,54 @@
#!/usr/bin/env python
"""
Print out the register of postings.
"""
# stdlib imports
import re, logging
from os.path import *
# beancount imports
from beancount import cmdline
from beancount.utils import render_tree
from beancount.wallet import Wallet
logging.warning("THIS IS INCORRECT, YOU NEED TO RECONCILE THE LIST OF TRANSACTIONS, "
"see app.py:register()")
def main():
import optparse
parser = optparse.OptionParser(__doc__.strip())
cmdline.select_addopts(parser)
parser.add_option('-v', '--verbose', action='store_true',
help="Verbosely print some extra stuff.")
opts, ledger, args = cmdline.main(parser)
# Filter out the selected postings.
postings = cmdline.select_postings(ledger, opts)
# Print a tree of the selected accounts.
if opts.verbose:
pred = lambda acc: getattr(acc, 'selected', 0)
print
for acc, branch, line in render_tree(ledger.get_root_account(), pred):
print branch + line
print
balance = Wallet()
for post in sorted(postings):
balance += post.amount
print post.fulldate(), post.txn.topline()
print post.pretty(ledger)
print '**********', balance.round()
print
if __name__ == '__main__':
main()

Show comments

@@ -0,0 +1,6 @@
#!/usr/bin/env python
"Simple web server to display the contents of some Ledger."
from beancount.web.serve import main
main()

Show comments

@@ -0,0 +1,226 @@
#!/usr/bin/env python
"""
Virtual shell for bean-count accounts hierarchy.
"""
# stdlib imports
import sys, os, cmd, optparse, traceback, logging
from operator import itemgetter
from itertools import izip, imap
from os.path import *
# misc imports
import psycopg2 as dbapi
# local imports
from fscmd import HierarchicalCmd
class BeanShell(HierarchicalCmd):
"A bean-counter database manipulation tool."
fmt_short = '%(name)s'
fmt_long = '%(id)05s %(sec)-16s %(isdebit)2s %(name)s'
def __init__(self, conn, *args):
# Connection to the database.
self.conn = conn
self.curs = conn.cursor()
HierarchicalCmd.__init__(self, *args)
def stat(self, node):
assert isinstance(node, int)
self.curs.execute("""
select * from account where id = %s
""", (node,))
if self.curs.rowcount == 1:
m = dict(izip(imap(itemgetter(0), self.curs.description),
self.curs.next()))
self.prepare_stat(m)
else:
m = None
return m
def stat_short(self, node):
assert isinstance(node, int)
self.curs.execute("""
select name, parent_id from account where id = %s
""", (node,))
if self.curs.rowcount == 1:
name, parent = self.curs.next()
else:
name, parent = None, None
return name, parent
def get_root_node(self):
# Get the root node.
self.curs.execute("""
select id from account where parent_id is null order by id
""")
if self.curs.rowcount == 0:
# The root node should always be available.
raise dbapi.DatabaseError("Non-existent root node.")
elif self.curs.rowcount > 1:
logging.warning("Warning: found accounts without parents.")
return self.curs.next()[0]
def get_node_parent(self, node):
"Return the parent of the given node."
assert node is not None
self.curs.execute("""
select parent_id from account where id = %s
""", (node,))
if self.curs.rowcount == 0:
# The root node should always be available.
raise dbapi.DatabaseError("Non-existent node: %s." % node)
assert self.curs.rowcount == 1
return self.curs.next()[0]
def get_child_node(self, parent, name):
self.curs.execute("""
select id from account where parent_id = %s and name = %s
""", (parent, name))
if self.curs.rowcount == 0:
child = None
else:
child = self.curs.next()[0]
return child
def listdir(self, node):
self.curs.execute("select id from account where parent_id = %s",
(node,))
return map(itemgetter(0), self.curs)
def create(self, parent, name, *args):
# Figure out which security to use.
if not args:
self.curs.execute("""
select sec from account where id = %s
""", (parent,))
sec = self.curs.next()[0]
self.stdout.write(
"Note: using security from parent: '%s'.\n" % sec)
else:
sec = args[0].upper()
# Create the node. Note: there is already a constraint about the
# uniqueness of subaccounts per directory.
try:
self.curs.execute("""
insert into account (name, parent_id, sec) values (%s, %s, %s)
""", (name, parent, sec))
except Exception, e:
self.perr(str(e))
self.conn.rollback()
else:
self.conn.commit()
def remove(self, node):
try:
self.curs.execute("""
delete from account where id = %s
""", (node,))
except dbapi.Error:
self.perr("Could not remove node %s. Maybe it is not empty?" % node)
self.conn.rollback()
else:
self.conn.commit()
def remove_node_contents(self, node):
trace('FIXME: todo -- remove_node_contents')
pass
def set_parent(self, node, parent):
try:
self.curs.execute("""
update account set parent_id = %s where id = %s
""", (parent, node))
except Exception, e:
self.perr(str(e))
self.conn.rollback()
else:
self.conn.commit()
def set_name(self, node, name):
try:
self.curs.execute("""
update account set name = %s where id = %s
""", (name, node))
except Exception, e:
self.perr(str(e))
self.conn.rollback()
else:
self.conn.commit()
def prepare_stat(self, m):
"Prepare the raw stat data for printing."
parent_id = m['parent_id']
m['parent_id'] = parent_id if parent_id is not None else ''
m['isdebit'] = 'De' if m['isdebit'] else 'Cr'
def splitargs(args):
"Split the cmdline into discrete elements."
return args.split() if args else None
## FIXME: deal with spaces (add quoting capability).
## FIXME: deal with globbing patterns.
def expand(fn, cwdpath):
"""
Expand the dots and dot-dots in filename 'fn'. All returned filenames are
absolute.
"""
assert isabs(cwdpath)
if not isabs(fn):
fn = join(cwdpath, fn)
assert isabs(fn)
comps = fn.split(os.sep)
assert comps[0] == ''
comps = comps[1:]
newcomps = []
for c in comps:
if c == '.':
continue
elif c == '..':
if not newcomps:
return os.sep
newcomps.pop()
else:
newcomps.append(c)
return os.sep + os.sep.join(newcomps)
def main():
parser = optparse.OptionParser(__doc__.strip())
parser.add_option('-d', '--database', action='store_true',
default='beancount.db',
help="Database name.")
opts, args = parser.parse_args()
conn = dbapi.connect(database=opts.database,
host='localhost',
password='pg',
user=os.environ.get('USER', None))
sh = BeanShell(conn)
try:
sh.cmdloop()
except KeyboardInterrupt:
print '\nInterrupted.'
if __name__ == '__main__':
main()

Show comments

@@ -0,0 +1,43 @@
#!/usr/bin/env python
"""
Testbed for new bean tools.
"""
# stdlib imports
import sys, os, logging, re
from decimal import Decimal
from datetime import date
from os.path import *
from itertools import count, izip
from operator import attrgetter
# beancount imports
from beancount.ledger import *
from beancount.utils import *
from beancount import cmdline
def main():
import optparse
parser = optparse.OptionParser(__doc__.strip())
parser.add_option('-a', '--all', action='store_true',
help="Show all accounts.")
opts, ledger, args = cmdline.main(parser)
root = ledger.get_root_account()
for x in itertree(root):
print x
## FIXME: build a command to list all the unique payees.
## FIXME: build an info command to list stats on the ledger.
## FIXME: make the parse method on the Ledger, it should add to the current
## Ledger object.
if __name__ == '__main__':
main()

Show comments

@@ -0,0 +1,5 @@
#!/bin/sh
# This command runs the beancount web server on a demo ledger file.
# Run this in a shell and use a web browser to access http://localhost:8000
# The demo file contains many example transactions.
python bin/bean-serve --debug examples/demo.ledger

Show comments

@@ -0,0 +1,33 @@
#!/usr/bin/env schema
MODELS = \
beancount.sql
.SUFFIXES: .txt .sql .db .create .test .clean
all: $(MODELS)
.txt.sql:
-rm -f $@
echo "-- This file is auto-generated by rst-literals. Do not modify." > $@
rst-literals -t sql $< >> $@
chmod a-w $@
.sql.clean:
dropdb $(<:.sql=.db)
.sql.create:
-dropdb $(<:.sql=.db) >/dev/null
createdb $(<:.sql=.db)
psql -f $< $(<:.sql=.db) 2>&1 | grep -v NOTICE:
.sql.test:
psql -f $(<:.sql=test.sql) $(<:.sql=.db) 2>&1 | grep -v NOTICE:
docs:
projects docs flair
examples:
rst-literals -s accounting.txt >/dev/null

Show comments

Large diffs are not rendered by default.

Show comments

Large diffs are not rendered by default.

Show comments

@@ -0,0 +1,355 @@
<?xml version="1.0" encoding="iso-8859-1" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<meta name="generator" content="Docutils 0.5: http://docutils.sourceforge.net/" />
<title>Bean Counter: A Simpler Double-Entry Accounting System via SQL</title>
<meta name="date" content="2008-03-29" />
<link rel="stylesheet" href="../style.css" type="text/css" />
</head>
<body>
<div id="project-header">
<a href="/"><img src="/home/furius-logo-w.png" id="logo"></a>
<div id="project-home"><a href="..">Project Home</a></div>
</div>
<div class="document" id="bean-counter-a-simpler-double-entry-accounting-system-via-sql">
<h1 class="title">Bean Counter: A Simpler Double-Entry Accounting System via SQL</h1>
<table class="docinfo" frame="void" rules="none">
<col class="docinfo-name" />
<col class="docinfo-content" />
<tbody valign="top">
<tr class="field"><th class="docinfo-name">Id:</th><td class="field-body">89339e54-5eca-40fe-a8fb-bf7787be3efb</td>
</tr>
<tr><th class="docinfo-name">Date:</th>
<td>2008-03-29</td></tr>
</tbody>
</table>
<div class="abstract topic">
<p class="topic-title first">Abstract</p>
<p>Simple design for a double-entry accounting system, using an SQL
database as a backend.</p>
</div>
<div class="contents topic" id="contents">
<p class="topic-title first">Contents</p>
<ul class="simple">
<li><a class="reference internal" href="#introduction" id="id1">Introduction</a></li>
<li><a class="reference internal" href="#description-of-objects" id="id2">Description of Objects</a><ul>
<li><a class="reference internal" href="#securities" id="id3">Securities</a></li>
<li><a class="reference internal" href="#accounts" id="id4">Accounts</a></li>
<li><a class="reference internal" href="#transactions" id="id5">Transactions</a></li>
<li><a class="reference internal" href="#posting" id="id6">Posting</a></li>
</ul>
</li>
<li><a class="reference internal" href="#inconsistencies" id="id7">Inconsistencies</a></li>
<li><a class="reference internal" href="#operations" id="id8">Operations</a><ul>
<li><a class="reference internal" href="#importing-postings" id="id9">Importing Postings</a></li>
<li><a class="reference internal" href="#categorizing-postings" id="id10">Categorizing Postings</a></li>
<li><a class="reference internal" href="#graph-of-account-relationships" id="id11">Graph of Account Relationships</a></li>
</ul>
</li>
<li><a class="reference internal" href="#tags" id="id12">Tags</a></li>
<li><a class="reference internal" href="#initialization" id="id13">Initialization</a></li>
<li><a class="reference internal" href="#other-issues" id="id14">Other Issues</a><ul>
<li><a class="reference internal" href="#issues-with-multiple-currencies" id="id15">Issues with Multiple Currencies</a></li>
<li><a class="reference internal" href="#import-batches" id="id16">Import Batches</a></li>
</ul>
</li>
<li><a class="reference internal" href="#ideas-for-automation" id="id17">Ideas for Automation</a></li>
</ul>
</div>
<!-- 1 Introduction
2 Description of Objects
2.1 Securities
2.2 Accounts
2.3 Transactions
2.4 Posting
3 Inconsistencies
4 Operations
4.1 Importing Postings
4.2 Categorizing Postings
4.3 Graph of Account Relationships
5 Tags
6 Initialization
7 Other Issues
7.1 Issues with Multiple Currencies
7.2 Import Batches
8 Ideas for Automation -->
<div class="section" id="introduction">
<h1><a class="toc-backref" href="#id1">Introduction</a></h1>
<p>This file contains a description of a relational database schema for a
simple double-entry accounting system. The motivation behind the
creation of a simple homegrown schema is that most available softwares
are either bloated, use an opaque database format, or are downright
buggy and impossible to use and trust. There is nothing very
complicated about the simplest aspects of a double-entry accouting
system, and this is what we provide here.</p>
</div>
<div class="section" id="description-of-objects">
<h1><a class="toc-backref" href="#id2">Description of Objects</a></h1>
<div class="section" id="securities">
<h2><a class="toc-backref" href="#id3">Securities</a></h2>
<p>Each account will contain postings which are valued in a number of
units of a specific currency. Some of the accounts will contain shares
of securities, like stocks. We need to create a table of unique
securities, some of which include the currency units:</p>
<pre class="literal-block">
#!sql
CREATE TABLE security (
-- Unique symbol, type and a name/description.
symbol VARCHAR(16),
name TEXT NOT NULL,
PRIMARY KEY (symbol)
);
</pre>
<p>Eventually, we will want to add much more information to the
securities themselves, and we will probably do this with an anciliary
table, but for now, just to have the securities defined is good enough
to carry out all the basic accounting classification tasks.</p>
</div>
<div class="section" id="accounts">
<h2><a class="toc-backref" href="#id4">Accounts</a></h2>
<p>All the transactions that can occur occur within an "account". Each
account is always denominated under a specific currency, and no
transaction may occur in another currency within that account.</p>
<p>Accounts may contain postings and sub-accounts (just like
filesystem directories contain files and other directories):</p>
<pre class="literal-block">
#!sql
CREATE TABLE account (
-- A unique and a long name/description.
id SERIAL,
name TEXT,
-- The parent account to which this account belongs.
parent_id INTEGER REFERENCES account(id),
-- The security that this account is denominated in.
sec VARCHAR(16) NOT NULL REFERENCES security(symbol),
-- Whether the account is a credit (True) or debug (False)
-- account.
isdebit BOOLEAN,
PRIMARY KEY (id),
UNIQUE (parent_id, name)
);
</pre>
</div>
<div class="section" id="transactions">
<h2><a class="toc-backref" href="#id5">Transactions</a></h2>
<p>Transactions are used as groupings of postings, which are meant
to be balanced individually. If all postings are linked to a
transaction, and all transactions balance, the system is insured to
balance:</p>
<pre class="literal-block">
#!sql
CREATE TABLE transaction (
-- A unique id to refer to the transaction.
id SERIAL,
-- An optional description that applies to the set of entries.
description TEXT,
-- A unqiue time for this set of transactions.
timestamp TIMESTAMP WITHOUT TIME ZONE,
PRIMARY KEY(id)
);
</pre>
<p>About times:</p>
<blockquote>
Note that a posting has a specific date/time, but its
corresponding transaction may have a different one. We need to
insure that each transaction is atomic, so that at any point in time
the system be coherent and balanced.</blockquote>
<p>About securities:</p>
<blockquote>
<p>A transaction may contain postings which are denominated in
different underlying currrencies. For example, this is how a stock
purchase is supposed to work out:</p>
<pre class="literal-block">
transaction
posting AAPL 3
posting Checking 301.00
</pre>
<p>The "exchange rate" between the two provides an implicit price for
the conversion. Here the price would be 101.00$/share.</p>
<p>Note that this price may not be the actual real price of the journal
item that occurred on a market. We will want to group commission and
fees in the same transaction, for example, which affects the
true "price":</p>
<pre class="literal-block">
transaction
posting AAPL 3
posting Expenses:Commission 3.00
posting Checking 303.00
</pre>
</blockquote>
</div>
<div class="section" id="posting">
<h2><a class="toc-backref" href="#id6">Posting</a></h2>
<p>All the activity that occurs in an account is called a "posting". A
posting can be placed in any account that has a matching base
security. Much of the work in filling up an accounting database is to
categorize to which account a posting belongs, and to pair up entries
with a transaction that balances.</p>
<p>Note that if for whatever reason a posting is not yet classified to an
account, it has to be placed in some sort of temporary account created
for this purpose. A posting can never exist without an account.</p>
<pre class="literal-block">
#!sql
CREATE TABLE posting (
-- A unique id to refer to the transaction.
id SERIAL,
-- Timestamp.
timestamp TIMESTAMP WITHOUT TIME ZONE,
-- The account to which this transaction belongs.
account_id INTEGER NOT NULL REFERENCES account(id)
ON DELETE SET NULL,
-- Which transaction this posting belongs to (possibly NULL).
trans_id INTEGER DEFAULT NULL REFERENCES transaction(id)
ON DELETE SET NULL,
-- The underlying currency or security.
symbol VARCHAR(16) REFERENCES security(symbol),
-- Amount of the transaction, in units of its currency.
-- Note that the sign indicates whether to increase or decrease
-- the owning account.
amount NUMERIC(16, 6),
-- Some text to identify this item. This is meant to be either
-- imported from some data file or entered manually by the user.
-- It may entirely be left empty, as it is not otherwise used by
-- the system.
description TEXT,
memo TEXT,
-- An optional unique GUID used at import time, to insure that
-- postings may not be imported twice.
guid VARCHAR(32) UNIQUE,
PRIMARY KEY(id)
);
</pre>
</div>
</div>
<div class="section" id="inconsistencies">
<h1><a class="toc-backref" href="#id7">Inconsistencies</a></h1>
<p>Here is a list of the kinds of inconsistencies that may occur in the
system, for which we need to provide convenient mechanisms to fix up:</p>
<ol class="arabic simple">
<li>postings without a transaction. This is the default state of
newly imported entries.</li>
<li>Transactions that don't balance. This is the main task that a user
wanting to reconcile everything must do.</li>
</ol>
<p>Here are the kinds of inconsistencies that may not occur:</p>
<ol class="arabic simple">
<li>postings without account. These could occur if we allowed
importing without an account, or if you could delete an account
without clearing the transactions it contains. The database schema
insures that all postings have some parent account.</li>
</ol>
</div>
<div class="section" id="operations">
<h1><a class="toc-backref" href="#id8">Operations</a></h1>
<div class="section" id="importing-postings">
<h2><a class="toc-backref" href="#id9">Importing Postings</a></h2>
<p>A tool should be provided to parse some of the available data file
formats (QIF, QFX, etc.) and to specify which account the postings
have to go into.</p>
<p>These importing tools should avoid importing the same items multiple
times, by using the GUIDs that the data file formats provide.</p>
</div>
<div class="section" id="categorizing-postings">
<h2><a class="toc-backref" href="#id10">Categorizing Postings</a></h2>
<p>Once the transactions are imported into the database, we need to
place each of them into a posting. This is done by creating a
matching transaction with another account, in the other direction.</p>
<p>A script can be run to allow the user to easily create these
transactions.</p>
</div>
<div class="section" id="graph-of-account-relationships">
<h2><a class="toc-backref" href="#id11">Graph of Account Relationships</a></h2>
<p>A graph of the relationships between the various accounts can be
obtained by looking at the transactions between accounts in a period
of time.</p>
</div>
</div>
<div class="section" id="tags">
<h1><a class="toc-backref" href="#id12">Tags</a></h1>
<p>Why limit ourselves to a tree structure? Tags can be used to make
queries on sets of accounts:</p>
<pre class="literal-block">
#!sql-alt
CREATE TABLE tag (
tagname VARCHAR(32),
account_id INTEGER REFERENCES account(id),
PRIMARY KEY(tagname)
);
</pre>
</div>
<div class="section" id="initialization">
<h1><a class="toc-backref" href="#id13">Initialization</a></h1>
<p>Some of the base securities to be created include:</p>
<pre class="literal-block">
#!sql
insert into security (symbol, name) values ('USD', 'US Dollar');
insert into security (symbol, name) values ('CAD', 'Canadian Dollar');
insert into security (symbol, name) values ('AUD', 'Australian Dollar');
insert into security (symbol, name) values ('JPY', 'Japanese Yen');
</pre>
</div>
<div class="section" id="other-issues">
<h1><a class="toc-backref" href="#id14">Other Issues</a></h1>
<div class="section" id="issues-with-multiple-currencies">
<h2><a class="toc-backref" href="#id15">Issues with Multiple Currencies</a></h2>
<p>At <a class="reference external" href="http://homepages.tcp.co.uk/~m-wigley/gc_wp_ded.html">http://homepages.tcp.co.uk/~m-wigley/gc_wp_ded.html</a>, a transaction
between multiple currencies is expressed as four entries, going
through a special "cash book" account. Is this useful?</p>
</div>
<div class="section" id="import-batches">
<h2><a class="toc-backref" href="#id16">Import Batches</a></h2>
<p>It would be nice to be able to refer to all the entries inserted
during a single import. We should create an anciliary table for
entries that would store that information, e.g.:</p>
<pre class="literal-block">
#!alt
CREATE TABLE import_batch (
batch_no INTEGER,
id INTEGER REFERENCES posting(id),
PRIMARY KEY (id)
);
</pre>
<p>We could then easily join that table with the accounts table to select
all the messages imported at once.</p>
</div>
</div>
<div class="section" id="ideas-for-automation">
<h1><a class="toc-backref" href="#id17">Ideas for Automation</a></h1>
<ul class="simple">
<li>Pre-fill amount using a matching description.</li>
<li>On import: scan for duplicate transactions (matching all fields).
Maybe we generate a unique checksum on the fields and use that later
on to detect duplicate entries.</li>
</ul>
</div>
</div>
</body>
</html>

Show comments

@@ -0,0 +1,92 @@
-- This file is auto-generated by rst-literals. Do not modify.
CREATE TABLE security (
-- Unique symbol, type and a name/description.
symbol VARCHAR(16),
name TEXT NOT NULL,
PRIMARY KEY (symbol)
);
CREATE TABLE account (
-- A unique and a long name/description.
id SERIAL,
name TEXT,
-- The parent account to which this account belongs.
parent_id INTEGER REFERENCES account(id),
-- The security that this account is denominated in.
sec VARCHAR(16) NOT NULL REFERENCES security(symbol),
-- Whether the account is a credit (True) or debug (False)
-- account.
isdebit BOOLEAN,
PRIMARY KEY (id),
UNIQUE (parent_id, name)
);
CREATE TABLE transaction (
-- A unique id to refer to the transaction.
id SERIAL,
-- An optional description that applies to the set of entries.
description TEXT,
-- A unqiue time for this set of transactions.
timestamp TIMESTAMP WITHOUT TIME ZONE,
PRIMARY KEY(id)
);
CREATE TABLE journal (
-- A unique id to refer to the transaction.
id SERIAL,
-- Timestamp.
timestamp TIMESTAMP WITHOUT TIME ZONE,
-- The account to which this transaction belongs.
account_id INTEGER NOT NULL REFERENCES account(id)
ON DELETE SET NULL,
-- Which entry this item belongs to (possibly NULL).
trans_id INTEGER DEFAULT NULL REFERENCES transaction(id)
ON DELETE SET NULL,
-- The underlying currency or security.
symbol VARCHAR(16) REFERENCES security(symbol),
-- Amount of the transaction, in units of its currency.
-- Note that the sign indicates whether to increase or decrease
-- the owning account.
amount NUMERIC(16, 6),
-- Some text to identify this item. This is meant to be either
-- imported from some data file or entered manually by the user.
-- It may entirely be left empty, as it is not otherwise used by
-- the system.
description TEXT,
memo TEXT,
-- An optional unique GUID used at import time, to insure that
-- journal entries may not be imported twice.
guid VARCHAR(32) UNIQUE,
PRIMARY KEY(id)
);
insert into security (symbol, name) values ('USD', 'US Dollar');
insert into security (symbol, name) values ('CAD', 'Canadian Dollar');
insert into security (symbol, name) values ('AUD', 'Australian Dollar');
insert into security (symbol, name) values ('JPY', 'Japanese Yen');

Show comments

@@ -0,0 +1,341 @@
====================================================================
Bean Counter: A Simpler Double-Entry Accounting System via SQL
====================================================================
:Id: 89339e54-5eca-40fe-a8fb-bf7787be3efb
:Date: 2008-03-29
:Abstract:
Simple design for a double-entry accounting system, using an SQL
database as a backend.
.. contents::
..
1 Introduction
2 Description of Objects
2.1 Securities
2.2 Accounts
2.3 Transactions
2.4 Posting
3 Inconsistencies
4 Operations
4.1 Importing Postings
4.2 Categorizing Postings
4.3 Graph of Account Relationships
5 Tags
6 Initialization
7 Other Issues
7.1 Issues with Multiple Currencies
7.2 Import Batches
8 Ideas for Automation
Introduction
============
This file contains a description of a relational database schema for a
simple double-entry accounting system. The motivation behind the
creation of a simple homegrown schema is that most available softwares
are either bloated, use an opaque database format, or are downright
buggy and impossible to use and trust. There is nothing very
complicated about the simplest aspects of a double-entry accouting
system, and this is what we provide here.
Description of Objects
======================
Securities
----------
Each account will contain postings which are valued in a number of
units of a specific currency. Some of the accounts will contain shares
of securities, like stocks. We need to create a table of unique
securities, some of which include the currency units::
#!sql
CREATE TABLE security (
-- Unique symbol, type and a name/description.
symbol VARCHAR(16),
name TEXT NOT NULL,
PRIMARY KEY (symbol)
);
Eventually, we will want to add much more information to the
securities themselves, and we will probably do this with an anciliary
table, but for now, just to have the securities defined is good enough
to carry out all the basic accounting classification tasks.
Accounts
--------
All the transactions that can occur occur within an "account". Each
account is always denominated under a specific currency, and no
transaction may occur in another currency within that account.
Accounts may contain postings and sub-accounts (just like
filesystem directories contain files and other directories)::
#!sql
CREATE TABLE account (
-- A unique and a long name/description.
id SERIAL,
name TEXT,
-- The parent account to which this account belongs.
parent_id INTEGER REFERENCES account(id),
-- The security that this account is denominated in.
sec VARCHAR(16) NOT NULL REFERENCES security(symbol),
-- Whether the account is a credit (True) or debug (False)
-- account.
isdebit BOOLEAN,
PRIMARY KEY (id),
UNIQUE (parent_id, name)
);
Transactions
------------
Transactions are used as groupings of postings, which are meant
to be balanced individually. If all postings are linked to a
transaction, and all transactions balance, the system is insured to
balance::
#!sql
CREATE TABLE transaction (
-- A unique id to refer to the transaction.
id SERIAL,
-- An optional description that applies to the set of entries.
description TEXT,
-- A unqiue time for this set of transactions.
timestamp TIMESTAMP WITHOUT TIME ZONE,
PRIMARY KEY(id)
);
About times:
Note that a posting has a specific date/time, but its
corresponding transaction may have a different one. We need to
insure that each transaction is atomic, so that at any point in time
the system be coherent and balanced.
About securities:
A transaction may contain postings which are denominated in
different underlying currrencies. For example, this is how a stock
purchase is supposed to work out::
transaction
posting AAPL 3
posting Checking 301.00
The "exchange rate" between the two provides an implicit price for
the conversion. Here the price would be 101.00$/share.
Note that this price may not be the actual real price of the journal
item that occurred on a market. We will want to group commission and
fees in the same transaction, for example, which affects the
true "price"::
transaction
posting AAPL 3
posting Expenses:Commission 3.00
posting Checking 303.00
Posting
-------
All the activity that occurs in an account is called a "posting". A
posting can be placed in any account that has a matching base
security. Much of the work in filling up an accounting database is to
categorize to which account a posting belongs, and to pair up entries
with a transaction that balances.
Note that if for whatever reason a posting is not yet classified to an
account, it has to be placed in some sort of temporary account created
for this purpose. A posting can never exist without an account.
::
#!sql
CREATE TABLE posting (
-- A unique id to refer to the transaction.
id SERIAL,
-- Timestamp.
timestamp TIMESTAMP WITHOUT TIME ZONE,
-- The account to which this transaction belongs.
account_id INTEGER NOT NULL REFERENCES account(id)
ON DELETE SET NULL,
-- Which transaction this posting belongs to (possibly NULL).
trans_id INTEGER DEFAULT NULL REFERENCES transaction(id)
ON DELETE SET NULL,
-- The underlying currency or security.
symbol VARCHAR(16) REFERENCES security(symbol),
-- Amount of the transaction, in units of its currency.
-- Note that the sign indicates whether to increase or decrease
-- the owning account.
amount NUMERIC(16, 6),
-- Some text to identify this item. This is meant to be either
-- imported from some data file or entered manually by the user.
-- It may entirely be left empty, as it is not otherwise used by
-- the system.
description TEXT,
memo TEXT,
-- An optional unique GUID used at import time, to insure that
-- postings may not be imported twice.
guid VARCHAR(32) UNIQUE,
PRIMARY KEY(id)
);
Inconsistencies
===============
Here is a list of the kinds of inconsistencies that may occur in the
system, for which we need to provide convenient mechanisms to fix up:
#. postings without a transaction. This is the default state of
newly imported entries.
#. Transactions that don't balance. This is the main task that a user
wanting to reconcile everything must do.
Here are the kinds of inconsistencies that may not occur:
#. postings without account. These could occur if we allowed
importing without an account, or if you could delete an account
without clearing the transactions it contains. The database schema
insures that all postings have some parent account.
Operations
==========
Importing Postings
------------------
A tool should be provided to parse some of the available data file
formats (QIF, QFX, etc.) and to specify which account the postings
have to go into.
These importing tools should avoid importing the same items multiple
times, by using the GUIDs that the data file formats provide.
Categorizing Postings
---------------------
Once the transactions are imported into the database, we need to
place each of them into a posting. This is done by creating a
matching transaction with another account, in the other direction.
A script can be run to allow the user to easily create these
transactions.
Graph of Account Relationships
------------------------------
A graph of the relationships between the various accounts can be
obtained by looking at the transactions between accounts in a period
of time.
Tags
====
Why limit ourselves to a tree structure? Tags can be used to make
queries on sets of accounts::
#!sql-alt
CREATE TABLE tag (
tagname VARCHAR(32),
account_id INTEGER REFERENCES account(id),
PRIMARY KEY(tagname)
);
Initialization
==============
Some of the base securities to be created include::
#!sql
insert into security (symbol, name) values ('USD', 'US Dollar');
insert into security (symbol, name) values ('CAD', 'Canadian Dollar');
insert into security (symbol, name) values ('AUD', 'Australian Dollar');
insert into security (symbol, name) values ('JPY', 'Japanese Yen');
Other Issues
============
Issues with Multiple Currencies
-------------------------------
At http://homepages.tcp.co.uk/~m-wigley/gc_wp_ded.html, a transaction
between multiple currencies is expressed as four entries, going
through a special "cash book" account. Is this useful?
Import Batches
--------------
It would be nice to be able to refer to all the entries inserted
during a single import. We should create an anciliary table for
entries that would store that information, e.g.::
#!alt
CREATE TABLE import_batch (
batch_no INTEGER,
id INTEGER REFERENCES posting(id),
PRIMARY KEY (id)
);
We could then easily join that table with the accounts table to select
all the messages imported at once.
Ideas for Automation
====================
- Pre-fill amount using a matching description.
- On import: scan for duplicate transactions (matching all fields).
Maybe we generate a unique checksum on the fields and use that later
on to detect duplicate entries.

Show comments

@@ -0,0 +1,82 @@
==============================================
Differences between Beancount and Ledger
==============================================
:Date: 2008-04-23
:Author: Martin Blais <[email protected]>
:Abstract:
A document that summarizes the differences between the valid syntax
for Beancount and the syntax of Ledger.
Introduction
============
Beancount is an accounting system that uses simple text files, whose
syntax is meant to be compatible with John Wigley's Ledger program.
Beancount supports a subset of the Ledger syntax, but also provides
some additional directives not found in Ledger. This document
summarizes those differences. (This document is meant to be
complementary to the Ledger documentation.)
Some of the unsupported syntax is due to the fact that only a partial
parser for the Ledger syntax was implement, because the intention is
to eventually use the Ledger program as a Python module to do the
parsing, but some of it is there because we felt that some features
were not necessary.
Unsupported Syntax
==================
Beancount...
- does not support expressions.
- does not support the syntax that uses symbols instead of explicit
commodities, e,g. ``$10.00``. You have to use ``10.00 USD``
- does not infer the format of the output for each commodity
This is by choice, the definition is ambiguous: if you use two
different syntaxes, which one is preferred? Instead, Beancount will
provide a way to "set" the syntax, and will do something reasonable
with defaults.
About Dates
-----------
All check dates are assumed to be at midnight at the start of the
given day. All transaction dates are assumed to occur something
withint the day. Therefore, when you place an @assert directive, you
are asserting this amount at the **beginning** of the day, before any
of the transactions of that day occur.
Features not found in Ledger
============================
FIXME todo
- @pad
The @pad directive automatically inserts an entry to make the
assertions that come before and after itself automatically match.
It is used to fill in for missing data in the file, and is
typically used to have an opening balances entry automatically
created for it.
Here is its format:
@pad <DATE> <ACCOUNT> <BALANCE-ACCOUNT>
- @assert

Show comments

Large diffs are not rendered by default.

Show comments

Large diffs are not rendered by default.

Show comments

@@ -0,0 +1,72 @@
Date Tue, 22 Apr 2008 3:11 PM ( 50 mins 15 secs ago ) Text view
Print view
Raw view
From "Raymond Hettinger" <[email protected]>
To "Martin Blais" <[email protected]>
Subject Accounting Package Show full header
Nice little package.
I look at your question list. Here's a few thoughts.
Usually, when there is a choice of several ways to record things (trade date vs settlement date, asset vs expense, cost basis vs
mark-to-market), the right answer depends on who is using the books and for what purpose. Traders usually think in terms of trade
dates because that is when the decision is made and the asset valuation risk begins. Accountants (including taxing authorities)
usually look to settlement dates because that is when the cash moves.
Often you can meet multiple needs by keeping multiple accounts for a single asset and then looking at either the individual accounts
or the sum depending on what you're trying to analyze
my_house:at_cost 200,000
my_house_mkt_adj 10,000
my_car:at_cost 30,000
my_car:depreciation <4,500>
For taxes, you record a liability and expense at the time the obligation is incurred. When you pay them, reduce the liability and
cash:
#2/20/2008
Tax Expense 1,000 Dr
Tax Payable 1,000 Cr
#1/15/2009
Tax Payable 8,000 Dr
Tax Expense 8,000 Cr
For multiple currencies, there simplest solution is to record all transactions in their native currency. Then add a dynamic
translation to USD or CAD using the current exchange rate before summing the trial balance. The balancing entry goes to an expense
for currency gains and losses. Given:
Cash:USD 1000 Dr
Cash:Euro 4000 Dr
Equity:USD 7,500 Cr
The dynamic trial balance with rate of (1.5 to 1) is:
Cash:USD 1000 Dr
Cash:Euro_Converted 6000 Dr
Equity:USD 7,500 Cr
CurrencyLosses 500 Dr
Rather than making every trading entry or food purchase in your books, the common approach is to book summary totals from a
single-entry subledger:
Burger King 10
McDonalds 20
Taco Bell 30
More junk 40
More tacos 50
Total 150
FoodExpense 150 Dr
Cash 150 Cr
FWIW, microsoft has a new accounting product that automatically books entries from PayPal and Ebay transactions. That feature
should not be hard to replicate.
Raymond
Inbox: 1 of 6 Go to: < Mailbox > Delete and: < Mailbox >
Need help? Start here. Read the FAQ. Talk to other users.

Show comments

@@ -0,0 +1,2 @@
#!/bin/sh

Show comments

@@ -0,0 +1,4 @@
README index.html @header
doc/beancount.txt @header
doc/accounting.txt @header
doc/pyledger.txt @header

Show comments

@@ -0,0 +1,59 @@
;;/usr/bin/env emacs
;;
;; Emacs setup for Ledger.
;;
;; Add the emacs path.
(add-to-list 'load-path
(concat project-dir "/lib/emacs"))
(require 'ledger)
(require 'ledger-plus)
(defun user-ledger-mode-hook ()
(set-fill-column 200)
;; (outline-minor-mode 1)
(setq outline-regexp "^;;;;; ")
(define-key ledger-mode-map [(control ?c) (control ?n)]
'outline-next-visible-heading)
(define-key ledger-mode-map [(control ?c) (control ?p)]
'outline-previous-visible-heading)
;; FIXME: we should make this work for the current entry when a region is not
;; selected.
(define-key ledger-mode-map [(control ?c) (control ?q)]
(lambda () (interactive) (ledger-align-amounts 80)))
;; Remove tab bindings that are injected illegally in ledger.el.
(define-key ledger-mode-map [tab] nil)
(define-key ledger-mode-map [(control ?i)] nil)
;; Bring back comment-region.
(define-key ledger-mode-map [(control ?c) (control ?c)] 'comment-region)
(setq comment-start "; ")
)
(add-hook 'ledger-mode-hook 'user-ledger-mode-hook)
(add-to-list 'auto-mode-alist '("\\.ledger$" . ledger-mode))
;; Support parsing Python logging errors, with a suitable logging.basicConfig()
;; format.
(unless (assq 'python-logging compilation-error-regexp-alist-alist)
(add-to-list
'compilation-error-regexp-alist-alist
'(python-logging "\\(ERROR\\|WARNING\\):\\s-*\\([^:]+\\):\\([0-9]+\\)\\s-*:" 2 3))
(add-to-list
'compilation-error-regexp-alist 'python-logging)
)

Show comments

@@ -0,0 +1,9 @@
#!/bin/sh
#
# My environment initialization for this project.
USERPATH=$USERPATH:$PROJDIR/bin
PYTHONPATH=$PYTHONPATH:\
$PROJDIR/lib/python:\
$PROJDIR/lib/python-fallback

Show comments

Binary file not shown.

Show comments

No changes.

Show comments

@@ -0,0 +1,58 @@
<?xml version="1.0" encoding="iso-8859-1" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<meta name="generator" content="Docutils 0.5: http://docutils.sourceforge.net/" />
<title>BeanCount: Command-line Accounting in Python</title>
<link rel="stylesheet" href="style.css" type="text/css" />
</head>
<body>
<div id="project-header">
<a href="/"><img src="/home/furius-logo-w.png" id="logo"></a>
</div>
<div class="document" id="beancount-command-line-accounting-in-python">
<h1 class="title">BeanCount: Command-line Accounting in Python</h1>
<div class="contents topic" id="contents">
<p class="topic-title first">Contents</p>
<ul class="simple">
<li><a class="reference internal" href="#description" id="id1">Description</a></li>
<li><a class="reference internal" href="#documentation" id="id2">Documentation</a></li>
<li><a class="reference internal" href="#copyright-and-license" id="id3">Copyright and License</a></li>
<li><a class="reference internal" href="#author" id="id4">Author</a></li>
</ul>
</div>
<!-- 1 Description
2 Documentation
3 Copyright and License
4 Author -->
<div class="section" id="description">
<h1><a class="toc-backref" href="#id1">Description</a></h1>
<p>A powerful accounting system that uses only a simple text file format
and a few Python scripts.</p>
</div>
<div class="section" id="documentation">
<h1><a class="toc-backref" href="#id2">Documentation</a></h1>
<ul class="simple">
<li><a class="reference external" href="CHANGES">CHANGES</a></li>
<li><a class="reference external" href="TODO">TODO</a></li>
<li><a class="reference external" href="/beancount/doc/pyledger.html">Notes on Ledger in Python</a></li>
<li><a class="reference external" href="/beancount/doc/accounting.html">Notes on acccounting</a></li>
<li><a class="reference external" href="/beancount/doc/beancount.html">SQL data model for simple accounting</a></li>
</ul>
</div>
<div class="section" id="copyright-and-license">
<h1><a class="toc-backref" href="#id3">Copyright and License</a></h1>
<p>Copyright (C) 2007-2008 Martin Blais. All Rights Reserved.</p>
</div>
<div class="section" id="author">
<h1><a class="toc-backref" href="#id4">Author</a></h1>
<p>Martin Blais <<a class="reference external" href="mailto:[email protected]">[email protected]</a>></p>
</div>
</div>
</body>
</html>

Show comments

@@ -0,0 +1,210 @@
;;; ledger-plus.el --- Additions to ledger.el for future integration.
;; Copyright (C) 2008 Martin Blais <[email protected]>
;;; Commentary:
(require 'ledger)
(defun ledger-accounts nil
"The list of accounts in the current buffer.")
(make-variable-buffer-local 'ledger-accounts)
(add-hook 'ledger-mode-hook 'ledger-plus-hook-function)
(defun ledger-insert-date ()
"A simpler, less smart, more convenient date insertion function
that just inserts today's date."
(interactive)
(insert
(time-stamp-string "%:y/%02m/%02d * ")
))
(defun ledger-plus-hook-function ()
"Hook to install on entering ledger mode."
;; Setup partial completion delimiters.
(make-local-variable 'PC-word-delimiters)
(setq PC-word-delimiters ": ")
(define-key ledger-mode-map [(control ?c) (control ?d)] 'ledger-insert-date)
(define-key ledger-mode-map [(control ?c) (control ?i)] 'ledger-insert-account)
)
(defun ledger-insert-account ()
"Prompts the user for an account to insert at point."
(interactive)
(let ((accounts (ledger-get-accounts (line-number-at-pos (point)))))
(insert
(ledger-completing-read "Account: " accounts))))
(require 'iswitchb)
(defun ledger-completing-read (prompt choices)
"(I can't get icomplete nor icicles to work nicely, this is
old, solid and lovely.)"
(let ((iswitchb-make-buflist-hook
(lambda ()
(setq iswitchb-temp-buflist choices)))
(iswitchb-case t))
(iswitchb-read-buffer prompt)))
;; FIXME: add history too.
;; (defun my-icompleting-read (prompt choices)
;; "Use iswitch as a completing-read replacement to choose from
;; choices. PROMPT is a string to prompt with. CHOICES is a list of
;; strings to choose from."
;; (let ((iswitchb-make-buflist-hook
;; (lambda ()
;; (setq iswitchb-temp-buflist choices))))
;; (iswitchb-read-buffer prompt)))
;;; (skip-syntax-backward "-" (save-excursion (forward-line 0) (point)))
;;; ;; Don't delete formfeeds, even if they are considered whitespace.
;;; (save-match-data
;;; (if (looking-at ".*\f")
;;; (goto-char (match-end 0))))
;;; (delete-region (point) (match-end 0))))))
(defun ledger-get-accounts (exclude-line)
"Heuristically obtain a list of all the accounts used in all the postings.
We ignore patterns seen the line 'exclude-line'."
(let ((accounts))
(save-excursion
(goto-char (point-min))
(while (re-search-forward
(concat "\\(?:"
"^[ \t]+\\([A-Z][A-Za-z0-9-_:]*\\)\\s-*"
"\\|"
"^@defaccount\\s-+\\(?:De\\|Cr\\)\\s-+\\([A-Z][A-Za-z0-9-_:]*\\)\\s-*"
"\\)"
) nil t)
(let ((no (if (match-string 1) 1 2)))
(when (not (= (line-number-at-pos (match-end no)) exclude-line))
(let ((acc (match-string no)))
(add-to-list 'accounts acc)
)))))
accounts))
;; The following two variables are used to implement cycling between
;; accounts.
(defvar ledger-last-account-matches nil
"The list of account names last matched by an invocation of
ledger-expand-account.")
(defun ledger-expand-account ()
"Look at the word before point and try to expand it into an account name."
(interactive)
;; Note: we use 'filename as a thing, because it accepts the : separators for
;; the underlying words we're looking for.
(let ((w (thing-at-point 'filename))
(bounds (bounds-of-thing-at-point 'filename)))
(if (and repeatable-repeated
ledger-last-account-matches)
;; Use the previous search string.
(let ((newmatch (or
(cadr (member w ledger-last-account-matches))
(car ledger-last-account-matches))))
(when newmatch
(kill-region (car bounds) (cdr bounds))
(insert newmatch)))
;; Do the search.
(let ((regexp (format ".*%s.*" w))
;; A regexp that would place the match in first priority.
(regexp-prio (format ".*\\(\b%s\\|%s\b\\)" w w))
(accounts (ledger-get-accounts (line-number-at-pos (point)))))
;; Filter the list of matches by our word's regexp.
(setq ledger-last-account-matches
;; Sort the list by preferring those matches which match the
;; word's boundary first.
(mapcar 'cdr
(sort
(mapcar ;; Schwartzian transform.
(lambda (x) (cons (string-match regexp-prio x)
x))
;; Filter in only those accounts that match the word.
(filter (lambda (x) (string-match regexp x))
accounts))
;; Comparison that takes into account the boundary match.
(lambda (x y)
(let ((px (car x))
(py (car y)))
(if (and px py)
(string< (cdr x) (cdr y))
(if px t nil))))
)))
(cond ((= (length ledger-last-account-matches) 1)
(kill-region (car bounds) (cdr bounds))
(insert (car ledger-last-account-matches)))
((> (length ledger-last-account-matches) 1)
(message
(concat (format "Many matches for %s: " w)
(mapconcat 'identity ledger-last-account-matches ", "))
)
(kill-region (car bounds) (cdr bounds))
(insert (car ledger-last-account-matches))
)
(t (message (format "No matches for '%s'." w)))
))
)))
;; FIXME: in the sorting order, we should look at the accounts used in the file,
;; right before our line number. Actually, sort the original account definitions
;; according to the distance from the current line.
(define-key ledger-mode-map [(control ?c) (?')] 'ledger-expand-account)
;; Allow cycling through the matching names.
(when (require 'repeatable nil t)
(repeatable-command-advice ledger-expand-account))
;; Make emacs able to go to the errors generated in our Python-code. With a
;; suitable logging.basicConfig(), this should work for other programs too.
(unless (assq 'python-logging compilation-error-regexp-alist-alist)
(add-to-list
'compilation-error-regexp-alist-alist
'(python-logging "\\(ERROR\\|WARNING\\):\\s-*\\([^:]+\\):\\([0-9]+\\)\\s-*:" 2 3))
(add-to-list
'compilation-error-regexp-alist 'python-logging)
)
(provide 'ledger-plus)
;;; ledger-plus.el ends here

Show comments

Large diffs are not rendered by default.

Show comments

@@ -0,0 +1,56 @@
module AnotherParser where
import List
import Data.Char
import System.IO
-- |A line is string of characters (end-of-line has been stripped) paired with
-- its line number.
type Line = (Int, String)
-- |A line group is either a list of a single line representing a directive or
-- a list of multiple lines representing an entry; in the latter case, the
-- head of the group is the entry declaration and the tail is the list of
-- transactions for the entry.
type LineGroup = [Line]
-- Put your favorite file name here
fileName = "fxt.ledger"
main = do input <- readFile fileName
let items = journal input
mapM_ (putStrLn . show) items
putStrLn ("Found " ++ show (length items) ++ " items")
journal :: String -> [LineGroup]
journal = groupLines . mkLines
-- |Break down the input into a list of lines, paired them up with line
-- numbers, and drop all comment and empty lines.
mkLines :: String -> [Line]
mkLines s = filter (not . isComment . snd) (zip [1..](lines s))
-- |A comment is a line that is empty or starts with ';' or only contains
-- white space.
isComment :: String -> Bool
isComment [] = True
isComment (c:cs) | c == ';' = True
| otherwise = and . map isSpace $ (c:cs)
-- |Lines are grouped so that all transactions lines for an entry are in the same
-- line group as the entry line.
-- We accomplish this by passing around an extra parameter used to accumulate
-- all the lines for the current group.
groupLines :: LineGroup -> [LineGroup]
groupLines = tail . groupLines' []
where
groupLines' cg [] = [cg]
groupLines' cg (l:ls) | isTrans . snd $ l = groupLines' (cg ++ [l]) ls
| otherwise = cg : groupLines' [l] ls
-- |A transaction is a line that starts with space.
-- This function should not be called on lines that consists only of white space.
isTrans :: String -> Bool
isTrans (c:cs) = isSpace c

Show comments

@@ -0,0 +1,223 @@
module Main where
import Text.ParserCombinators.Parsec
import Data.Time.Calendar
import Data.Time.Calendar.Julian
import Data.Char (isSpace, isDigit)
-- Put your favorite file name here
fname = "blais.ledger"
main = do st <- startState
input <- readFile fname
let res = runParser journal st fname input
case res of
Left err -> do putStr "parse error at " ; print err
Right rs -> mapM_ (putStrLn . show) rs
-- I use mapM_ to print one record per line; otherwise, calling
-- show of a list will put everything on the same line.
type Directive = String
data Entry = Ent { entDate :: Date
, entEffectiveDate :: Date
, entStatus :: Status
, entCode :: Code
, entPayee :: Payee
, entNote :: Note
, entTransactions :: [Transaction]
} deriving Show
type Date = Day
data Status = Unbalanced | Pending | Cleared
deriving Show
type Code = String
data Transaction = Tran { tranStatus :: Status
, tranAccount :: Account
, tranAmount :: Maybe Amount
, tranNote :: Note
} deriving Show
type Payee = String
type Account = [String]
-- The second Amount' is used as a price per unit. If a price per unit is not specified,
-- it will be set to one unit of the original commodity (the one given in the first Amount').
type Amount = (Amount', Amount')
type Amount' = (Quantity, Commodity)
type Quantity = Float
type Commodity = String
type Note = String
newtype JrnlState = JS { year :: Integer }
type JrnlParser = GenParser Char JrnlState
startState :: IO JrnlState
startState = return JS { year = 2008 }
-- This is used to build lexeme parsers, i.e. parsers that consume trailing white space so
-- that the next non-white character is ready for the next parser to consume.
lexeme :: JrnlParser a -> JrnlParser a
lexeme p = do res <- p
many (oneOf " \t")
return res
journal :: JrnlParser [Entry]
journal = do xs <- many ( entry
<|> directive
<|> comment
<|> emptyLine )
eof
return [x | Just x <- xs]
-- WARNING: this has to be the last production or it will not parse
-- The reason is that the parser could choose this production by matching
-- empty and it will not be able to backtrack once it discovers empty is not
-- followed by a newline.
emptyLine :: JrnlParser (Maybe Entry)
emptyLine = do many (oneOf " \t")
newline
return Nothing
<?> "empty line"
comment :: JrnlParser (Maybe Entry)
comment = do oneOf ";*hb"
skipMany (satisfy (/= '\n'))
newline
return Nothing
<?> "comment"
directive :: JrnlParser (Maybe Entry)
directive = do satisfy (\c -> not (';' == c || isSpace c || isDigit c))
newline
return Nothing
<?> "directive"
entry :: JrnlParser (Maybe Entry)
entry = do e <- plainEntry
return (Just e)
plainEntry :: JrnlParser Entry
plainEntry = do (d, ed) <- lexeme fullDate
st <- option Unbalanced (lexeme status)
cd <- option "" (lexeme code)
p <- payee
n <- option "" note
newline
ts <- many1 transaction
return (Ent { entDate = d,
entEffectiveDate = ed,
entStatus = st,
entCode = cd,
entPayee = p,
entNote = n,
entTransactions = ts } )
fullDate :: JrnlParser (Date, Date)
fullDate = do d <- date <?> "date"
ed <- option d (char '=' >> date) <?> "effective date"
return (d, ed)
date :: JrnlParser Date
date = do sy <- count 4 digit
dateSep
sm <- count 2 digit
dateSep
sd <- count 2 digit
let y = read sy
m = read sm
d = read sd
if m > 12 || d > (julianMonthLength y m) then
fail "invalid date"
else
return (fromGregorian y m d)
dateSep :: JrnlParser ()
dateSep = do oneOf "/-."
return ()
status :: JrnlParser Status
status = (char '!' >> return Pending) <|> (char '*' >> return Cleared) <?> "status"
code :: JrnlParser Code
code = between (char '(') (char ')') (many (noneOf ")\n")) <?> "code"
payee :: JrnlParser Payee
payee = many (satisfy (/= '\n')) <?> "payee"
transaction :: JrnlParser Transaction
transaction = do many1 (oneOf " \t")
st <- option Unbalanced (lexeme status)
acc <- lexeme account
amt <- optionMaybe (lexeme amount)
n <- option "" note
newline
return (Tran { tranStatus = st,
tranAccount = acc,
tranAmount = amt,
tranNote = n } )
account :: JrnlParser Account
account = accountNameList
<|> between (char '(') (char ')') accountNameList
<|> between (char '[') (char ']') accountNameList
<?> "account"
accountNameList :: JrnlParser Account
accountNameList = do sepBy1 accountName (char ':')
-- The 'try' combinator provides backtracking when its parser fails. Here, the failure will
-- stop the mutual recursion and result in an account name free of trailing white space.
accountName :: JrnlParser String
accountName = do w <- many1 (noneOf ":)] \t;\n")
ws <- (try accountName') <|> return ""
return (w ++ ws)
return w
-- An account name may contain white space, except if the white space is followed by the
-- beginning of the <amount>, <note>, or newline (whatever can legally follow the account
-- production). The notFollowedBy combinator makes it so easy! It fails if its parser
-- succeeds; in this case, the entire accountName' parser fails, giving the try combinator
-- above a chance to backtrack; if notFollowedBy succeeds, parsing of the account name
-- continues on with mutual recursion.
accountName' :: JrnlParser String
accountName' = do s1 <- many1 (oneOf " \t")
notFollowedBy (digit <|> oneOf "-;\n")
s2 <- accountName
return (s1 ++ s2)
amount :: JrnlParser Amount
amount = do (q, c) <- amount'
pr <- option (1, c) (lexeme (char '@') >> amount') -- price per unit
return ((q, c), pr)
amount' :: JrnlParser Amount'
amount' = do q <- lexeme quantity
c <- lexeme commodity
return (q, c)
quantity :: JrnlParser Quantity
quantity = do s <- option ' ' (char '-')
d <- many1 digit
f <- option "0" (char '.' >> many1 digit)
return (read (s:d ++ "." ++ f))
<?> "quantity"
commodity :: JrnlParser Commodity
commodity = do c <- letter
cs <- many alphaNum
return (c:cs)
<?> "commodity"
note :: JrnlParser String
note = do char ';'
cs <- many (satisfy (/= '\n'))
return cs
<?> "note"

Show comments

@@ -0,0 +1,182 @@
module JournalParser where
import Text.ParserCombinators.Parsec
import qualified Text.ParserCombinators.Parsec.Token as P
import Text.ParserCombinators.Parsec.Language (haskellStyle)
import Text.ParserCombinators.Parsec.Expr
import Data.Ratio
import Data.Char
import Data.Time.Calendar
import Data.Time.Calendar.Julian
import Control.Monad (when)
import qualified Account as A
lexer :: P.TokenParser ()
lexer = P.makeTokenParser (haskellStyle {
P.reservedNames = ["buy", "sell", "split", "exchange", "n"]
, P.reservedOpNames = ["+", "-", "*", "/"]
})
whiteSpace = P.whiteSpace lexer
lexeme = P.lexeme lexer
colon = P.colon lexer
decimal = P.decimal lexer
identifier = P.identifier lexer
parens = P.parens lexer
reserved = P.reserved lexer
reservedOp = P.reservedOp lexer
symbol = P.symbol lexer
stringLiteral = P.stringLiteral lexer
journalParser :: Parser [A.Account ()]
journalParser = do whiteSpace
as <- many activity
eof
return as
activity :: Parser (A.Account ())
activity = buy <|> sell <|> split <|> exchange
buy :: Parser (A.Account ())
buy = do reserved "buy"
dat <- date
sym <- asset
qty <- quantity
cst <- cost qty
fail (show cst)
return (A.buy dat sym qty cst)
sell :: Parser (A.Account ())
sell = do reserved "sell"
dat <- date
sym <- asset
qty <- quantity
pro <- proceeds qty
return (A.sell dat sym qty pro)
split :: Parser (A.Account ())
split = do reserved "split"
dat <- date
sym <- asset
rat <- splitRatio
return (A.split dat sym rat)
exchange :: Parser (A.Account ())
exchange = do reserved "exchange"
dat <- date
sym <- asset
qty <- quantity
sym' <- asset
qty' <- quantity
return (A.exchange dat sym qty sym' qty')
date :: Parser Day
date = lexeme date' <?> "date as 'yyyy/mm/dd'"
date' :: Parser Day
date' = do sy <- count 4 (digit <?> "year as 'yyyy'")
sep <- oneOf "/:.-" <?>
"date field separator (one of '/', ':', '.', or '-')"
m <- oneOrTwoDigits <?> "month as 'mm'"
char sep <?> ("date field separator " ++ show sep)
d <- oneOrTwoDigits <?> "day as 'dd'"
let y = read sy
when (m > 12 || d > (julianMonthLength y m))
(fail ("invalid date: " ++ sy ++ [sep] ++ show m ++ [sep] ++ show d))
return (fromGregorian y m d)
oneOrTwoDigits :: Parser Int
oneOrTwoDigits = do d <- digit
secondDigit (digitToInt d)
secondDigit :: Int -> Parser Int
secondDigit n = do d <- digit <?> ""
return (10 * n + digitToInt d)
<|> return n
asset :: Parser A.Symbol
asset = identifier <|> stringLiteral <?> "symbol or asset description"
splitRatio :: Parser Rational
splitRatio = do n <- lexeme decimal
colon <?> "split ratio separator ':'"
d <- lexeme decimal
when (n <= 0 || d <= 0)
(fail ("invalid split ratio: " ++ show n ++ ":" ++ show d))
return (n % d)
<?> "split ratio as 'new-units : old-units'"
quantity :: Parser A.Quantity
quantity = do q <- value
when (q <= 0) (fail "number of units must be greater than zero")
return q
<?> "number of units (fractional units are permitted)"
cost :: A.Quantity -> Parser A.Cost
cost qty = do cst <- (valueExt qty <?> "transaction cost (including commissions)")
<|> pricePerUnit qty
when (cst < 0) (fail "cost cannot be negative")
return cst
proceeds :: A.Quantity -> Parser A.Proceeds
proceeds qty = do pro <- (valueExt qty <?> "transaction proceeds (net of commissions)")
<|> pricePerUnit qty
when (pro < 0) (fail "proceeds cannot be negative")
return pro
pricePerUnit :: A.Quantity -> Parser A.Value
pricePerUnit qty = do symbol "@"
pr <- valueExt qty
return (qty * pr)
<?> "price per unit as '@ number'"
valueExt :: A.Quantity -> Parser Rational
valueExt qty = lexeme value'
<|> parens (exprExt qty)
exprExt :: A.Quantity -> Parser Rational
exprExt qty = buildExpressionParser table (factorExt qty) <?> "expression"
factorExt :: A.Quantity -> Parser Rational
factorExt qty = parens (exprExt qty)
<|> lexeme value'
<|> do reserved "n"
return qty
<?> "simple expression (use 'n' for number of units)"
value :: Parser Rational
value = lexeme value'
<|> parens expr
expr :: Parser Rational
expr = buildExpressionParser table factor <?> "expression"
factor :: Parser Rational
factor = parens expr
<|> lexeme value'
<?> "simple expression"
table = [[op "*" (*) AssocLeft, op "/" (/) AssocLeft],
[op "+" (+) AssocLeft, op "-" (-) AssocLeft]
]
where op s f assoc =
Infix (do { reservedOp s; return f } <?> "operator") assoc
value' :: Parser Rational
value' = do d <- decimal
f <- option 0 fraction <?> "optional fractional part"
return (toRational d + f)
fraction :: Parser Rational
fraction = do char '.'
d <- decimal <?> "digits after decimal point"
return (d % ceiling10 d)
where ceiling10 0 = 1
ceiling10 1 = 1
ceiling10 n = 10 * ceiling10 (n `div` 10)
-----------------------------------------
pt = parseTest (do journalParser; return ())

Show comments

@@ -0,0 +1,183 @@
module Main where
import Text.ParserCombinators.Parsec
import qualified Text.ParserCombinators.Parsec.Token as P
import Text.ParserCombinators.Parsec.Language (javaStyle)
import Data.Time.Calendar
import Data.Char (isSpace)
import System.IO
-- Put your favorite file name here
fname = "fxt.ledger"
main = do res <- parseFromFile journal fname
case res of
Left err -> do putStr "parse error at " ; print err
Right rs -> mapM_ (putStrLn . show) rs
-- I use mapM_ to print one record per line; otherwise, calling
-- show of a list will put everything on the same line.
type Entry = String
type Directive = String
type Transaction = String
type Code = String
type Payee = String
type Amount = String
type Commodity = String
type Quantity = String
type Annotation = String
type Account = [String]
type Date = String
journal :: Parser [Entry]
journal = do xs <- many ( entry
<|> directive
<|> comment
<|> emptyLine )
eof
return [x | Just x <- xs]
-- WARNING: this has to be the last production or it will not parse
-- The reason is that the parser could choose this production by matching
-- empty and it will not be able to backtrack once it discovers empty is not
-- followed by a newline.
emptyLine :: Parser (Maybe Entry)
emptyLine = do many (oneOf " \t")
newline
return Nothing
comment :: Parser (Maybe Entry)
comment = do oneOf ";*hb"
skipMany (satisfy (/= '\n'))
newline
return Nothing
directive :: Parser (Maybe Entry)
directive = do (oneOf "!*" >> wordDirective) <|> charDirective
newline
return Nothing
wordDirective :: Parser Directive
wordDirective = (string "include" >> longText)
<|> (string "account" >> longText)
<|> (string "end")
<|> (string "alias" >> identifier >> char '=' >> longText)
<|> (string "def" >> longText)
-- WARNING: should this really include trailing white space?
longText :: Parser String
longText = many1 (satisfy (/= '\n'))
-- WARNING: not sure about this one
identifier :: Parser String
identifier = many1 (satisfy (not . isSpace))
charDirective :: Parser Directive
charDirective = do oneOf "iIoO"
date
time
longText
return "i|I|o|O directive"
<|> do char 'D'
amount
return "D directive"
<|> do char 'A'
longText
return "A directive"
<|> do char 'C'
commodity
char '='
amount
return "C directive"
<|> do char 'P'
date
time
commodity
amount
return "P directive"
<|> do char 'N'
commodity
return "N directive"
<|> do char 'Y'
count 4 digit
return "Y directive"
<|> do string "--"
identifier
longText
return "-- directive"
date :: Parser Date
date = do count 4 digit
dateSep
count 2 digit
dateSep
count 2 digit
return "date"
dateSep :: Parser ()
dateSep = do oneOf "/-."
return ()
time :: Parser ()
time = do count 2 digit
char ':'
count 2 digit
char ':'
count 2 digit
return ()
commodity = identifier
-- WARNING: I need to define amount correctly
amount = identifier
entry :: Parser (Maybe Entry)
entry = do e <- plainEntry
return (Just e)
plainEntry :: Parser Entry
plainEntry = do date
optional (char '=' >> date)
optional status
optional code
fullString
optional note
newline
many1 transaction
return "plain entry"
status :: Parser Char
status = oneOf "*!"
code :: Parser String
code = between (char '(') (char ')') (many1 (noneOf ")\n"))
note :: Parser String
note = do char ';'
cs <- longText
return cs
-- WARNING: this is not defined in grammar.y
fullString :: Parser String
fullString = many1 (noneOf ";\n")
transaction :: Parser Transaction
transaction = do many1 (oneOf " \t")
optional status
account
-- missing values_opt from grammar.y
optional note
newline
return "transaction"
account :: Parser String
account = accountName
<|> between (char '(') (char ')') accountName
<|> between (char '[') (char ']') accountName
accountName :: Parser String
accountName = do ns <- sepBy1 accountName' (char ':')
return (concat ns)
accountName' :: Parser String
accountName' = many1 (noneOf ")]:;\n")

Show comments

@@ -0,0 +1,12 @@
This code has been written by Filippo Tampieri,
with respective Copyright.
Notes from Filippo:
int := sign digits
starting symbols: '-' or (empty)
octocat-spinner-128.gif

report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK