netherdrake / ga-bitbot Goto Github PK

Hello =)

    Since most of us use ga-bitbot on multi-purpose machines, it would be very nice to be able to limit individual proceses to certain cores on your processor [one clone of your project on google code was made for exactly the same purpose, but seems abandoned =] and set their priority. Since different OSes have different ways and utilities for accomplishing the same purpose, it would be great if a config file [maybe global_config.json =] had two additional keys for 'process prefix' which would when be used by gal.py as a prefix before launching monitored and unmonitored processes. Anyone could then just edit that value at one place, according to his/her needs and OS, and hopefully preserve it through git updates =]

Grateful in advance!

Add ranged variable capability to the genetic script to support negative 
numbers and offsets 

book.templ - extra '/' on new paths.
issue 45

What steps will reproduce the problem?
1. python gal.py
2. All processes start, reports are being generated
3. After a while I get this output:
WATCHDOG: PID d3934bc2 EXPIRED
Traceback (most recent call last):
  File "gal.py", line 143, in <module>
    monitor[pid]['process'].terminate()
  File "c:\Python27\lib\subprocess.py", line 1016, in terminate
    _subprocess.TerminateProcess(self._handle, 1)
WindowsError: [Error 5] Access is denied
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "c:\Python27\lib\atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "gal.py", line 79, in shutdown
    monitor[pid]['process'].terminate()
  File "c:\Python27\lib\subprocess.py", line 1016, in terminate
    _subprocess.TerminateProcess(self._handle, 1)
WindowsError: [Error 5] Access is denied
Error in sys.exitfunc:
Traceback (most recent call last):
  File "c:\Python27\lib\atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "gal.py", line 79, in shutdown
    monitor[pid]['process'].terminate()
  File "c:\Python27\lib\subprocess.py", line 1016, in terminate
    _subprocess.TerminateProcess(self._handle, 1)
WindowsError: [Error 5] Access is denied

What version of the product are you using?
ga-bitbot.0.2.zip
python-2.7.2.msi
pypy-1.8-win32.zip
pycrypto-2.3.win32-py2.7.zip

On what operating system?
Windows 7 64-bit SP1

Please provide any additional information below.
The command prompt (cmd) from which the 'python gal.py' was started has admin 
privileges.

This link may provide some tips on how to solve the problem: 
http://stackoverflow.com/questions/1682447/cross-platform-way-to-terminate-a-pro
cess-in-python

Currently, the bcbookie [and possibly the whole algo] rounds the prices down to 
the 2 decimal places. That may be a sound decision for determining an optimal 
trade strategy, but bcbookie real life performance could be improved by 
increasing the price it buys at by a minimal amount [0.00342 seems to me like a 
good start] and subtracting the same amount from the sell price. That-a-way 
ga-bitbot orders would be performed before other orders rounded up to the 2 
decimal places, resulting in more trades, and the profit/loss calculations of 
the whole system would be affected minimally. Additional advantage would be 
that such orders [made by bcbookie, not by humans/other bots] could easily be 
identified/distinguished when looking at the current order list by one mtgox 
user or the whole mtgox list.

Grateful in advance ;)

What steps will reproduce the problem?
1. running bcbookie.py
2.
3.

What is the expected output? What do you see instead?
When buy orders are filled they should enter a 'held' state until the commit 
price is met. In some cases filled orders are being closed in the book and 
reported as 'buy_cancel: pending state (insuf funds?)' 

This issue has been verified by comparing the trade_ids for 'buy_cancel: 
pending state (insuf funds?)' orders in the book to the account history 
available on MtGox.

Hello!

    I was very excited to see the addition of bid_maker, but unfortunately it seems that either my db is contaminated, or the my bid_maker and bcbookie refuse to cooperate. bid_maker seems to run fine, but bcbookie reports errors like pasted below all the time [output abbreviated a bit]:
______
update: current price: 13.320  account market value: 195.6005 @ Wed Jan  2 
10:39:58 2013
checking open orders
checking held positions
main: Availble Funds (USDS,BTCS) :(186.5505, 0.67942741)
scanning for active bids by priority
66830eb427d01cd3d163e01ebe7b9778 priority: 0 enabled: 1 target: $0.0 discount: 
100.0
c3305d462059672f4f6a8daa3f885728 priority: 0 enabled: 1 target: $12.10925 
discount: 9.08971471471
d561bc709dfea4059ebefe200ddc168b priority: 0 enabled: 1 target: $-1.0 discount: 
107.507507508
5fd57626d7ae95bb45a454f78ed6ce67 priority: 0 enabled: 1 target: $12.86118 
discount: 3.44459459459
0bdb6a52c098156a12aa17d0511717d6 priority: 0 enabled: 1 target: $12.75955 
discount: 4.20758258258
d062b161818d6e8ab37c46c6d5ec1e2b priority: 0 enabled: 1 target: $-1.0 discount: 
107.507507508
fd88f00ecdac76785fffb05ee69099f9 priority: 0 enabled: 1 target: $0.0 discount: 
100.0
82ff966f847c86839ec2a9110373d320 priority: 0 enabled: 1 target: $-1.0 discount: 
107.507507508
main: warning - ignoring invalid target order:
{u'bid_maker_time_stamp': 1357119350.424017, u'buy': -1.0, u'score': 0, 
u'gene_id': u'c869ac6e8acd82d1-d8d41286d56607fd'}
main: monitor mode or target order not validated.
sleeping...
__________________________________________________________________________

Please, advise how to proceed. Should we also be running the report_gen along 
side of the bcbookie or could we just forget about it if we do not care about 
the graphs of active gene_def?

Grateful in advance.

P.S.
    All the best wishes for 2013! ;)

Hello,

    This one seems like a minor one:

although I set the...
"enable_text_messaging":0
...in file...
config/bcbookie_main.json
...when I run the bcbookie it still reports:
AWS_SNS requires the boto package to be installed.

Thank you!

The 1st and 2nd quartile are having problems gathering information from mtgo.
With the last updates the application is reporting a high number of expired 
connections (usually are the same two connections that are being dropped and 
recreated), while the gene visualizer show that the first 2 quartiles have much 
less information than the other two.
I am attaching the gene visualizer of a old version (commit 320b6b39cc15) and a 
new one (commit 11fbbefe3aa3) both with a difference of a few hours.

Add a simulator mode to bcbookie in order to gauge the profitability of the 
system as it evolves with the latest trade data.

Hi all,

I've created a commit that adds a .gitignore that would allow for easier usage 
of a git checkout for actually running the code.

I've also change most py files to follow PEP 8, by changing tabs into spaces. 
There are a few typo fixes in there as well.

You can get the commit here: 
https://github.com/timstoop/ga-bitbot/commit/f345506c54454fa8afd115dea804b6caa48
dbb7c

Hope this helps!

In today's version the gene_def states:
"commision": 0.06,

Shouldn't it be 0.006? ;)

Keep up the great work! =)

[using latest git, but seen the issue in previous versions]
Hello,

    The output of report_gen.py and the reports it creates differ from the data the 'system monitor' displays. Number of trades in each of the quartiles differs a lot [some are even reported as empty], and all the scores differ a lot. It does display the correct score in the 'header' of it's stdout output, for example:
Quartile: 4 (29725.3666)
...but few lines down it states:
Score:  -1445.21898857
...and continues to state wrong number of orders, and the resulting htmls also 
have trades and scores that do not make sense to me. Could it be it's not using 
the default gene_def?

Grateful in advance!

My experience is that somewhere around 20-25 gts.py instances [on remote 
machines] most of the time make it crash within 10-15 minutes of the runtime.

Unfortunately, no error is to be seen, and the gal.py does not restart it 
[after all, it is 'unmonitored' =]

The problems can escalate if there are open orders at that moment, because the 
bcbookie also terminates when it can't contact the gene server, and leaves the 
orders open.

Adding gene server to monitored proceses [and assuming that it loads it's 
archived database automatically when [re]started by the gal.py] would lessen 
the impact.

bcbookie could cancel all the orders upon detecting an critical error [for 
example gene server not responding] before terminating.

Thank you =]

All the clients caclulate the scores as -0.12346. Latest git. confirmed with 
old db, blank db and both gts.py 'y' and 'n'.

Thank you for all the hard work!

[version <=0.1.2b]

Brian confirmed that we should 'check the gts.py file line 58 and set
load_throttle = 0'.

Will probably be fixed in the next version.

What steps will reproduce the problem?

1. Start the system through gal.py (pypy)
2. Wait a few minutes
3. When a process dies, an Access Denied error will be thrown and the entire 
system will exit

What is the expected output? What do you see instead?

The process should terminate and the system continue to run normally.


What version of the product are you using? On what operating system?

latest trunk (master) branch


Please provide any additional information below.

Running Windows 7 x64. UAC has no impact on this issue. Google seems to suggest 
this is some kind of known/common problem on Windows platforms. I've attached a 
patch that seems to work and should also be cross-platform compatible still, 
although I haven't tested it on others yet. I'm not Python expert so feel free 
to take or leave the patch, just one possible solution to the problem I could 
work out.

What steps will reproduce the problem?
1. pypy gts.py 1 y v
2. wait
3. Wonder why did it halt ;) Should it not continue reevaluating results 
considering the new data that bcfeed.py gathers?

What is the expected output? What do you see instead?
Expected - lot of numbers flying past =]
Actual - ...MAX_GENE is gone...Halted.

What version of the product are you using? On what operating system?
0.1.2b or the one befire that one [how do I figure out the current ver? ]

Please provide any additional information below.
Last lines of output are attached to the post.

extra forward slashes on new paths

What steps will reproduce the problem?
1. python ./gal.py
2. wait a couple of hours
3. WATCHDOG: PID 0d4a9dfd EXPIRED

What is the expected output? What do you see instead?
This error-message appear:
WATCHDOG: PID 97ff6095 EXPIRED
Traceback (most recent call last):
  File "gal.py", line 152, in <module>
    monitor.update({npl[0]:{'cmd':cmd_line,'process':p}})   #store the pid/cmd_line/process
IndexError: list index out of range


What version of the product are you using? On what operating system?
0.2

What steps will reproduce the problem?
1. running the system launcher (gal.py) 

When the gene_server starts up the default gene_def is set to "UNDEFINED". The 
gene_server then uses the gene_def hash from the first client registration to 
set the default gene_def_hash / database. 

Right now the the first client to register is report_gen.py which requests the 
current default gene_def_hash from the gene_server on every reporting cycle.  
This means the "UNDEFINED" db will be set as the default.

report_gen will then try to report out the "UNDEFINED" database, which in 
itself isn't a problem. The issue is that if no data is available report_gen 
will wait in a loop and under normal operating conditions the "UNDEFIEND" db 
isn't used so no data will ever become available. 

Even if the default config is changed on the server, report_gen wouldn't pick 
up the change until the next cycle which will never come.

What steps will reproduce the problem?
1. launch bct.py
2.
3.

What is the expected output? What do you see instead?
Result:
Bitcoin trade simulator v0.02a
Traceback (most recent call last):
  File "bct.py", line 615, in <module>
    te = prof.runcall(test)
  File "/usr/lib/python2.7/hotshot/__init__.py", line 78, in runcall
    return self._prof.runcall(func, args, kw)
  File "bct.py", line 595, in test
    te.input(float(t),float(r))
  File "bct.py", line 430, in input
    self.ai()       #call the trade ai
  File "bct.py", line 276, in ai
    if self.balance > (current_price * self.shares) and self.buy_delay == 0 and self.quartile == self.market_class[self.period][1]:
IndexError: list index out of range


What version of the product are you using? On what operating system?
latest source

Please provide any additional information below.
gene_server, gts, report_gen, and bcfeed where running

What steps will reproduce the problem?
1. run the application for a long time (more than a few hours)
2. This issue is worse in the dev version than the current stable version
3. Stable version could go a day or longer before this was an issue, the dev 
version is effected within a few hours of operation.


What is the expected output? What do you see instead?
Only the wc_server and report_gen scripts are running after hours of execution. 
 No bts scripts are running or being restarted.

What version of the product are you using? On what operating system?
Tried with current stable and dev version (from git).

OS: Windows Server 2008 R2 (x64)
pypy is used in-lue of python.

Please provide any additional information below.

Since I stopped running/updating my old pool due to multiple errors and started 
a new one, I was wondering how could I grab a few of my old champions and 
insert them into my new environment. One of my gts.py processes was forgotten 
[and running the whole time] during the switchover, and already passed the 
'old' genes to the new environment. I could manually parse the gene server db 
backup files and copy/paste few of the highest scoring genes from the old to 
the new environment, but I'm sure that it could be done in a much more 
efficient way. Maybe the new genes have much to learn from some of the old ones 
that do not have very high scores ;)

If you think about it for a moment.. What's stopping us from exchanging the 
genes between different people that run ga-bitbot? =) One python script that 
would merge them in a smart way is indeed all we need =]

Anyone interested in exchanging the genes [please, be serious, we're talking 
about the 1s and zeroes ;] please reply to this thread/issue, and maybe we can 
spark Brian's interest =]

What steps will reproduce the problem?
1. Run gal.py for a while.
2. Graphs will no longer be updated.

What is the expected output? What do you see instead?
I expect the graphs to be updated more frequently. Currently, I'm still getting 
the ones from 12 hours ago.

What version of the product are you using? On what operating system?
I'm using the HEAD git repository.

Please provide any additional information below.
Is there any way to force a redraw of the gene visualizer and the Quartile 
reports?

After commit 981929597d44 the WATCHDOG function report every process as EXPIRED 
after on minute.
The process probably arent being launched because "top" doesn't report any 
signal of pypy activity, as usual.
I am using ubuntu 12.10 (x86) with stock packages, except pypy that is from 
pypy weekly build ppa.

What steps will reproduce the problem?
1. bcfeed_synch.py -d to bootstrap data.
2. invoke gal.py, processes continue and bot starts up.
3. Terminate bot
4. invoke gal.py

What is the expected output? What do you see instead?
Expected:

Checking potential for incremental update...
Downloading mtgox historic data...
Download complete.
Processing input...
Writing output file...
Updating the datafeed directory directly...no need to manualy move the output 
file
Writing incremental update...
Done.


Seen:
Checking potential for incremental update...
Downloading mtgox historic data...
Download complete.
Processing input...
Traceback (most recent call last):
  File "bcfeed_synch.py", line 58, in <module>
    last_t = d[0].split(',')[0]
IndexError: list index out of range
Starting the live datafeed capture script...

What version of the product are you using? On what operating system?
Version: git clone* latest revisions.
OS:Ubuntu 10.4_x86_64 w/python2.7

Please provide any additional information below.

Deleting datafeed files, copying datafeed_reboot file into datafeed folder, and 
running bcfeed_synch.py -d allows startup with gal.py and expected result.

Hello!

    I have checked out todays version from the git tree and tried running it the old way [manualy started each component]. It seemed to work correctly. Afterwards, I tried running the new gal.py script and it failed to start. The output is pasted at the end of this description. Please, ignore/close this issue if you already know about this =]

Feel free to ask for any testing I may help you with.

Kind regards!

----
user@machine:~/tmp/ga-bitbot$ pypy gal.py
ga-bitbot system launcher v0.01a
Synching the local datafeed...

Bitcoin Data Download (CSV) Processor v0.01a

    Converts the data into a weighted price 1min data feed format

To automaticaly download and process the mtgox usd historic data run with the 
following option:
python process -d

Without the -d switch the output will only be written to the datafeed_reboot 
folder

Checking potential for incremental update...
Downloading mtgox historic data...
Download complete.
Processing input...
Writing output file...
Updating the datafeed directory directly...no need to manualy move the output 
file
Writing incremental update...
Done.
Starting the live datafeed capture script...
Launching the xmlrpc server...
Connected to 127.0.0.1 : 9854
Load archived gene database? (y/n)
y
Loading the gene database...
NOK
Launching GA Clients...
Monitored Process Launched (PID: 360a27b7 CMD: pypy gts.py 1 n )
Monitored Process Launched (PID: c1cc9163 CMD: pypy gts.py 2 n )
Monitored Process Launched (PID: 49d65d51 CMD: pypy gts.py 3 n )
Monitored Process Launched (PID: 0cae0ab7 CMD: pypy gts.py 4 n )
Traceback (most recent call last):
  File "app_main.py", line 53, in run_toplevel
  File "gal.py", line 106, in <module>
    monitor.update({npl[0]:{'cmd':cmd_line,'process':p}})   #store the pid/cmd_line/process
IndexError: list index out of range
user@machine:~/tmp$ pypy --version
Python 2.7.1 (?, Oct 08 2011, 14:11:45)
[PyPy 1.6.0 with GCC 4.5.2]

What steps will reproduce the problem?
1. let bcbookie submit bids
2. manually log into mtgox and cancel the bid 

What is the expected output? What do you see instead?
bcbookie should detect this case and report that the order was canceled 
Instead, bcbookie reports the order as 'error: insuf funds' or 'buy_cancel: 
pending state (insuf funds?)' depending if the order was pending or open.

Need to implement a solution to detect this condition in order to accurately 
report the reason for closing the position in the book.

Hello,

    During the last two months I have not succeded to convince ocl_gts.py to calculate correct scores. It shows -10000 in both 'previews' of the top 10 and as a local maximum reached. After I noticed that part, it was not a surprise anymore to me that it never sends any BOBs to the gene server =( Confirmed the behavior on both 64 bit linux machine and 64 bit win machine. I have sent you more detailed description by the mail.

Feel free to ask for any further details that may help and thank you for 
helping us all this time =)

If you want to, you can allow me to edit the FAQ adding the clarifications you 
write to us as a issue resolutions and email conversations. That would save you 
some time, and hopefully my additions would not be seen by anyone but you 
before you approve them. If not, just ignore my offer and close this issue ;)

What steps will reproduce the problem?
1. run bcbookie.py
2.
3.

What is the expected output? What do you see instead?
bcbookie should take into consideration open buy orders when determining if 
enough funds are available to support new orders.

Instead bcbookie only considers the account balance when screening orders which 
can result in buy orders with insufficient funds.

What steps will reproduce the problem?
1. run the ga-bitbot system
2. wait until OS failure (ubuntu 11.10)
3. corrupt pickle data gets written to disk 

- This issue is repeatable.

What is the expected output? What do you see instead?
I would expect the OS not to write bad data to disk.

After running the system for a few days the operating system reports the disk 
is full (it's not) and the pickle module will write invalid data to disk. This 
destroys all bcbookie records resulting in lost trade transaction data.

Need to implement an automated backup strategy to safeguard for this condition.

Hello,

    I ran one gal.py 'server' instance and two 'client' instances. All three of them were synced to today's git tree. After I stop one of the client instances using CTRL+c [@linux, with pypy], I get the output as follows:
-----
Traceback (most recent call last):
  File "app_main.py", line 51, in run_toplevel
  File "gal.py", line 237, in <module>
    sleep(5) #check one pid every n seconds.
KeyboardInterrupt
-----
...so far seems everything is working as expected. But.. The other client 
instance immidiately stops working:
-----
Traceback (most recent call last):
  File "app_main.py", line 53, in run_toplevel
  File "gal.py", line 238, in <module>
    if server.pid_check(pid,WATCHDOG_TIMEOUT) == "NOK":
  File "/usr/share/pypy-1.6/lib-python/2.7/xmlrpclib.py", line 1224, in __call__
    return self.__send(self.__name, args)
  File "/usr/share/pypy-1.6/lib-python/2.7/xmlrpclib.py", line 1570, in __request
    verbose=self.__verbose
  File "/usr/share/pypy-1.6/lib-python/2.7/xmlrpclib.py", line 1264, in request
    return self.single_request(host, handler, request_body, verbose)
  File "/usr/share/pypy-1.6/lib-python/2.7/xmlrpclib.py", line 1292, in single_request
    self.send_content(h, request_body)
  File "/usr/share/pypy-1.6/lib-python/2.7/xmlrpclib.py", line 1439, in send_content
    connection.endheaders(request_body)
  File "/usr/share/pypy-1.6/lib-python/2.7/httplib.py", line 937, in endheaders
    self._send_output(message_body)
  File "/usr/share/pypy-1.6/lib-python/2.7/httplib.py", line 797, in _send_output
    self.send(msg)
  File "/usr/share/pypy-1.6/lib-python/2.7/httplib.py", line 759, in send
    self.connect()
  File "/usr/share/pypy-1.6/lib-python/2.7/httplib.py", line 740, in connect
    self.timeout, self.source_address)
  File "/usr/share/pypy-1.6/lib-python/modified-2.7/socket.py", line 616, in create_connection
    raise err
error: [Errno 111] Connection refused
-----
...ooops! Checked the server instance and saw that the gene server is no longer 
up =( Unfortunately, there is no new entry to be seen in 
gene_server_error_log.txt =(

Tried it once again and confirmed the results. This is not the first time I 
experienced such a problem, but is the first time I reported it. Please, look 
into it when you decide it is time to figure it out, and if you are unable to 
reproduce the issue, I would be glad to help in any way I can.

Grateful in advance!

When trying to create the api key an error appear with the following output:

Generating the encrypted API KEY file...
Traceback (most recent call last):
  File "encrypt_api_key.py", line 59, in <module>
    encryptor = AES.new(hash_pass, AES.MODE_CBC)
  File "/usr/lib/python2.7/dist-packages/Crypto/Cipher/AES.py", line 94, in new
    return AESCipher(key, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/Crypto/Cipher/AES.py", line 59, in __init__
    blockalgo.BlockAlgo.__init__(self, _AES, key, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/Crypto/Cipher/blockalgo.py", line 141, in __init__
    self._cipher = factory.new(key, *args, **kwargs)
ValueError: IV must be 16 bytes long

This happens because with AES with CBC mode the parameter IV isn't optional 
anymore.

https://bugs.launchpad.net/pycrypto/+bug/1018283

What steps will reproduce the problem?
1. invoke bfceed_synch.py -d 
2. invoke gal.py
3. Load archived gene database? (y/n) y

What is the expected output? What do you see instead?
Expected:
    update: current price 6.770 @ Fri Jan  6 15:15:22 2012
checking open orders
checking held positions
main: Availble Funds (USDS,BTCS) :(u'100.0093', u'50')
main: No GA order available.
sleeping...

Seen:
    update: current price 6.750 @ Fri Jan  6 14:58:07 2012
checking open orders
checking held positions
main: Availble Funds (USDS,BTCS) :(u'100.0093', u'50')
Traceback (most recent call last):
  File "bcbookie.py", line 482, in <module>
    commit = ((t['target'] - t['buy']) * 0.8) + t['buy'] #commit sell order at 80% to target
KeyError: 'target'


What version of the product are you using? On what operating system?
Version: Git Clone w/ revisions
OS: Ubuntu 10.4_x86_64 w/ python2.7

Please provide any additional information below.

If you load archived gene database files previous to bcfeed_synch.py -d data, 
this error pops up. Boot strapping data and selecting no on load archived gene 
database stops the error from happening.

Hello,

    I'm running the latest git release. After successfully running the 
gal.py with [PyPy 1.6.0 with GCC 4.5.2], I have imported the API key and 
manually ran the bcbookie.py with [Python 2.7.1+]. After a few sleep cycles it 
tries to buy a few shares and crashes with an error message pasted below. 
Please advise what to do next, or feel free to ask for more data or more 
detailed troubleshooting.

Grateful in advance!

---------

sleeping...
________________________________________________________________________________



________________________________________________________________________________
    update: current price: 6.300  account market value: 4.4100 @ Mon Jan  9 17:10:41 2012
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
checking open orders
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
checking held positions
--------------------------------------------------------------------------------
main: Availble Funds (USDS,BTCS) :(u'0', u'0.7')
buy: order validated
Traceback (most recent call last):
  File "bcbookie.py", line 550, in <module>
    b.buy(0.5,t['buy'],commit,t['target'],t['stop'],60 * 5,t['stop_age'])
  File "bcbookie.py", line 278, in buy
    if (self.funds() - self.balance_committed) > cost and qty >= 0.01:
TypeError: unsupported operand type(s) for -: 'unicode' and 'int'

gene.templ - extra '/' on new paths.
issue 45

Need to check for partial order fills for expiring buy orders. 

Currently if the buy order expires before the complete order is filled the 
position is canceled. 

Instead, the order amount should be adjusted to the fill amount and enter the 
held state.

Various crashes and problems could be more easily identified if ga-bitbot 
logged output of it's components to log files. Currently, various utilities do 
output to stdout/screen various messages when ran directly [as in pypy 
gts.py..], but not when gal.py runs them. It would be great if they stashed 
their output to log files when gal.py starts them.

Thank you =]

Since you modularized your code, and individual components indeed do 
communicate over the network, it just begs to be executed on many machines 
simultaneously =]

It would be more convinient to be able to run just one script on the 'server' 
machine [bcfeed.py, wc_server.py & report_gen.py] and the other script that 
runs and monitors gts.py processes on client machines [and the server machine 
if it has enough resources =]

I really do not want to impose my self on your project, but if you would be 
willing to accept code contributions, I can give it a try. Of course, it would 
work better if you wrote it yourself, but I do have a feeling that you tend to 
concentrate on more complex and important issues, so this one may linger at the 
bottom of the todo list for an eternity =]

Hello,

I'm using the current git version. After a dozen of successful trades I checked 
my book.html, and found out five orders are being 'held' and a few of previous 
purchases are being sold. Seems everything would be on right track if it 
weren't for those held orders. One of them has trade id that ends in 590. mtgox 
records show: "BTC bought: [tid:xxx590] 1.00000000 BTC at $4.69222". Hmmm... 
the book.html says that commit is 4.93791000, the stop column contains 0, the 
'target' is 5.01981000, and column 'book' is 'held'.

As far as I got it, bookie tried to buy @4.93, somehow bought @4.69, and 
figured out that it makes no sense to sell again @5.01, so it held the order.

Please, advise. If you need my bookie_records.pkl or any other log file, I'd be 
glad to send it to you by email. If you want my bookie's ear, that can be also 
arranged ;)

Grateful in advance!

The intention from the beginning was to make the system completely modular with 
each component independent of one another. This would allow easy 
re-appropriation of the code base for other uses or end user implementation of 
alternative trade engines. 

Over time some modules became coupled to one another; In particular between 
bct, which implements the trade strategy, and gts, which configures and feeds 
bct. 

Most of the decoupling between other modules can be handled by assigning 
variables to  'magic numbers' and moving them into external configuration files 
(already covered by another enhancement request)

The purpose of this enhancement is to define a clear interface for which 
fitness classes can adhere. This will facilitate modular programming and 
decouple bct (the fitness class in this case) from the rest of the system.

system.templ - extra '/' on new paths.
issue 45

My machines running gts.py started running into the following error:
Traceback (most recent call last):
  File "app_main.py", line 51, in run_toplevel
  File "gts.py", line 327, in <module>
    g.next_gen()
  File "C:\bin\ga-bitbot\genetic.py", line 315, in next_gen
    mf = self.merge_multi(f_l)  #multi parent merge - female
  File "C:\bin\ga-bitbot\genetic.py", line 179, in merge_multi
    sums[i] += 1
IndexError: list index out of range

I've performed a fresh checkout of ga-bitbot source, and manually ran gts.py on 
my second machine [the first one runs the gene server], and after a while I get 
the same error. The error occurred while running gts.py on the second quartile. 
Although I checked out clean version, one of my other machines was still 
processing the 'old' fourth quartile, so it managed to sneak in one bob into 
the 4th quartile of the new setup =]

Both environments showed the same behavior repeatedly multiple times within at 
max an hour of runtime, and I am willing to send the gene server db backups if 
needed.

Grateful in advance!

What steps will reproduce the problem?
1. launch "report_gen"

What is the expected output? What do you see instead?
Expected report generation.
Program crash with this error:

Traceback (most recent call last):
  File "report_gen.py", line 140, in <module>
    te.chart("/home/emfb/public_html/bc/chart.templ","/tmp/chart_test_%s.html"%str(quartile))
  File "/home/ale/ga-bitbot/bct.py", line 510, in chart
    f = open(template,'r')
IOError: [Errno 2] No such file or directory: 
'/home/emfb/public_html/bc/chart.templ'


What version of the product are you using? On what operating system?
git latest version

Please provide any additional information below.

What steps will reproduce the problem?
1. potential out of synch issue with "max_length" variables across modules 
(report_gen & gts)

What is the expected output? What do you see instead?
The system configuration should be outside of the modules. Instead magic 
numbers for the same variable are included in multiple modules, leading to a 
potential variable synch issue.

Need to complete a review of the code to identify and move all in module 
configuration variables shared between modules into a shared configuration file.

What steps will reproduce the problem?
1. Have a buy order placed (book = held)
2. Have this order committed (old buy order oid, book = closed:commit)


What is the expected output? What do you see instead?
Expected result
record_synch: OID: 284fa068-1c29-4a98-bdfa-fd2be93df6fc  active
Seen:
!!!!!!! record_synch: orphaned or manual order found: TYPE: 1 AMOUNT: 0.1 
PRICE: 



Please provide any additional information below.
Perhaps the new sell order oid is not registered in self.records.
Old buy order oid (tagged as closed:commit) not removed from self.records, so 
it remains in book.html report

What steps will reproduce the problem?
1. Run bcfeed_sync.py


What is the expected output? What do you see instead?

bcfeed_synch: Checking potential for incremental update...
bcfeed_synch: Downloading mtgox historic data...
Traceback (most recent call last):
  File "bcfeed_synch.py", line 41, in <module>
    data = urllib2.urlopen(link).read()
  File "/usr/lib/python2.7/urllib2.py", line 127, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 407, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 520, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 445, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 379, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 528, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 503: Service Temporarily Unavailable

What version of the product are you using? On what operating system?

9a3eb1cce9da

Please provide any additional information below.

Causes gal.py to crash and causes all the clients to crash.

What steps will reproduce the problem?
1. Run the ga-bitbot system and accumulate held positions
2. Wait until someone dumps a large volume of sell orders into a weak bid 
market depth.
3. ga-bitbot executes stop loss orders on held positions, selling into the crash
4. Cascading stop loss orders from this and other bots drives the price down 
lower yet.

What is the expected output?
Add a delay (wait for lower volitility?) before executing the stop loss order.