Submitted by: @pcisar

This affects mostly binary BLOBs, but it's general problem. Shows up in both, materialized and stream BLOBs.

Commits: dd7d0a0 FirebirdSQL/fbt-repository@b229988

Submitted by: @pcisar

Should be interface-compatible to KInterbasDB's implementation..

Submitted by: Philipp Wojke (pw)

Attachments:
fbcore.py.patch
objgraph.png
fbcore.py.2.patch
fbcore.py.3.patch

http://fbcore.py uses weakrefs for breaking reference cycles, but fails to do it properly. On several occasions a weakref callback is used, that introduces a circular reference through the implicit self of the bound method.

This leads to multiple memory leaks, the most obvious one affecting PreparedStatement and every call to execute with a different operation. This memory leak is intensified by the fact that PreparedStatement holds a database resource handle, the prepared statement. In scripts that heavily pound the database this leads to error -904 'too many open handles to database'.

To break the cycle an additional weakref can be used. Each time a callback to a method bound to self is used, this must be wrapped in a weakref proxy:

Wrong: self.cursor = weakref.proxy(cursor,self.__cursor_deleted)
Right: self.cursor = weakref.proxy(cursor,weakref.proxy(self.__cursor_deleted))

No additional handling should be necessary, because the referenced method lives at least as long as the weakref calling the callback.

This change fixed multiple previously unexplained program terminations and memory problems in our environment.

Commits: 148f23f FirebirdSQL/fbt-repository@2f06fd5

Submitted by: Cagatay Tengiz (ctengiz)

in http://fbcore.py line 1257:

pairCount = len(buf) / pairSize

in http://fbcore.py line 1260

for i in range(pairCount):

but pairCount is not integer.

I think this is because of : http://stackoverflow.com/questions/19507808/python3-integer-division

So converting pairCount to integer implicitly solves problem..

Commits: 40a5d41 FirebirdSQL/fbt-repository@823398c

Submitted by: @pmakowski

Assigned to: @pmakowski

#⁠coding:utf-8

import fdb

con = fdb.connect(dsn='localhost:employee', user='sysdba', password='masterkey')

cur = con.cursor()
SELECT = "select COUNTRY, CURRENCY from country"

cur.execute(SELECT)
for row in cur:
print('%s - %s.' % (row[0], row[1]))
print('end1')
cur.execute(SELECT)
print('start2')
for row in cur:
print('%s - %s.' % (row[0], row[1]))
print('end2')
cur.execute(SELECT)
print('start3')
for row in cur:
print('%s - %s.' % (row[0], row[1]))
print('end3')

the third iteration fails
start3
Traceback (most recent call last):
File "test_bla.py", line 22, in <module>
for row in cur:
File "/usr/lib/python2.7/site-packages/fdb-0.7.1-py2.7.egg/fdb/fbcore.py", line 1633, in next
row = self.fetchone()
File "/usr/lib/python2.7/site-packages/fdb-0.7.1-py2.7.egg/fdb/fbcore.py", line 1697, in fetchone
return self._ps._fetchone()
File "/usr/lib/python2.7/site-packages/fdb-0.7.1-py2.7.egg/fdb/fbcore.py", line 1599, in _fetchone
"Cursor.fetchone: Unknown status returned by fetch operation:")
fdb.fbcore.ProgrammingError: ('Cursor.fetchone: Unknown status returned by fetch operation:\n- SQLCODE: -502\n- The cursor identified in an OPEN statement is already open.\n- Dynamic SQL Error', -502, 335544569)

if you close the cursor, all is ok
cur.execute(SELECT)
for row in cur:
print('%s - %s.' % (row[0], row[1]))
print('end1')
cur.execute(SELECT)
print('start2')
for row in cur:
print('%s - %s.' % (row[0], row[1]))
print('end2')
cur.close() #⁠ here added a close
cur.execute(SELECT)
print('start3')
for row in cur:
print('%s - %s.' % (row[0], row[1]))
print('end3')

Commits: 324a05a

Submitted by: Douglas Hammond (wizhippo)

The version check default to using isc_info_firebird_version which does not work with interbase server.

When switched to using isc_info_version instead it works,

Here is the version string I get when using isc_info_version WI-V8.1.1.333

This may be the case with other Interbase server. Perhaps this driver will not support connection to an intebase sever but only a firebird server?

Submitted by: Rajko Thon (rthon)

Attachments:
fdb_test.zip

For a database stress test we run a python script which spawns multiple threads with multiple connections per thread.
After some short time all threads stop working (hang).
See attached python script which reproduces the error reliably.
The issue seems to occur for multiple threads only. One thread alone is working fine.
A similar program using freepascal does not have any problem with the same database.
So we think it is a problem in the python driver which also only occurs under high load.

For 4 threads: See faulthandler traceback-output on Linux after: kill -USR1 <pid>.
Surprisingly one thread suddenly seems to jump from __XSQLDA2Tuple into the destructor.
The subsequent close seems to block all other threads.

Faulthandler output:

Thread 0x00007f3039020700 (most recent call first):
File "/usr/local/lib/python3.4/dist-packages/fdb/fbcore.py", line 3028 in _close
File "/usr/local/lib/python3.4/dist-packages/fdb/fbcore.py", line 3149 in __del__
File "/usr/local/lib/python3.4/dist-packages/fdb/fbcore.py", line 2383 in __XSQLDA2Tuple
File "/usr/local/lib/python3.4/dist-packages/fdb/fbcore.py", line 3070 in _execute
File "/usr/local/lib/python3.4/dist-packages/fdb/fbcore.py", line 3353 in execute
File "http://fbcall.py", line 70 in run
File "/usr/lib/python3.4/threading.py", line 920 in _bootstrap_inner
File "/usr/lib/python3.4/threading.py", line 888 in _bootstrap

Thread 0x00007f3039821700 (most recent call first):
File "/usr/local/lib/python3.4/dist-packages/fdb/fbcore.py", line 3062 in _execute
File "/usr/local/lib/python3.4/dist-packages/fdb/fbcore.py", line 3353 in execute
File "http://fbcall.py", line 70 in run
File "/usr/lib/python3.4/threading.py", line 920 in _bootstrap_inner
File "/usr/lib/python3.4/threading.py", line 888 in _bootstrap

Thread 0x00007f303a022700 (most recent call first):
File "/usr/local/lib/python3.4/dist-packages/fdb/fbcore.py", line 3062 in _execute
File "/usr/local/lib/python3.4/dist-packages/fdb/fbcore.py", line 3353 in execute
File "http://fbcall.py", line 70 in run
File "/usr/lib/python3.4/threading.py", line 920 in _bootstrap_inner
File "/usr/lib/python3.4/threading.py", line 888 in _bootstrap

Thread 0x00007f303a823700 (most recent call first):
File "/usr/local/lib/python3.4/dist-packages/fdb/fbcore.py", line 3062 in _execute
File "/usr/local/lib/python3.4/dist-packages/fdb/fbcore.py", line 3353 in execute
File "http://fbcall.py", line 70 in run
File "/usr/lib/python3.4/threading.py", line 920 in _bootstrap_inner
File "/usr/lib/python3.4/threading.py", line 888 in _bootstrap

Current thread 0x00007f303eaa2700 (most recent call first):
File "/usr/lib/python3.4/threading.py", line 1076 in _wait_for_tstate_lock
File "/usr/lib/python3.4/threading.py", line 1060 in join
File "http://fbcall.py", line 226 in <module>

Submitted by: Mikhail Tchervonenko (rusmikle)

if system x64 and FB x32 path for fbclient.dll in registry HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Firebird Project\Firebird Server\Instances\DefaultInstance (now HKEY_LOCAL_MACHINE\SOFTWARE\Firebird Project\Firebird Server\Instances\DefaultInstance)
FDB driver does not find the library fbclient.dll
please add in http://ibase.py section for sys.platform == 'win64'
.........

Commits: 1c79dcd FirebirdSQL/fbt-repository@dbea24e

Submitted by: asmith (asmith)

Any exception in the event_process method causes the event thread started by fdb to die, but this error is in no way propagated, so process continues to run, but events are no longer consumed.

@40000000531fed1d03a23aac waiting for events
@40000000531fee592f1640ac Exception in thread Thread-1:
@40000000531fee592f16487c Traceback (most recent call last):
@40000000531fee592f164c64 File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner
@40000000531fee592f16504c self.run()
@40000000531fee592f165434 File "/usr/lib/python2.6/threading.py", line 484, in run
@40000000531fee592f16581c self.__target(*self.__args, **self.__kwargs)
@40000000531fee592f165c04 File "/usr/lib/python2.6/site-packages/fdb/fbcore.py", line 1775, in event_process
@40000000531fee592f97e814 events = data.count_and_reregister()
@40000000531fee592f97ebfc File "/usr/lib/python2.6/site-packages/fdb/fbcore.py", line 1734, in count_and_reregister
@40000000531fee592f97efe4 self.__wait_for_events()
@40000000531fee592f97f3cc File "/usr/lib/python2.6/site-packages/fdb/fbcore.py", line 1720, in __wait_for_events
@40000000531fee592f97f7b4 "Error while waiting for events:")
@40000000531fee592f980f24 DatabaseError: ('Error while waiting for events:\n- SQLCODE: -902\n- Error reading data from t
he connection.', -902, 335544726)
@40000000531fee592f98130c

Submitted by: Arthur Lee (arthurlee)

If database connection is lost while waiting at an event conduit, it will abort the application with the assertion

Assertion failed: cb\_node\-\>updated\_buf \!= NULL, file c:\\kinterbasdb\\kinterbasdb\-3\.3\.0\\\_kievents\_infra\.c, line 959

This application has requested the Runtime to terminate it in an unusual way\.
Please contact the application's support team for more information\.

Perhaps, an exception should be raised instead.

Use the following to reproduce the failure:

#⁠-------------------------------------------------------------
import kinterbasdb

cn = kinterbasdb.connect(dsn='localhost:anydatabase', user='sysdba',
password='masterkey!')

conduit = cn.event_conduit(['fake_event'])

try:
print 'waiting for events... Stop Firebird now.'
events = conduit.wait()

print '\*\* Not reachable\. Received', events

except:
print '** Not reachable. Exception'

print '** Not reachable. Bye.'
#⁠-------------------------------------------------------------

Submitted by: @tonal

Attachments:
Py_ssize_t_STRING_FORMAT.path

Log for build:
...
In file included from _kiconversion.c:81,
from _kinterbasdb.c:235:
_kiconversion_blob_streaming.c: In function 'conv_in_blob_from_pyfilelike':
_kiconversion_blob_streaming.c:816: warning: unknown conversion type character 'z' in format
_kiconversion_blob_streaming.c:816: warning: too many arguments for format
In file included from _kinterbasdb.c:245:
_kicore_connection.c: In function 'pyob_Connection_connect':
_kicore_connection.c:1175: warning: unknown conversion type character 'z' in format
_kicore_connection.c:1175: warning: too many arguments for format
_kicore_connection.c:1188: warning: unknown conversion type character 'z' in format
_kicore_connection.c:1188: warning: too many arguments for format
In file included from _kinterbasdb.c:249:
_kicore_cursor.c: In function 'pyob_Cursor_arraysize_set':
_kicore_cursor.c:1876: warning: unknown conversion type character 'z' in format
_kicore_cursor.c:1876: warning: too many arguments for format
...
Pach for correct:
Index: _kinterbasdb.h

--- _kinterbasdb.h (revision 1088)
+++ _kinterbasdb.h (working copy)
@@ -166,7 +166,11 @@
#⁠define PyInt_FromSsize_t PyInt_FromLong
#⁠define PyInt_AsSsize_t PyInt_AsLong
#⁠else
- #⁠define Py_ssize_t_STRING_FORMAT "%zd"
+ #⁠ifndef PLATFORM_WINDOWS
+ #⁠define Py_ssize_t_STRING_FORMAT "%zd"
+ #⁠else
+ #⁠define Py_ssize_t_STRING_FORMAT "%Id"
+ #⁠endif
#⁠define Py_ssize_t_EXTRACTION_CODE "n"
#⁠endif
#⁠define SIZE_T_TO_PYTHON_SIZE(x) ((Py_ssize_t)(x))

Submitted by: Boris Savelev (boris)

Votes: 1

http://downloads.sourceforge.net/firebird/kinterbasdb-3.3.0.tar.bz2

x86_64-alt-linux-gcc -pthread -fno-strict-aliasing -DNDEBUG -pipe -Wall -O2
-fPIC -DPIC -D_GNU_SOU RCE -fPIC -UNDEBUG -I/usr/include/python2.6
-I/usr/include/python2.6 -c _kiservices.c -o build/tem
p.linux-x86_64-2.6/_kiservices.o -pedantic -g -std=c99 -fno-strict-aliasing
-pthread -O3
In file included from /usr/include/string.h:658,
from /usr/include/python2.6/Python.h:38,
from _kinterbasdb.h:33,
from _kiservices.h:21,
from _kiservices.c:18:
In function 'memcpy',
inlined from 'pyob_query_base' at _kiservices.c:363:
/usr/include/bits/string3.h:52: error: call to __builtin___memcpy_chk will
always overflow destination buffer

probably length on 'unsigned' different on x86 and x86_64

Submitted by: Dirk Rothe (d.rothe)

Attachments:
test_dates_fdb.py

Raw string date input parameters fail with fdb 0.8 whereas kinterbasdb converts them implicitly.

Commits: 50b699b

Submitted by: @pcisar

Direct dependency on fbclient library has two issues:

1. prevents successful installation when library cannot be found.
2. it's not possible to specify custom client library to be used.

Solution should solve both issues.

Commits: 3354699

Submitted by: Dominik Psenner (dpsenner)

Looks like the conversion does not handle the datetime.date datatype properly.

Traceback (most recent call last):in __read
File "fdb\\fbcore.pyo", line 3338, in execute
File "fdb\\fbcore.pyo", line 3006, in _execute
File "fdb\\fbcore.pyo", line 2812, in __Tuple2XSQLDA
File "fdb\\fbcore.pyo", line 2270, in _convert_timestamp
AttributeError: 'datetime.date' object has no attribute 'date'

Commits: a3d6310 FirebirdSQL/fbt-repository@7ed6220

Submitted by: @pavel-zotov

Attachments:
python-crash-screen-when-using-fbclient-of-fb3.png
python-crash-dumps-when-using-fbclient-of-fb3.zip

Steps to reproduce:

0) install FB 2.5 and FB 3.0 on the same Windows host (hereafter: FB 2.5 was installed in C:\1INSTALL\FIREBIRD\fb25sC\ and listening port 3255; FB 3.0 was installed in C:\1INSTALL\FIREBIRD\fb30sS\ and listening port 3333; `PATH` variable contains first folder to FB3: PATH=C:\1INSTALL\FIREBIRD\fb30sS;D:\Python27\;D:\Python27\Scripts;D:\W2K\system32;D:\W2K; . . .)

1) create .fbt file, e.g. `core_4386.fbt`, and save it in the folder %fbt-repo-home%\tests\bugs\
File content:
{
'id': '',
'qmid': None,
'tracker_id': '',
'title': '',
'description': '',
'min_versions': '3.0',
'versions': [
{
'firebird_version': '3.0',
'platform': 'All',
'init_script':
"""
""",
'test_type': 'Python',
'test_script':'''c1=db_conn.cursor()
c1.execute("select sign(current_connection) as att from rdb$database");
printData(c1)
''',
'expected_stdout':
"""
ATT
-----------
1
""",
'substitutions':[('-.*','')]
}
]
}

2) open cmd.exe window and do there:

C:\FBTESTING\qa\fbt-repo>set pbak=%path%
C:\FBTESTING\qa\fbt-repo>set p25=C:\1INSTALL\FIREBIRD\fb25sC\bin
C:\FBTESTING\qa\fbt-repo>set path=%p25%;%pbak%

// check that we indeed have folder to FB 2.5 client at the beginning of PATH list:
C:\FBTESTING\qa\fbt-repo>path
PATH=C:\1INSTALL\FIREBIRD\fb25sC\bin;C:\1INSTALL\FIREBIRD\fb30sS;D:\Python27\;D:\Python27\Scripts;D:\W2K\system32; . . .

// now run test which should check FB-3 instance but do connect using client of 2.5:

C:\FBTESTING\qa\fbt-repo>fbt_run -b C:\1INSTALL\FIREBIRD\fb30sS bugs.core_4386 -o localhost/3333
.
----------------------------------------------------------------------
Ran 1 tests in 0.781s

OK

// now restore PATH to previous value, so the first folder in it will be "C:\1INSTALL\FIREBIRD\fb30sS" -- i.e. to FB3 instance:
C:\FBTESTING\qa\fbt-repo>set path=%pbak%

C:\FBTESTING\qa\fbt-repo>path
PATH=C:\1INSTALL\FIREBIRD\fb30sS;D:\Python27\;D:\Python27\Scripts;D:\W2K\system32;D:\W2K; . . .

// and repeat fbt_run once more:

C:\FBTESTING\qa\fbt-repo>fbt_run -b C:\1INSTALL\FIREBIRD\fb30sS bugs.core_4386 -o localhost/3333

Last action can finish with either exception followed by "OK"

Exception WindowsError: 'exception: access violation reading 0x00000010' in <bound method PreparedStatement.__del__ of <
fdb.fbcore.PreparedStatement object at 0x02308E90>> ignored
.
----------------------------------------------------------------------
Ran 1 tests in 0.906s

OK

-- or with Windows message about failed process (Python), and in that case test will not run at all.
Sometimes I get only empty output: neither exception from Python nor Windows abend message.
No any rows in firebird.log appear when this fault occur.

Attached files: screenshot and additional dumps that are specified in it.

PS.

`fbtest` was updated 07-JUN-2015, but I saw this error before.

Config of FB3:

RemoteServicePort = 3333
AuthServer = Legacy_Auth, Srp, Win_Sspi
UserManager = Legacy_UserManager
AuthClient = Legacy_Auth
WireCrypt = Disabled
SharedCache = true
SharedDatabase = false
ExternalFileAccess = Full

Config of FB 2.5:

RemoteServicePort = 3255
DefaultDbCachePages = 256
ExternalFileAccess = Restrict C:\1INSTALL\FIREBIRD\fb25sC;C:\FBTESTING\qa\fbt-repo\tmp;C:\FBTESTING\OLTP2014
FileSystemCacheThreshold = 65536K
LockHashSlots = 22111
MaxUserTraceLogSize = 99999

Submitted by: Baskin Tapkan (baskint)

Have tried running the "python http://setup.py install" command on two different systems. I am getting the below error on both. Would you please help? Thanks.

c:\src\python\fdb-0.9.1>python http://setup.py install
Traceback (most recent call last):
File "http://setup.py", line 7, in <module>
from fdb import __version__
File "c:\src\python\fdb-0.9.1\fdb\__init__.py", line 23, in <module>
from fdb.fbcore import *
File "c:\src\python\fdb-0.9.1\fdb\http://fbcore.py", line 26, in <module>
from . import ibase
File "c:\src\python\fdb-0.9.1\fdb\http://ibase.py", line 39, in <module>
fb_library = WinDLL(fb_library_name)
File "C:\Python27\lib\ctypes\__init__.py", line 353, in __init__
self._handle = _dlopen(self._name, mode)
TypeError: expected string or Unicode object, NoneType found

Submitted by: @pcisar

Other python database adapters support specifying a name for the parameters in the SQL string and passing a dict to execute(), to bind the parameters to its values. Something like this:

> cur.execute('select * from customers where id = %(cid)d', {'cid': 3})

Submitted by: @pcisar

Commits: e760a21

Submitted by: @pcisar

Releasing the Cursor instance does not permanently free the prepared statement on the server side. During the destructor of the Cursor instance the internal managed PreparedStatement instances will also be released. This will call the method _close from PreparedStatement. Inside the _close Method the release of the server side statement resource will only be triggered if there is an open connection. But to get this connection the PreparedStatement instance uses the cursor Attribute which is a weak reference to the Cursor instance. But this Cursor instance is no longer available because the program execution is inside the destructor (__del__) of this cursor. The weak reference is dead.

It seems that Cursor and PreparedStatement instances are released on the client side but the following statement from the _close method of
PreparedStatement doesn't get executed.

api.isc_dsql_free_statement(self._isc_status, stmt_handle,
ibase.DSQL_drop)

Commits: b59ab8f bee6db8 43545f1 FirebirdSQL/fbt-repository@16788b7 FirebirdSQL/fbt-repository@2dc5a90 FirebirdSQL/fbt-repository@1061333

Submitted by: brad clements (bkc)

I have code that calls kinterbasdb.init(type_conv=199, concurrency_level=2)

this doesn't work with importing fdb as kinterbasdb

is there an equivalent init method for fdb ?

Submitted by: @artyom-smirnov

Regression after commit 61195.

FDB unit tests full of error like:

.Exception ReferenceError: 'weakly-referenced object no longer exists' in <bound method PreparedStatement.__del__ of <fdb.fbcore.PreparedStatement object at 0x7fb1bb9b8e90>> ignored
.Exception ReferenceError: 'weakly-referenced object no longer exists' in <bound method Cursor.__del__ of <fdb.fbcore.Cursor object at 0x7fb1bb9b8e90>> ignored

Commits: c363a56 FirebirdSQL/fbt-repository@d9444cb

Submitted by: Uwe Grauer (uweg)

Attachments:
kinterbasdb-3.2.1-patched-diff.txt

There was a bugfix for kinterbasdb 3.2 from David Rushby back in May 2007.
See:
http://sourceforge.net/forum/message.php?msg_id=4288153
Also the thread "kinterbasdb 3.2.1":
http://tech.groups.yahoo.com/group/firebird-python/message/69

I already sent a diff for KI 3.2.1 to Philippe Makowski.

Submitted by: Virgo Pärna (virgo)

fbclient.dll is not found by _dlopen.

Traceback (most recent call last):
File "test_fb.py", line 4, in <module>
import fdb
File "D:\Python33\lib\site-packages\fdb-1.0-py3.3.egg\fdb\__init__.py", line 23, in <module>
from fdb.fbcore import *
File "D:\Python33\lib\site-packages\fdb-1.0-py3.3.egg\fdb\http://fbcore.py", line 26, in <module>
from . import ibase
File "D:\Python33\lib\site-packages\fdb-1.0-py3.3.egg\fdb\http://ibase.py", line 41, in <module>
fb_library = WinDLL(fb_library_name)
File "D:\Python33\lib\ctypes\__init__.py", line 353, in __init__
self._handle = _dlopen(self._name, mode)
TypeError: bad argument type for built-in operation

I fixed it for myselt by adding:
elif sys.platform in ['win32']:
fb_library_name = find_library('fbclient.dll')

into http://ibase.py. But i don't know, if it's needed for every Python versions/Windows version. I don't have 64 bit python installed, so I don't know, if it is issue there do. In that case probably 64 bit Windows should probably have special behavior also.

Commits: 2335712

Submitted by: Dan Casper (g.d.d.c)

I have this domain defined:

CREATE DOMAIN txtparm AS BLOB SUB_TYPE 1 DEFAULT NULL;

And a table using it was raising this error:

Unicode strings are not acceptable input for a non-textual BLOB column.

From ! line 2851 in http://fbcore.py. The if statement requires an additional check that sqlvar.sqlsubtype != 1 before raising that error is reasonable. Is there a .git repository I can clone / pull from or likewise so I can submit changes? Or is notifying someone enough?

Commits: c678adf FirebirdSQL/fbt-repository@75818c8

Submitted by: @pcisar

Assigned to: @pmakowski

Commits: 571fa2a

Submitted by: @pcisar

FDB should support context managers out of the box whenever it's reasonable (at least for transaction management).

Commits: 16d93bd FirebirdSQL/fbt-repository@754839f

Submitted by: @pcisar

New, accurate documentation for FDB is needed. We can't go with KInterbasDB documentation any longer.

Submitted by: Werner F Bruhin (werner)

Attachments:
fdbAndPy3.patch

When trying to do a backup and exception is thrown, details please see the thread "Python 3.4 and FDB - backup" on mailto:[email protected]

Commits: a3d6310 FirebirdSQL/fbt-repository@7ed6220

Submitted by: Oscar Micheli (oscar.micheli)

The problem is the following:

If I try to read from a varchar column a string longer than 128 chars i get an empty string. The same string in a BLOB TEXT return the correct content.

I made a table for testing:

CREATE TABLE FDBTEST (
TEST80 VARCHAR(80),
TEST128 VARCHAR(128),
TEST255 VARCHAR(255),
TEST1024 VARCHAR(1024),
TESTCLOB BLOB SUB_TYPE 1 SEGMENT SIZE 255
);

and the floowing program to test

import fdb
conn = fdb.connect(dsn='localhost:e:/apps/db/test.fdb', user='sysdba', password='masterkey')

cur = conn.cursor()
cur.execute("SELECT TEST255 FROM FDBTEST")
for c in cur.fetchall():
print len(c[0]),c[0]
conn.close()

The output with test data is the following:

42 012345678901234567890123456789021234567890
126 012345678901234567890123456789021234567890012345678901234567890123456789021234567890012345678901234567890123456789021234567890
127 0123456789012345678901234567890212345678900123456789012345678901234567890212345678900123456789012345678901234567890212345678901
0
0
0

The last three rows should be 128,129 and 130 instead of 0.

Of course the same program using kinterbasdb gives the correct result.

I suppose there must be some limit buried somwhere in the code.

Commits: 19f3bcb 0c3c629

Submitted by: @pcisar

Current Services API implementation has numerous issues:

1) doesn't conform to Python code standards (method names etc.)
2) bugs.
3) bad design in some cases.

Submitted by: @pcisar

Should be interface-compatible to KInterbasDB's implementation.

Submitted by: Geovani de Souza (geovanisouza92)

Hello guys,

I'm trying to deploy a Django project that uses django-firebird app, that uses fdb.

I'm deploying on http://flynn.io/, an alternative to heroku, using heroku-buildpack-multi, because I need to install libfbclient2 to enable fdb.

The problem is that heroku-buildpack-apt that I use to install libfbclient2 puts the lib on "/app/.apt/usr/lib/x86_64-linux-gnu" and add this folder to LD_LIBRARY_PATH, but the class "fdb.ibase.fbclient_API" doesn't reach the fbclient lib, because "ctypes.util.find_library" is unable to read alternative folder from LD_LIBRARY_PATH, for example (http://bugs.python.org/issue2936).

Reading the source, I saw that "fdb.ibase.fbclient_API.__init__()" accepts a parameter "fb_library_name". This is passed from "fbcore.load_apit()" that is called from "fbcore.connect()" and "fbcore.create_database()", but the last two doesn't uses this parameter.

My suggestion is to add the "fb_library_name" to "fbcore.connect()" and "fbcore.create_database()" passing it to "fbcore.load_api()".

This way, I can set "settings.DATABASES['myfbdb']['OPTIONS']['fb_library_name']" on my Django project settings, that is passed through django-firebird to fb ("fbcore.connect()" to be specific).

Commits: 4bfa810 FirebirdSQL/fbt-repository@cd987ae

Submitted by: @pmakowski

Crash during Firebird test suite
kernel: [ 724.669973] fbt_run[21560]: segfault at 0 ip (null) sp 00007fff305ac9f8 error 14 in python2.7

My last attempt with Superserver show the crash during test about CORE2307
My other attempt with Classic show the crash during test about functional.monitoring.04

No problem with FDB at rev 59844

More info on Firebird devel list and logs at http://ci.ibphoenix.com

the test about CORE2307 is :

- create a database
In the database :

set term ^ ;
EXECUTE BLOCK AS
DECLARE VARIABLE I INTEGER = 0;
BEGIN
WHILE (I < 1000)
DO
BEGIN
EXECUTE statement 'CREATE TABLE T' || cast(:I as varchar(5)) || ' (C INTEGER)';
I = I + 1 ;
END
END ^
commit ^

EXECUTE BLOCK AS
DECLARE VARIABLE I INTEGER = 0;
BEGIN
WHILE (I < 1000)
DO
BEGIN
EXECUTE statement 'INSERT INTO T' || cast(:I as varchar(5)) || ' (C) VALUES (1)';
I = I + 1 ;
END
END ^
commit ^

After that the python script is doing :

import fdb

db_conn = fdb.connect(
host='localhost', database='/temp/test.db',
user='sysdba', password='masterkey'
)

sql = """EXECUTE BLOCK AS
DECLARE VARIABLE N VARCHAR(31);
DECLARE VARIABLE I INTEGER;
BEGIN
FOR SELECT RDB$RELATION_NAME
FROM RDB$RELATIONS
INTO :N
DO
BEGIN
EXECUTE statement 'SELECT FIRST 1 1 FROM ' || :N INTO :I;
END
END"""
c = db_conn.cursor()
c.execute(sql)
print (db_conn.db_info(kdb.isc_info_read_seq_count))

The test about functional.monitoring.04 is a connection to a database with SYSDBA and with another user :

import fdb

con_1 = fdb.connect(
host='localhost', database='/temp/test.db',
user='sysdba', password='masterkey'
)
con_1.begin()
c1 = con_1.cursor()
con_2 = fdb.connect(
host='localhost', database='/temp/test.db',
user='TEST', password='test'
)
con_2.begin()
c2 = con_2.cursor()
c2.execute("SELECT TRIM(T2.MON$USER), T1.MON$ISOLATION_MODE FROM MON$TRANSACTIONS T1 JOIN MON$ATTACHMENTS T2 USING (MON$ATTACHMENT_ID) ORDER BY 1")
print (c2.fetchall())
c1.execute("SELECT TRIM(T2.MON$USER), T1.MON$ISOLATION_MODE FROM MON$TRANSACTIONS T1 JOIN MON$ATTACHMENTS T2 USING (MON$ATTACHMENT_ID) ORDER BY 1")
print (c1.fetchall())

Submitted by: Marius Ionescu (hector_imb)

Votes: 1

1. I create an initial migration with south:
python manage schemamigration myapp --initial

2. Then I execute:
python http://manage.py migrate myapp

3. Then a "Segmentation fault" message is received.

Tried the same thing on multiple Linux boxes (e.g. Debian).

Without being able to run database migrations, basically I cannot use firebird at all. I had to temporarily switch to MySQL.

Submitted by: mike bayer (zzzeek)

The following script illustrates a difference in concurrency behavior between kinterbasdb and fdb. I've tried various FDB settings in order to work around this but I can't seem to get one. Basically, when a particular DBAPI connection has rollback() called, it should not be retaining any kind of transactional state that would interfere with a table being dropped. With kinterbasdb this works, with FDB it does not. The issue here is somewhat of a blocker for FDB support in SQLAlchemy, as we can't get our unit test suite to pass completely.

import fdb
import kinterbasdb

def run_test(dbapi, user, password, host, dbname):
print("Running with dbapi: %s" % dbapi)

raw\_conn\_one = dbapi\.connect\(user=user, host=host, password=password, database=dbname\)
raw\_conn\_two = dbapi\.connect\(user=user, host=host, password=password, database=dbname\)

cursor = raw\_conn\_one\.cursor\(\)
cursor\.execute\("CREATE TABLE foo \(id integer\)"\)
cursor\.close\(\)
raw\_conn\_one\.commit\(\)

cursor = raw\_conn\_two\.cursor\(\)
cursor\.execute\("insert into foo \(id\) values \(1\)"\)
cursor\.close\(\)
#⁠ rollback connection two, but dont actually close it\.
#⁠ if we close it, then both DBAPIs proceed\.
raw\_conn\_two\.rollback\(\)

cursor = raw\_conn\_one\.cursor\(\)
cursor\.execute\("drop table foo"\)
cursor\.close\(\)

print\("about to commit\.\.\."\)
raw\_conn\_one\.commit\(\)

#⁠ FDB will not get here until raw\_conn\_two is closed\.
#⁠ kinterbasdb will\.
print\("done\!"\)

run_test(kinterbasdb, "sysdba", "masterkey", "localhost", "//Users/classic/foo.fdb")
run_test(fdb, "sysdba", "masterkey", "localhost", "//Users/classic/foo.fdb")

the test above will complete when the "kinterbasdb" call proceeds. On the "fdb" call, it hangs:

classic$ python http://test2.py
Running with dbapi: <module 'kinterbasdb' from '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/kinterbasdb/__init__.pyc'>
about to commit...
done!
Running with dbapi: <module 'fdb' from '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/fdb/__init__.pyc'>
about to commit...

I'm not sure if fdb has some architectural issue that makes it work this way, but if not a bug, some guidance on correct practices would be helpful. thanks !

Commits: a3d6310 FirebirdSQL/fbt-repository@7ed6220

Submitted by: Dirk Rothe (d.rothe)

Attachments:
fdb_test.py

Executing an insert statement and omitting a NOT NULL parameter that should be filled by an Insert Trigger fails.

Commits: 1040913

Submitted by: Uwe Grauer (uweg)

Relate to CORE2033

I installed kinterbasdb 3.2.1 from source with "python http://setup.py install".
python -c "import kinterbasbd" gives:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/local/lib/python2.5/site-packages/kinterbasdb/__init__.py", line 109, in <module>
import _kinterbasdb as _k
ImportError: /usr/lib/libfbclient.so.2: undefined symbol: _Unwind_GetIP

See also thread starting with:
http://tech.groups.yahoo.com/group/firebird-python/message/69

Submitted by: Dominik Psenner (dpsenner)

Attachments:
brokenDbEvents.py
EventBlock-0.png
EventBlock-19.png

A memory leak in the class EventBlock(object), defined in http://dbcore.py at line 1687 most probably caused by the cyclic reference introduced in its constructor at lines 1705 and 1709:

1705: self.__queue.put((ibase.OP_RECORD_AND_REREGISTER,self))
1709: self.__queue = queue

See here for a more detailed description:

https://groups.yahoo.com/neo/groups/firebird-python/conversations/messages/742

This may be related to the issue outlined at https://groups.yahoo.com/neo/groups/firebird-python/conversations/messages/742 . Sometimes the event_conduit(events) invoke raises exceptions in the EventBlock object and thus it is impossible to invoke close() on the event_conduit that has been created. The exceptions observed so far are:

event\_listener = con\.event\_conduit\(eventNames\)

File "C:\Python27\lib\site-packages\fdb-1.4.1-py2.7.egg\fdb\http://fbcore.py", line 1601, in event_conduit
conduit = EventConduit(self._db_handle,event_names)
File "C:\Python27\lib\site-packages\fdb-1.4.1-py2.7.egg\fdb\http://fbcore.py", line 1811, in __init__
self.__event_blocks.append(EventBlock(self.__queue,db_handle,block_events))
File "C:\Python27\lib\site-packages\fdb-1.4.1-py2.7.egg\fdb\http://fbcore.py", line 1726, in __init__
self.__wait_for_events()
File "C:\Python27\lib\site-packages\fdb-1.4.1-py2.7.egg\fdb\http://fbcore.py", line 1735, in __wait_for_events
"Error while waiting for events:")
DatabaseError: ('Error while waiting for events:\n- SQLCODE: -902\n- Error reading data from the connection.', -902, 335544726)

event\_listener = con\.event\_conduit\(eventNames\)

File "C:\Python27\lib\site-packages\fdb-1.4.1-py2.7.egg\fdb\http://fbcore.py", line 1601, in event_conduit
conduit = EventConduit(self._db_handle,event_names)
File "C:\Python27\lib\site-packages\fdb-1.4.1-py2.7.egg\fdb\http://fbcore.py", line 1811, in __init__
self.__event_blocks.append(EventBlock(self.__queue,db_handle,block_events))
File "C:\Python27\lib\site-packages\fdb-1.4.1-py2.7.egg\fdb\http://fbcore.py", line 1726, in __init__
self.__wait_for_events()
File "C:\Python27\lib\site-packages\fdb-1.4.1-py2.7.egg\fdb\http://fbcore.py", line 1735, in __wait_for_events
"Error while waiting for events:")
DatabaseError: ('Error while waiting for events:\n- SQLCODE: -902\n- Error writing data to the connection.', -902, 335544727)

File "http://dbEvents.py", line 90, in run
event_listener = con.event_conduit(eventNames)
File "C:\Python27\lib\site-packages\fdb-1.4.1-py2.7.egg\fdb\http://fbcore.py", line 1601, in event_conduit
conduit = EventConduit(self._db_handle,event_names)
File "C:\Python27\lib\site-packages\fdb-1.4.1-py2.7.egg\fdb\http://fbcore.py", line 1811, in __init__
self.__event_blocks.append(EventBlock(self.__queue,db_handle,block_events))
File "C:\Python27\lib\site-packages\fdb-1.4.1-py2.7.egg\fdb\http://fbcore.py", line 1726, in __init__
self.__wait_for_events()
File "C:\Python27\lib\site-packages\fdb-1.4.1-py2.7.egg\fdb\http://fbcore.py", line 1735, in __wait_for_events
"Error while waiting for events:")
DatabaseError: ('Error while waiting for events:\n- SQLCODE: 0\n- unknown ISC error 0', 0, 0)

This issue relates also to PYFB41.

Commits: 23f90ec 65160c3 8de82ce fc5dd5a a3d6310 FirebirdSQL/fbt-repository@5462470 FirebirdSQL/fbt-repository@a54cdd7 FirebirdSQL/fbt-repository@1b5c1cf FirebirdSQL/fbt-repository@83703c1 FirebirdSQL/fbt-repository@7ed6220

Submitted by: @pmakowski

Assigned to: @pmakowski

Commits: 5001795

Submitted by: Werner F Bruhin (werner)

Attachments:
useregtogetlocation.patch

When installing fdb 1.4 using either 'http://setup.py install' or 'easy_install fdb' it does not find the fbclient.dll if it is not on a path. The error shown does not clearly identify what is causing the error.

ctypes.find_library fails to find it as the path is not including the FB server install and there is also this issue http://bugs.python.org/issue16283

Commits: fcc6548 FirebirdSQL/fbt-repository@100f245

Submitted by: nepomnyashiy evgeniy (nepomnyashiy evgeniy)

Sometimes I receive strange exception

File "c:\tools\reestr\env\Lib\site-packages\fdb\http://fbcore.py", line 3040, in execute
PreparedStatement(operation, self, True))
File "c:\tools\reestr\env\Lib\site-packages\fdb\http://fbcore.py", line 1701, in __init__
self.__coerce_XSQLDA(self.out_sqlda)
File "c:\tools\reestr\env\Lib\site-packages\fdb\http://fbcore.py", line 2009, in __coerce_XSQLDA
sqlvar.sqlind = ctypes.pointer(ISC_SHORT(0))
TypeError: incompatible types, LP_c_short instance instead of LP_c_short instance

After I receive it first time, every query to database ends with it. Creating new connection or something else dont help - I need to restart apache.

On my development machine (without apache and mod_wsgi) I never receive such exception.

Submitted by: @pcisar

Should be interface-compatible to KInterbasDB's implementation.

Submitted by: nepomnyashiy evgeniy (nepomnyashiy evgeniy)

I have table with field
"field" "Text" /* "Text" = VARCHAR(5000) */,

One record have text of about 2500 chars in this field.
If I select this record, I receive only 213 chars.

cur = con.cursor()
cur.execute('SELECT "field" from "table" where id=?', [id])
for m in cur.itermap():
print m["field"]

With kinterbasdb this works fine

Commits: 3354699

Submitted by: Paulo Freitas (paulofreitas)

I'm having trouble to create a table with BIGINT data type with fdb module. See that StackOverflow question: http://stackoverflow.com/questions/18156780/importing-a-large-sql-file-into-a-local-firebird-database

I've no idea why client SQL dialect is not being set here, shouldn't sql_dialect argument of fdn.create_database() work for that?

Am I missing something?

Thanks!

Submitted by: Cagatay Tengiz (ctengiz)

Is duplicated by PYFB63

Bug in : http://schema.py line 2852

for p in self.input_params:

I think it should be :

for p in self.output_params:

ps : In fact my fdb version is 1.4.8 but jira does not have this version

Commits: 40a5d41 084dd42 FirebirdSQL/fbt-repository@823398c FirebirdSQL/fbt-repository@25cd595

Submitted by: @pcisar

FDB should be refactored to use abstract, higher level pythonic interface for communication with Firebird, and currect direct communication via Firebird Client Library should be reworked as default implementation of this interface.

This step will allow other implementations, notably direct communication using FB wire protocol or communication using new Firebird "object" API introduced in Firebird 3.0.

Submitted by: @pcisar

According to DB API 2.0 spec:

An Error <http://www.python.org/dev/peps/pep-0249/#error> (or subclass) exception is raised if the previous call to .execute*() <http://www.python.org/dev/peps/pep-0249/#id14> did not produce any result set or no call was issued yet.

Instead reporting correct error, FDB response to fetch on executed statement that doesn't return any result ends with memory access violation error.

Commits: 74637bc a336033 FirebirdSQL/fbt-repository@07c370d FirebirdSQL/fbt-repository@85ea220

Submitted by: Treeve Jelbert (treeve)

Assigned to: @pmakowski

isc_info_firebird_version fails with TypeError: 'an integer is required'

I tried using the python debugger and got the following data

$ ./test
>
/usr/lib/python2.7/site-packages/sqlalchemy/dialects/firebird/fdb.py(155)_get_server_version_in$
-> fbconn = connection.connection
(Pdb) c
> /usr/lib/python2.7/site-packages/fdb/fbcore.py(780)database_info()
-> if ord2(res_buf[i]) != isc_info_end:
(Pdb) p res_buf
'g\x80\x00\x03\x1cLI-V2.5.2.26455 Firebird 2.50LI-V2.5.2.26455 Firebird 2.5/tcp
(Gemini-64)/P120LI-V2.5.2.26455 Firebird 2.5/tcp
(Gemini-64)/P12\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00$
(Pdb) p i
131
(Pdb) s
> /usr/lib/python2.7/site-packages/fdb/fbcore.py(783)database_info()
-> if request_buffer[0] != res_buf[0]:
(Pdb) p request_buffer[0]
'g'
(Pdb) s
> /usr/lib/python2.7/site-packages/fdb/fbcore.py(785)database_info()
-> if result_type.upper() == 'I':
(Pdb) s
> /usr/lib/python2.7/site-packages/fdb/fbcore.py(787)database_info()
-> elif (result_type.upper() == 'S'
(Pdb)
> /usr/lib/python2.7/site-packages/fdb/fbcore.py(788)database_info()
-> and info_code not in _DATABASE_INFO__KNOWN_LOW_LEVEL_EXCEPTIONS):
(Pdb)
> /usr/lib/python2.7/site-packages/fdb/fbcore.py(792)database_info()
-> return ctypes.string_at(res_buf[3:], i - 3)
(Pdb)
--Call--
> /usr/lib/python2.7/ctypes/__init__.py(495)string_at()
-> def string_at(ptr, size=-1):
(Pdb)
> /usr/lib/python2.7/ctypes/__init__.py(499)string_at()
-> return _string_at(ptr, size)
(Pdb)
--Return--
> /usr/lib/python2.7/ctypes/__init__.py(499)string_at()->'\x03\x1cLI-V...emini-64)/P12'
-> return _string_at(ptr, size)
(Pdb)
--Return-->
/usr/lib/python2.7/site-packages/fdb/fbcore.py(792)database_info()->'\x03\x1cLI-V...emini-64)/P$
-> return ctypes.string_at(res_buf[3:], i - 3)
(Pdb)
> /usr/lib/python2.7/site-packages/fdb/fbcore.py(892)db_info()
-> pos = 1
(Pdb)
> /usr/lib/python2.7/site-packages/fdb/fbcore.py(894)db_info()
-> versionStringLen = (struct.unpack('B',
(Pdb)
> /usr/lib/python2.7/site-packages/fdb/fbcore.py(895)db_info()

(Pdb)
> /usr/lib/python2.7/site-packages/fdb/fbcore.py(895)db_info()
-> int2byte(buf[pos]))[0])
(Pdb) unt
--Return--
> /usr/lib/python2.7/site-packages/fdb/fbcore.py(895)db_info()->None
-> int2byte(buf[pos]))[0])
(Pdb)
TypeError: 'an integer is required'

Commits: 9bf1220

Submitted by: @pcisar

Assigned to: @pmakowski

Commits: 2a2aeda