cython / cython Goto Github PK

The most widely used Python to C compiler

License: Apache License 2.0

Python 50.07% C 9.02% C++ 0.38% Makefile 0.03% Smalltalk 0.01% PowerShell 0.05% Batchfile 0.02% sed 0.01% Starlark 0.04% Shell 0.09% Cython 40.30%

big-data c cpp cpython cpython-extensions cython performance python

cython's Introduction

Welcome to Cython!

Cython is a Python compiler that makes writing C extensions for Python as easy as Python itself. Cython is based on Pyrex, but supports more cutting edge functionality and optimizations.

Cython translates Python code to C/C++ code, but additionally supports calling C functions and declaring C types on variables and class attributes. This allows the compiler to generate very efficient C code from Cython code.

This makes Cython the ideal language for wrapping external C libraries, and for fast C modules that speed up the execution of Python code.

Official website: https://cython.org/
Documentation: https://docs.cython.org/
Github repository: https://github.com/cython/cython
Wiki: https://github.com/cython/cython/wiki

Cython has about 30 million downloads per month on PyPI. You can support the Cython project via Github Sponsors or Tidelift.

Installation:

If you already have a C compiler, just run following command:

pip install Cython

otherwise, see the installation page.

License:

The original Pyrex program was licensed "free of restrictions" (see below). Cython itself is licensed under the permissive Apache License.

See LICENSE.txt.

Contributing:

Want to contribute to the Cython project? Here is some help to get you started.

Differences to other Python compilers

Started as a project in the early 2000s, Cython has outlived most other attempts at producing static compilers for the Python language.

Similar projects that have a relevance today include:

PyPy, a Python implementation with a JIT compiler.
- Pros: JIT compilation with runtime optimisations, fully language compliant, good integration with external C/C++ code
- Cons: non-CPython runtime, relatively large resource usage of the runtime, limited compatibility with CPython extensions, non-obvious performance results
Numba, a Python extension that features a JIT compiler for a subset of the language, based on the LLVM compiler infrastructure (probably best known for its clang C compiler). It mostly targets numerical code that uses NumPy.
- Pros: JIT compilation with runtime optimisations
- Cons: limited language support, relatively large runtime dependency (LLVM), non-obvious performance results
Pythran, a static Python-to-C++ extension compiler for a subset of the language, mostly targeted at numerical computation. Pythran can be (and is probably best) used as an additional backend for NumPy code in Cython.
mypyc, a static Python-to-C extension compiler, based on the mypy static Python analyser. Like Cython's pure Python mode, mypyc can make use of PEP-484 type annotations to optimise code for static types.
- Pros: good support for language and PEP-484 typing, good type inference, reasonable performance gains
- Cons: no support for low-level optimisations and typing, opinionated Python type interpretation, reduced Python compatibility and introspection after compilation
Nuitka, a static Python-to-C extension compiler.
- Pros: highly language compliant, reasonable performance gains, support for static application linking (similar to cython_freeze but with the ability to bundle library dependencies into a self-contained executable)
- Cons: no support for low-level optimisations and typing

In comparison to the above, Cython provides

fast, efficient and highly compliant support for almost all Python language features, including dynamic features and introspection
full runtime compatibility with all still-in-use and future versions of CPython
"generate once, compile everywhere" C code generation that allows for reproducible performance results and testing
C compile time adaptation to the target platform and Python version
support for other C-API implementations, including PyPy and Pyston
seamless integration with C/C++ code
broad support for manual optimisation and tuning down to the C level
a large user base with thousands of libraries, packages and tools
almost two decades of bug fixing and static code optimisations

Get the full source history:

Note that Cython used to ship the full version control repository in its source distribution, but no longer does so due to space constraints. To get the full source history from a downloaded source archive, make sure you have git installed, then step into the base directory of the Cython source distribution and type:

make repo

The following is from Pyrex:

This is a development version of Pyrex, a language for writing Python extension modules.

For more info, take a look at:

Doc/About.html for a description of the language
INSTALL.txt for installation instructions
USAGE.txt for usage instructions
Demos for usage examples

Comments, suggestions, bug reports, etc. are most welcome!

The latest version of Pyrex can be found here.

Greg Ewing, Computer Science Dept
University of Canterbury
Christchurch, New Zealand

A citizen of NewZealandCorp, a wholly-owned subsidiary of USA Inc.

cython's People

Contributors

Stargazers

Watchers

Forkers

markflorisson certik vitek snaury jpeel dagss robertwb bzzzz dalejung qsnake wking rafekettler bhy pv francescalted jasongrout mistobaan elistevens njsmith jpe tkopczuk larsmans denik amcmorl scoder blackvladimir thewtex cournape matthagy bfroehle adamklein markpeek jramnani marklodato kif mikegraham vlasovskikh matthew-brett gorakhargosh macdentalr12 rp- jparyani andreasvc minrk dhirschfeld msabramo yarikoptic jakevdp echlebek mongi3 tshirtman 87 jamzo bertrandbordage grothesque pepin ldallolio samb javbaezga thespooler joonro exarkun carl4 dmj111 juliantaylor strohel barroque ahojnnes juntalis joshayers piti118 steinn hhsprings forivall-old-repos zyv johntyree jrray geggo alexis-mignon belevtsoff rgommers martin-magang ohanar vmx seberg inkymail gandalf013 anotherthomas tecki mindw glarrain sciumo bgrant xqat gvol felix-salfelder sturlamolden bluehorn appledarwin jpaalasm

cython's Issues

function pointers not accepted as function arguments

The following should work.

cdef int foo(int x):
    return x*x*x-x

cdef call_foo(int x, int (*f)(int)):
    return f(x)

print call_foo(3, &foo)

Migrated from http://trac.cython.org/ticket/31

cpdef memory leak

On Jul 7, 2008, at 8:37 PM, Guillaume Chereau wrote:

Hello all,

I am currently using cython for a project related to openmoko cell
phone : http://charlie137-2.blogspot.com/2008/07/introducing-tichy.html

I noticed that the memory is constantly increasing when I use cython. I
tracked down the problem to the case where I create a subclass of a
cython class and then redefine a cpdef method, asking it to call the
parent method, but ONLY if the cpdef method then call an other cpdef
method !

Here is the smallest example I could come with that fails :


== test.pyx ==

cdef class A:
    cpdef func(self):
        return

    cpdef test(self):
         self.func()


== main.py ==

import test
import gc

class B(test.A):
    def test(self):
        test.A.test(self)

b = B()

for i in range(10):
    b.test()
    gc.collect()
    print len(gc.get_objects())


The output will show that some objects are not released.

Is it a known bug ? Is there a way to avoid it ?

Migrated from http://trac.cython.org/ticket/24

Cython calls ExtType.init() as Python function

When subclasses of extension types call the __init__() method of their supertype, Cython generates code that looks up the "__init__" attribute of the instance and then calls it through Python using arg tuple/kwargs. This is because the special __init__() method ("tp_init" slot) uses this call signature.

Cython should recognise calls to this method and at least call it directly.

In the (presumably very common) case that the arguments do not use starargs, a more advanced approach would be to split the __init__() method into an internal plain C-function replacement and a tp_init wrapper, and then call the internal function directly, without doing any tuple packing etc. Not sure if it's worth it, though, as the call is already preceded by an (expensive) object allocation.

Migrated from http://trac.cython.org/ticket/3

Cython needs better support for API documentation

Currently, Python functions and methods in Cython generated C modules cannot provide their signature to interactive introspection. This is a major problem for the "help()" function in the interpreter (where function signatures look like "parse(...)"), but also to API doc generation tools like epydoc.

epydoc supports a special first line in the docstring here that mimics the look of the signature:

http://epydoc.sourceforge.net/api/epydoc.docstringparser-module.html#parse_function_signature
http://epydoc.sourceforge.net/api/epydoc.docstringparser-pysrc.html#parse_function_signature

Cython could generate such a line based on the original source signature.

Migrated from http://trac.cython.org/ticket/2

Error in default values

The following doesn't work:

class Bar(object):
    CONST = 25
    def foo(self, value=CONST):
        pass

Migrated from http://trac.cython.org/ticket/17

Several inplace operators not defined

The following produce error messages (regardless of a and n):

a <<= n
a >>= n
a **= n
a //= n

http://docs.python.org/lib/module-operator.html

This is because Pyrex only supported single and two-character operators in its lexicon. (Of course, his should change.)

Migrated from http://trac.cython.org/ticket/1

inspection of compilation namespace for IF/ELIF/ELSE/DEF

I tried to post this to cython-dev twice and got one awaiting
moderator approval and one silent no-op. So, I'll try it here.

It would be nice to have conditional compilation
directives that make multiple inclusion idempotent.
The usual guards in the C/C++ world would be:

#ifndef FOO_H
#define FOO_H
...
#endif

In Cython, I have been doing
DEF FOO_1 = 1
DEF FOO_2 = 1
IF FOO_1 == FOO_2:
DEF FOO_2 = 2
...

That seems kind of ugly, though. Beyond aesthetics,
it is probably nice in other contexts besides include
idempotency to test for a name before using it.

So, I propose a tiny addition to add a new compile-
time builtin called DEFINED() that just evaluates
to a boolean based on whether its string argument
exists in the compile-time environment.

This allows the above construct to be written both
more simply and more clearly:

IF not DEFINED("FOO"):
    DEF FOO = 1
    ...

Name choice follows the principle of least surprise
since the lowercase cpp name has similar functionality,
just like the other conditional compilation features.

Here is a very simple patch that implements this,
attached as well.

diff -ruw Cython-0.9.8.orig/Cython/Compiler/Scanning.py Cython-0.9.8/Cython/Compiler/Scanning.py
--- Cython-0.9.8.orig/Cython/Compiler/Scanning.py 2008-06-11 14:25:34.000000000 -0400
+++ Cython-0.9.8/Cython/Compiler/Scanning.py 2008-07-17 16:39:27.691882798 -0400
@@ -186,6 +186,9 @@
else:
raise

def defined(self, name):
```
   return name in self.entries
```
def initial_compile_time_env():
benv = CompileTimeScope()
names = ('UNAME_SYSNAME', 'UNAME_NODENAME', 'UNAME_RELEASE',
@@ -201,6 +204,7 @@
for name in names:
benv.declare(name, getattr(builtin, name))
denv = CompileTimeScope(benv)
denv.declare('DEFINED', denv.defined)
return denv

------------------------------------------------------------------

At 2008-07-19T15:44:09Z [email protected] added attachment defined.patch

Migrated from http://trac.cython.org/ticket/27

Casting is slow because of TypeCheck Call

For instance, consider:

 149:                 top = (<SymbolicPowArithmetic>x)._operands[       __pyx_1 = __Pyx_GetItemInt(((PyObject *)((struct __pyx_obj_4sage_9symbolics_9operators_10arithmetic_SymbolicPowArithmetic *)__pyx_v_x)->__pyx_base.__pyx_base.__pyx_base._operands), 1, 0); if (!__pyx_1) {__pyx_filename = __pyx_f[0](1]

); __pyx_lineno = 149; __pyx_clineno = __LINE__; goto __pyx_L1;}
        if (!(__Pyx_TypeTest(__pyx_1, __pyx_ptype_4sage_9symbolics_10expression_TypedSymbolicExpression))) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 149; __pyx_clineno = __LINE__; goto __pyx_L1;}
        Py_DECREF(((PyObject *)__pyx_v_top));
        __pyx_v_top = ((struct __pyx_obj_4sage_9symbolics_10expression_TypedSymbolicExpression *)__pyx_1);
        __pyx_1 = 0;

There needs to be a way to cast without the Py_TypeCheck call inside of the cast. I spend more time in sage-symbolics in TypeChecks then actually computing things. This is a critical problem, because in almost all cases I know that I can bypass this check.

Migrated from http://trac.cython.org/ticket/37

[patch] Fix for raising exceptions from iterator bug

See the attached test-case -- there was a bug related to raising exceptions from within an iterator.

At 2008-05-14T19:59:24Z @dagss added attachment iteratorexceptionbug.patch

Migrated from http://trac.cython.org/ticket/10

slicing string literals is broken

I added a test case "run/literalslicing.pyx" to show that slicing string literals is broken. Basically, when you do

py_result = "abc"[2]

Cython will generate C code that accesses the C byte string directly and retrieves the second char in it. Fast and simple, but this triggers two bugs.

The result is a C char, not a Python string. Converting it to a Python object will return an int instead of a single-character string.
Doing the same with a unicode string will access the (multi-byte) UTF-8 representation of the string and thus return the wrong byte if the string contains non-ASCII characters.

Migrated from http://trac.cython.org/ticket/46

no mangling of double underscore names in class

When defining a class attribute that starts with double underscores, CPython mangles the name from __bla to _classname_bla, but Cython produced classes don't.

Minimal example:

test.pyx and good.py are the same, like so:
class Test(object):
    __bla = 1

alon@alfajor:~/src/bugs/cython/doubleunderscore$ python
Python 2.5.2 (r252:60911, Feb 26 2008, 15:04:22)
[4.2.3 (Ubuntu 4.2.3-2ubuntu1)](GCC) on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import test
>>> test.Test
<class 'test.Test'>
>>> test.Test.__bla
1
>>>
alon@alfajor:~/src/bugs/cython/doubleunderscore$
alon@alfajor:~/src/bugs/cython/doubleunderscore$ ls
build setup.py test.c test.pyx test.so
alon@alfajor:~/src/bugs/cython/doubleunderscore$ cp test.pyx good.py
alon@alfajor:~/src/bugs/cython/doubleunderscore$ python
Python 2.5.2 (r252:60911, Feb 26 2008, 15:04:22)
[4.2.3 (Ubuntu 4.2.3-2ubuntu1)](GCC) on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import good
>>> good.Test
<class 'good.Test'>
>>> good.Test.__bla
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: type object 'Test' has no attribute '__bla'
>>> good.Test._Test__bla
1
>>>

Migrated from http://trac.cython.org/ticket/5

Cython .bat-script for windows

Starting Cython from Windows PowerShell with

cython.py <arguments>

yields a new console window with Cython output, which closes immedeately after Cython finished.
To get Cython's output one would always have to write something like

python C:\Python25\Scripts\cython.py <arguments>

Therefore, i wrote a simple batch file that does fairly the same as cython.py and allows you to start cython simply via

cython <arguments>

You can find it attached to this ticket. It would be great if such a script could become part of future Cython releases.

At 2008-07-24T16:52:18Z [email protected] added attachment cython.bat

Migrated from http://trac.cython.org/ticket/30

Type checking in method overriding broken for "ctypedef extern class" types

It seems if you have a method in a class with an argument typed to be
an "ndarray":

cdef class base:
    cdef void func(self, ndarray arr): pass

and you then try to override the method in a derived class, you get
errors. Things work fine if the type of the argument is an "int", or
if both base and derived class are in the same file.

I think this happens because "ndarray" is not a cdef class kind of a
type, but comes in from an outside library -- numpy -- as a:

ctypedef extern class numpy.ndarray [PyArrayObject](object):

Here is what happens when you run:

python cython.py base.pyx
gcc -c -fPIC -O6 -fno-strict-aliasing -I/usr/include/python2.3 base.c
gcc -shared -lm base.o -o base.so
python cython.py main.pyx

Error converting Pyrex file to C:
------------------------------------------------------------
...

cdef class item(base.base):

    #cdef void method(me, int arr): pass

    cdef void method(me, ndarray arr): pass     ^
------------------------------------------------------------

/home/dg/w/pex/Tickets/01_ndarray_as_overridden_arg/main.pyx:8:6: Signature not compatible with previous declaration

Error converting Pyrex file to C:
------------------------------------------------------------
...
include "numpy.pxi"

cdef class base:

    #cdef void method(me, int arr)
    cdef void method(me, ndarray arr)                ^
------------------------------------------------------------

/home/dg/w/pex/Tickets/01_ndarray_as_overridden_arg/base.pxd:7:17: Previous declaration is here

At 2008-06-19T14:56:17Z @anonymous added attachment files.zip

Migrated from http://trac.cython.org/ticket/20

req: support cdef class attrib initialisation

A common Python programming style is for classes to be configured via initialised class attributes, which allows users to create subclasses with different initial values, for example:

class Foo:
   bar = 5
   baz = "someval"

class MyFoo(Foo):
   bar = 7
   baz = "anotherval"

However, this does not presently work with Cython/Pyrex.

This filing is a request for support of initialised cdef class attributes, eg:

cdef class Foo:
    cdef public int bar = 5
    cdef public object baz = "someval"
...
cdef class MyFoo(Foo):
    cdef public int bar = 7
    cdef public object baz = "anotherval"
...
class AnotherFoo(Foo):
    bar = 8
    baz = "yetanotherval"

In this example, we subclass Foo twice - once with an extension subclass containing initialised cdef class attributes, and the other as a pure-python subclass containing initialised class attributes in pure-python syntax.

Ideally, both idioms would be supported. In the former case, the int value 7 would be written to the attribute 'bar' prior to init, and in the latter case, the int object 8 would be converted to a C integer then written to the attribute bar.

The advantage of supporting these cdef class attribute initialisations is that it would allow users to stick with the python idiom of 'initialised class attributes as configuration option', while allowing the speed advantage of struct-level attribute access (as opposed to the slower getattr at the pure-python level).

Migrated from http://trac.cython.org/ticket/18

Cython annotation breaks on certain pointer casts

When compiling the following text with '''cython -a''', I get a UnicodeDecodeError:

cdef extern from "Python.h":
    ctypedef int Py_intptr_t

print '  stride %d:' % <Py_intptr_t>3

Traceback (most recent call last):
  File "/Users/stefan/bin/cython", line 8, in <module>
    main(command_line = 1)
  File "/Users/stefan/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 342, in main
    result = context.compile(source, options)
  File "/Users/stefan/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 217, in compile
    tree.process_implementation(scope, options, result)
  File "/Users/stefan/lib/python2.5/site-packages/Cython/Compiler/ModuleNode.py", line 51, in process_implementation
    self.generate_c_code(env, options, result)
  File "/Users/stefan/lib/python2.5/site-packages/Cython/Compiler/ModuleNode.py", line 246, in generate_c_code
    code.save_annotation(result.c_file[+ "pyx") # change?
  File "/Users/stefan/lib/python2.5/site-packages/Cython/Compiler/Annotate.py", line 78, in save_annotation
    lines[line_no](:-1]) = line[+ item + line[col:](:col])
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf0 in position 0: ordinal not in range(128)

I'm running 726:9527edc22cb6 from Jun 13 20:50:39 2008 +0200.

Migrated from http://trac.cython.org/ticket/19

Bug accessing internals of Python long

I am trying to access the internal structure of a Python long. The code
fragment below works when I use Pyrex 0.9.8.4 but generates a compile error
when using Cython 0.9.8.

The code:

cdef extern from "Python.h":
ctypedef struct PyTypeObject:
pass
ctypedef struct PyObject:
Py_ssize_t ob_refcnt
PyTypeObject * ob_type
cdef extern from "longintrepr.h":
cdef struct _longobject:
int ob_refcnt
PyTypeObject * ob_type
int ob_size
unsigned int * ob_digit
def test(temp = long(0)):
cdef _longobject *l
l = <_longobject *> temp
print sizeof(l.ob_size)
print sizeof(l.ob_digit[0])

Migrated from http://trac.cython.org/ticket/41

Cython long and long long integer suffixes

In Cython, 100L has different meaning than in C. Also, 100LL is not accepted. This should probably be changed (resturning an actaul long object when coerced to Python, but outputing the L when used in C.

To force an object, one can now type <object>100.

Migrated from http://trac.cython.org/ticket/33

Missing opening parenthesis in code generation

Hi,

There is a missing parenthesis in module level code generation. Here is a trivial patch to fix compilation under 2.6

At 2008-08-15T14:24:47Z [email protected] added attachment fix_parenthesis.patch

Migrated from http://trac.cython.org/ticket/49

C++ compile error when exception part declared on function

The following results in gcc 4.01 reporting "error: invalid conversion from ‘const char_’ to ‘char_’" when using Cython 0.9.8

test.pyx:

cdef extern from "test.h":
  ctypedef struct c_Test "Test":
    char *getString() except +RuntimeError
                     ## All OK if no 'except +RuntimeError'
cdef class Test:
  cdef c_Test *thisptr
  def getString(self):
    return self.thisptr.getString()

test.h:

static const char *astring = "123" ;
class Test {
 public:
  const char *getString(void) { return astring ; }
  } ;

setup.py:

from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext

setup(
  name = 'biosigpy',
  ext_modules = [   Extension('test', ['test.pyx'](
),
      language='c++',
      extra_link_args=[     )
    ]("-g"],
),
  cmdclass = {'build_ext': build_ext}
  )

Migrated from http://trac.cython.org/ticket/42

The following code causes mysterious exceptions

Consider:

cdef PyObject* fast_tp_alloc(RichPyTypeObject *t, Py_ssize_t):

This gives the following traceback

Traceback (most recent call last):
  File "/home/gfurnish/sage-3.0.6/local/bin/cython", line 8, in <module>
    main(command_line = 1)
  File "/home/gfurnish/sage-3.0.6/local/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 527, in main
    result = compile(sources, options)
  File "/home/gfurnish/sage-3.0.6/local/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 505, in compile
    return compile_multiple(source, options)
  File "/home/gfurnish/sage-3.0.6/local/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 472, in compile_multiple
    result = context.compile(source, options)
  File "/home/gfurnish/sage-3.0.6/local/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 327, in compile
    tree.process_implementation(scope, options, result)
  File "/home/gfurnish/sage-3.0.6/local/lib/python2.5/site-packages/Cython/Compiler/ModuleNode.py", line 48, in process_implementation
    self.analyse_declarations(env)
  File "/home/gfurnish/sage-3.0.6/local/lib/python2.5/site-packages/Cython/Compiler/ModuleNode.py", line 45, in analyse_declarations
    self.body.analyse_declarations(env)
  File "/home/gfurnish/sage-3.0.6/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 242, in analyse_declarations
    stat.analyse_declarations(env)
  File "/home/gfurnish/sage-3.0.6/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 986, in analyse_declarations
    name_declarator, type = self.declarator.analyse(base_type, env, self.body is not None)
  File "/home/gfurnish/sage-3.0.6/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 382, in analyse
    return self.base.analyse(ptr_type, env, nonempty = nonempty)
  File "/home/gfurnish/sage-3.0.6/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 434, in analyse
    name_declarator, type = arg_node.analyse(env, nonempty = nonempty)
  File "/home/gfurnish/sage-3.0.6/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 531, in analyse
    return self.declarator.analyse(base_type, env, nonempty = nonempty)
  File "/home/gfurnish/sage-3.0.6/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 344, in analyse
    self.name = base_type.name
AttributeError: CPySSizeTType instance has no attribute 'name'

This should give a more user friendly error.

Migrated from http://trac.cython.org/ticket/35

annotation broken in dagss branch?

Migrated from http://trac.cython.org/ticket/47

from package cimport module doesn't work

Is this form of cimport:

from <package_dir> cimport

officially not supported? It gives "'package.pxd' not found" error.

You can however do

cimport <package_dir>. as

which comes to the same thing...

Migrated from http://trac.cython.org/ticket/4

Cython doesn't catch double declaration with args

On Jun 19, 2008, at 2:15 PM, Jonathan Bober wrote:

Thus bug is different, in that I am definitely doing something wrong. I
suppose that cython should catch it before gcc does, though.

bober@bober:~/cython_test$ cython bug2.pyx
bober@bober:~/cython_test$ gcc -c bug2.c -I/home/bober/sage/local/include/python2.5
bug2.c: In function ‘__pyx_pf_4bug2_foo’:
bug2.c:114: error: redeclaration of ‘__pyx_v_foo2’ with no linkage
bug2.c:113: error: previous declaration of ‘__pyx_v_foo2’ was here
bober@bober:~/cython_test$

At 2008-07-16T08:34:38Z @robertwb added attachment bug2.pyx

Migrated from http://trac.cython.org/ticket/26

bad declarations do not produce warning

The following code does not produce any errors, which it should because of the double declaration of different types.

        cdef int i, j, oplen
        cdef Py_ssize_t i, j, k

Migrated from http://trac.cython.org/ticket/44

Problems with automatic pxd inclusion

Joost Cassee wrote:

Cython (0.9.6.13.1) allow pyx files to be placed in the package directory. When I do so, the accompanying pxd file is not found. An strace of cython shows it uses the package directory twice in building the path:

~/src$ strace cython package/module.pyx
...
stat64("/home/user/src/package/package/module.pxd", 0xbf8b1f48) = -1 ENOENT (No such file or directory)
...

On the other hand, if I name the module as in python I get a cython error:

~/src$ cython package.module.pyx
...
IOError: [2](Errno) No such file or directory: '/home/user/src/package.module.pyx'

On the other hand, an strace shows that the pxd file was found. :-)

Migrated from http://trac.cython.org/ticket/7

cython generates horrible bint comparison code

Consider:

cdef bint foo
if foo == True:

This generates a raw rich compare

cdef bint foo
if foo==1:

This generates a C if statement.
This minor problem causes a massive performance loss in inner loops.

Migrated from http://trac.cython.org/ticket/34

cdef public attributes don't work for extension types

On Jul 11, 2008, at 11:50 PM, Joshua Kantor wrote:

cdef readonly actually works with the numpy arrays,
cdef public does not however. In principle it should be possible to assign a to it, but you of course could only assign a numpy array.

There is code to do c-type conversion on assignment, it shouldn't be hard to do type checking too

Migrated from http://trac.cython.org/ticket/23

Bad code generation

 147:             x = <TypedSymbolicExpression>PyList_GET_ITEM(operands, i)

      Py_INCREF(((PyObject *)((struct __pyx_obj_4sage_9symbolics_10expression_TypedSymbolicExpression *)PyList_GET_ITEM(__pyx_v_operands, __pyx_v_i))));
      Py_DECREF(((PyObject *)__pyx_v_x));
      __pyx_v_x = ((struct __pyx_obj_4sage_9symbolics_10expression_TypedSymbolicExpression *)PyList_GET_ITEM(__pyx_v_operands, __pyx_v_i));

This should use a temporary instead of calling the thing being casted twice.

Migrated from http://trac.cython.org/ticket/36

Potential problems for extern cdefs in argument parsing

Consider this code:

cdef extern from "t.h":
     ctypedef long hullo

def foo(hullo h): pass

I noticed that it creates code like this:
PyArg_ParseTupleAndKeywords(__pyx_args, __pyx_kwds, "l", __pyx_argnames, 
&__pyx_v_h))

Which could probably destroy the stack if we don't have the exact 
typedef, right? Which means: One should either write our own parsing 
code instead (generate for each parameter signature), or generate the 
typestring at runtime, or insert C checks for the sizeof(the typedefs). 
The latter will destroy the current approach for numpy.int64 and friends 
and remove what I consider a nice feature.

Just as I took all that care to allow approximate typedefs for buffers :-)

Just noting it down, I suppose for now we'll live with it. But need to 
be more careful about recommending approximate typedefs.

-- 
Dag Sverre

See thread at http://codespeak.net/pipermail/cython-dev/2008-July/001760.html

Migrated from http://trac.cython.org/ticket/28

Emit code to guard declarations for this module's public functions

This is a one-line patch:

Emit code to guard declarations for this module's public functions.

It appears that the machinery is all there to protect the declaration
of inconsistent linkage types (i.e. dllimport vs. dllexport) by using
a guard unique to the pyx file. This patch simply emits that guard
for the current module to prevent the error "inconsistent linkage types"
on a Windows-based system.

At 2008-05-24T01:01:48Z JimKleckner added attachment guarddll.hg

Migrated from http://trac.cython.org/ticket/14

future feature division is not defined

% echo "from future import division" > foo.pyx
% cython foo.pyx

Error converting Pyrex file to C:

...
from future import division

^

/home/zaphod/shiny/glint/foo.pyx:1:31: future feature division is not defined

Cython 0.9.8 reports the above error. Cython 0.9.6.14 compiles the module successfully.

Migrated from http://trac.cython.org/ticket/22

Mysterious error with conflicting types

If you declare, for example, _operands to be a list, and cdef _operands(self) to be a function in the same class, mysterious errors can be produced such as:

Traceback (most recent call last):
  File "/home/gfurnish/sage-3.0.6/local/bin/cython", line 8, in <module>
    main(command_line = 1)
  File "/home/gfurnish/sage-3.0.6/local/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 527, in main
    result = compile(sources, options)
  File "/home/gfurnish/sage-3.0.6/local/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 505, in compile
    return compile_multiple(source, options)
  File "/home/gfurnish/sage-3.0.6/local/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 472, in compile_multiple
    result = context.compile(source, options)
  File "/home/gfurnish/sage-3.0.6/local/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 327, in compile
    tree.process_implementation(scope, options, result)
  File "/home/gfurnish/sage-3.0.6/local/lib/python2.5/site-packages/Cython/Compiler/ModuleNode.py", line 59, in process_implementation
    self.generate_c_code(env, options, result)
  File "/home/gfurnish/sage-3.0.6/local/lib/python2.5/site-packages/Cython/Compiler/ModuleNode.py", line 243, in generate_c_code
    self.body.generate_function_definitions(env, code, options.transforms)
  File "/home/gfurnish/sage-3.0.6/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 252, in generate_function_definitions
    stat.generate_function_definitions(env, code, transforms)
  File "/home/gfurnish/sage-3.0.6/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 2051, in generate_function_definitions
    self.entry.type.scope, code, transforms)
  File "/home/gfurnish/sage-3.0.6/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 252, in generate_function_definitions
    stat.generate_function_definitions(env, code, transforms)
  File "/home/gfurnish/sage-3.0.6/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 892, in generate_function_definitions
    exc_check = self.caller_will_check_exceptions()
  File "/home/gfurnish/sage-3.0.6/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 1162, in caller_will_check_exceptions
    return self.entry.type.exception_check
AttributeError: BuiltinObjectType instance has no attribute 'exception_check'

Migrated from http://trac.cython.org/ticket/43

Better dependancy tracking

Pyrex implemented this here

http://hg.cython.org/pyrex/rev/9a5d15e9acd3

Unfortunately the attached patch (modified a bit for Cython) doesn't apply cleanly.

At 2008-05-28T06:34:54Z @robertwb added attachment dependancies.patch

Migrated from http://trac.cython.org/ticket/15

'make all' to build Scanners.so

At 2008-07-23T09:13:55Z [email protected] added attachment make-all.patch

Migrated from http://trac.cython.org/ticket/29

Bad list constructor code

Consider:

 405:     _operands = list()

  __pyx_1 = PyObject_Call(((PyObject*)&PyList_Type), ((PyObject *)__pyx_empty_tuple), NULL); if (unlikely(!__pyx_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 405; __pyx_clineno = __LINE__; goto __pyx_L1;}
  Py_DECREF(__pyx_v__operands);
  __pyx_v__operands = __pyx_1;
  __pyx_1 = 0;

This is wildly inefficient compared to a single PyList_New(0) call.

Migrated from http://trac.cython.org/ticket/39

__stdcall define wrong for Windows platform

The previous patch that fixed the redefinition of !__stdcall on Cygwin caused
the Windows platform to fail to recognize the calling convention.

The attached patch adds a check for MS_WINDOWS as well before !__stdcall is turned off.

At 2008-05-23T00:42:15Z JimKleckner added attachment stdcall.hg

At 2008-07-25T17:23:34Z JimKleckner added attachment windows.jek.patch

Migrated from http://trac.cython.org/ticket/12

Non-optimal code generation for equality test

Consider:

 151:                 if top._expression_type==enum_SymbolicConstant and PyObject_TypeCheck((<SymbolicConstant>top)._obj,Integer) and (<SymbolicConstant>top)._obj<0:

        __pyx_1 = __Pyx_PyBool_FromLong((__pyx_v_top->_expression_type == __pyx_e_4sage_9symbolics_10expression_enum_SymbolicConstant)); if (unlikely(!__pyx_1)) {__pyx_filename = __pyx_f[__pyx_lineno = 151; __pyx_clineno = __LINE__; goto __pyx_L1;}
        __pyx_2 = __Pyx_PyObject_IsTrue(__pyx_1); if (unlikely(__pyx_2 < 0)) {__pyx_filename = __pyx_f[0](0];); __pyx_lineno = 151; __pyx_clineno = __LINE__; goto __pyx_L1;}
        if (__pyx_2) {

Note it is converting the result of a C boolean compare to a Python boolean object, then converting it back to a C boolean.

Migrated from http://trac.cython.org/ticket/38

Generate shorter string literals

This small patch split docstrings around ¥n when put into C code, because some compilers barf at long string literals (VS 2003, for once).

At 2008-08-15T14:57:33Z @anonymous added attachment donotuselongstring.patch

Migrated from http://trac.cython.org/ticket/50

bug in char* declaration


Attached is a simple file that causes the bug. I compile with 

cython test.pyx

and

gcc -c test.c -I/home/bober/local/sage/include -I/home/bober/local/include/csage -I/home/bober/sage/local/include/python2.5

and get the error:

test.c: In function ‘inittest’:
test.c:285: error: ‘a’ undeclared (first use in this function)
test.c:285: error: (Each undeclared identifier is reported only once
test.c:285: error: for each function it appears in.)

At 2008-07-16T08:33:14Z @robertwb added attachment test.pyx

Migrated from http://trac.cython.org/ticket/25

mcubnte

For example

__pyx_1 = (__pyx_skip_dispatch = 1, ((struct
__pyx_vtabstruct_4ship_7icewing_7plugins_PluginInstance *)((struct
__pyx_obj_4ship_7icewing_7plugins_PluginInstance
*)__pyx_v_self)->__pyx_vtab)->init(((struct
__pyx_obj_4ship_7icewing_7plugins_PluginInstance *)__pyx_v_self), ((void
*)&((struct
__pyx_opt_args_4ship_7icewing_7plugins_14PluginInstance_init){1,__pyx_v_args}))));

if (unlikely(!__pyx_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 93;
goto __pyx_L1;}

gives

build/temp.linux-x86_64-2.4/pyrex/plugins.c: In function 'PyObject*
__pyx_pf_4ship_7icewing_7plugins_14PluginInstance_init(PyObject*,
PyObject*, PyObject*)':
build/temp.linux-x86_64-2.4/pyrex/plugins.c:1228: error: invalid lvalue
in unary '&'
error: command 'x86_64-pc-linux-gnu-g++' failed with exit status 1

Migrated from http://trac.cython.org/ticket/21

patch: RFC: constify Cython output all over the place (newbie approach)

You know, when developing code, it is very tedious to look for meaningful
errors and warnings in presence of tons of noise like

warning: deprecated conversion from string constant to â€˜char*â€™

And you know what? It seems in the next version of gcc, this deprecation
warnings will be turned into errors.

Python sources already use 'const' keyword freely, so I think it's time to add
constify bits all over the place.

At 2008-05-14T17:18:34Z [email protected] added attachment constify.patch

Migrated from http://trac.cython.org/ticket/9

Inefficient comparison code generation

Consider the following code:

                if left._name==(<SymbolicVariable_class>right)._name:

Which generates:

     __pyx_2 = PyObject_RichCompare(__pyx_v_left->__pyx_base._name, ((struct _\
_pyx_obj_4sage_9symbolics_14variable_class_SymbolicVariable_class *)__pyx_v_rig\
ht)->__pyx_base._name, Py_EQ); if (unlikely(!__pyx_2)) {__pyx_filename = __pyx_\
f[__pyx_lineno = 126; __pyx_clineno = __LINE__; goto __pyx_L1;}
      __pyx_1 = __Pyx_PyObject_IsTrue(__pyx_2); if (unlikely(__pyx_1 < 0)) {__p\
yx_filename = __pyx_f[0](0];); __pyx_lineno = 126; __pyx_clineno = __LINE__; goto __\
pyx_L1;}

Is there a reason we don't use the

int PyObject_RichCompareBool(   PyObject *o1, PyObject *o2, int opid)

functionality in python to avoid the extra python boolean step?

Migrated from http://trac.cython.org/ticket/45

Better error for casts

x = <foo>None
       ^
------------------------------------------------------------

/Users/robert/sage/cython/cython-0.9.8.1/Demos/test.pyx:2:8: Declarator should be empty

Migrated from http://trac.cython.org/ticket/48

Minor patch set for Python 2.6a3 on Windows VS2008

The attached patch set should be reviewed for inclusion.

Note particularly the change to belchenko1 and comment whether it is appropriate.

At 2008-05-23T21:50:12Z JimKleckner added attachment py26.hg

Migrated from http://trac.cython.org/ticket/13

Annotate mode disables inline source comments

When I turned on the annotate mode for sage, the inline comments that show the cython code are no longer embedded in the C file. This makes it really hard to use the C file for debugging (with gdb) while performance tuning (with the annotated code). Is it possible to not have these two modes disable each other?

Migrated from http://trac.cython.org/ticket/40

IF clauses inside definitions

Normally, IF only works in the global scope.
Defining conditional members of a struct necessitates redefining the whole structure:

IF UNAME_SYSNAME == "Windows":
      ctypedef struct ohmygod:
            int common
            int greece
ELSE:
      ctypedef struct ohmygod:
            int common
            float pelvis

This would be more convenient, especially for big structures:

ctypedef struct ohmygod:
     int common
     IF UNAME_SYSNAME == "Windows":
                 int greece
     ELSE:
                 float pelvis

Migrated from http://trac.cython.org/ticket/16

gcc warns of unused variables in module initialization

I create foo.pyx containing either or both of the following statements:

  cdef double INFINITY = float('inf')
  assert True

I build it using Cython 0.9.6.13.1 and gcc 4.1.3. The build succeeds and the code works, but gcc issues the following warning:

  foo.c: In function ‘initfoo’:
  foo.c:130: warning: unused variable ‘__pyx_4’
  foo.c:129: warning: unused variable ‘__pyx_3’

I see no such warnings when building the same code under Cython 0.9.6.12.

Migrated from http://trac.cython.org/ticket/6

DEF X = -1 raises TypeError in ExprNodes.py

The fix for bug http://trac.cython.org/ticket/220108 introduces a problem when defining a negative
integer:

DEX X = -1

...
File "/usr/lib/python2.5/site-packages/Cython-0.9.6.14-py2.5.egg/
Cython/Compiler/Parsing.py", line 1267, in p_DEF_statement
value = expr.compile_time_value(denv)
File "/usr/lib/python2.5/site-packages/Cython-0.9.6.14-py2.5.egg/
Cython/Compiler/ExprNodes.py", line 674, in compile_time_value
return int(self.value, 0)
TypeError: int() can't convert non-string with explicit base

Cython version 0.9.6.13 worked fine.

Migrated from http://trac.cython.org/ticket/8

[patch] Transform utilities

It should now be possible to do stuff like:


class WithTransform(VisitorTransform):
     # from with transform PEP...
     with_fragment = TreeFragment(u"""
_mgr = (EXPR)
_exit = mgr.__exit__
_value = mgr.__enter__()
_exc = True
try:
     try:
         VAR = _value
         BLOCK
...
<snip>
...
""")

     def process_WithStatementNode(self, node):
         return self.with_fragment.substitute({
             "EXPR" : node.expr,
             "VAR" : node.var,
             "BLOCK" : node.body
         })

:-)

(The above is simplified, there's not always a VAR. Also it needs
another feature before it can be completely streamlined (automatic
"temporaries" that won't clash in the namespace; basically,
"with_fragment.substitute(..., temps=("_mgr", ...)). When that is done,
supporting the with statement is about as much work as extending the
parser, the transform/implementation comes for free.

I've already discussed the CodeWriter. It only supports a limited
subset (with some holes, ~30% perhaps) at this time; but it's what I
need for now (for unit tests). I might work further on that too.
Some changes to Transform.py which I hope goes through... there's a
Visitor object there; using the "process_ClassName" pattern (I think
that was the conclusion for future performance reasons).
A clone_node method on Node for proper node copying (shallow object
copy except child node lists, which are also copied).
Here's the controversial bit:

In order to be able to provide proper error messages for string-based
code snippets like the above (which are passed to Parsing.py...); I've
changed the pointer to the source code (used as the first element in the
position tuples found everywhere...) from being a string filename to
being a SourceDescriptor object.

A SourceDescriptor can currently be a FileSourceDescriptor, in which
case things work like before (it gives the filename on str so much
code needed not change), or a StringSourceDestriptor which I use for my
new code...

I hope you see the advantages to this from the above code. (There are
less intrusive ways to do this, but they would only be hacky and
postpone the problem. Better do it properly...? BTW this pattern is
rather common, consider for instance Source in the XML Transform APIs/TrAX.)

At 2008-05-16T16:42:18Z @dagss added attachment dagss-branch-changes.diff

At 2008-05-16T16:45:26Z @dagss added attachment dagss-branch.bundle

Migrated from http://trac.cython.org/ticket/11

unsigned integer suffix not parsed

In C one can type integers such as 123U to designate integer constants as unsigned. One should be able to do in Cython as well.

Migrated from http://trac.cython.org/ticket/32

cython / cython Goto Github PK

cython's Introduction

Welcome to Cython!

Installation:

License:

Contributing:

Differences to other Python compilers

Get the full source history:

The following is from Pyrex:

cython's People

Contributors

Stargazers

Watchers

Forkers

cython's Issues

------------------------------------------------------------------

Error converting Pyrex file to C:

^

Recommend Projects

Recommend Topics

Recommend Org