pearu / f2py Goto Github PK

A couple of minor problems prevented parsing of SciPy's .pyf files [1]. 
Attaching the patch I'm using to fix it, with it it works like a charm.

If I can get a word that this is correct I can push it myself.

Ignore this message.

[emason@marula r_tools]$ f2py -verbose  -c -m --fcompiler=intelem *.F      
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
build_src
building extension "rfact.F" sources
f2py options: []
f2py:> /tmp/tmp_QlXua/src.linux-x86_64-2.7/rfact/Fmodule.c
creating /tmp/tmp_QlXua/src.linux-x86_64-2.7
creating /tmp/tmp_QlXua/src.linux-x86_64-2.7/rfact
Reading fortran codes...
        Reading file 'roms_read_write.F' (format:fix,strict)
Traceback (most recent call last):
  File "/usr/bin/f2py", line 28, in <module>
    main()
  File "/usr/lib64/python2.7/site-packages/numpy/f2py/f2py2e.py", line 648, in main
    run_compile()
  File "/usr/lib64/python2.7/site-packages/numpy/f2py/f2py2e.py", line 633, in run_compile
    setup(ext_modules=[ext])
  File "/usr/lib64/python2.7/site-packages/numpy/distutils/core.py", line 169, in setup
    return old_setup(**new_attr)
  File "/usr/lib64/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/usr/lib64/python2.7/site-packages/numpy/distutils/command/build.py", line 47, in run
    old_build.run(self)
  File "/usr/lib64/python2.7/distutils/command/build.py", line 127, in run
    self.run_command(cmd_name)
  File "/usr/lib64/python2.7/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/usr/lib64/python2.7/site-packages/numpy/distutils/command/build_src.py", line 147, in run
    self.build_sources()
  File "/usr/lib64/python2.7/site-packages/numpy/distutils/command/build_src.py", line 164, in build_sources
    self.build_extension_sources(ext)
  File "/usr/lib64/python2.7/site-packages/numpy/distutils/command/build_src.py", line 326, in build_extension_sources
    sources = self.f2py_sources(sources, ext)
  File "/usr/lib64/python2.7/site-packages/numpy/distutils/command/build_src.py", line 563, in f2py_sources
    ['-m', ext_name]+f_sources)
  File "/usr/lib64/python2.7/site-packages/numpy/f2py/f2py2e.py", line 408, in run_main
    postlist = callcrackfortran(files, options)
  File "/usr/lib64/python2.7/site-packages/numpy/f2py/f2py2e.py", line 329, in callcrackfortran
    postlist = crackfortran.crackfortran(files)
  File "/usr/lib64/python2.7/site-packages/numpy/f2py/crackfortran.py", line 3216, in crackfortran
    readfortrancode(files, crackline)
  File "/usr/lib64/python2.7/site-packages/numpy/f2py/crackfortran.py", line 511, in readfortrancode
    dowithline(finalline)
  File "/usr/lib64/python2.7/site-packages/numpy/f2py/crackfortran.py", line 748, in crackline
    % (groupcounter))
Exception: crackline: groupcounter(=0) is nonpositive. Check the blocks.
[emason@marula r_tools]$

This patch ensures that the parser looks for ForAllStmt before ForAllConstruct, 
since after fixing the typo reported in Issue 34, forall constructs match the 
regular expression used for finding forall statements, but then cause an 
AnalyzeError since for a forall statement there is not matching EndForall.

diff -r 732d6b540026 -r be4de39a1147 fparser/block_statements.py
--- a/fparser/block_statements.py       Fri Jan 27 14:29:58 2012 +0000
+++ b/fparser/block_statements.py       Fri Jan 27 14:31:54 2012 +0000
@@ -1249,8 +1249,8 @@
 # GeneralAssignment = Assignment + PointerAssignment
 # EndFunction, EndProgram, EndSubroutine - part of the corresponding blocks

-executable_construct = [ Associate, Do, ForallConstruct, IfThen,
-    Select, WhereConstruct ] + action_stmt
+executable_construct = ([ Associate, Do, IfThen, Select, WhereConstruct ] +
+                        action_stmt + [ ForallConstruct ])
 #Case, see Select

 execution_part_construct = executable_construct + [ Format, Entry,

This patch walks up the parse tree from a Variable to find the containing 
module, rather than assuming all Variables are two levels below the module, 
which breaks for module-level variables.

diff -r c5b88e996b11 -r 732d6b540026 fparser/base_classes.py
--- a/fparser/base_classes.py   Fri Jan 27 14:26:10 2012 +0000
+++ b/fparser/base_classes.py   Fri Jan 27 14:29:58 2012 +0000
@@ -282,7 +282,11 @@
     def is_private(self):
         if 'PUBLIC' in self.attributes: return False
         if 'PRIVATE' in self.attributes: return True
-        return self.parent.parent.check_private(self.name)
+
+        mod = self.parent
+        while not hasattr(mod, 'check_private'):
+            mod = mod.parent
+        return mod.check_private(self.name)
     def is_public(self): return not self.is_private()

     def is_allocatable(self): return 'ALLOCATABLE' in self.attributes

Hello,

I am able to generate, but not to use the DLLs generated by f2py, in Fortran 90.

Here is my system:
- Windows Vista Business 64 bits
- Python 2.6.5
- numpy 1.5.1rc1
- scipy 0.8.0
- MinGW + gfortran

So to begin with, I wrote a hello.f Fortran file (http://www.scipy.org/F2py). 
Then I compiled it into a python module using the following command:

f2py.py -c --fcompiler=gnu95 --compiler=mingw32 -lmsvcr71 -m hello hello.f

Up to here, everything's alright. However, when I try "import hello" from 
Python, then I get the following error:

Traceback (most recent call last):
  File "<console>", line 1, in <module>
ImportError: DLL load failed: %1 is not a valid Win32 application

On the other hand, everything works fine if I use the gnu compiler (not gnu95) 
without the -lmsvcr71 option:

f2py.py -c --fcompiler=gnu --compiler=mingw32 -m hello hello.f

The problem is that I need to wrap plenty of Fortran 90 code, not Fortran 77...

Help please!!

Thanks in advance

Bruno

Hi,
A possible application of the fparser is 
as a basic library to re-write fortran codes. 
For example, it will be possible to
add intent to all variables, or we can insert
some comments at the head of each subroutines; the comments
explains used variables and so on in some format.

Or the re-written fortran code can be some hyper-text or document by
mark-up language.

In anyway, I think a mechanism to preserve original documents
in a simple manner is useful. 
Then we can control such re-writer programs.

What steps will reproduce the problem?

Output is compiler dependent, ifort works, gfortran produces garbage
in case with external function (test2.f90). 

1) Intel Fortran Compiler ifort - linux

f2py -c --fcompiler=intelem  -m ffct ffct.f90 

f2py -c --fcompiler=intelem  -m test1 ffct.f90 test1.f90 

f2py -c --fcompiler=intelem  -m test2 ffct.f90 test2.f90

python test.py

Output:
(-8.23547535068e-06-1.08608608614e-05j)
(-8.23547535068e-06-1.08608608614e-05j)

2) gfortran - linux

f2py -c --fcompiler=gnu95 -m ffct ffct.f90 

f2py -c --fcompiler=gnu95  -m test1 ffct.f90 test1.f90 

f2py -c --fcompiler=gnu95  -m test2 ffct.f90 test2.f90

python test.py

Output:
(-8.23547535068e-06-1.08608608614e-05j)
(-0.000624818723635+2.1849065056e-319j) <- WRONG

What is the expected output? What do you see instead?

Described above. 

What version of the product are you using? On what operating system?

Version:     2
numpy Version: 1.8.2
Debian

Please provide any additional information below.

I put
-------test.F------------
      subroutine bndfp()
      include "events.ins"
      logical mlog
      end
------------------------
and
----- events.ins------
      integer i
----------------------
in the same directory, and then do parser. Then it caused ERROR.

Withoug events.ins, only WARNING (as I asked to you; so no problem).

Hi Pearu,

When the analyse() function of the Use statement is called it will try to find 
the file of the module and call the parser to parse and analyse this module. 
However, i noticed that the ignore_comments parameter to the constructor of 
FortranParser is not set. So if the user is parsing with ignore_comments=False, 
then this is not carried through to the parser instances called within the 
cascade of analyse() calls.

Here's my proposed fix:

diff -r ec5eb01a4e3e fparser/statements.py
--- a/fparser/statements.py Fri Sep 03 20:20:51 2010 +0300
+++ b/fparser/statements.py Mon Sep 06 22:36:52 2010 +0200
@@ -913,7 +913,7 @@
                 from parsefortran import FortranParser
                 self.info('looking module information from %r' % (fn))
                 reader = FortranFileReader(fn, include_dirs=self.reader.include_dirs, source_only=self.reader.source_only)
-                parser = FortranParser(reader)
+                parser = 
FortranParser(reader,ignore_comments=self.top.parent.ignore_comments)
                 parser.parse()
                 parser.block.a.module.update(modules)
                 parser.analyze()


best,

omar

What steps will reproduce the problem?
1.expande attatched file.
2.Put takao_test_read.py just above the f2py directoy.
3.Then type "takao_test_read.py Issue7.F"

What is the expected output? What do you see instead?
This is what I got in the above procedure.
=======================================
Not implemented: Defined_Binary_Op
Not implemented: Defined_Binary_Op
Defined_Operator not defined used by Generic_Spec
['Issue7.F']
@@@@@ Issue7.F @@@@@ start -----
(1, 1) Line subroutine bndfp()
(2, 8) Line use m_struc_def
(3, 3) C-
(9, 9) Line end
@@@@@ Issue7.F @@@@@ end -------
=======================================
The first things in each line are reader.span.
You can see that (3, 3) is after (2, 8).
This seems to occur when C followed by '- 'is a comment line.

What version of the product are you using? On what operating system?
r17

Please provide any additional information below.

Hi, 
thank you so much for your effort.
I observed most of all problems are removed for me.
But I still have a problem on modules.
Error message 'is_public' are shown in cases.

=============================================
What steps will reproduce the problem?

1. copy this small test program as fatest.py
------------fatest.py ---------------------------
#!/usr/bin/env python
import sys,os
#sys.path.append('/home/takao/ecal/TOOLS/f2py')
from fparser.api import parse
nargv = len(sys.argv) -1
argset= sys.argv[1:]
print argset
for ffile in argset:
    print ffile
    tree = parse(ffile,isfree=False,isstrict=False,ignore_comments=False)
    print tree.content
sys.exit()
---------------------------------------

2.run
$fatest.py m_rdctrl.F, which is included in an attached file.

Then I got
-----------------------------------
takao@takaot61:~/ecal/TEST$ ~/ecal/TOOLS/fatest.py m_rdctrl.F 
Not implemented: Defined_Binary_Op
Not implemented: Defined_Binary_Op
Defined_Operator not defined used by Generic_Spec
['m_rdctrl.F']
m_rdctrl.F
WARNING while processing 'm_rdctrl.F' (mode='fix90')..
    4:C # is commented # 1 "subs/m_rdctrl.F"
    5:      module m_rdctrl
    6:      use m_struc_def
    7:C     implicit none
    8:
    9:C ... Fixed parameters <== no information about the module
'm_struc_def' in use statement
   10:      integer,parameter:: NULLI =-99999
   11:      real(8),parameter::    NULLR =-99999
WARNING while processing 'm_rdctrl.F' (mode='fix90')..
  268:Cu   09 Aug 07
  269:C ----------------------------------------------------------------------
  270:      use m_toksw <== no information about the module 'm_toksw' in
use statement
  271:      use m_gtv
  272:C     implicit none
WARNING while processing 'm_rdctrl.F' (mode='fix90')..
  269:C ----------------------------------------------------------------------
  270:      use m_toksw
  271:      use m_gtv
  272:C     implicit none
  273:C ... Passed parameters <== no information about the module 'm_gtv'
in use statement
  274:      character*(*):: prgn
  275:      character(6):: vstrn(2)
While processing
  Line('module m_rdctrl',(5, 5),'')
with <function analyze at 0x7fec0534f578> the following exception was raised:
Traceback (most recent call last):
  File "/home/takao/ecal/TOOLS/f2py/fparser/parsefortran.py", line 86, in
analyze
    self.block.analyze()
  File "/home/takao/ecal/TOOLS/f2py/fparser/utils.py", line 191, in new_func
    func(self)
  File "/home/takao/ecal/TOOLS/f2py/fparser/block_statements.py", line 216,
in analyze
    stmt.analyze()
  File "/home/takao/ecal/TOOLS/f2py/fparser/utils.py", line 191, in new_func
    func(self)
  File "/home/takao/ecal/TOOLS/f2py/fparser/block_statements.py", line 309,
in analyze
    stmt.analyze()
  File "/home/takao/ecal/TOOLS/f2py/fparser/utils.py", line 191, in new_func
    func(self)
  File "/home/takao/ecal/TOOLS/f2py/fparser/block_statements.py", line 617,
in analyze
    if self.is_public():
AttributeError: 'Subroutine' object has no attribute 'is_public'
FATAL ERROR: STOPPED ANALYSING <closed file 'm_rdctrl.F', mode 'r' at
0x7fec05371c68> CONTENT
-----------------------------------
I also attached other *.F files which caused similer error.
Try fatest.py *.F

regards, 
takao 

Hi,
I have two things.
One is my own change (A), the other is bug reports (B).

As for (A), I now want to keep fortran code as it is in self.item.
In order to do so, I made a little modification to readfortran.py.
If possible, I hope you will include this as an option in your fparser.

---preparation to reproduce my results---
In a directy, expand IssueCommet.tar.gz.
Then copy fatest.py just above f2py.
Then replace f2py/fparser/readfortran.py (r25) with readfortran.py in
IssueCommet.tar.gz. (Try diff of these two readfortran.py.).


==== (A) Changes in readfortran.py======
Perform
  fatest.py CommentTest.F.
There is some difference when we use original readfortran.py (r25).

1. lines with '#' at 1st column are recognized as comment lines.
   As far as fortran codes are grammatically correct even 
   when "#foo"s are treated as comments, we can analyze the codes
  (I rewrote my codes so).

   Related to this, I show my opinions below.--

    In future, I hope that fparser can recognize #foo as 
    fpp directives (especially it is necessary to distinguish #if block
    structure). In order to do so, it might be better that 
    not FortranFileReader but FortranParser should identify CommentBlock
    from others on the same footing as other block_statements.
    Inline comments (starting from !) can be an attribute of each line.

    I think CommentBlock is not so good identification.
    Fortran comments can be an attribute of a line --- 
    then we can treat inline comment and traditional comment lines 
    on the same footings. Class of a traditional comment line may be 
    named as DoNothing.
    (Then we need another attribute --- span of the comments). 
    It is not so meaningful to distinguish two comment lines;
    one starts from "!" at the first column, and the other starts 
    from second column.

2. I added "\n" at the end of CommentBlock.comment.
   So the output of fatest.py looks like item=Commnt('...\n',span).

3. Even a line with nothing is recognized as a comment line.
   So we see 
     item=Comment('c test abb\nc\n# 1232\n!\n\nc\ncabi\n',(3, 9))
     item=Comment('\n',(14, 14)) 
   for example.

===(B) Bug reports===
1. Perform
  fatest.py Issue_r25_1.F 
 Then I can see item=Comment('!hhhh2\n!hhhh2\n',(12, 12)).
 This looks strange.

2. Perform
  fatest.py Issue_r25_2.F 
 This caused a stop (this occurs when a nothing line is in the middle of
format).

best regards,
takao kotani

p.s. 
I would like to set up a Fortran-code analyzer 
(Module dependency, call-caller tree, recursive input/output check of
subroutine. 
 Block analysis. Data flow, or so.)
The tree =
parse(ffile,isfree=False,isstrict=False,ignore_comments=False,analyze=True)
can treated as a database. It can be pickled, 
and refreshed when a fortran file is modified.

I should apply some operation to do analysis based on the database.
or I hope that you will start such a project 
you as a leader(but maybe too busy to you?).

I have attatched a fortran source.
No problem to scan it by r29.

But r31 caused a stop by 
-------------------------
ERROR while processing '/home/takao/ecal/lm7K/lmf2gw.F' (mode='fix90')..
  504:      i2=0
  505:      return
  506:  2   i2=len(str)+1
  507:      do 3 i=len(str),1,-1 <== exception triggered here: <type
'exceptions.Exception'> 'Line' object has no attribute 'analyze'
  508:        if(str(i:i).ne.' ') then
  509:          i2=i
  510:          goto 4
-------------------------
Could you fix this?

Hi Attatched is a simple test system for fparser.

Usege:
  expand fparsertest.tar.gz in an empty directory.
  Then run ./job_fatest. No arguments.
  This does svn checkout http://f2py.googlecode.com/svn/trunk/ f2py-read-only
  Then parse fortran codes which are in FfixSample/.

It takes a minute or less. You may remove some codes 
to reduce computational time from FfixSample. Look into job_fatest first.

r33 caused some errors. (r32 did not).
If necessary, I will look into ERRORs furthermore.

Hi, I have a problem. Probably it is a bug.
I need your help. Could you look into it?

What steps will reproduce the problem?
1. Expand an attached file in a directory just above f2py.
2. Run a command in "job" file.

What is the expected output? What do you see instead?
See dat and err.
Do "grep f2py dat". This shows strange results (I think).
For example, dat contains lines as
-----------------------------------------
5 (485, 486) call    rxs2(f2py_expr_tuple_124) ---> {}
-----------------------------------------
As you can easily see, this means that
an item.strlinemap do not contain a thing corresponding to
f2py_expr_tuple_124. 
Actually it is included in its previous item. Strange.
From the beginning, why "f2py_expr_tuple_124" is in lower case?

What version of the product are you using? On what operating system?
r58,ubuntu9.10


Please provide any additional information below.


The followings are another story; so no reply required.
----------
I am gradually using your fparser; I like it. 
It will become very useful for Fortran Rewriter/Analyzer. 

For high performance calculations, I imagine an idea 
"a slight extension of fortran language",
which is a translator to usual fortran codes (human readable).
This can be rather practical than f2py from the view of fortran
writers.

For example, "Fortran without declear variables" can be possible;
or a sentence like "a,b = subroutine_xxx(input_1,input_2)" 
can be possible. Inputs and Outputs should be distinguished clearly.
Complexed index sets can be used as index of a matrix.

What steps will reproduce the problem?
1.Use the latest version r16
2.Copy this as Issue5.F
---------------------------
      subroutine bndfp()
      implicit none
      character strn*120,plbopt*120,mulopt*120,clsopt*120,strn2*120,dc*1
      end
---------------------------
3. Analyze this. by fparser as I did in Issue 4.

What is the expected output? What do you see instead?
---------------------------
ERROR while processing 'Issue5.F' (mode='fix90')..
    1:      subroutine bndfp()
    2:      implicit none
    3:      character
strn*120,plbopt*120,mulopt*120,clsopt*120,strn2*120,dc*1 <== analyze error:
Implicit rules mapping is null while getting 'strn' type
    4:      end
---------------------------
and so on.

What version of the product are you using? On what operating system?
r16

Please provide any additional information below.
If no implicit none, no ERROR messages.

This code fails to parse (analyze):

==============
MODULE testa

CONTAINS

SUBROUTINE f(arga)
INTEGER :: arga

CONTAINS

SUBROUTINE subf(argx)
INTEGER :: argx
END SUBROUTINE subf

END SUBROUTINE f

END MODULE testa
==============

The reason is that at block_statements.py:624 it is assumed that after the 
filter_stmts function on line 620 is executed, there are no blank lines or 
comments between the 'CONTAINS' and the 'END SUBROUTINE f' line. 

see attached file for my proposed patch - I didn't dare to commit to mercurial 
yet, will do it myself once I'm feeling more comfortable, besides.. maybe my 
fix sucks:

It seems that Fortran2003.py is not used now.
Can I remove it?

Or it is still meaningful in some sense?
How it works?

I expect that you will include a text4developers.txt, which
is to explain idea of fparser and structure of code.

takao

Hy all,

I would like to get the last version of f2py. I run the commands shown above. 
What do I have to do next to obtain the script 'f2py', that I can run?

Thank you,

David

hg clone https://f2py.googlecode.com/hg/ f2py
cd f2py
./setup.py install --prefix=~/last_f2py_version
cd ~/last_f2py_version
find .
.
./lib
./lib/python2.6
./lib/python2.6/site-packages
./lib/python2.6/site-packages/f2py
./lib/python2.6/site-packages/f2py/setup.pyc
./lib/python2.6/site-packages/f2py/setup.py
./lib/python2.6/site-packages/f2py/version.pyc
./lib/python2.6/site-packages/f2py/__init__.pyc
./lib/python2.6/site-packages/f2py/version.py
./lib/python2.6/site-packages/f2py/__init__.py
./lib/python2.6/site-packages/fparser
./lib/python2.6/site-packages/fparser/sourceinfo.pyc
./lib/python2.6/site-packages/fparser/statements.pyc
./lib/python2.6/site-packages/fparser/pattern_tools.pyc
./lib/python2.6/site-packages/fparser/block_statements.pyc
./lib/python2.6/site-packages/fparser/sourceinfo.py
./lib/python2.6/site-packages/fparser/setup.pyc
./lib/python2.6/site-packages/fparser/utils.pyc
./lib/python2.6/site-packages/fparser/setup.py
./lib/python2.6/site-packages/fparser/block_statements.py
./lib/python2.6/site-packages/fparser/Fortran2003.pyc
./lib/python2.6/site-packages/fparser/api.py
./lib/python2.6/site-packages/fparser/script_options.py
./lib/python2.6/site-packages/fparser/typedecl_statements.py
./lib/python2.6/site-packages/fparser/readfortran.pyc
./lib/python2.6/site-packages/fparser/script_options.pyc
./lib/python2.6/site-packages/fparser/base_classes.pyc
./lib/python2.6/site-packages/fparser/splitline.pyc
./lib/python2.6/site-packages/fparser/pattern_tools.py
./lib/python2.6/site-packages/fparser/utils.py
./lib/python2.6/site-packages/fparser/parsefortran.py
./lib/python2.6/site-packages/fparser/typedecl_statements.pyc
./lib/python2.6/site-packages/fparser/statements.py
./lib/python2.6/site-packages/fparser/splitline.py
./lib/python2.6/site-packages/fparser/__init__.pyc
./lib/python2.6/site-packages/fparser/base_classes.py
./lib/python2.6/site-packages/fparser/parsefortran.pyc
./lib/python2.6/site-packages/fparser/api.pyc
./lib/python2.6/site-packages/fparser/Fortran2003.py
./lib/python2.6/site-packages/fparser/readfortran.py
./lib/python2.6/site-packages/fparser/log.config
./lib/python2.6/site-packages/fparser/__init__.py
./lib/python2.6/site-packages/f2py-0.3.1.dev-py2.6.egg-info

What steps will reproduce the problem?

Attempting to parse following Fortran 90 code containing an overloaded 
assignment interface fails:

>>> s = """module foo
... 
... interface assignment(=)
...    module procedure baa
... end interface assignment(=)
...  
... end module foo"""
>>> from fparser import api
>>> print api.parse(s)
WARNING: While processing 'string-4457787584' (mode='free')..
    3:interface assignment(=)
    4:   module procedure baa
    5:end interface assignment(=) <== no parse pattern found for "end interface assignment(F2PY_EXPR_TUPLE_2)" in 'Interface' block.
WARNING: While processing 'string-4457787584' (mode='free')..
    5:end interface assignment(=)
    6:
    7:end module foo <== no parse pattern found for "end module foo" in 'Interface' block.
WARNING: While processing 'string-4457787584' (mode='free')..
    1:module foo
    2:
    3:interface assignment(=) <== failed to find the end of block
    4:   module procedure baa
    5:end interface assignment(=)
WARNING: While processing 'string-4457787584' (mode='free')..
    1:module foo <== failed to find the end of block
    2:
    3:interface assignment(=)
ERROR: While processing 'string-4457787584' (mode='free')..
    1:module foo
    2:
    3:interface assignment(=) <== exception triggered here: <type 'exceptions.Exception'> 'Line' object has no attribute 'analyze'
    4:   module procedure baa
    5:end interface assignment(=)
    6:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/jameskermode/lib/python/fparser/api.py", line 157, in parse
    parser.analyze()
  File "/Users/jameskermode/lib/python/fparser/parsefortran.py", line 88, in analyze
    self.block.analyze()
  File "/Users/jameskermode/lib/python/fparser/utils.py", line 204, in new_func
    func(self)
  File "/Users/jameskermode/lib/python/fparser/block_statements.py", line 222, in analyze
    stmt.analyze()
  File "/Users/jameskermode/lib/python/fparser/utils.py", line 204, in new_func
    func(self)
  File "/Users/jameskermode/lib/python/fparser/block_statements.py", line 323, in analyze
    stmt.analyze()
  File "/Users/jameskermode/lib/python/fparser/utils.py", line 204, in new_func
    func(self)
  File "/Users/jameskermode/lib/python/fparser/block_statements.py", line 498, in analyze
    stmt.analyze()
AttributeError: 'Line' object has no attribute 'analyze'

What is the expected output? What do you see instead?

The problem is caused by:

 (i)  regular expression in EndInterface.match not matching parenthesised characters
 (ii) interface name not matching end interface name since bracketed expressions are
      converted to F2PY_EXPR_TUPLE_1 and F2PY_EXPR_TUPLE_2, causing test in 
      base_classes.EndStatement.process_item() to fail.

The attached patch fixes both of these issues. After applying patch, output of 
above commands is:

>>> s = """module foo
... 
... interface assignment(=)
...    module procedure baa
... end interface assignment(=)
...  
... end module foo"""
>>> from fparser import api
>>> print api.parse(s)
!BEGINSOURCE <cStringIO.StringI object at 0x10d33f8d0> mode=free
  MODULE foo
    INTERFACE assignment(=)
      MODULE PROCEDURE baa
    END INTERFACE assignment(=)
  END MODULE foo

What version of the product are you using? On what operating system?

Latest mercurial repository revision (changeset:  83:15a65747d6c6), on Mac OS X 
10.7.

Kurt (kwmsmith) has suggested to switch f2py svn repository to mercurial 
repository. Googlecode provides instructions for that

http://code.google.com/p/support/wiki/ConvertingSvnToHg

Now I have the following questions:

  1) Does anyone have any objections for the switch? It would be nice if all f2py commiters could give a short answer whether they agree or disagree with the switch.

  2) Do we want to keep full log history?

Seems to be a missing apply_map, as patched below.

Best,
Andreas

This patch adds the contents of module_interface to the list of all the public 
symbols provided by a module (all_mod_provides)


diff -r c5b88e996b11 -r 732d6b540026 fparser/statements.py
--- a/fparser/statements.py     Fri Jan 27 14:26:10 2012 +0000
+++ b/fparser/statements.py     Fri Jan 27 14:29:58 2012 +0000
@@ -930,6 +930,7 @@
         norenames = [item for item in self.items if '=>' not in item]
         all_mod_provides = dict(module.a.module_provides)
         all_mod_provides.update(module.a.use_provides)
+        all_mod_provides.update(module.a.module_interface)
         if self.isonly:
             # populate use_provides with items/renames only.
             for rename, orig in renames:

I wrote a small program to generate callcaller tree for Fortran codes.
 f_calltree.F file1 file2,...  >callcaller.dat 2>callcaller.err
Generate call caller for fortran codes.
Put this just above f2py directory.

I hope you will include this in your fparser package if possible.
(Warning: this cause an error if computer codes are gramatically wrong 
 when we comment out preprocessor lines. Further, include is not treated well).

Another point.--
fparser uses yellowtext and so, and ubyte in numpy.
So people need to install numpy only in order to use fparser.
I think this is a little too demanding for people who just want 
to use fparser only. So I hope fparser works without numpy.

What steps will reproduce the problem?
1. Run a Fortran program wrapped using f2py in Python; any STOP command in 
Fortran will kill the entire Python instance


What version of the product are you using? On what operating system?
Version 2

Please provide any additional information below.

I am trying to wrap a large, existing Fortran program 
(http://www.itp.uzh.ch/~teyssier/ramses/RAMSES.html) with multiple files and 
modules in f2py and call it from a Python package. The Fortran program makes 
liberal use of STOP commands in both upper and lower case and is under active 
development, so removing the STOP commands manually will be a pain. I would 
like to keep the Python instance active after the Fortran program ends, rather 
than having the Fortran program kill the Python script upon exit. Putting the 
f2py call in a separate thread doesn't appear to help; a STOP command in a 
spawned thread will kill the Python instance that spawned it. If you can 
suggest work-arounds that don't involve calling the program from the command 
line, let me know!

running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
build_src
building extension "dcuhre" sources
f2py options: []
f2py:> /tmp/tmpvlbnqe4n/src.linux-x86_64-3.6/dcuhremodule.c
creating /tmp/tmpvlbnqe4n/src.linux-x86_64-3.6
Reading fortran codes...
	Reading file 'dcuhre.f90' (format:free)
rmbadname1: Replacing "index" with "index_bn".
rmbadname1: Replacing "index" with "index_bn".
rmbadname1: Replacing "index" with "index_bn".
Traceback (most recent call last):
  File "/home/maxnoe/.local/anaconda3/bin/f2py", line 28, in <module>
    main()
  File "/home/maxnoe/.local/anaconda3/lib/python3.6/site-packages/numpy/f2py/f2py2e.py", line 648, in main
    run_compile()
  File "/home/maxnoe/.local/anaconda3/lib/python3.6/site-packages/numpy/f2py/f2py2e.py", line 633, in run_compile
    setup(ext_modules=[ext])
  File "/home/maxnoe/.local/anaconda3/lib/python3.6/site-packages/numpy/distutils/core.py", line 169, in setup
    return old_setup(**new_attr)
  File "/home/maxnoe/.local/anaconda3/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/home/maxnoe/.local/anaconda3/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/home/maxnoe/.local/anaconda3/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/maxnoe/.local/anaconda3/lib/python3.6/site-packages/numpy/distutils/command/build.py", line 47, in run
    old_build.run(self)
  File "/home/maxnoe/.local/anaconda3/lib/python3.6/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/home/maxnoe/.local/anaconda3/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/home/maxnoe/.local/anaconda3/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/maxnoe/.local/anaconda3/lib/python3.6/site-packages/numpy/distutils/command/build_src.py", line 148, in run
    self.build_sources()
  File "/home/maxnoe/.local/anaconda3/lib/python3.6/site-packages/numpy/distutils/command/build_src.py", line 165, in build_sources
    self.build_extension_sources(ext)
  File "/home/maxnoe/.local/anaconda3/lib/python3.6/site-packages/numpy/distutils/command/build_src.py", line 327, in build_extension_sources
    sources = self.f2py_sources(sources, ext)
  File "/home/maxnoe/.local/anaconda3/lib/python3.6/site-packages/numpy/distutils/command/build_src.py", line 564, in f2py_sources
    ['-m', ext_name]+f_sources)
  File "/home/maxnoe/.local/anaconda3/lib/python3.6/site-packages/numpy/f2py/f2py2e.py", line 408, in run_main
    postlist = callcrackfortran(files, options)
  File "/home/maxnoe/.local/anaconda3/lib/python3.6/site-packages/numpy/f2py/f2py2e.py", line 329, in callcrackfortran
    postlist = crackfortran.crackfortran(files)
  File "/home/maxnoe/.local/anaconda3/lib/python3.6/site-packages/numpy/f2py/crackfortran.py", line 3246, in crackfortran
    readfortrancode(files, crackline)
  File "/home/maxnoe/.local/anaconda3/lib/python3.6/site-packages/numpy/f2py/crackfortran.py", line 510, in readfortrancode
    dowithline(finalline)
  File "/home/maxnoe/.local/anaconda3/lib/python3.6/site-packages/numpy/f2py/crackfortran.py", line 781, in crackline
    analyzeline(m, pat[1], line)
  File "/home/maxnoe/.local/anaconda3/lib/python3.6/site-packages/numpy/f2py/crackfortran.py", line 1087, in analyzeline
    last_name = updatevars(typespec, selector, attr, edecl)
  File "/home/maxnoe/.local/anaconda3/lib/python3.6/site-packages/numpy/f2py/crackfortran.py", line 1543, in updatevars
    el = [x.strip() for x in markoutercomma(entitydecl).split('@,@')]
  File "/home/maxnoe/.local/anaconda3/lib/python3.6/site-packages/numpy/f2py/crackfortran.py", line 821, in markoutercomma
    assert not f, repr((f, line, l, cc))
AssertionError: (2, ' dim2w(14,5) = reshape( (/   0.3379692360134460d-01,  0.9508589607597761d-01, 0.1176006468056962d+00,  0.2657774586326950d-01, 0.1701441770200640d-01,  0.0000000000000000d+00, 0.1626593098637410d-01,  0.1344892658526199d+00, 0.1328032165460149d+00,  0.5637474769991870d-01, 0.3908279081310500d-02,  0.3012798777432150d-01, 0.1030873234689166d+00, 0.6250000000000000d-01, ', ' dim2w(14,5) = reshape( (/   0.3379692360134460d-01,  0.9508589607597761d-01, 0.1176006468056962d+00,  0.2657774586326950d-01, 0.1701441770200640d-01,  0.0000000000000000d+00, 0.1626593098637410d-01,  0.1344892658526199d+00, 0.1328032165460149d+00,  0.5637474769991870d-01, 0.3908279081310500d-02,  0.3012798777432150d-01, 0.1030873234689166d+00, 0.6250000000000000d-01, ', ')')

What steps will reproduce the problem?
1. Wrap some code that contains WRITE statements with f2py
2. Try to capture this output on Python stdout / stderr descriptors
3. Watch yourself fail

What is the expected output? What do you see instead?

I would expect to be able to capture the text printed by wrapped routines from 
inside Python.

What version of the product are you using? On what operating system?

Latest NumPy / SciPy / f2py / Python 2.7 / Linux + Windows

Please provide any additional information below.

See e.g. http://projects.scipy.org/scipy/ticket/1350 for discussion:

In essence, SciPy wraps a lot around Fortran code, e.g. for optimizers. 
Unfortunately, these codes print a lot of interesting debugging (not only 
debugging, actually) information on stdout / stderr and this output can not be 
captured / parsed automatically from within Python.

It is possible to modify the wrappers to return more information, but it is 
tedious and not practical to change every function that uses WRITE so that it 
adds text to a global string that will be returned to Python.

The solution that I have in mind is to implement an extra function from within 
f2py, e.g. WRITE_PYTHON which would be identical to WRITE, but actually use 
PySys_WriteStdout to write to the same descriptors that Python has opened.

Probably this will also require the modification of the Fortran codes, but mass 
replacement of WRITE with WRITE_PYTHON already looks much more reasonable.

Discussion and alternative solutions are welcome!

Thanks!

I don't know whether this is actually an f2py issue, but:

if you try to compile the following code with:

$> f2py -c -m test test.f90

subroutine array(n, A, B)
    implicit none
    integer n
    double precision A(n,n)
    double precision B(n*n) !B(n**2)
    !f2py intent(in) n
    !f2py intent(out) A
    !f2py intent(out) B
    A(:,:) = 1.0
    B(:) = 2.0
    print *, n
end subroutine

This code works with a multiplication in the dimension of B (i.e. B(n*n)) but 
it does not work with the power operator (i.e. B(n**2)). As far as I know using 
the power works in common Fortran.

The error msg that I get is:

"
In function ‘f2py_rout_test_array’:
error: invalid type argument of unary ‘*’ (have ‘int’)
"

=================
What version of the product are you using? On what operating system?

I am using f2py Version: 2, numpy Version 1.6.1 Python 2.7.3 on Ubuntu 12.04

Hi Pearu,

When declaring a variable fortran allows to pass the KIND intrinsic function 
directly to the 
intrinsic type specifier, e.g.

REAL(KIND(1.0D0)    :: a

the function TypeDeclarationStatement._parse_kind_selector does not allow for 
this case.

I have added an example fortran code and a possible patch

cheers,

omar

I have two points. I have simplified problems.

(1) For a program,
-----------------------------------
      subroutine bndfp(ax,i)
      logical:: ax
      if(ax) i=1
      end  
-----------------------------------
Your parse in api.py classify 'if(ax)i' as a variable.
---------------------------------------------------
  Subroutine
    args=['ax', 'i']
    item=Line('subroutine bndfp(ax,i)',(2, 2),None,None,<reader>)
    a=AttributeHolder:
variables=<dict with keys ['i', 'ax']>
    content:
      Logical
        selector=('', '')
        entity_decls=['ax']
        item=Line('logical:: ax',(3, 3),None,None,<reader>)
      Assignment
        variable='if(ax)i'
        sign='='
        expr='1'
        item=Line('if(ax) i=1',(4, 4),None,None,<reader>)
  EndSubroutine
    blocktype='subroutine'
    name='bndfp'
    item=Line('end',(5, 5),None,None,<reader>)
---------------------------------------------------


(2) For a program,
-----------------------------------
      subroutine bndfp(ax,i)
      logical:: ax
      if(ax) call bb()
      end  
-----------------------------------
Your parse in api.py shows
-----------------------------------
If
  blocktype='if'
  name='__IF__'
  item=Line('if(ax) call bb()',(3, 3),None,None,<reader>)
  content:
    Call
      item=Line('call bb()',(3, 3),None,None,<reader>)
-----------------------------------
. This is a little problematic because 'call bb()'
is contained in both of Line. I think  'call bb()' should
not be contained in the If instance.
(it is the same as if then block...).

takao

In rev58, I did
    reader = FortranFileReader(ffile)
    reader.set_mode(isfree=False,isstrict=False)
    parser=FortranParser(reader,ignore_comments=False)
    parser.parse()
    parser.analyze()
where ffile is bndfp.F, which I attatched in Issue 21.
In other words, I just insered line parser.analyze() in Issue 21.
Then it ends up with
   AttributeError: 'Line' object has no attribute 'analyze'.

When working with a bunch of fortran files, each containing a module and 
modules USEing each 
other it is very well possible that a variable in a module is parsed and 
analyzed more than once.. 

that happens at least when one module is used more than once by different 
modules.

When that happens, analyze raises an exception. I have traced the problem back 
to 
Variable.update(self, *attrs) line 304 (file base_classes.py). The assert on 
this line failes if the 
variable was parsed and analyzed before. What I find a bit strange is that this 
assertion only 
exists for the dimension modifier in the type declaration of a variable, not 
for the other ones

I suggest to do one of the following two: either remove this assertion (I don't 
see its use), or not 
reanalyze a variable if it exists already in the attribute dictionary of its 
container. However I don't 
quite understand yet how those data structures are managed and hence I am not 
sure if this is 
really a good idea. At least for me, removing the assertion seems to fix the 
problem.

thanks,

omar

Hi pearu,
I wanted to know how to extract instances recursively.
For the purpose, I made a function,

def toreprx(block, depth=-1, incrtab=''):
    if depth==0 or not block.content:return
    for c in block.content:
        print 'xxxxxx ', depth, c.__class__.__name__
        toreprx(c,depth-1,incrtab)
    return

I think this should work when we supply parser.block to this routine. 
But it gives strange answer. block.content of Comment is not None. 
Why so? For me it is easy (e.g. to set up call-caller trees) if this works.

What I did is
1. Put takao_blocktest.py just above f2py/
2. do
>takao_blocktest.py takao_blocktest.F

takao

Hi pearu,
I looked into r40. It works OK.But two point if you can modify.

1. When I parsed a code, I can not find the include statement.
It seems that there is no class for the include statement.
I think the include statement should make depth one-rank deeper
(as an attribute of the "Include" Class, included file name. 
 Maybe relative path from the source is good from 
 the view of protability).
---Now we see strange results; span gives a line number in the included file. 

2.Then I found that the include statement in a fortran code is automatically
  classified to a comment. I think this is wrong procedure.
  It makes difficult to use fparser as a rewriter.
  If you introduce a class for include, then its instance 
  of included file is just empty, and no content.

So two problems above is systematically solved by adding a "Include" Class
in block_statement.py, I think

Hi there,

I'm opening this bug to let you all know that I've started a Python 3 port of 
this, here:

https://bitbucket.org/inducer/f2py

It's not completely finished, but out of the 313 tests, there are only 3 errors 
and 2 failures as of right now, which makes it work well enough for my needs 
for the moment.

Andreas

With fatest.py (or anything else), 
By r29, it fails to parse
------------------------------------
      module m_rdctrl
      use m_struc_def
      contains

      subroutine readctrl(prgn,vstrn,vn)

      end subroutine readctrl

      end module
------------------------------------

Last year, I hacked your code a little so as to fparse a few of my *.F well
by myself. What I did is in fparsertestTakao1.tar.gz; just do
TEST_TAKAO.py, which is
------------------------
#!/usr/bin/env python
from api import parse
fff='./xxx.F'
tree = parse(fff,isfree=False,isstrict=False,ignore_comments=False)
print tree.content
------------------------
Anyway, I found it is not so easy for me to fix kinds of problems in fparser
--- then I stopped. But I really need to disentangle our spaghetti code.
So I want to start it again.


================================
fparserIssueTakao1.tar.gz:
   : contains all required files
    (Fortran file names are *.o under */obj.gfortran.fpp/) 
Usually we use fpp (fortran version cpp). But it makes parsing so
difficult;It is another step. So I now packed all files (fixed form) 
after preprocessed.
See fparserIssueTakao1.tar.gz; Note that Fortran file names
are *.o instead of *.F, because I used -E option in my makefile in gfortran.

Our full package is at http://github.com/tkotani/ecal.
But not useful to you, maybe.

best regards,
takao kotani

    tree = fparser.api.parse(stream.read(), isfree=True, isstrict=True)
  File ".../vendor/f2py/fparser/api.py", line 158, in parse
    parser.analyze()
  File ".../vendor/f2py/fparser/parsefortran.py", line 90, in analyze
    self.block.analyze()
  File ".../vendor/f2py/fparser/utils.py", line 209, in new_func
    func(self)
  File ".../vendor/f2py/fparser/block_statements.py", line 223, in analyze
    stmt.analyze()
  File ".../vendor/f2py/fparser/utils.py", line 209, in new_func
    func(self)
  File ".../vendor/f2py/fparser/block_statements.py", line 324, in analyze
    stmt.analyze()
  File ".../vendor/f2py/fparser/utils.py", line 209, in new_func
    func(self)
  File ".../vendor/f2py/fparser/block_statements.py", line 1098, in analyze
    args,rest = parse_bind(spec)
  File ".../vendor/f2py/fparser/utils.py", line 89, in parse_bind
    newline = newline[4:].lstrip()
UnboundLocalError: local variable 'newline' referenced before assignment

Hi Pearu,

I couldn't figure out how to fix this one yet.. but here a test case that makes 
the parser unhappy:

=========
MODULE testa

TYPE t
    INTEGER :: x
END TYPE

CONTAINS

SUBROUTINE f(arga)
INTEGER :: arga
TYPE(t), DIMENSION(2) :: v

v(1)%x = 23
v(2)%x = 42

END SUBROUTINE f
END MODULE testa
=========

cheers,

omar

r32, When I parse this 
------------------------------------
      subroutine bndfp()
      include "events.ins"
      end
------------------------------------
, standard output contains WARNING. 
It should go to error output.

I am not so sure whether you think it is worth to fix it or not.

What steps will reproduce the problem?
1.
--------------------------
C test ccc
      subroutine bndfp1()
      end

      subroutine bndfp2()
      end

--------------------------
2. parse it with fatest.py as Issue4.

What is the expected output? What do you see instead?
This is the result by r17.
print tree.content gives not clean block structure as
[        CommentBlock
          item=Comment('C test ccc',(1, 1)),         Subroutine
          item=Line('subroutine bndfp1()',(2, 2),'')
          content:
        EndSubroutine
          blocktype='subroutine'
          name='bndfp1'
          item=Line('end',(3, 4),''),         Subroutine
          item=Line('subroutine bndfp2()',(5, 5),'')
          content:
        EndSubroutine
          blocktype='subroutine'
          name='bndfp2'
          item=Line('end',(6, 6),'')]


What version of the product are you using? On what operating system?
r17

Please provide any additional information below.
The output of print tree.content looks a little ugly
 ("Subroutine" is end of lines).

What you said is not a solution.I made an easy test system.
So please look into it again it. For your convenience.
I set up quick test system.

I have fparserIssueTakao2/ directory in an attatched file. Then
1. I prepare some of my *.F codes which are no # at the begining of each
   line. So, standard fortran codes.
2. Copy your all files in fparser/ to fparserIssueTakao2/.
3. Then I made a TAKAOTEST2.py.
   It works with TAKAOTEST2.py ztoy.F, TAKAOTEST2.py *.F or so.

Then I typed "TAKAOTEST2.py ztoy.F". It works fine. Good.
But "TAKAOTEST2.py *.F" shows many WARNINGS and fparser stops.

I hope you may modify fparser in some way with minimum fixing for my purpose.

For my purpose, just only showing item='....' is somehow helpful; 
how to show only these line analysys only.

regards,
takao

This patch fixes assignments to variables named "type" or "entry" for which 
were misinterpreted as TypeDecl constructs and Entry statements, by adding a 
negative lookahead for "=" to the regexps, and corrects a typo in 
ForallConstruct ("forar" -> "forall)

diff -r 9958619dab6a -r c5b88e996b11 fparser/block_statements.py
--- a/fparser/block_statements.py       Fri Jan 27 14:15:00 2012 +0000
+++ b/fparser/block_statements.py       Fri Jan 27 14:26:10 2012 +0000
@@ -842,7 +842,7 @@
     <forall-assignment-stmt> = <assignment-stmt> | <pointer-assignment-stmt>
     """
     end_stmt_cls = EndForall
-    match = re.compile(r'forarr\s*\(.*\)\Z',re.I).match
+    match = re.compile(r'forall\s*\(.*\)\Z',re.I).match
     name = ''
     def process_item(self):
         self.specs = self.item.get_line()[6:].lstrip()[1:-1].strip()
@@ -1035,7 +1035,7 @@
     <type-attr-spec> = <access-spec> | EXTENDS ( <parent-type-name> )
                        | ABSTRACT | BIND(C)
     """
-    match = re.compile(r'type\b\s*').match
+    match = re.compile(r'type\b(?!\s*=)').match
     end_stmt_cls = EndType

     a = AttributeHolder(extends = None,
diff -r 9958619dab6a -r c5b88e996b11 fparser/statements.py
--- a/fparser/statements.py     Fri Jan 27 14:15:00 2012 +0000
+++ b/fparser/statements.py     Fri Jan 27 14:26:10 2012 +0000
@@ -1364,7 +1364,7 @@
     <language-binding-spec> = BIND ( C [ , NAME = <scalar-char-initialization-expr> ] )
     <dummy-arg> = <dummy-arg-name> | *
     """
-    match = re.compile(r'entry\b', re.I).match
+    match = re.compile(r'entry\b(?!\s*[%=])', re.I).match
     def process_item(self):
         line = self.item.get_line()[5:].lstrip()
         m = re.match(r'\w+', line)