Comments (11)
Thank you for your replies @drreynolds and balos1.
-The reason I turned to Cvode from Intel ode solver was that FCvode (version 2003) was thread safe. It means a lot that Pr. Reynolds confirmed that Fcvode (version 2003) is thread safe. Unfortunately the past month I didn't succeed to parallelize my code through openmp. I am using threadprivate allocatable arrays to feed data to Fcvode inside openmp regions and I get inconsistent results with lots of crashes due to either Floating point exceptions or Memory access violations. My code works perfectly in serial compilation (absence of any openmp directives). So Its either the latest Intel OneApi Fortran compiler has made small changes in the latest openmp implementation that I am not aware of or using Fcvode forced my compiler to use F2003 Fortran that may be incompatible with existing code.
-At the moment I am trying to produce a simple example that clearly exhibits inconsistent results between serial and openmp version of my code. If I have something concrete I 'll create a new issue in hope for resolution.
-I replied in the current thread cause I am not quite sure the compilation process for producing Fcvode is absolutely correct. I worry that internal optimizations of fcvode or Cvode may spawn extra threads inside openmp regions and create unexpected results. Though this shouldn't be the case.
Best Regards
from sundials.
At least, does anyone tried to compile CVode with Visual studio (2013 version or later) and encountered similar problems where Fortran objects are not compiled because the Fortran and C objects must be in different projects?
from sundials.
There was a previous user report on the sundials email list about the issue with Visual Studio and Fortran projects. It seems that you have found a resolution for that though. I don't quite follow what you have done to allow it to compile, but If you had to change code in SUNDIALS, posting the diff or a pull request would be appreciated.
As far as the run-time issue in Debug mode... I believe this is because the FN_VGetArrayPointer
function calls c_f_pointer
with a shape of [1]
here. In Debug mode there is likely a bounds check that does not like this.
You can try to update the FN_VGetArrayPointer
function to:
function FN_VGetArrayPointer(v) &
result(swig_result)
use, intrinsic :: ISO_C_BINDING
real(C_DOUBLE), dimension(:), pointer :: swig_result
type(N_Vector), target, intent(inout) :: v
type(C_PTR) :: fresult
type(C_PTR) :: farg1
type(C_INT64_T) :: length
farg1 = c_loc(v)
fresult = swigc_FN_VGetArrayPointer(farg1)
length = FN_VGetLength(v)
call c_f_pointer(fresult, swig_result, [length])
end function
If this works, then we can try to incorporate it into the next SUNDIALS release.
from sundials.
Dear Balos,
Yep we have a winner!
After 1.5 month of efforts CVode on Fortran runs!
The code you proposed fixed the problem with out of boundaries exception.
After updating the "FN_VGetArrayPointer" Function the example ("examples/cvode/F2003_serial/cv_analytic_sys_dns_f2003.f90") ran properly and after doing some debug the "yvec" is an array with size of 3 elements as it was supposed to be.
A couple of things to note:
- The fix you proposed has a typo:
function FN_VGetArrayPointer(v) &
result(swig_result)
use, intrinsic :: ISO_C_BINDING
real(C_DOUBLE), dimension(:), pointer :: swig_result
type(N_Vector), target, intent(inout) :: v
type(C_PTR) :: fresult
type(C_PTR) :: farg1
type(C_INT64_T) :: length
farg1 = c_loc(v)
fresult = swigc_FN_VGetArrayPointer(farg1)
length = FN_VGetLength(v)
call c_f_pointer(fresult, swig_result, [length])
end function
The line "type(C_INT64_T) :: length " is supposed to be "integer(C_INT64_T) :: length " else it won't compile, so the correct update would be:
function FN_VGetArrayPointer(v) &
result(swig_result)
use, intrinsic :: ISO_C_BINDING
real(C_DOUBLE), dimension(:), pointer :: swig_result
type(N_Vector), target, intent(inout) :: v
type(C_PTR) :: fresult
type(C_PTR) :: farg1
integer(C_INT64_T) :: length
farg1 = c_loc(v)
fresult = swigc_FN_VGetArrayPointer(farg1)
length = FN_VGetLength(v)
call c_f_pointer(fresult, swig_result, [length])
end function
- The subroutine "c_f_pointer" is used in a similar way elsewhere is the code and may need an update/fix too.
Please have a look:
function FN_VGetDeviceArrayPointer(v) &
result(swig_result)
use, intrinsic :: ISO_C_BINDING
real(C_DOUBLE), dimension(:), pointer :: swig_result
type(N_Vector), target, intent(inout) :: v
type(C_PTR) :: fresult
type(C_PTR) :: farg1
farg1 = c_loc(v)
fresult = swigc_FN_VGetDeviceArrayPointer(farg1)
call c_f_pointer(fresult, swig_result, [1])
end function
Best regards,
Bill
from sundials.
Dear Balos,
Regarding the steps I followed to compile under Visual Studio 2013 and 2019 unfortunately I didn't update any code. I just did some manual changes on the visual studio project that the Cmake-gui had generated. This means that every time the generate button is pressed on Cmake all the changes I did, must be redone. For the a simple version of Cvode (no examples or tests) this requires about 50 mins of copy paste. To fix properly Cvode, extensive changes on Cmake files are needed, so that the C and Fortran files are not anymore in the same project and instead are separated in different projects. The compilation order should be, first build the Fortran project and then the C targets.
Since the Sundials Cmake files seemed to me very complicated (so many targets with different options)
- I just created a simple Fortran project and I added all .f90 files I could find in Cvode. I duplicated the extension of Fortran file from .f90 to .f90.f90.
- I imported the Fortran project in the existing solution the Cmake-gui produced.
- I found all targets that pointed to a .f90.obj file and I changed the location of where the linker should search to find them. So for example in the solution explorer (right top hand side of visual studio) there is target (project) named "sundials_fcvode_mod_static" and if you expand it (expand the object folder too) this contains many objects like "fcvode_mod.f90.obj", fnvector_serial_mod.f90.obj and so on. If you look at the properties of these objects (right hand side of screen, under solution explorer) you will see a field called "Relative Path". This path contains the path where the linker should look to locate these objects. If you change this path to the "debug" or "release" folder of the Fortran project created above, then all will be ok. The linker will find the fcvode_mod.f90.obj objects and the "All_Build" will compile properly.
- To build the "Install" target I just created a new folder named "Fortran_Static" in the "build_dir" directory and inserted inside it all the .mod files created from the Fortran project above. Then hit build the "Install" target and everything worked.
I hope this helped a bit on the process I followed to compile Cvode with Visual Studio. If there is a need for more clarification let me know. It took me some time compile Cvode and I hope this is useful somehow.
Cheers,
Bill
from sundials.
@Bill-gitt Thanks for the detailed writeup that others can refer to for a workaround. We will try to fix both issues (the Visual Studio build, and FN_VGetArrayPointer problem) in a future SUNDIALS release.
from sundials.
Dear Balos,
It seems I have encountered another issue with the FCvode. It is related to its thread-safety, it seems independent threads access shared memory when executing memory release functions (FCVodeFree or FSUNMatDestroy etc...) and the program crashes. So, should I create a new issue in github or continue here in the current thread (it may be compile related)?
Regards,
Bill
from sundials.
from sundials.
@drreynolds is correct that FCVODE (the FORTRAN 77 interface to CVODE) is not theadsafe, but I think you are referring to the FCVode
function from the sundials_fcvode_mod
module (i.e. the F2003 interfaces), correct @Bill-gitt?
If you are indeed referring to the FCVode
function from the sundials_fcvode_mod
module, please start a new issue for it.
from sundials.
Dear balos1,
After lots of effort, I finally managed to make Fcvode work properly inside openmp regions. This is a success I believe (unless something else comes up later).
To give a bit of backstory so others may benefit from it, this is what happened. After trying to run simple Fcvode examples in parallel yielded random results. Most of the time everything worked out well but one out ten runs 1/10 the program could give erroneous results. Thusly I decided to use Intel inspector on my code. What I found out was eye opening, more than 25 warnings where revealed inside Fcvode libraries (irrelevant of the code I have written) that indicated failure of thread safely. I knew though that Fcvode should be thread safe and thusly the reason for failing should be in compilation. Searching a bit further and testing lots of different compilation flags I found out what was missing. The directive "qopenmp" was missing from the compilation of Fcvode interface. Searching online in Intel I found this
(1) “Static local data” is shared by each thread, i.e. each thread may access the same data location! This is potentially unsafe.
(2) “Automatic data” is independent between each thread; it has its own, independent copy on its own stack.
With Intel Fortran when compiled and running serially, the defaults are local scalar variables are automatic and local arrays are static. But when compiling with -qopenmp, local arrays are automatic by default. It is the same as compiling with -auto. This may require an increase in the maximum stack size. If the stack size is too small, there will be a segmentation fault at runtime.
Which means that when not using qopenmp local arrays are not thread safe...
After using qopenmp and running intel inspector again the 25 warnings were now only 2 which seemed nothing serious.
So for anyone that wants to run Fcvode inside openmp regions make sure that you compile code with qopenmp.
That's it for now. I have managed to compile Fcvode with Intel Visual studio and use its thread safety features, successfully.
Best regards
from sundials.
@Bill-gitt Thanks for posting your findings.
from sundials.
Related Issues (20)
- 6.7.0 failed to build with open-mpi 5.0 HOT 4
- library does not support the 'bdf' method HOT 2
- Duplicated symbols in libsundials_ida.a and libsundials_sunlinsolklu.a HOT 2
- Type conversion (maybe corrupting data)? HOT 5
- Possible incorrect build order when building FORTRAN interface HOT 5
- CVODES returns at wrong time when integrating backwards HOT 3
- API inconsistency: some SUNDIALS "Space" functions use `sunindextype` and some use `long int` HOT 5
- Crash when using dense linear solver and Netwon nonlinear solver with relative large number of equations HOT 3
- 7.0.0 release build failure HOT 1
- Update SUNDIALS source tree figure in documentation HOT 1
- Error when building both static and shared libs on Windows with MSVC HOT 4
- ARKODE error messages are being cutoff HOT 10
- Build Error with MPI and C++ Examples Enabled HOT 5
- Error Handler Memory Leak HOT 3
- Fortran interface on 32 bits machines? HOT 2
- Getting Error when trying to Build Paradis with KINSOL HOT 2
- Question about KINSOL behavior HOT 16
- IDA inconsistent results on MacOS HOT 1
- [BUG] SUNDIALS fails to build due to a linker error with MSYS+gfortran on Windows HOT 1
- Check system installation of sundials HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sundials.