Comments (15)
getopt and getopt_long are not thread-safe, so they will not work with Adaptive MPI (and possibly other runtimes). I believe argtable uses getopt, too. I don't know of any better command line parsing libraries, but I thought you might want to consider this before implementing it!
from kernels.
@SamTWhite Thanks for bringing this to our attention! I guess I can write my own argument parser. I would normally suggest pthread_once
could be used here, but since AMPI is user-level threads, I doubt that it respects POSIX semantics in this respect.
from kernels.
If you want to follow the letter of the MPI standard, you should parse at process 0 and broadcast the parameters, because MPI doesn't guarantee that all of the processes will get the args (or private copies of the args). If you use that approach, getopt should be fine, even for a threaded MPI implementation like AMPI.
from kernels.
@jdinan We already do that. For example: https://github.com/ParRes/Kernels/blob/master/MPI1/Transpose/transpose.c#L223.
from kernels.
@jdinan Even if we can fix this for AMPI and FG-MPI, it is possible that we will need a thread-safe argument parser for other models where such serialization is harder to implement.
from kernels.
Which programming models are broken, such that you can't adequately synchronize, have one thread parse, and then share or broadcast the parameters? It's not that I don't believe you; I'm just curious.
from kernels.
I don't know for sure, but if any of the implementations do the weird trick where they turn main()
into a vector of tasks, then it might be an issue. Of course, one could wait until after parsing to create said vector of tasks, but that is not always what the dynamic runtime systems people want to do.
Even if this is not a problem for us, there is no good argument against having a thread-safe input argument parser, since it seems that the standard options do not meet this requirement.
from kernels.
One argument against is that if a programming model is going to do such and thing and not provide a solution to the resulting synchronization problem, then how viable is it as a generally usable programming model? It could be that leaving the thread-unsafe argument parsing in place poses a good (and practical) challenge to such programming models.
from kernels.
from kernels.
My point is that if a programming model breaks long-standing assumptions about thread safety (as exemplified by the lack of reentrant getopt), then it should also provide the functionality needed to resolve the resulting concurrency issues. So, for me, you are fixing a problem that should not actually need to be fixed. And, if it does need to be fixed for some programming model, I want to know about it.
from kernels.
The problem here is getopt
not being reentrant, not the lack of a serialization capabilities in some yet-to-be-determined parallel programming model.
from kernels.
I think I made my point already. If the programming model doesn't give you a way to deal with a sequence of operations that need to be serialized (argument parsing with getopt is only one such example), then there's a significant gap in the programming model.
from kernels.
About the comment by Jeff that we already follow the rule of having rank 0 do the argument parsing in MPI, that is actually not completely true. In the case of Synch_p2p, where the highest numbered rank carries out the verification test, we also let that rank do the parsing. Thanks for pointing out that that is not strictly conforming, Jim. We can easily make it so, and I'll do that.
from kernels.
Ah, yes I didn't realize that you were already parsing on only 1 rank, making the getopt issue, practically speaking, a non-issue for at least AMPI and the other runtimes that I know about.
from kernels.
The ETA for this exceeds the lifetime of the universe.
from kernels.
Related Issues (20)
- redo Travis CI infrastructure
- fix Numba p2p
- Broken link in Readme to docs HOT 1
- debug multi-GPU SYCL nstream HOT 1
- add DG1 helper
- Use LoopVectorization in julia stencil / transpose HOT 7
- Scala deprecation warnings HOT 1
- ALPAKA support HOT 8
- nvhpc offloading on RH8/p9+V100 system: supported? HOT 5
- Data Race due to num_error variable HOT 3
- CUDA stencil inefficiency (compared to SYCL) HOT 3
- pic-sycl.cc uses C99 HOT 1
- OpenMP Synch_p2p depends on x86 HOT 7
- Fortran MPI p2p is buggy HOT 1
- transpose-petsc crashes HOT 1
- Cannot find -lkokkos HOT 6
- MPI transpose with datatypes HOT 1
- prk_ranges.h:59:24: error: ‘ranges::views’ has not been declared HOT 7
- hip platform run stencil occurs "Segementation fault(core dumped)" HOT 3
- Kokkos kernels don't work under Kokkos version 4.3.0
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kernels.