Comments (9)
I had discussed a slightly bigger vision with Keshav along the same lines: Separate Galois apps from library, but also split the library into a hierarchy of components/libraries. The motivation being:
- not everybody needs everything.
- The pieces we carve out can be useful stand-alone libraries and tools (I think there's an issue you opened asking to separate tools).
If you have some cycles for it, we can work together. I haven't had a chance to work on this yet, but I'm interested.
from galois.
I agree with @amberhassaan. The issue you mentioned above is #54.
We just had our release meeting this morning, and decided that we will move the cmake configuration in Galois from "directory-based" to "target-based". Specifically, our plan follows:
- Remove the
app
macro in the root CMakeLists.txt. - Let each application/target specify whatever targets it wants to link against; cmake should then take care of the transitive closure/hierarchy of related targets.
If the above changes are in place and #45 is resolved, Galois core can be used as libraries by both external users and Lonestar apps/tools. This means #54 will be automatically resolved.
For the Galois core, we will have multiple library targets so that users can choose only parts essential to them. For example, libgpu
and libdist
are not required for purely shared-memory applications. For some applications, e.g., python programs using Galois backend, .so
is required; this should be addressable by the same mechanism we want here. Certainly a library can be built from other sub-libraries if there is no cyclic dependencies.
Separating repos for Galois core and Lonestar apps helps external users to reduce the code base they need to interact with should they decide to include Galois core as a git submodule. This is the case for OpenSTA.
from galois.
By the way @insertinterestingnamehere and @ddn0 are leading the effort for Release 6.0 and they are working on this front. Feel free to get in touch with them if interested; my bandwidth now mainly goes to #89.
from galois.
I think target based linking is a good idea. I'd like to take it further to using dependencies (galois libraries) as submodule just like you mentioned (for OpenSTA?).
I'd like to split at least apps, dist-apps, libdist, libshared into their repos. The apps can use libdist/libshared as submodules. Similarly, tools can be a repo of its own. It may be difficult right now to split the repos right this moment if multiple people are pushing commits, so either everyone stops for a few days or we do this after the release.
I'm sorry I'm not contributing actively at the moments but thoughts from others are welcome, esp. @ddn0 & @insertinterestingnamehere
from galois.
I'd like to split at least apps, dist-apps, libdist, libshared into their repos. The apps can use libdist/libshared as submodules. Similarly, tools can be a repo of its own. It may be difficult right now to split the repos right this moment if multiple people are pushing commits, so either everyone stops for a few days or we do this after the release.
From my perspective, the must-have is to be able to use "core Galois" out of the source tree (for all the reasons everyone has mentioned above). For me, "core Galois" are the shared memory, GPU and distributed memory libraries. And this will be a matter of just following standard CMake tooling (importable targets, target-based linking, flags, etc.).
On having separate repos for core Galois, I'm not super keen because I think the overall goal of the Galois project is to have a unified substrate for compute and there might be linked changes to, for example, support multiple GPUs that require linked changes across libdist, libgalois, ... But I can be convinced otherwise.
For the apps themselves, I am more ambivalent about where they should go and there is merit for either case. Core Galois should transparently support either. For me, the biggest factor is how "hardened" the code is.
FWIW, the plan is to still keep a mono-repo along the following lines:
lib/galois
lib/dist
lib/...
lonestar/analytics/bfs/dist
lonestar/...
from galois.
@ddn0 , the plan you outlined is a good next step.
For me, separating the repos has some cosmetic advantages. Esp. for the new user, who let's say wants to "galoise" his/her app. The steps would be 1) add submodules for shared-galois and if the app is distributed, add dist-galois. 2) do Cmake hookups, which should be as simple as "add_subdirectory" but may be in reality we need to specify a few build flags as well. And that's it, I hope.
If we go this route and do these steps for lonestar apps (in their own separate repo), we can serve that as an example to follow for future users. Also, a minor advantage is that understanding the code structure becomes easier, but that may already be achieved by your new directory structure.
I do agree that there's a code dependency problem, where say a change in shared-galois breaks dist-galois. I don't have an answer for that and I'm still thinking about this pitfall. One the plus side, the git-submodule mechanism ensures that working build won't break until you update one or more submodules.
from galois.
We have decided to use the same repo for both library and apps. So should this issue be closed?
from galois.
Perhaps a more concrete question would be: @yishanlu is the current state of the repo sufficient to enable the sharing use cases you raised about OpenSTA and Cyclone?
from galois.
For Cyclone I don't need the repos to be separated, since Cyclone is actually an app inside of Galois right now.
For OpenSTA, the current form works, though a repo with only Galois core libraries would be easier to handle with as a git submodule.
from galois.
Related Issues (20)
- Compilation Problems HOT 2
- Question about sssp-pull.cpp HOT 1
- What is the format of mastersFile? HOT 1
- How to add new code to Galois and modify the Makefile HOT 2
- Question of the MPI Network Layer (is the order guaranteed for the message?) HOT 6
- BFS hangs HOT 1
- Compiling Galois HOT 1
- Installation error HOT 1
- did you forget to ‘#include <optional>’? HOT 2
- Performance degradation for distributed PageRank HOT 8
- Is this project still alive? HOT 1
- papiGetTID error HOT 1
- Cmake error HOT 1
- "endian.h" file not found error while building BFS Application
- conversion undefined for void graphs HOT 1
- Cannot seem to get correct results for Connected Components
- Where is the directory of your HyperNetVec's implementation in corresponding paper?
- Does it support running on mac with m series chips
- Did this framework support SIMD instruction for parallelization?
- The server with the inputs requires a secured https connection
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from galois.