Giter Club home page Giter Club logo

hwlocality's People

Contributors

daschl avatar dependabot[bot] avatar fsck avatar hadrieng2 avatar ichbinjoe avatar johalun avatar lu-zero avatar nazar-pc avatar trolldemorted avatar uglyoldbob avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

hwlocality's Issues

Expose GPU/heterogeneous API interop

hwloc provides interoperability with a bunch of GPU/heterogeneous system APIs. The amount of support code needed for each is small, but each will need to get a dedicated feature and suitable cfg+doc(cfg)+cfg(doc) annotations.

Also, testing is going to be problematic since CI nodes don't have GPUs and I don't even have local setups for some of these APIs.

Clarify dependencies required

I am trying to get GitHub actions and container image builds to succeed and looks like documentation right now is not quite accurate in terms of dependencies necessary.

About Linux it current says:

In addition to a valid C build environment, this requires autotools on Unices and CMake on Windows

While I noticed that for Ubuntu automake,libtool and pkg-config are all necessary for compilation.

Didn't get to macOS yet, but I expect similar surprises there.

Would be great to make documentation more exhaustive regarding dependencies.

UPD: libtool and pkg-config are also needed on macOS.

Not working on Alpine Linux (libc::pthread_t != std::os::linux::raw::pthread_t)

On Alpine Linux, pthread_t type is mismatched between libc and std.

  • (libc) ThreadId = libc::pthread_t's type is *mut libc::c_void!
  • (std) std::os::unix::thread::JoinHandleExt::as_pthread_t's return type is RawPthread = u64.

Error reconstruction

docker run --rm -it alpine:3.19  # Digest: x86_64 => sha256:6457d53fb065d6f250e1504b9bc42d5b6c65941d57532c072d929dd0628977d0

# In the docker container, we'll do the rest
apk add cargo git hwloc-dev eudev-dev

git clone "https://github.com/HadrienG2/hwlocality.git"
cd hwlocality
cargo test --all  # Error!

Logs

   Compiling hwlocality-sys v0.4.0 (/hwlocality/hwlocality-sys)
   Compiling hwlocality v1.0.0-alpha.1 (/hwlocality)
error[E0277]: `*mut c_void` doesn't implement `std::fmt::Display`
   --> src/cpu/binding.rs:805:62
    |
805 |             Self::Thread(id) => format!("the thread with TID {id}"),
    |                                                              ^^^^ `*mut c_void` cannot be formatted with the default formatter
    |
    = help: the trait `std::fmt::Display` is not implemented for `*mut c_void`
    = note: in format strings you may be able to use `{:?}` (or {:#?} for pretty-print) instead
    = note: this error originates in the macro `$crate::__export::format_args` which comes from the expansion of the macro `format` (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0308]: mismatched types
   --> src/lib.rs:288:9
    |
286 |     fn as_thread_id(&self) -> ThreadId {
    |                               -------- expected `*mut c_void` because of return type
287 |         use std::os::unix::thread::JoinHandleExt;
288 |         self.as_pthread_t()
    |         ^^^^^^^^^^^^^^^^^^^ expected `*mut c_void`, found `u64`
    |
    = note: expected raw pointer `*mut c_void`
                      found type `u64`

Some errors have detailed explanations: E0277, E0308.
For more information about an error, try `rustc --explain E0277`.
error: could not compile `hwlocality` (lib) due to 2 previous errors
warning: build failed, waiting for other jobs to finish...
error[E0277]: `*mut libc::c_void` doesn't implement `std::fmt::Display`
   --> src/cpu/binding.rs:805:62
    |
805 |             Self::Thread(id) => format!("the thread with TID {id}"),
    |                                                              ^^^^ `*mut libc::c_void` cannot be formatted with the default formatter
    |
    = help: the trait `std::fmt::Display` is not implemented for `*mut libc::c_void`
    = note: in format strings you may be able to use `{:?}` (or {:#?} for pretty-print) instead
    = note: this error originates in the macro `$crate::__export::format_args` which comes from the expansion of the macro `format` (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0308]: mismatched types
   --> src/lib.rs:288:9
    |
286 |     fn as_thread_id(&self) -> ThreadId {
    |                               -------- expected `*mut libc::c_void` because of return type
287 |         use std::os::unix::thread::JoinHandleExt;
288 |         self.as_pthread_t()
    |         ^^^^^^^^^^^^^^^^^^^ expected `*mut c_void`, found `u64`
    |
    = note: expected raw pointer `*mut libc::c_void`
                      found type `u64`

error: could not compile `hwlocality` (lib test) due to 2 previous errors

Expected Solution

If we cast the type ThreadId to *mut libc::c_void, then all examples would be failed because ThreadId is not thread-safe.

So I think the plausible option is: type ThreadId = u64.

Consider exposing component/plugin API

In principle, it is possible to extend hwloc by writing plugins for it. However, the compile-time constant used to evaluate ABI compatibility is a header define, which means I can't check it from Rust, so I'm not sure if exposing this from the Rust side is a good idea. It will probably require at least a little build.rs hackery to get back the define value into Rust.

In any case, here's the docs:

Topology::cpuset causes segfault

I have a function that looks like this:

pub fn example(mut tp: hwlocality::topology::Topology) {
    let cpuset = tp.cpuset();

    for index in cpuset.iter_set().map(usize::from).collect::<Vec<_>>() {
        println!("cpu {}", index);
    }
}

I call it like this:

example(tp.clone());

Here is a backtrace:

#0  0x00005555556bc6e5 in hwloc_bitmap_next (set=0x0, prev_cpu=0) at /home/thomas/hwlocality/hwloc/hwloc/bitmap.c:1402
#1  0x00005555556edbe3 in hwlocality::bitmaps::{impl#0}::next_set::{closure#0} (bitmap=0x0, prev=0) at src/bitmaps/mod.rs:868
#2  0x00005555556edaed in hwlocality::bitmaps::Bitmap::next<hwlocality::bitmaps::{impl#0}::next_set::{closure_env#0}> (self=0x7fffffffce18, index=..., next_fn=...)
    at src/bitmaps/mod.rs:854
#3  0x00005555556a0d46 in hwlocality::bitmaps::Bitmap::next_set (self=0x7fffffffce18, index=...) at src/bitmaps/mod.rs:867
#4  0x00005555556678d2 in hwlocality::bitmaps::{impl#32}::next<&hwlocality::bitmaps::Bitmap> (self=0x7fffffffcd90)
    at /home/thomas/hwlocality/src/bitmaps/mod.rs:1189
#5  0x0000555555615fe3 in core::iter::adapters::map::{impl#2}::next<usize, hwlocality::bitmaps::BitmapIterator<&hwlocality::bitmaps::Bitmap>, fn(hwlocality::bitmaps::indices::BitmapIndex) -> usize> (self=0x7fffffffcd90) at /rustc/8c74a5d27c644a0f7a22bb2fa8dd3ff8257bc220/library/core/src/iter/adapters/map.rs:103
#6  0x0000555555686c19 in alloc::vec::Vec<usize, alloc::alloc::Global>::extend_desugared<usize, alloc::alloc::Global, core::iter::adapters::map::Map<hwlocality::bitmaps::BitmapIterator<&hwlocality::bitmaps::Bitmap>, fn(hwlocality::bitmaps::indices::BitmapIndex) -> usize>> (self=0x7fffffffcd30, iterator=...)
    at /rustc/8c74a5d27c644a0f7a22bb2fa8dd3ff8257bc220/library/alloc/src/vec/mod.rs:2795
#7  0x00005555556895da in alloc::vec::spec_extend::{impl#0}::spec_extend<usize, core::iter::adapters::map::Map<hwlocality::bitmaps::BitmapIterator<&hwlocality::bitmaps::Bitmap>, fn(hwlocality::bitmaps::indices::BitmapIndex) -> usize>, alloc::alloc::Global> (self=0x7fffffffcd30, 
    iter=<error reading variable: Cannot access memory at address 0x0>) at /rustc/8c74a5d27c644a0f7a22bb2fa8dd3ff8257bc220/library/alloc/src/vec/spec_extend.rs:17
#8  0x0000555555685ddd in alloc::vec::spec_from_iter_nested::{impl#0}::from_iter<usize, core::iter::adapters::map::Map<hwlocality::bitmaps::BitmapIterator<&hwlocality::bitmaps::Bitmap>, fn(hwlocality::bitmaps::indices::BitmapIndex) -> usize>> (iterator=...)
    at /rustc/8c74a5d27c644a0f7a22bb2fa8dd3ff8257bc220/library/alloc/src/vec/spec_from_iter_nested.rs:43
#9  0x000055555568962d in alloc::vec::spec_from_iter::{impl#0}::from_iter<usize, core::iter::adapters::map::Map<hwlocality::bitmaps::BitmapIterator<&hwlocality::bitmaps::Bitmap>, fn(hwlocality::bitmaps::indices::BitmapIndex) -> usize>> (iterator=<error reading variable: Cannot access memory at address 0x0>)
    at /rustc/8c74a5d27c644a0f7a22bb2fa8dd3ff8257bc220/library/alloc/src/vec/spec_from_iter.rs:33
#10 0x0000555555689575 in alloc::vec::{impl#14}::from_iter<usize, core::iter::adapters::map::Map<hwlocality::bitmaps::BitmapIterator<&hwlocality::bitmaps::Bitmap>, fn(hwlocality::bitmaps::indices::BitmapIndex) -> usize>> (iter=...) at /rustc/8c74a5d27c644a0f7a22bb2fa8dd3ff8257bc220/library/alloc/src/vec/mod.rs:2695
#11 0x000055555561626d in core::iter::traits::iterator::Iterator::collect<core::iter::adapters::map::Map<hwlocality::bitmaps::BitmapIterator<&hwlocality::bitmaps::Bitmap>, fn(hwlocality::bitmaps::indices::BitmapIndex) -> usize>, alloc::vec::Vec<usize, alloc::alloc::Global>> (
    self=<error reading variable: Cannot access memory at address 0x0>)
    at /rustc/8c74a5d27c644a0f7a22bb2fa8dd3ff8257bc220/library/core/src/iter/traits/iterator.rs:1895
#12 0x000055555562a9dd in benchmark::cpuload::example (tp=...) at src/cpuload.rs:88
#13 0x000055555568a7ce in benchmark::main () at src/main.rs:45

Disagreement of membind:set_thisthread_membind

Hi,

I'm trying to bind and migrate all of my thread's memory to a numanode, but I'm getting an unsupported feature error despite my system supporting it.

assertion failed: support::MemoryBindingSupport::default().set_current_thread() == true
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
fatal runtime error: failed to initiate panic, error 5
$ pacman -Q | grep hwloc
hwloc 2.10.0-1

$ hwloc-info --support | grep thisthread_membind
membind:set_thisthread_membind = 1
membind:get_thisthread_membind = 1

How should I go about binding and migrating my thread's memory to the desired numa node?

I'm using main in the above test with the latest feature gate.

hwlocality = { git="https://github.com/HadrienG2/hwlocality", branch="main", features=["hwloc-latest"]}

P.S. - this library is awesome, makes setting up locality a breeze. Thanks for your hard work :>

Expose object distances

Add a musl build to CI

To avoid issues like #114 in the future, I should add a musl build to CI. Hopefully replicating the dockerfile listed there, but with clippy lints and maybe tests on latest hwloc + rustc, should be enough to catch the obvious stuff.

Try to make bundled feature work on macOS, without dynamic linking

The bundled feature is currently hardcoded to error out on macOS (and this combination is disabled in CI) because if I let such a build run through, it will bomb at binary linking time due to the fact that for some unknown reason, the resulting internal hwloc build does not correctly set the -framework flags that it needs in its pkgconfig file.

I do not have a Mac around, so I cannot investigate this on my side. Patches from Mac users are welcome!

Road to v1

While I have commented on several other issues about what's needed before a final v1 release, I just realized that a single issue that centralizes this knowledge would make this info more visible, so here goes.

Before releasing v1.0.0 of hwlocality and expending more energy on advertising this project to the world, I want to...

  • Have tests for all modules.
  • Do a last round of API review to make sure I'm comfortable with stabiizing this API.
    • Error handling in particular could use some extra care, IIRC I still have some panics in places that should return Result.
    • Cross-check that common traits are implemented where sensible/possible for idiomatic ergonomics and add static assertions for that.
  • Add community-oriented docs.
    • Changelog.
    • Contribution guide.
    • Architecture overview.
    • See openSSF best practices checklist for more inspiration.
  • Have more usage examples in the docs.
  • Have a checklist for future releases.

...and I think that's it. Will expand if anything else comes to mind.

For vendored hwloc, allow to build the minimal version

The way configuration of hwloc works by default is that it looks at some of the system libraries (that is my assumption anyway) and based on what it find, requires extra dependencies.

For example in some cases it needs cudart and in others it doesn't.

My use case is to only know CPU topology and to be able to pin threads to a set of CPU cores, so all of the extra stuff just makes it harder to build.

At the same time configuration of hwloc has some options to disable CUDA and other things, which I think might have been helpful to expose. For example build minimal version by default and require vendored-full or something for default build with complete feature-set (just because features in Cargo are additive).

WDYT?

Add support for hwloc v2.10

hwloc tagged their 2.10 release a couple days ago.

The changelog is pretty short, and not super explicit about the new heterogeneous memory stuff, but a quick diff of the include directory suggests that the affected API surface is minimal and will be easy to wrap. The big change is removal of netloc, which hwlocality never supported to begin with.

I'm waiting for openSUSE Tumbleweed to package this release before getting started.

`rustls-tls` instead of OpenSSL for reqwest

Latest version of hwlocality-sys pulls in reqwest, which by default pulls OpenSSL, which means OpenSSL is now an installation requirement, which is a huge dependency with very bad implication in terms of developer experience for Rust developers.

It is possible to opt-in into vendored version of OpenSSL, but reqwest also supports rustls-tls, as an alternative.

Consider disabling reqwest's default features and using rustls-tls feature instead.

Expose Linux specifics and libnuma interop

Expose Linux-specific API functions and libnuma interoperability behind suitable cfg flags with associated doc(cfg)/cfg(doc):

libnuma support should also be gated behind a suitable feature, and will depend on the availability of a minimally working binding.

The fully qualified file name must be less than 260 characters

There is some kind of test test that in my project (when building for a specific target with -Z build-std) exceeds path limit:

C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppBuild.targets(382,5): error MSB3491: Could not write lines to file "hwloc_get_largest_objs_inside_cpuset.dir\Release\hwloc_ge.C408B399.tlog\hwloc_get_largest_objs_inside_cpuset.lastbuildstate". Path: hwloc_get_largest_objs_inside_cpuset.dir\Release\hwloc_ge.C408B399.tlog\hwloc_get_largest_objs_inside_cpuset.lastbuildstate exceeds the OS max path limit. The fully qualified file name must be less than 260 characters. [C:\actions-runner_work\subspace\subspace\target\x86_64-pc-windows-msvc\production\build\hwlocality-sys-aa1805acd0c31c09\out\build\tests\hwloc_get_largest_objs_inside_cpuset.vcxproj]

Do you think it is possible to shorten it?

Also I'm wondering why tests are built when I just use it as a library dependency.

UPD: I have a fix, will submit a bit later.

Find a way around sourceforge blocking automated downloads of pkgconfiglite

Without hacking, windows CI is broken by the fact that pkgconfiglite binary releases are hosted on SourceForge and that host is constantly blocking downloads from github CI nodes.

As a stopgap solution, I'm putting a savage mirror of the binaries as an attachment to this issue: pkg-config-lite-0.28-1_bin-win32.zip.

If someone managed to figure out a way to get around SourceForge's download blocus, suggestions are most welcome!

Update description to emphasize that bundled version exists

I'm working on a software that can't expect user to install hwlock library and the fact that hwlocality has bundled (it is typically called vendored though) version is a big win.

However, I noticed it much later because the first line in prerequisites is:

You will need a system with hwloc >=2.0.0 and associated development packages installed.

I think it should mention right there something like "or optionally vendored version can be used".

Similarly https://crates.io/crates/hwlocality-sys mentions bindings, but not the fact that it has vendored option too.

Without reading cargo features and/or source code it is not possible to discover that, while for me the fact that I can statically link to a recent version of the library is a major benefit of the library comparing to the alternatives.

Aarch64 compilation error

Tried to cross-compile for aarch64 Linux and got this:

#22 108.6    Compiling hwlocality v1.0.0 (https://github.com/HadrienG2/hwlocality?rev=3141847b0a463f38adcf623a2d720931757a38ae#3141847b)
#22 109.0 error[E0308]: mismatched types
#22 109.0    --> /root/.cargo/git/checkouts/hwlocality-e92c49001d785139/3141847/src/ffi/mod.rs:107:22
#22 109.0     |
#22 109.0 107 |             snprintf(buf.as_mut_ptr(), buf.len()),
#22 109.0     |             -------- ^^^^^^^^^^^^^^^^ expected `*mut u8`, found `*mut i8`
#22 109.0     |             |
#22 109.0     |             arguments to this function are incorrect
#22 109.0     |
#22 109.0     = note: expected raw pointer `*mut u8`
#22 109.0                found raw pointer `*mut i8`
#22 109.0 
#22 109.0 error[E0277]: the trait bound `Box<[u8]>: From<Vec<i8>>` is not satisfied
#22 109.0    --> /root/.cargo/git/checkouts/hwlocality-e92c49001d785139/3141847/src/ffi/mod.rs:116:13
#22 109.0     |
#22 109.0 116 |         buf.into()
#22 109.0     |             ^^^^ the trait `From<Vec<i8>>` is not implemented for `Box<[u8]>`
#22 109.0     |
#22 109.0     = help: the following other types implement trait `From<T>`:
#22 109.0               <Box<OsStr> as From<Cow<'_, OsStr>>>
#22 109.0               <Box<OsStr> as From<OsString>>
#22 109.0               <Box<OsStr> as From<&OsStr>>
#22 109.0               <Box<Path> as From<Cow<'_, Path>>>
#22 109.0               <Box<Path> as From<PathBuf>>
#22 109.0               <Box<Path> as From<&Path>>
#22 109.0               <Box<CStr> as From<Cow<'_, CStr>>>
#22 109.0               <Box<CStr> as From<CString>>
#22 109.0             and 18 others
#22 109.0     = note: required for `Vec<i8>` to implement `Into<Box<[u8]>>`
#22 109.0 
#22 109.0 Some errors have detailed explanations: E0277, E0308.
#22 109.0 For more information about an error, try `rustc --explain E0277`.
#22 109.1 error: could not compile `hwlocality` (lib) due to 2 previous errors
#22 109.1 warning: build failed, waiting for other jobs to finish...
#22 ERROR: process "/bin/sh -c /root/.cargo/bin/cargo build         --locked         -Z build-std         --profile $PROFILE         --bin subspace-farmer         --target aarch64-unknown-linux-gnu &&     mv target/*/*/subspace-farmer subspace-farmer &&     rm -rf target" did not complete successfully: exit code: 101
------
 > [stage-0 16/16] RUN     /root/.cargo/bin/cargo build         --locked         -Z build-std         --profile production         --bin subspace-farmer         --target aarch64-unknown-linux-gnu &&     mv target/*/*/subspace-farmer subspace-farmer &&     rm -rf target:
109.0               <Box<Path> as From<&Path>>
109.0               <Box<CStr> as From<Cow<'_, CStr>>>
109.0               <Box<CStr> as From<CString>>
109.0             and 18 others
109.0     = note: required for `Vec<i8>` to implement `Into<Box<[u8]>>`
109.0 
109.0 Some errors have detailed explanations: E0277, E0308.
109.0 For more information about an error, try `rustc --explain E0277`.
109.1 error: could not compile `hwlocality` (lib) due to 2 previous errors
109.1 warning: build failed, waiting for other jobs to finish...

Looks like there are some platform-dependent type?

Use runtime env vars CARGO_CFG_* for probing target, os and flavor in build.rs

build.rs is built for the host system, and the cfg variables are set accordingly, when cross compiling the same information is presented at build script runtime via env vars CARGO_CFG_*.

Cases that may trip:

  • cross compiling from windows to linux
  • cross compiling from linux to windows or macos
  • cross compiling from macos to linux (I took care of it already)

Cross-compilation on macOS fails

Tried to compile on x86-64 macOS for aarch64 and it failed like this:

  configure: error: in `/Users/runner/work/subspace/subspace/target/aarch64-apple-darwin/production/build/hwlocality-sys-5187208e9a266943/out/build':
  configure: error: cannot run C compiled programs.
  If you meant to cross compile, use `--host'.
  See `config.log' for more details
  thread 'main' panicked at /Users/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/autotools-0.2.6/src/lib.rs:781:5:

Not sure if this is hwlocality's fault yet, but this is the first package that failed in the project.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.