juliaparallel / ucx.jl Goto Github PK

View Code? Open in Web Editor NEW

18.0 18.0 7.0 254 KB

License: MIT License

Julia 100.00%

ucx.jl's People

Contributors

Stargazers

Watchers

Forkers

juliatagbot adharshkamath zeta1999 lcw standardgalactic ranocha

ucx.jl's Issues

Support callbacks for requests

See #6

Update UCX_jll to 1.9

x-ref: JuliaPackaging/Yggdrasil#2459

Add bindings for Active Messages

IIUC UCX has support for AM and that is probably the best layer to implement RPC ala Distributed.jl over,
it seems that AM's are not part of the UCP API (despite being advertised as such) and are instead part of the UCT API.

Build UCX with support for ROCM

blocked by JuliaGPU/AMDGPU.jl#6

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

`examples/client_server.jl` returns corrupted data with more than one client

vchuravy@odin ~/s/UCX (master)> julia --project=. examples/client_server.jl test 2

ERROR: LoadError: TaskFailedException:
AssertionError: String(buffer) == data
Stacktrace:
 [1] start_client(::Int64) at /home/vchuravy/src/UCX/examples/client_server.jl:58
 [2] (::var"#5#7"{Int64})() at ./task.jl:356

...and 1 more exception(s).

Build UCX with Shmem support

IIUC UCX has 3 different kinds of shmem support:

knem: https://knem.gitlabpages.inria.fr/
xpmem: https://github.com/hjelmn/xpmem
cma: CMA uses features added in Linux 3.2 and glibc 2.15

knem and xpmem seem to be a tough call for Yggdrasil since they are partially implemented as kernel modules.
cma right now is available for the ppc64le builds, but not the x86_64 builds since those build against glibc 2.12

Active message handler should not call `progress`

If one calls progress during an AM handler, hilarity ensues and everything becomes recursive.

This means if we happen to switch to a different task on the same thread and that task call progress we are in deep trouble.

Add check for recursion and turns this into an error
Add lock to prevent
Can we disable task switch from am handler? Or make task switch an error

Build UCX with CUDA support

Polling vs non polling

Currently UCX.jl uses FileWatching.poll_fd in order to wait on incoming messages, this relies on Julia usage of libuv to allow the watcher task to suspend. This is the preferred approach since it doesn't introduce a Julia task that looks artificially busy to the scheduler.

This does depend on UCP_FEATURE_WAKEUP which currently leads to shared memory not being used by UCX (openucx/ucx#5322)

In symmetric use-cases like tagged messages and streams not using polling is fine, but in asymmetric use-cases like active messages polling is essential, otherwise we will not call progress on the UCXWorker often enough. Even with polling I have seen timing improvements by switching the watcher task into busy waiting (which is bad for the rest of the system).

Alternatives that I thought about:

LibUV timer, but wake ups occur not often enough
LibUV idle276874d -- need to benchmark again.
@threadcall: progress will invoke callbacks into Julia code, can't execute that on a libuv thread

Configuration parsing and setting support

Port selection without using `listenany`

UCX.jl/src/UCX.jl

Lines 276 to 281 in 6a369d8

 # Choose free port 

 if port === nothing || port == 0 

 port_hint = 9000 + (getpid() % 1000) 

 port, sock = listenany(UInt16(port_hint)) 

 close(sock) # FIXME: https://github.com/rapidsai/ucx-py/blob/72552d1dd1d193d1c8ce749171cdd34d64523d53/ucp/core.py#L288-L304 

 end

State what this packages does in the README?

The Julia survey asked about what tools we use for parallelism and it mentioned UCX.jl. As I never heard of it before, I was curious and tried to find out what it is. But this repository doesn't really help with that :-(.

I am guessing that perhaps it is related to this https://github.com/openucx/ucx ?

	# Choose free port
	if port === nothing \|\| port == 0
	port_hint = 9000 + (getpid() % 1000)
	port, sock = listenany(UInt16(port_hint))
	close(sock) # FIXME: https://github.com/rapidsai/ucx-py/blob/72552d1dd1d193d1c8ce749171cdd34d64523d53/ucp/core.py#L288-L304
	end