Comments (14)
navidcy
commented on June 1, 2024
Can you try using the branch ncc/use-julia-v1.9.4 which, despite its original name, uses Julia v1.10.0?
from oceananigans.jl.
navidcy
commented on June 1, 2024
on tartarus with the above-mentioned branch things seem OK
navidcy:Oceananigans.jl/ |ncc/use-julia-v1.9.4 ✓|$ julia-1.10 --project
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.10.0 (2023-12-25)
_/ |\__'_|_|_|\__'_| |
|__/ |
julia> using Oceananigans
[ Info: Oceananigans will use 48 threads
julia> grid = RectilinearGrid(GPU(),
size = (16, 16, 16),
x = (0, 1),
y = (0, 1),
z = (-1, 0),
topology = (Periodic, Periodic, Bounded))
16×16×16 RectilinearGrid{Float64, Periodic, Periodic, Bounded} on GPU with 3×3×3 halo
├── Periodic x ∈ [0.0, 1.0) regularly spaced with Δx=0.0625
├── Periodic y ∈ [0.0, 1.0) regularly spaced with Δy=0.0625
└── Bounded z ∈ [-1.0, 0.0] regularly spaced with Δz=0.0625
julia> model = NonhydrostaticModel(; grid)
NonhydrostaticModel{GPU, RectilinearGrid}(time = 0 seconds, iteration = 0)
├── grid: 16×16×16 RectilinearGrid{Float64, Periodic, Periodic, Bounded} on GPU with 3×3×3 halo
├── timestepper: QuasiAdamsBashforth2TimeStepper
├── tracers: ()
├── closure: Nothing
├── buoyancy: Nothing
└── coriolis: Nothing
julia> u, v, w = model.velocities
NamedTuple with 3 Fields on 16×16×16 RectilinearGrid{Float64, Periodic, Periodic, Bounded} on GPU with 3×3×3 halo:
├── u: 16×16×16 Field{Face, Center, Center} on RectilinearGrid on GPU
├── v: 16×16×16 Field{Center, Face, Center} on RectilinearGrid on GPU
└── w: 16×16×17 Field{Center, Center, Face} on RectilinearGrid on GPU
julia> maximum(u)
0.0
julia> maximum(w)
0.0
julia> maximum(v)
0.0
julia> maximum(abs, u)
0.0
julia> maximum(abs, w)
0.0
julia> maximum(abs, v)
0.0from oceananigans.jl.
navidcy
commented on June 1, 2024
While using main indeed I can reproduce the error above...
navidcy:Oceananigans.jl/ |main ✓|$ julia-1.10 --project
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.10.0 (2023-12-25)
_/ |\__'_|_|_|\__'_| |
|__/ |
julia> using Oceananigans
┌ Warning: The active manifest file has dependencies that were resolved with a different julia version (1.9.3). Unexpected behavior may occur.
└ @ ~/Oceananigans.jl/Manifest.toml:0
┌ Warning: The project dependencies or compat requirements have changed since the manifest was last resolved.
│ It is recommended to `Pkg.resolve()` or consider `Pkg.update()` if necessary.
└ @ Pkg.API ~/julia-1.10/usr/share/julia/stdlib/v1.10/Pkg/src/API.jl:1800
Precompiling Oceananigans
1 dependency successfully precompiled in 21 seconds. 143 already precompiled.
[ Info: Oceananigans will use 48 threads
julia> grid = RectilinearGrid(GPU(),
size = (16, 16, 16),
x = (0, 1),
y = (0, 1),
z = (-1, 0),
topology = (Periodic, Periodic, Bounded))
16×16×16 RectilinearGrid{Float64, Periodic, Periodic, Bounded} on GPU with 3×3×3 halo
├── Periodic x ∈ [0.0, 1.0) regularly spaced with Δx=0.0625
├── Periodic y ∈ [0.0, 1.0) regularly spaced with Δy=0.0625
└── Bounded z ∈ [-1.0, 0.0] regularly spaced with Δz=0.0625
julia> model = NonhydrostaticModel(; grid)
NonhydrostaticModel{GPU, RectilinearGrid}(time = 0 seconds, iteration = 0)
├── grid: 16×16×16 RectilinearGrid{Float64, Periodic, Periodic, Bounded} on GPU with 3×3×3 halo
├── timestepper: QuasiAdamsBashforth2TimeStepper
├── tracers: ()
├── closure: Nothing
├── buoyancy: Nothing
└── coriolis: Nothing
julia> u, v, w = model.velocities
NamedTuple with 3 Fields on 16×16×16 RectilinearGrid{Float64, Periodic, Periodic, Bounded} on GPU with 3×3×3 halo:
├── u: 16×16×16 Field{Face, Center, Center} on RectilinearGrid on GPU
├── v: 16×16×16 Field{Center, Face, Center} on RectilinearGrid on GPU
└── w: 16×16×17 Field{Center, Center, Face} on RectilinearGrid on GPU
julia> maximum(u)
0.0
julia> maximum(w)
0.0
julia> maximum(v)
0.0
julia> maximum(abs, u)
0.0
julia> maximum(abs, w)
ERROR: CUDA error: too many resources requested for launch (code 701, ERROR_LAUNCH_OUT_OF_RESOURCES)
Stacktrace:
[1] throw_api_error(res::CUDA.cudaError_enum)
@ CUDA ~/.julia/packages/CUDA/nbRJk/lib/cudadrv/libcuda.jl:27
[2] check
@ ~/.julia/packages/CUDA/nbRJk/lib/cudadrv/libcuda.jl:34 [inlined]
[3] cuLaunchKernel
@ ~/.julia/packages/CUDA/nbRJk/lib/utils/call.jl:26 [inlined]
[4] (::CUDA.var"#867#868"{Bool, Int64, CUDA.CuStream, CUDA.CuFunction, CUDA.CuDim3, CUDA.CuDim3})(kernelParams::Vector{Ptr{Nothing}})
@ CUDA ~/.julia/packages/CUDA/nbRJk/lib/cudadrv/execution.jl:69
[5] macro expansion
@ ~/.julia/packages/CUDA/nbRJk/lib/cudadrv/execution.jl:33 [inlined]
[6] macro expansion
@ ./none:0 [inlined]
[7] pack_arguments(::CUDA.var"#867#868"{…}, ::CUDA.KernelState, ::CartesianIndices{…}, ::CartesianIndices{…}, ::CUDA.CuDeviceArray{…}, ::Oceananigans.AbstractOperations.ConditionalOperation{…})
@ CUDA ./none:0
[8] launch(f::CUDA.CuFunction, args::Vararg{…}; blocks::Union{…}, threads::Union{…}, cooperative::Bool, shmem::Integer, stream::CUDA.CuStream) where N
@ CUDA ~/.julia/packages/CUDA/nbRJk/lib/cudadrv/execution.jl:62 [inlined]
[9] #872
@ CUDA ~/.julia/packages/CUDA/nbRJk/lib/cudadrv/execution.jl:136 [inlined]
[10] macro expansion
@ CUDA ~/.julia/packages/CUDA/nbRJk/lib/cudadrv/execution.jl:95 [inlined]
[11] macro expansion
@ CUDA ./none:0 [inlined]
[12] convert_arguments
@ CUDA ./none:0 [inlined]
[13] #cudacall#871
@ CUDA ~/.julia/packages/CUDA/nbRJk/lib/cudadrv/execution.jl:135 [inlined]
[14] cudacall
@ CUDA ~/.julia/packages/CUDA/nbRJk/lib/cudadrv/execution.jl:134 [inlined]
[15] macro expansion
@ CUDA ~/.julia/packages/CUDA/nbRJk/src/compiler/execution.jl:223 [inlined]
[16] macro expansion
@ CUDA ./none:0 [inlined]
[17] call(::CUDA.HostKernel{…}, ::typeof(identity), ::typeof(max), ::Nothing, ::CartesianIndices{…}, ::CartesianIndices{…}, ::Val{…}, ::CUDA.CuDeviceArray{…}, ::Oceananigans.AbstractOperations.ConditionalOperation{…}; call_kwargs::@Kwargs{…})
@ CUDA ./none:0
[18] (::CUDA.HostKernel{…})(::Function, ::Vararg{…}; threads::Int64, blocks::Int64, kwargs::@Kwargs{…})
@ CUDA ~/.julia/packages/CUDA/nbRJk/src/compiler/execution.jl:345
[19] macro expansion
@ ~/.julia/packages/CUDA/nbRJk/src/compiler/execution.jl:106 [inlined]
[20] mapreducedim!(f::typeof(identity), op::typeof(max), R::SubArray{…}, A::Oceananigans.AbstractOperations.ConditionalOperation{…}; init::Nothing)
@ CUDA ~/.julia/packages/CUDA/nbRJk/src/mapreduce.jl:271
[21] mapreducedim!(f::typeof(identity), op::typeof(max), R::SubArray{…}, A::Oceananigans.AbstractOperations.ConditionalOperation{…})
@ CUDA ~/.julia/packages/CUDA/nbRJk/src/mapreduce.jl:169
[22] mapreducedim!(f::Function, op::Function, R::SubArray{…}, A::Oceananigans.AbstractOperations.ConditionalOperation{…})
@ GPUArrays ~/.julia/packages/GPUArrays/EZkix/src/host/mapreduce.jl:10
[23] #maximum!#860
@ Base ./reducedim.jl:1034 [inlined]
[24] maximum!(f::Function, r::Field{…}, a::Oceananigans.AbstractOperations.ConditionalOperation{…}; condition::Nothing, mask::Float64, kwargs::@Kwargs{…})
@ Oceananigans.Fields ~/Oceananigans.jl/src/Fields/field.jl:618
[25] maximum(f::Function, c::Field{…}; condition::Nothing, mask::Float64, dims::Function)
@ Oceananigans.Fields ~/Oceananigans.jl/src/Fields/field.jl:648
[26] maximum(f::Function, c::Field{…})
@ Oceananigans.Fields ~/Oceananigans.jl/src/Fields/field.jl:637
[27] top-level scope
@ REPL[9]:1
[28] top-level scope
@ ~/.julia/packages/CUDA/nbRJk/src/initialization.jl:205
Some type information was truncated. Use `show(err)` to see complete types.
julia> maximum(abs, v)
ERROR: CUDA error: too many resources requested for launch (code 701, ERROR_LAUNCH_OUT_OF_RESOURCES)
Stacktrace:
[1] throw_api_error(res::CUDA.cudaError_enum)
@ CUDA ~/.julia/packages/CUDA/nbRJk/lib/cudadrv/libcuda.jl:27
[2] check
@ ~/.julia/packages/CUDA/nbRJk/lib/cudadrv/libcuda.jl:34 [inlined]
[3] cuLaunchKernel
@ ~/.julia/packages/CUDA/nbRJk/lib/utils/call.jl:26 [inlined]
[4] (::CUDA.var"#867#868"{Bool, Int64, CUDA.CuStream, CUDA.CuFunction, CUDA.CuDim3, CUDA.CuDim3})(kernelParams::Vector{Ptr{Nothing}})
@ CUDA ~/.julia/packages/CUDA/nbRJk/lib/cudadrv/execution.jl:69
[5] macro expansion
@ ~/.julia/packages/CUDA/nbRJk/lib/cudadrv/execution.jl:33 [inlined]
[6] macro expansion
@ ./none:0 [inlined]
[7] pack_arguments(::CUDA.var"#867#868"{…}, ::CUDA.KernelState, ::CartesianIndices{…}, ::CartesianIndices{…}, ::CUDA.CuDeviceArray{…}, ::Oceananigans.AbstractOperations.ConditionalOperation{…})
@ CUDA ./none:0
[8] launch(f::CUDA.CuFunction, args::Vararg{…}; blocks::Union{…}, threads::Union{…}, cooperative::Bool, shmem::Integer, stream::CUDA.CuStream) where N
@ CUDA ~/.julia/packages/CUDA/nbRJk/lib/cudadrv/execution.jl:62 [inlined]
[9] #872
@ CUDA ~/.julia/packages/CUDA/nbRJk/lib/cudadrv/execution.jl:136 [inlined]
[10] macro expansion
@ CUDA ~/.julia/packages/CUDA/nbRJk/lib/cudadrv/execution.jl:95 [inlined]
[11] macro expansion
@ CUDA ./none:0 [inlined]
[12] convert_arguments
@ CUDA ./none:0 [inlined]
[13] #cudacall#871
@ CUDA ~/.julia/packages/CUDA/nbRJk/lib/cudadrv/execution.jl:135 [inlined]
[14] cudacall
@ CUDA ~/.julia/packages/CUDA/nbRJk/lib/cudadrv/execution.jl:134 [inlined]
[15] macro expansion
@ CUDA ~/.julia/packages/CUDA/nbRJk/src/compiler/execution.jl:223 [inlined]
[16] macro expansion
@ CUDA ./none:0 [inlined]
[17] call(::CUDA.HostKernel{…}, ::typeof(identity), ::typeof(max), ::Nothing, ::CartesianIndices{…}, ::CartesianIndices{…}, ::Val{…}, ::CUDA.CuDeviceArray{…}, ::Oceananigans.AbstractOperations.ConditionalOperation{…}; call_kwargs::@Kwargs{…})
@ CUDA ./none:0
[18] (::CUDA.HostKernel{…})(::Function, ::Vararg{…}; threads::Int64, blocks::Int64, kwargs::@Kwargs{…})
@ CUDA ~/.julia/packages/CUDA/nbRJk/src/compiler/execution.jl:345
[19] macro expansion
@ ~/.julia/packages/CUDA/nbRJk/src/compiler/execution.jl:106 [inlined]
[20] mapreducedim!(f::typeof(identity), op::typeof(max), R::SubArray{…}, A::Oceananigans.AbstractOperations.ConditionalOperation{…}; init::Nothing)
@ CUDA ~/.julia/packages/CUDA/nbRJk/src/mapreduce.jl:271
[21] mapreducedim!(f::typeof(identity), op::typeof(max), R::SubArray{…}, A::Oceananigans.AbstractOperations.ConditionalOperation{…})
@ CUDA ~/.julia/packages/CUDA/nbRJk/src/mapreduce.jl:169
[22] mapreducedim!(f::Function, op::Function, R::SubArray{…}, A::Oceananigans.AbstractOperations.ConditionalOperation{…})
@ GPUArrays ~/.julia/packages/GPUArrays/EZkix/src/host/mapreduce.jl:10
[23] #maximum!#860
@ Base ./reducedim.jl:1034 [inlined]
[24] maximum!(f::Function, r::Field{…}, a::Oceananigans.AbstractOperations.ConditionalOperation{…}; condition::Nothing, mask::Float64, kwargs::@Kwargs{…})
@ Oceananigans.Fields ~/Oceananigans.jl/src/Fields/field.jl:618
[25] maximum(f::Function, c::Field{…}; condition::Nothing, mask::Float64, dims::Function)
@ Oceananigans.Fields ~/Oceananigans.jl/src/Fields/field.jl:648
[26] maximum(f::Function, c::Field{…})
@ Oceananigans.Fields ~/Oceananigans.jl/src/Fields/field.jl:637
[27] top-level scope
@ REPL[10]:1
[28] top-level scope
@ ~/.julia/packages/CUDA/nbRJk/src/initialization.jl:205
Some type information was truncated. Use `show(err)` to see complete types.That suggests that it's because the package dependencies on main were resolved with Julia v1.9.3.
┌ Warning: The active manifest file has dependencies that were resolved with a different julia version (1.9.3). Unexpected behavior may occur.This issue will be resolved when #3403 is merged.
from oceananigans.jl.
glwagner
commented on June 1, 2024
It looks like the conditional reduction is too heavy for mapreduce. Perhaps @simone-silvestri has ideas to resolve this.
from oceananigans.jl.
simone-silvestri
commented on June 1, 2024
The operation should not be too large since the grid is very small. Probably this is a symptom of a bug that does not affect the results but results in a waste of computational resources somewhere in conditional operation. I ll have a look
from oceananigans.jl.
glwagner
commented on June 1, 2024
I think the size dependence has to do with how mapreduce works; it breaks the reduction into chunks and (10, 10, 10) might be just one chunk.
from oceananigans.jl.
glwagner
commented on June 1, 2024
from oceananigans.jl.
josuemtzmo
commented on June 1, 2024
I also had this issue, as new into GPU running, I was super confused about this error. It will be helpful if this issue is not fixable, to at least point out in the documentation.
I encountered this error by running a simulation based on the tutorial (Langmuir turbulence) in GPUs. Note that the print function prints the maximum(abs, u), maximum(abs, v), maximum(abs, w):
msg = @sprintf("i: %04d, t: %s, Δt: %s, umax = (%.1e, %.1e, %.1e) ms⁻¹, wall time: %s\n",
iteration(simulation),
prettytime(time(simulation)),
prettytime(simulation.Δt),
maximum(abs, u), maximum(abs, v), maximum(abs, w),
prettytime(simulation.run_wall_time))
thus resulting in the error:
LoadError: CUDA error: too many resources requested for launch
For reference, the code works once the maximum functions are removed:
msg = @sprintf("i: %04d, t: %s, �~Tt: %s, wall time: %s\n",
iteration(simulation),
prettytime(time(simulation)),
prettytime(simulation.�~Tt),
prettytime(simulation.run_wall_time))
from oceananigans.jl.
navidcy
commented on June 1, 2024
reopening this
from oceananigans.jl.
glwagner
commented on June 1, 2024
@simone-silvestri has declared an interest in fixing this
from oceananigans.jl.
simone-silvestri
commented on June 1, 2024
can you try maximum without abs?
from oceananigans.jl.
glwagner
commented on June 1, 2024
I think its the abs (probably any function) that's the main issue
from oceananigans.jl.
josuemtzmo
commented on June 1, 2024
@simone-silvestri, effectively if I try maximum without abs the printing function works well. @glwagner is right, any function within the maximum creates the same issue (I tested with sum).
from oceananigans.jl.
glwagner
commented on June 1, 2024
Well sum definitely won't work (it has to be a simple single-argument transformation) but you could try a function like
square(x) = x * xor log if you want to be adventurous
from oceananigans.jl.
Related Issues (20)
- `bottom_heights` from the immersed boundary method don't look right with a coarse grid HOT 1
- 2D turbulence example is outputting `NaNs` HOT 1
- Doing better than `overwriting_existing=true` with output? HOT 5
- Lagrangian particles on Flat topology HOT 3
- `ImmersedPoissonSolver` is slow HOT 10
- `@eval` considered harmful HOT 3
- Noise below the mixed layer when using `AnisotropicMinimumDissipation` HOT 7
- `Clock` and `QuasiAdamsBashforth2` both have a property representing the previous time-step
- What's the point of neglecting the computation of tendencies when taking a time-step?
- Boundary conditions and diffusion with background fields HOT 17
- Bug in determination of indices of parent array by `parent_index_range` HOT 14
- Broadcasting over regions for `MultiRegionField`s HOT 2
- Forcing functions that depend on internal model fields besides velocities and tracers HOT 18
- Typo in docs on callbacks
- Call it "Simulation setup" in the docs, not "Model setup"
- `BackgroundField` is a confusing name HOT 2
- Derivatives of a reduced field over an Immersed boundary HOT 5
- Pressure has extremely high gradients in random chunks of simulation using `NonhydrostaticModel` with `ImmersedBoundaryGrid` and `BuoyancyTracer` HOT 17
- Simulation tips may be a little misleading
- Lagrangian_particles (tracked_fields and using immersed boundaries) HOT 9
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from oceananigans.jl.