juliaimages / imagetransformations.jl Goto Github PK
View Code? Open in Web Editor NEWGeometric transformations on images for Julia
License: Other
Geometric transformations on images for Julia
License: Other
Observed from c09f576
It seems that JuliaArrays/OffsetArrays.jl#213 breaks this:
using ImageTransformations
using OffsetArrays
using Test
A = reshape(collect(1:60), 5, 4, 3)
Ao = OffsetArray(A, (-2,1,0))
@inferred(restrict(Ao, 1)) # inferred Any
Yet I haven't investigated where the patch should be put.
cc: @jishnub
Is there a way to get the restrict function with faster load times?
julia> @time using ImageTransformations: restrict
3.119409 seconds (6.69 M allocations: 376.407 MiB, 4.46% gc time)
julia> exit()
...
julia> @time using ImageTransformations: restrict
3.212386 seconds (6.69 M allocations: 376.411 MiB, 4.51% gc time)
It looks like the tests fail for new some reason. (I have yet to look into it)
autorange: Error During Test
Got an exception of type MethodError outside of a @test
MethodError: Rotations.RotMatrix{2,Float64,4}(::StaticArrays.SArray{Tuple{2,2},Float64,2,4}) is ambiguous. Candidates:
(::Type{Rotations.RotMatrix{N,T,L}})(x::AbstractArray) where {N, T, L} in Rotations at /home/csto/.julia/v0.6/Rotations/src/core_types.jl:78
(::Type{SA})(a::StaticArrays.StaticArray) where SA<:StaticArrays.StaticArray in StaticArrays at /home/csto/.julia/v0.6/StaticArrays/src/convert.jl:4
Possible fix, define
(::Type{Rotations.RotMatrix{N,T,L}})(::StaticArrays.StaticArray)
Stacktrace:
[1] macro expansion at /home/csto/.julia/v0.6/ImageTransformations/test/autorange.jl:108 [inlined]
I am interested to know if warp function defined in this repository work similar to the _warp_fast function available in scipy for homography ?
img = rand(RGB{N0f8}, 1000, 1000)
img2 = imrotate(img, π/2)
size(img) # (1000, 1000)
size(img2) # (1002, 1002)
I realize it's pretty naive to just ask "why doesn't this work as expected?". I get the sense that "simply" rotating an image is not so simple. (I know very little about image processing)
I'm wondering how I should get the desired result for rotations where the degree is multiples of pi/2
. This might be related to #119 although I'm not sure.
Would it make sense to just define a new imrotate specifically for these factors of pi/2
?
I warp the same source image using Kornia and ImageTransformations, using the same homography matrix. I use a slightly modified function warp_me2
, and the whole thing can be reproduced using this repo and running the mytests.jl
file in it.
This is the image that I get in Kornia, which is actually very accurate -
This is the image I get after warping it using the warp_me2
function -
I've tried checking the individual indices to which it get warped, and their values, but they were all looking correct. I don't know why am I getting an issue here.
Any suggestions as to what is going wrong? Can someone checkout if they are facing the same issues?
The Kornia notebook can be accessed here. The required files are all present in the reproduction repository above.
Some serious memory allocation and regression in Julia 0.7. Here is a MWE.
julia> img = rand(100, 100);
julia> tfm = recenter(RotMatrix(-pi/4), center(img));
In Julia 0.6,
julia> @btime warp(img, tfm);
422.876 μs (5 allocations: 157.72 KiB)
In Julia 0.7,
julia> @btime warp(img, tfm);
8.478 ms (261626 allocations: 9.07 MiB)
Looking at the code of warp!
,
function warp!(out, img::AbstractExtrapolation, tform)
@inbounds for I in CartesianIndices(axes(out))
out[I] = _getindex(img, tform(SVector(I.I)))
end
out <==== should not return anything
end
Obviously, warp!
should not return out
. But eliminating the return only helps marginally. But I cannot dig any further. Might be some problem with _getindex
.
This is added to our documentation as an example, but it would be nice to have it as a standard function in ImageTransformations.
It would be great if someone adds interpolation_type argument in imresize
function as in PIL package of Python.
Is it possible/recommended to add an overload to warp that takes a lookup table instead of a transform. The functionality would be similar to OpenCVs remap found here.
I'm trying to convert a RGB{N0f8}
image to the YUV420 format for ffmpeg. (YUV420 is YCbCr, where the Cb and Cr channels are half resolution)
When trying to use restrict to downsample YCbCr{Float64} the result is RGB{Float64}.
img = rand(RGB{N0f8},100,200)
img_YCbCr = convert.(YCbCr{Float64}, img)
@show typeof(img_YCbCr) #Array{YCbCr{Float64},2}
img_YCbCr_half = restrict(img_YCbCr)
@show typeof(img_YCbCr_half) #Array{RGB{Float64},2}
Also, is there a better way to do RGB{N0f8} -> YUV420 (UInt8)
I'm getting a surprising result when calling warp
with an IdentityTransformation
. Is this a bug?
using Images, CoordinateTransformations, Interpolations
img = rand(4,8)
img[:,1:2] = NaN
img[:,7:8] = NaN
imgw = warp(img, IdentityTransformation(), indices(img), Linear(), Flat())
@show img
#Notice that column 6 gets set to NaN
@show parent(imgw)
Note that this doesn't happen when Constant()
interpolation is used for the degree. As for the fill parameter, Flat()
and NaN
give the same result.
reported by @johnnychen94 at JuliaImages/Images.jl#742
The document of
imresize
doesn't say sz must beTuple
type, so I just try the MATLAB way of usingimresize
, and it throws an unexpected StackOverflowError.using Images, TestImages lena = testimage("lena_gray_256") imresize(lena, (128,128)) # work as expected imresize(lena, [128,128]) # throw StackOverflowError
The merge of JuliaLang/METADATA.jl#11972 will break some existing files
The CoordinateTransform compat bump on master would be good to release
It's better to have Projective
transformation to ImageTransformations.jl
.
# Definition of projective transformation
"""
Projective transformation.
"""
struct Projective <: Transformation
H::SArray{Tuple{3,3},Float64,2,9}
end
"""
Definition of projective transformation.
"""
function (p::Projective)(x)
x_ = SA[x[1],x[2],1.0]
x′_1, x′_2, x′_3 = p.H*x_
return SA[x′_1/x′_3, x′_2/x′_3]
end
"""
Inverse of projective transformation.
"""
function Base.inv(p::Projective)
Projective(inv(p.H))
end
Discussions in Slack:
https://julialang.slack.com/archives/CB1R90P8R/p1626929588093000
https://julialang.slack.com/archives/CB1R90P8R/p1626983453103200
I feel the right place to add this is in CoordinateTransformations.jl
Or we could 1) first introduce a high-level API in ImageTransformations.jl with this Projective as internal type, and then 2) migrate it to upstream CoordinateTransformations, and finally 3) use the upstream version.
(The quoted words are by @johnnychen94)
The error below occurs when tfm
is not based on a StaticArray
. It's a one-line fix, I'll do a PR shortly.
ImageTransformations.autorange(h, tfm)
ERROR: MethodError: Cannot `convert` an object of type Array{Float64,1} to an object of type Tuple
This may have arisen from a call to the constructor Tuple(...),
since type constructors fall back to convert methods.
Stacktrace:
[1] autorange(::CartesianRange{CartesianIndex{2}}, ::CoordinateTransformations.AffineMap{Array{Float64,2},Array{Float64,1}}) at /home/cody/.julia/v0.6/ImageTransformations/src/autorange.jl:16
[2] autorange(::OffsetArrays.OffsetArray{ColorTypes.Gray{FixedPointNumbers.Normed{UInt8,8}},2,Array{ColorTypes.Gray{FixedPointNumbers.Normed{UInt8,8}},2}}, ::CoordinateTransformations.AffineMap{Array{Float64,2},Array{Float64,1}}) at /home/cody/.julia/v0.6/ImageTransformations/src/autorange.jl:3
The new warp
function is bit different than the old transform
from AffineTransforms
. Suppose I have an array and I want to translate the array by (x, y)
and fill the moved out rows and columns with 0
. How would I do that?
If I want to translate and rotate the array, and keep the same size as the original array and fill with 0
, how should I use warp
?
Thanks!
is there something that performs this transformation?
[1 0; 0 0] -> [1 1 0 0; 1 1 0 0; 0 0 0 0; 0 0 0 0]
when I try to imresize(img, ratio=100)
it seems to be interpolating but I just want to make the image bigger without degrading the pixels.
thanks
I am trying to use ForwardDiff to compute the gradient of a cost function defined as the Sum of Square Differences (SSD) between two images with respect to the parameters of an image transformation. In the code shown below, one of the images has been translated by a small amount. This is basically just a toy problem that I designed to understand if ForwardDiff.jl and ImageTransformations.jl play well together. Eventually, I will compute derivatives relative the parameters of different image warps. Notice that I do not want to compute gradients of the pixel intensities, but, say, the 2x2 Jacobian of the translation, like:
using Images
using TestImages
using ImageTransformations
using ForwardDiff
img1 = testimage("cameraman.tif")
y_shift = 0.33
x_shift = -1.76
t = ImageTransformations.Translation(y_shift, x_shift)
img2 = ImageTransformations.warp(img1, t, ImageTransformations.indices_spatial(img1), 0)
function cost(Δ::Vector{T}) where T <: Real
t = ImageTransformations.Translation(Δ[2], Δ[1])
img2 = ImageTransformations.warp(img1, t, ImageTransformations.indices_spatial(img1), 0)
imgg1 = Gray.(img1)
imgg2 = Gray.(img2)
mat1 = convert(Array{Float64}, imgg1)
mat2 = convert(Array{Float64}, imgg2)
@. mat1 = (mat1 - mat2)^2
SSD = sum(vec(mat1))
return SSD
end
I’m getting this error:
StackOverflowError:
Stacktrace:
[1] make_typealias(x::Type)
@ Base ./show.jl:531
[2] show_typealias(io::IOBuffer, x::Type)
@ Base ./show.jl:661
[3] show(io::IOBuffer, x::Type)
@ Base ./show.jl:816
[4] show_datatype(io::IOBuffer, x::DataType)
@ Base ./show.jl:928
[5] show(io::IOBuffer, x::Type)
@ Base ./show.jl:819
[6] print(io::IOBuffer, x::Type)
@ Base ./strings/io.jl:35
[7] print_to_string(::String, ::Vararg{Any, N} where N)
@ Base ./strings/io.jl:135
[8] string
@ ./strings/io.jl:174 [inlined]
[9] floattype
@ ~/.julia/packages/FixedPointNumbers/HAGk2/src/deprecations.jl:4 [inlined]
[10] _default_digits(#unused#::Type{ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}}) (repeats 43235 times)
@ ImageTransformations ~/.julia/packages/ImageTransformations/xYRLH/src/autorange.jl:82
[11] __round(x::StaticArrays.SVector{2, ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}})
@ ImageTransformations ~/.julia/packages/ImageTransformations/xYRLH/src/autorange.jl:100
[12] (::ImageTransformations.var"#6#7"{Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, CoordinateTransformations.Translation{StaticArrays.SVector{2, ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}}}})(i::Int64)
@ ImageTransformations ~/.julia/packages/ImageTransformations/xYRLH/src/autorange.jl:90
[13] iterate
@ ./generator.jl:47 [inlined]
[14] _collect
@ ./array.jl:691 [inlined]
[15] collect_similar
@ ./array.jl:606 [inlined]
[16] map
@ ./abstractarray.jl:2294 [inlined]
[17] _round(tform::CoordinateTransformations.Translation{StaticArrays.SVector{2, ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}}}; kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ ImageTransformations ~/.julia/packages/ImageTransformations/xYRLH/src/autorange.jl:89
[18] _round(tform::CoordinateTransformations.Translation{StaticArrays.SVector{2, ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}}})
@ ImageTransformations ~/.julia/packages/ImageTransformations/xYRLH/src/autorange.jl:89
[19] warp!(out::Matrix{Gray{N0f8}}, img::Interpolations.FilledExtrapolation{Gray{N0f8}, 2, Interpolations.BSplineInterpolation{Gray{N0f8}, 2, Matrix{Gray{N0f8}}, Interpolations.BSpline{Interpolations.Linear}, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}}, Interpolations.BSpline{Interpolations.Linear}, Gray{N0f8}}, tform::CoordinateTransformations.Translation{StaticArrays.SVector{2, ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}}})
@ ImageTransformations ~/.julia/packages/ImageTransformations/xYRLH/src/warp.jl:92
[20] warp(img::Interpolations.FilledExtrapolation{Gray{N0f8}, 2, Interpolations.BSplineInterpolation{Gray{N0f8}, 2, Matrix{Gray{N0f8}}, Interpolations.BSpline{Interpolations.Linear}, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}}, Interpolations.BSpline{Interpolations.Linear}, Gray{N0f8}}, tform::CoordinateTransformations.Translation{StaticArrays.SVector{2, ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}}}, inds::Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}})
@ ImageTransformations ~/.julia/packages/ImageTransformations/xYRLH/src/warp.jl:88
[21] warp(img::Matrix{Gray{N0f8}}, tform::CoordinateTransformations.Translation{StaticArrays.SVector{2, ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}}}, inds::Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}, args::Int64; kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ ImageTransformations ~/.julia/packages/ImageTransformations/xYRLH/src/warp.jl:101
[22] warp(img::Matrix{Gray{N0f8}}, tform::CoordinateTransformations.Translation{StaticArrays.SVector{2, ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}}}, inds::Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}, args::Int64)
@ ImageTransformations ~/.julia/packages/ImageTransformations/xYRLH/src/warp.jl:100
[23] cost(Δ::Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}})
@ Main ./In[36]:18
[24] vector_mode_dual_eval(f::typeof(cost), x::Vector{Float64}, cfg::ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2, Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}}})
@ ForwardDiff ~/.julia/packages/ForwardDiff/QOqCN/src/apiutils.jl:37
[25] vector_mode_gradient(f::typeof(cost), x::Vector{Float64}, cfg::ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2, Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}}})
@ ForwardDiff ~/.julia/packages/ForwardDiff/QOqCN/src/gradient.jl:106
[26] gradient(f::Function, x::Vector{Float64}, cfg::ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2, Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}}}, ::Val{true})
@ ForwardDiff ~/.julia/packages/ForwardDiff/QOqCN/src/gradient.jl:19
[27] gradient(f::Function, x::Vector{Float64}, cfg::ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2, Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}}}) (repeats 2 times)
@ ForwardDiff ~/.julia/packages/ForwardDiff/QOqCN/src/gradient.jl:17
[28] top-level scope
@ In[37]:1
[29] eval
@ ./boot.jl:360 [inlined]
[30] include_string(mapexpr::typeof(REPL.softscope), mod::Module, code::String, filename::String)
@ Base ./loading.jl:1094
Any ideas?
I was trying to implement more complicated interpolation options in imrotate, but seem to be restricted to passing arguments accepted by the BSpline interpolation class. Is it possible to use other interpolation classes like below?
using Interpolations imrotate(img,1.2,Lanczos4OpenCV())
(this produces an error message)
In warp.jl a docstring example for imrotate
says:
# rotate with bilinear interpolation and with cropping
julia> imrotate(img, π/4, axes(img)
This works, but is confusing for two reasons. First, the option to put axes into the arguments isn't in the usage line of the docstring. Second, it fails when specifying degree and/or extrapolation other than the defaults. A more robust alternative is
imrotate(img, π/4)[axes(img)...]
Also, the references to Constant()
, etc. and Flat()
, etc. are unknown unless the user has imported Interpolations.jl
. Maybe clarify, or just reexport that?
Since the introduction of _round
(commit a4f2190), warp
(or rather autorange
) only accepts <:CoordinateTransformations.Transformation
. Is there any way we can allow for more generic transformations? I, for one, use Interpolations
to describe more complicated transformations (from camera calibrations), which now do not work.
Of course, one alternative would be to increase the kind of transformations CoordinateTransformations
can produce.
Thank you for this awesome package!
I'm auto-tracking animal movements in videos. I'm using VideoIO.jl
to extract frames from the video, resize them (uniform scale and fixing the Storage Aspect Ratio of the video), and then save a diagnostic short video of the results.
I'm using imresize
to do that, but can't help thinking that there must be a way to save some of the computations if the transform is identical for each frame: the original images as well as the resized images are always the same size. Is there a way to calculate the transformation once, and then apply it multiple times to each of the frames?
using ImageTransformations
using Colors
@show imresize([ARGB{N0f8}(1,1,1),ARGB{N0f8}(1,1,1)], ratio=2)
@show imresize([ARGB32(1,1,1),ARGB32(1,1,1)], ratio=2)
ARGB{N0f8} <: AlphaColor, ARGB32 <: AlphaColor
imresize([ARGB{N0f8}(1, 1, 1), ARGB{N0f8}(1, 1, 1)], ratio = 2) = ARGB{Normed{UInt8,8}}[ARGB{N0f8}(1.0,1.0,1.0,1.0), ARGB{N0f8}(1.0,1.0,1.0,1.0), ARGB{N0f8}(1.0,1.0,1.0,1.0), ARGB{N0f8}(1.0,1.0,1.0,1.0)]
TypeError: in Type{...} expression, expected UnionAll, got Type{ARGB32}
Stacktrace:
[1] imresize_type(::ARGB32) at /mnt/cache/qaz/.julia/packages/ImageTransformations/J3Mno/src/resizing.jl:309
[2] imresize(::Array{ARGB32,1}, ::Tuple{Int64}) at /mnt/cache/qaz/.julia/packages/ImageTransformations/J3Mno/src/resizing.jl:281
......
Probably a good idea. Surely someone at METADATA will point out the lack of travis testing :)
This difference is originally identified by @andrew-saydjari in #116 (comment) and I believe we could optimize it away:
img = testimage("cameraman")
imrotate(img, 0.3; method=Linear()) == imrotate(img, 0.3; method=BSpline(Linear())) # true
@btime imrotate($img, 0.3; method=Linear()); # 3.479 ms (21 allocations: 403.88 KiB)
@btime imrotate($img, 0.3; method=BSpline(Linear())); # 3.491 ms (23 allocations: 660.02 KiB)
Since the reference test code is duplicated among three packages by now I made a little stand alone package for the reference tests. Once registered in METADATA I plan to replace the old manual code, which includes this package.
https://github.com/Evizero/ReferenceTests.jl
Unless there are objections of course. That said this will only be a test/REQUIRE
dependency, and not really affect the actual package code
Right now imresize
performs something that is pretty similar to simple pixel sub-sampling. Naturally the quality of the result isn't particularly breathtaking.
I open this issue to brainstorm on what would be a good way to improve the quality for downsizing and image to an arbitrary specified size.
The straight forward way of doing it would be in two steps: 1.) Filter image, 2.) sub-sample. However, this would add an additional temporary array that I suspect could probably be avoided.
Would be useful
Hi,
I have a Python script in which there is a call to scipy.ndimage.rotate
for rotating a 2D array, similar to the following
using PyCall
ndimage = pyimport("scipy.ndimage")
# test array
img = Float64[1 2 3 4; 5 6 7 8; 9 10 11 12; 13 14 15 16]
img_90 = ndimage.rotate(img, 90, reshape=false)
I came across this package when searching for the same kind of operation on a plain 2D array in Julia. However, soon I got confused by the rotation methods provided here, especially imrotate
.
using ImageTransformations
using Rotations: RotMatrix
using CoordinateTransformations: recenter
# test array
img = Float64[1 2 3 4; 5 6 7 8; 9 10 11 12; 13 14 15 16]
# define transformation
trfm = recenter(RotMatrix(pi/2), center(img))
imgw = warp(img, trfm) # this is what I expected, same as in scipy
imgr = imrotate(img, pi/2) # this gives a 6x6 OffsetArray with values that I don't quite understand
where imrotate
returned
6×6 OffsetArray(::Matrix{Float64}, 0:5, 0:5) with eltype Float64 with indices 0:5×0:5:
NaN NaN NaN NaN NaN NaN
NaN NaN NaN 4.99995 NaN NaN
NaN 14.0 9.99998 5.99999 NaN NaN
NaN NaN 11.0 7.00002 3.00003 NaN
NaN NaN 12.0 NaN NaN NaN
NaN NaN NaN NaN NaN NaN
I tried the example given in the doc for imrotate
, and it looked just fine.
Can you kindly point out what I missed here? What is the closest possible translation from scipy.ndimage.rotate
?
I guess the following example illustrates the problem quite well.
julia> restrict([0 1 0][:])
2-element Array{Float64,1}:
0.25
0.25
julia> restrict([0 1 0])
1×2 Array{Float64,2}:
0.25 1.75543e13
While this is obviously not the intended use case, it happened to me accidentally not realizing that I should use a Vector instead of a 2D Array.
In #24 we began discussing a possible breaking change: inverting the "sense" in which a transformation is used/applied. Currently pixel values are computed as img[inv(tfm)(x)]
, but perhaps we should consider switching to img[tfm(x)]
. For a rotation, the former can be thought of as rotating the image, whereas the latter is rotating the coordinate axes. Let me acknowledge that I was the one who established the current convention; if we decide that the current system is a mess (I fear it may be), you know who to blame...
At the outset, let's acknowledge that from a standpoint of pure mathematics, either choice is viable. This is because (in my opinion) we should focus on diffeomorphisms and all diffeomorphisms are invertible. However, from a practical standpoint some are easier to invert than others. For example, affine transformations are easy to invert, but diffeomorphisms like this one are less straightforward. The latter might arise from a deformable image registration algorithm, so I think we have to take such seriously.
So, let me make the case for switching. Note here that I'm assuming that the goal of registration is to find a diffeomorphism ϕ
such that fixed[x] ≈ moving[ϕ(x)]
; if someone has a preferred framework for thinking about registration, now is a really, really good time to mention it.
For the function warp
in general you need both the forward and inverse, because (given that framework above)
warped[x] = moving[ϕ(x)]
and consequently warped
should be defined for all x
such that ϕ(x)
is within the domain of moving
, i.e., the bounding box of ϕ⁻¹(y)
for all y
within the domain of moving
. Since we need both the forward and the reverse, again it doesn't make a great deal of difference which we choose, with one exception: given that ϕ
is a diffeomorphism, we can compute that bounding box using only the edges of moving
, and so consequently the inversion only needs to be computed for a small fraction of points. If the inversion is computationally expensive (e.g., invokes a nonlinear solver), then it would be a win to supply ϕ
rather than ϕ⁻¹
However, for image registration we are likely to only care about warped
over the domain of fixed
, in which case we can allocate an empty warped
and then call warp!(warped, moving, ϕ)
. In this case, the behavior should be to iterate over all entries in warped
, in which case we don't even need ϕ⁻¹
. In this case, imposing the requirement that we can (and do) compute ϕ⁻¹
seems like an unnecessary burden.
I know it's just reverse(img;dim=2) in 2 different dims but would be nice to have it.
Mentioned here: https://julialang.slack.com/archives/CB1R90P8R/p1671316725407619
I think it makes sense to introduce the concept of fixed point to better enhance the imresize
function.
A fixedpoint p
is where imgr[p] == img[p]
holds.
Three new methods will be introduced as new API:
const FixedPointType = Union{Dims, CartesianIndex}
imresize(img, sz, p::FixedPointType)
imresize(img, inds, p::FixedPointType)
imresize(img, p::FixedPointType; ratio)
There are many use cases of this, I'll just list one thing that made me propose the idea:
With this, we can improve (change) the behavior to OffsetArray
. Currently, when the input is an OffsetArray
:
using OffsetArray
x = OffsetArray(rand(5, 5), -3, -3)
imresize(x, (-5:5, -5:5)) |> axes # (-5:5, -5:5)
imresize(x, (10, 10)) |> axes # (1:10, 1:10)
imresize(x; ratio=2) |> axes # (1:10, 1:10)
This loss the axes information of our OffsetArray
, we could change the behavior of it to
imresize(img::OffsetArray, sz::Dims) = imresize(img, sz, topleft(img))
imresize(img::OffsetArray; ratio) = imresize(img, topleft(img); ratio)
Use top left point as the default value because it is consistent with the case that the fixed point for imresize(img::Array, sz)
is also the topleft. (1-based indexing).
Happens with julia v1.0.1. Digging through Base, it looks like IdentityUnitRange was introduced in 1.1. Is my only option to update my julia install?
Stacktrace:
julia> using ImageTransformations
[ Info: Precompiling ImageTransformations [02fcd773-0e25-5acc-982a-7f6622650795]
ERROR: LoadError: LoadError: UndefVarError: IdentityUnitRange not defined
Stacktrace:
[1] getproperty(::Module, ::Symbol) at ./sysimg.jl:13
[2] top-level scope at none:0
[3] include at ./boot.jl:317 [inlined]
[4] include_relative(::Module, ::String) at ./loading.jl:1041
[5] include at ./sysimg.jl:29 [inlined]
[6] include(::String) at /Users/tomer/.julia/packages/ImageTransformations/rVbxR/src/ImageTransformations.jl:1
[7] top-level scope at none:0
[8] include at ./boot.jl:317 [inlined]
[9] include_relative(::Module, ::String) at ./loading.jl:1041
[10] include(::Module, ::String) at ./sysimg.jl:29
[11] top-level scope at none:2
[12] eval at ./boot.jl:319 [inlined]
[13] eval(::Expr) at ./client.jl:389
[14] top-level scope at ./none:3
in expression starting at /Users/tomer/.julia/packages/ImageTransformations/rVbxR/src/resizing.jl:158
in expression starting at /Users/tomer/.julia/packages/ImageTransformations/rVbxR/src/ImageTransformations.jl:30
ERROR: Failed to precompile ImageTransformations [02fcd773-0e25-5acc-982a-7f6622650795] to /Users/tomer/.julia/compiled/v1.0/ImageTransformations/Pfac4.ji.
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] macro expansion at ./logging.jl:313 [inlined]
[3] compilecache(::Base.PkgId, ::String) at ./loading.jl:1187
[4] macro expansion at ./logging.jl:311 [inlined]
[5] _require(::Base.PkgId) at ./loading.jl:944
[6] require(::Base.PkgId) at ./loading.jl:855
[7] macro expansion at ./logging.jl:311 [inlined]
[8] require(::Module, ::Symbol) at ./loading.jl:837
If I understand it right, then this package would have a lot of overlap with https://github.com/timholy/AffineTransforms.jl.
Am I correct to assume that this package is planned to be a reboot of sorts, making use of 0.5 goodies and specialized backends such as CoordinateTransformations?
ImageTransformations doesn't build the transformation matrix on its own, instead, it uses CoordinateTransformations and Rotations to do it.
imrotate
is an example of this, and we should probably document this so that users of ImageTransformations knows where to find it.
This issue is used to trigger TagBot; feel free to unsubscribe.
If you haven't already, you should update your TagBot.yml
to include issue comment triggers.
Please see this post on Discourse for instructions and more details.
If you'd like for me to do this for you, comment TagBot fix
on this issue.
I'll open a PR within a few hours, please be patient!
https://juliaimages.org/ImageTransformations.jl/stable returns 404.
We can add the old documentation by adding tags with metadata such as v0.8.12+doc1
.
cf. https://juliadocs.github.io/Documenter.jl/stable/man/hosting/#Documentation-Versions
The warped image size is vulnerable to numerical stability. For the purpose of image processing, unstable size is really a headache.
julia> using Rotations
julia> tfm = RotMatrix(0.1)
2×2 RotMatrix2{Float64} with indices SOneTo(2)×SOneTo(2):
0.995004 -0.0998334
0.0998334 0.995004
julia> inv(tfm) * tfm
2×2 RotMatrix2{Float64} with indices SOneTo(2)×SOneTo(2):
1.0 0.0
0.0 1.0
julia> inv(tfm) * tfm
2×2 RotMatrix2{Float64} with indices SOneTo(2)×SOneTo(2):
1.0 -6.47893e-18
6.47893e-18 1.0
This breaks the current test:
img_camera = testimage("camera"); # (512, 512)
tfm = recenter(RotMatrix(-pi/8), center(img_camera))
wv3 = InvWarpedView(img_camera, inv(tfm) ∘ tfm); # (513, 512)
One strategy to work around this is to use the same discretization trick in imrotate
:
ImageTransformations.jl/src/warp.jl
Line 128 in 560ab77
(ImageTransformations) pkg> st
Project ImageTransformations v0.8.7
Status `~/Documents/Julia/ImageTransformations.jl/Project.toml`
[13072b0f] AxisAlgorithms v1.0.0
[c3611d14] ColorVectorSpace v0.8.7
[150eb455] CoordinateTransformations v0.6.1
[bbac6d45] IdentityRanges v0.3.1
[a09fc81d] ImageCore v0.8.19
[a98d9a8b] Interpolations v0.13.1
[6fe1bfb0] OffsetArrays v1.4.1
[6038ab10] Rotations v1.0.2
[90137ffa] StaticArrays v1.0.1
cc: @c42f
In the docstring of imresize
:
This interpolates the values at sub-pixel locations. If you are shrinking the image, you risk aliasing unless you low-pass filter
img
first.
Yet we don't have a good showcase to tell potential users how this can be done.
Typically, we just apply a low-pass Gaussian filter to it. However, the example in the docstring is not very illustrative as it creates a over-blurred result:
using ImageFiltering
σ = map((o,n)->0.75*o/n, size(img), sz)
kern = KernelFactors.gaussian(σ)
imgr = imresize(imfilter(img, kern, NA()), sz)
Mind if we tag a new version with the current state? Could use it to make CI pass on a project I am working on and I am pressed on time a little
I'm trying to understand and use the image transformation method described in http://juliaimages.github.io/latest/indexing.html
I'm trying to apply the same rotation as described in the above doc:
using Images, ImageView, TestImages, ImageTransformations, CoordinateTransformations
img = testimage("mandrill")
imgg = convert(Image{Gray}, img)
tfm = recenter(RotMatrix(pi/8), center(imgg))
imgrot = warp(imgg, tfm);
img0 = similar(imgrot);
fill!(img0, 0);
img0[1:512, 1:512] = img;
imgov = colorview(RGB, img0, imgrot, zeroarray)
although this works, when I try to display the image using
imshow(imgov)
I get the following error:
ERROR: BoundsError: attempt to access -78:591×-78:591 ColorView{RGB}(::ImageCore.StackedView{Float64,3,Tuple{MappedArrays.MappedArray{Float64,2,OffsetArrays.OffsetArray{ColorTypes.Gray{Float64},2,Array{ColorTypes.Gray{Float64},2}},MappedArrays.##1#3{Float64},MappedArrays.##2#4{ColorTypes.Gray{Float64}}},MappedArrays.MappedArray{Float64,2,OffsetArrays.OffsetArray{ColorTypes.Gray{Float64},2,Array{ColorTypes.Gray{Float64},2}},MappedArrays.##1#3{Float64},MappedArrays.##2#4{ColorTypes.Gray{Float64}}},ImageCore.ZeroArray{Float64,2,UnitRange{Int64}}}}) with element type ColorTypes.RGB{Float64} at index [1:670,1:670] ...
This seems to have to do with the returned OffsetArray having negative indices.
Julia version: 0.5.0
Images: 0.8.0
ImageView: 0.3.1
TestImages: 0.2.0
ImageTransformations: 0.1.0
CoordinateTransformations: 0.4.0
Thank you.
Hi everybody!
This is just to let you know that recently we added code for nearest neighbour and bilinear upsampling to NNlib here in these PRs 1, 2. The code uses WHCN memory layout (and works on the first two dimensions). The nearest version is single threaded for CPU and works out of the box for GPU. The bilinear version is multi-threaded for CPU and has a GPU implementation here. Note that bilinear upsampling aligns the corners, so I think it's a bit different from what you are doing, but I intend to make the behaviour optional to be able to match your implementation. Both versions are "super fast" but I haven't benchmarked against ImageTransformation.jl. RGB data is currently not supported out of the box, but writing a wrapper should be simple.
See also JuliaMath/Interpolations.jl#396 - would be cool to recycle code where possible, perhaps some day in the future :)
WARNING: deprecated syntax "typealias FloatLike{T<:AbstractFloat} Union{T,Gray{T}}" at /home/juliohm/.julia/v0.6/ImageTransformations/src/ImageTransformations.jl:10.
Use "FloatLike{T<:AbstractFloat} = Union{T,Gray{T}}" instead.
WARNING: deprecated syntax "typealias FloatColorant{T<:AbstractFloat} Colorant{T}" at /home/juliohm/.julia/v0.6/ImageTransformations/src/ImageTransformations.jl:11.
Use "FloatColorant{T<:AbstractFloat} = Colorant{T}" instead.
Are there any plans to add support for custom image image transformations?
imrotate
is a high-level interface to warp
. WarpedView
is a lazy version of warp
, and it makes sense to also have a lazy version of imrotate
built on top of WarpedView
.
When tweaking the performance, the lazy version is usually a good choice because:
getindex
)The lazy version is not always the best choice because if you call getindex
repeatedly, the computation happens repeatedly.
Edit:
Similarily, we can support lazy imresize
.
b_img = ones(N0f8, (100, 100))
b_img = colorview(Gray, b_img)
mosaicview(
b_img,
imrotate(b_img, π),
imrotate(b_img, π, axes(b_img)),
imrotate(b_img, 179 / 180 * π),
imrotate(b_img, 180 / 180 * π),
imrotate(b_img, 181 / 180 * π),
nrow=2, ncol=3, npad=20, fillvalue=colorant"white"
)
Regardless of image size, black bars are at least two pixels in size.
Julia Version 1.7.1
Commit ac5cc99908* (2021-12-22 19:35 UTC)
Platform Info:
OS: Linux (x86_64-pc-linux-gnu)
"Manjaro Linux"
uname: Linux 5.10.84-1-MANJARO JuliaImages/Images.jl#1 SMP PREEMPT Wed Dec 8 09:50:30 UTC 2021 x86_64 unknown
CPU: Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz:
speed user nice sys idle irq
JuliaImages/Images.jl#1-64 1200 MHz 64345068 s 16198 s 16151708 s 1720339615 s 739238 s
Memory: 991.7728500366211 GB (93890.00390625 MB free)
Uptime: 2.83035374e6 sec
Load Avg: 2.23 2.05 2.72
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-13.0.0 (ORCJIT, cascadelake)
Environment:
HOME = /home/myuan
TERM = xterm-256color
PATH = /usr/local/sbin:/usr/local/bin:/usr/bin:/var/lib/snapd/snap/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl
(@v1.7) pkg> st
Status `~/.julia/environments/v1.7/Project.toml`
[c7e460c6] ArgParse v1.1.4
[6e4b80f9] BenchmarkTools v1.2.2
[336ed68f] CSV v0.9.11
[82f2e89e] ClickHouse v0.2.1
[150eb455] CoordinateTransformations v0.6.2
[5789e2e9] FileIO v1.11.2
[7073ff75] IJulia v1.23.2
[82e4d734] ImageIO v0.5.9
[6218d12a] ImageMagick v1.2.2
[02fcd773] ImageTransformations v0.8.13
[916415d5] Images v0.24.1
[4138dd39] JLD v0.12.5
[23992714] MAT v0.10.2
[7eb4fadd] Match v1.2.0
[b98c9c47] Pipe v1.3.0
[91a5bcdd] Plots v1.25.3
[438e738f] PyCall v1.92.5
[6038ab10] Rotations v1.1.1
[90137ffa] StaticArrays v1.3.2
[731e570b] TiffImages v0.5.2
[ddb6d928] YAML v0.4.7
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.