Probably a good idea. Surely someone at METADATA will point out the lack of travis testing :)

Is it possible/recommended to add an overload to warp that takes a lookup table instead of a transform. The functionality would be similar to OpenCVs remap found here.

Hi everybody!

This is just to let you know that recently we added code for nearest neighbour and bilinear upsampling to NNlib here in these PRs 1, 2. The code uses WHCN memory layout (and works on the first two dimensions). The nearest version is single threaded for CPU and works out of the box for GPU. The bilinear version is multi-threaded for CPU and has a GPU implementation here. Note that bilinear upsampling aligns the corners, so I think it's a bit different from what you are doing, but I intend to make the behaviour optional to be able to match your implementation. Both versions are "super fast" but I haven't benchmarked against ImageTransformation.jl. RGB data is currently not supported out of the box, but writing a wrapper should be simple.

See also JuliaMath/Interpolations.jl#396 - would be cool to recycle code where possible, perhaps some day in the future :)

In #24 we began discussing a possible breaking change: inverting the "sense" in which a transformation is used/applied. Currently pixel values are computed as img[inv(tfm)(x)], but perhaps we should consider switching to img[tfm(x)]. For a rotation, the former can be thought of as rotating the image, whereas the latter is rotating the coordinate axes. Let me acknowledge that I was the one who established the current convention; if we decide that the current system is a mess (I fear it may be), you know who to blame...

At the outset, let's acknowledge that from a standpoint of pure mathematics, either choice is viable. This is because (in my opinion) we should focus on diffeomorphisms and all diffeomorphisms are invertible. However, from a practical standpoint some are easier to invert than others. For example, affine transformations are easy to invert, but diffeomorphisms like this one are less straightforward. The latter might arise from a deformable image registration algorithm, so I think we have to take such seriously.

So, let me make the case for switching. Note here that I'm assuming that the goal of registration is to find a diffeomorphism ϕ such that fixed[x] ≈ moving[ϕ(x)]; if someone has a preferred framework for thinking about registration, now is a really, really good time to mention it.

For the function warp in general you need both the forward and inverse, because (given that framework above)

warped[x] = moving[ϕ(x)]

and consequently warped should be defined for all x such that ϕ(x) is within the domain of moving, i.e., the bounding box of ϕ⁻¹(y) for all y within the domain of moving. Since we need both the forward and the reverse, again it doesn't make a great deal of difference which we choose, with one exception: given that ϕ is a diffeomorphism, we can compute that bounding box using only the edges of moving, and so consequently the inversion only needs to be computed for a small fraction of points. If the inversion is computationally expensive (e.g., invokes a nonlinear solver), then it would be a win to supply ϕ rather than ϕ⁻¹

However, for image registration we are likely to only care about warped over the domain of fixed, in which case we can allocate an empty warped and then call warp!(warped, moving, ϕ). In this case, the behavior should be to iterate over all entries in warped, in which case we don't even need ϕ⁻¹. In this case, imposing the requirement that we can (and do) compute ϕ⁻¹ seems like an unnecessary burden.

It would be great if someone adds interpolation_type argument in imresize function as in PIL package of Python.

img = rand(RGB{N0f8}, 1000, 1000)
img2 = imrotate(img, π/2)
size(img) # (1000, 1000)
size(img2) # (1002, 1002)

I realize it's pretty naive to just ask "why doesn't this work as expected?". I get the sense that "simply" rotating an image is not so simple. (I know very little about image processing)

I'm wondering how I should get the desired result for rotations where the degree is multiples of pi/2. This might be related to #119 although I'm not sure.

Would it make sense to just define a new imrotate specifically for these factors of pi/2?

I'm getting a surprising result when calling warp with an IdentityTransformation. Is this a bug?

using Images, CoordinateTransformations, Interpolations
img = rand(4,8)
img[:,1:2] = NaN
img[:,7:8] = NaN
imgw = warp(img, IdentityTransformation(), indices(img), Linear(), Flat())
@show img
#Notice that column 6 gets set to NaN
@show parent(imgw)

Note that this doesn't happen when Constant() interpolation is used for the degree. As for the fill parameter, Flat() and NaN give the same result.

Would be useful

Hi,

I have a Python script in which there is a call to scipy.ndimage.rotate for rotating a 2D array, similar to the following

using PyCall
ndimage = pyimport("scipy.ndimage")
# test array
img = Float64[1 2 3 4; 5 6 7 8; 9 10 11 12; 13 14 15 16]
img_90 = ndimage.rotate(img, 90, reshape=false)

I came across this package when searching for the same kind of operation on a plain 2D array in Julia. However, soon I got confused by the rotation methods provided here, especially imrotate.

using ImageTransformations
using Rotations: RotMatrix
using CoordinateTransformations: recenter

# test array
img = Float64[1 2 3 4; 5 6 7 8; 9 10 11 12; 13 14 15 16]

# define transformation
trfm = recenter(RotMatrix(pi/2), center(img))
imgw = warp(img, trfm) # this is what I expected, same as in scipy

imgr = imrotate(img, pi/2) # this gives a 6x6 OffsetArray with values that I don't quite understand

where imrotate returned

6×6 OffsetArray(::Matrix{Float64}, 0:5, 0:5) with eltype Float64 with indices 0:5×0:5:
 NaN  NaN    NaN        NaN        NaN        NaN
 NaN  NaN    NaN          4.99995  NaN        NaN
 NaN   14.0    9.99998    5.99999  NaN        NaN
 NaN  NaN     11.0        7.00002    3.00003  NaN
 NaN  NaN     12.0      NaN        NaN        NaN
 NaN  NaN    NaN        NaN        NaN        NaN

I tried the example given in the doc for imrotate, and it looked just fine.

Can you kindly point out what I missed here? What is the closest possible translation from scipy.ndimage.rotate?

I am trying to use ForwardDiff to compute the gradient of a cost function defined as the Sum of Square Differences (SSD) between two images with respect to the parameters of an image transformation. In the code shown below, one of the images has been translated by a small amount. This is basically just a toy problem that I designed to understand if ForwardDiff.jl and ImageTransformations.jl play well together. Eventually, I will compute derivatives relative the parameters of different image warps. Notice that I do not want to compute gradients of the pixel intensities, but, say, the 2x2 Jacobian of the translation, like:

using Images
using TestImages
using ImageTransformations
using ForwardDiff

img1 = testimage("cameraman.tif")

y_shift = 0.33
x_shift = -1.76
t = ImageTransformations.Translation(y_shift, x_shift)

img2 = ImageTransformations.warp(img1, t, ImageTransformations.indices_spatial(img1), 0)

function cost(Δ::Vector{T}) where T <: Real
    t = ImageTransformations.Translation(Δ[2], Δ[1])
    img2 = ImageTransformations.warp(img1, t, ImageTransformations.indices_spatial(img1), 0)

    imgg1 = Gray.(img1)
    imgg2 = Gray.(img2)

    mat1 = convert(Array{Float64}, imgg1)
    mat2 = convert(Array{Float64}, imgg2)
    
    @. mat1 = (mat1 - mat2)^2
    SSD = sum(vec(mat1))
    
    return SSD
end

I’m getting this error:

StackOverflowError:

Stacktrace:
  [1] make_typealias(x::Type)
    @ Base ./show.jl:531
  [2] show_typealias(io::IOBuffer, x::Type)
    @ Base ./show.jl:661
  [3] show(io::IOBuffer, x::Type)
    @ Base ./show.jl:816
  [4] show_datatype(io::IOBuffer, x::DataType)
    @ Base ./show.jl:928
  [5] show(io::IOBuffer, x::Type)
    @ Base ./show.jl:819
  [6] print(io::IOBuffer, x::Type)
    @ Base ./strings/io.jl:35
  [7] print_to_string(::String, ::Vararg{Any, N} where N)
    @ Base ./strings/io.jl:135
  [8] string
    @ ./strings/io.jl:174 [inlined]
  [9] floattype
    @ ~/.julia/packages/FixedPointNumbers/HAGk2/src/deprecations.jl:4 [inlined]
 [10] _default_digits(#unused#::Type{ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}}) (repeats 43235 times)
    @ ImageTransformations ~/.julia/packages/ImageTransformations/xYRLH/src/autorange.jl:82
 [11] __round(x::StaticArrays.SVector{2, ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}})
    @ ImageTransformations ~/.julia/packages/ImageTransformations/xYRLH/src/autorange.jl:100
 [12] (::ImageTransformations.var"#6#7"{Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, CoordinateTransformations.Translation{StaticArrays.SVector{2, ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}}}})(i::Int64)
    @ ImageTransformations ~/.julia/packages/ImageTransformations/xYRLH/src/autorange.jl:90
 [13] iterate
    @ ./generator.jl:47 [inlined]
 [14] _collect
    @ ./array.jl:691 [inlined]
 [15] collect_similar
    @ ./array.jl:606 [inlined]
 [16] map
    @ ./abstractarray.jl:2294 [inlined]
 [17] _round(tform::CoordinateTransformations.Translation{StaticArrays.SVector{2, ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}}}; kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ ImageTransformations ~/.julia/packages/ImageTransformations/xYRLH/src/autorange.jl:89
 [18] _round(tform::CoordinateTransformations.Translation{StaticArrays.SVector{2, ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}}})
    @ ImageTransformations ~/.julia/packages/ImageTransformations/xYRLH/src/autorange.jl:89
 [19] warp!(out::Matrix{Gray{N0f8}}, img::Interpolations.FilledExtrapolation{Gray{N0f8}, 2, Interpolations.BSplineInterpolation{Gray{N0f8}, 2, Matrix{Gray{N0f8}}, Interpolations.BSpline{Interpolations.Linear}, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}}, Interpolations.BSpline{Interpolations.Linear}, Gray{N0f8}}, tform::CoordinateTransformations.Translation{StaticArrays.SVector{2, ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}}})
    @ ImageTransformations ~/.julia/packages/ImageTransformations/xYRLH/src/warp.jl:92
 [20] warp(img::Interpolations.FilledExtrapolation{Gray{N0f8}, 2, Interpolations.BSplineInterpolation{Gray{N0f8}, 2, Matrix{Gray{N0f8}}, Interpolations.BSpline{Interpolations.Linear}, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}}, Interpolations.BSpline{Interpolations.Linear}, Gray{N0f8}}, tform::CoordinateTransformations.Translation{StaticArrays.SVector{2, ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}}}, inds::Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}})
    @ ImageTransformations ~/.julia/packages/ImageTransformations/xYRLH/src/warp.jl:88
 [21] warp(img::Matrix{Gray{N0f8}}, tform::CoordinateTransformations.Translation{StaticArrays.SVector{2, ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}}}, inds::Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}, args::Int64; kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ ImageTransformations ~/.julia/packages/ImageTransformations/xYRLH/src/warp.jl:101
 [22] warp(img::Matrix{Gray{N0f8}}, tform::CoordinateTransformations.Translation{StaticArrays.SVector{2, ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}}}, inds::Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}, args::Int64)
    @ ImageTransformations ~/.julia/packages/ImageTransformations/xYRLH/src/warp.jl:100
 [23] cost(Δ::Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}})
    @ Main ./In[36]:18
 [24] vector_mode_dual_eval(f::typeof(cost), x::Vector{Float64}, cfg::ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2, Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}}})
    @ ForwardDiff ~/.julia/packages/ForwardDiff/QOqCN/src/apiutils.jl:37
 [25] vector_mode_gradient(f::typeof(cost), x::Vector{Float64}, cfg::ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2, Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}}})
    @ ForwardDiff ~/.julia/packages/ForwardDiff/QOqCN/src/gradient.jl:106
 [26] gradient(f::Function, x::Vector{Float64}, cfg::ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2, Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}}}, ::Val{true})
    @ ForwardDiff ~/.julia/packages/ForwardDiff/QOqCN/src/gradient.jl:19
 [27] gradient(f::Function, x::Vector{Float64}, cfg::ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2, Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(cost), Float64}, Float64, 2}}}) (repeats 2 times)
    @ ForwardDiff ~/.julia/packages/ForwardDiff/QOqCN/src/gradient.jl:17
 [28] top-level scope
    @ In[37]:1
 [29] eval
    @ ./boot.jl:360 [inlined]
 [30] include_string(mapexpr::typeof(REPL.softscope), mod::Module, code::String, filename::String)
    @ Base ./loading.jl:1094

Any ideas?

Observed from c09f576

It seems that JuliaArrays/OffsetArrays.jl#213 breaks this:

using ImageTransformations
using OffsetArrays
using Test

A = reshape(collect(1:60), 5, 4, 3)
Ao = OffsetArray(A, (-2,1,0))
@inferred(restrict(Ao, 1)) # inferred Any

Yet I haven't investigated where the patch should be put.

cc: @jishnub

I guess the following example illustrates the problem quite well.

julia> restrict([0 1 0][:])
2-element Array{Float64,1}:
 0.25
 0.25

julia> restrict([0 1 0])
1×2 Array{Float64,2}:
 0.25  1.75543e13

While this is obviously not the intended use case, it happened to me accidentally not realizing that I should use a Vector instead of a 2D Array.

I was trying to implement more complicated interpolation options in imrotate, but seem to be restricted to passing arguments accepted by the BSpline interpolation class. Is it possible to use other interpolation classes like below?

using Interpolations imrotate(img,1.2,Lanczos4OpenCV())

(this produces an error message)

This difference is originally identified by @andrew-saydjari in #116 (comment) and I believe we could optimize it away:

img = testimage("cameraman")

imrotate(img, 0.3; method=Linear()) == imrotate(img, 0.3; method=BSpline(Linear())) # true

@btime imrotate($img, 0.3; method=Linear()); # 3.479 ms (21 allocations: 403.88 KiB)
@btime imrotate($img, 0.3; method=BSpline(Linear())); # 3.491 ms (23 allocations: 660.02 KiB)

I'm trying to convert a RGB{N0f8} image to the YUV420 format for ffmpeg. (YUV420 is YCbCr, where the Cb and Cr channels are half resolution)

When trying to use restrict to downsample YCbCr{Float64} the result is RGB{Float64}.

img = rand(RGB{N0f8},100,200)
img_YCbCr = convert.(YCbCr{Float64}, img)
@show typeof(img_YCbCr) #Array{YCbCr{Float64},2}

img_YCbCr_half = restrict(img_YCbCr)
@show typeof(img_YCbCr_half) #Array{RGB{Float64},2}

Also, is there a better way to do RGB{N0f8} -> YUV420 (UInt8)

I warp the same source image using Kornia and ImageTransformations, using the same homography matrix. I use a slightly modified function warp_me2, and the whole thing can be reproduced using this repo and running the mytests.jl file in it.

This is the image that I get in Kornia, which is actually very accurate -

This is the image I get after warping it using the warp_me2 function -

I've tried checking the individual indices to which it get warped, and their values, but they were all looking correct. I don't know why am I getting an issue here.

Any suggestions as to what is going wrong? Can someone checkout if they are facing the same issues?
The Kornia notebook can be accessed here. The required files are all present in the reproduction repository above.

The CoordinateTransform compat bump on master would be good to release

I think it makes sense to introduce the concept of fixed point to better enhance the imresize function.

A fixedpoint p is where imgr[p] == img[p] holds.

Three new methods will be introduced as new API:

const FixedPointType = Union{Dims, CartesianIndex}

imresize(img, sz, p::FixedPointType)
imresize(img, inds, p::FixedPointType)
imresize(img, p::FixedPointType; ratio)

There are many use cases of this, I'll just list one thing that made me propose the idea:

With this, we can improve (change) the behavior to OffsetArray. Currently, when the input is an OffsetArray:

using OffsetArray

x = OffsetArray(rand(5, 5), -3, -3)

imresize(x, (-5:5, -5:5)) |> axes # (-5:5, -5:5)

imresize(x, (10, 10)) |> axes # (1:10, 1:10)
imresize(x; ratio=2) |> axes # (1:10, 1:10)

This loss the axes information of our OffsetArray, we could change the behavior of it to

imresize(img::OffsetArray, sz::Dims) = imresize(img, sz, topleft(img))
imresize(img::OffsetArray; ratio) = imresize(img, topleft(img); ratio)

Use top left point as the default value because it is consistent with the case that the fixed point for imresize(img::Array, sz) is also the topleft. (1-based indexing).

The error below occurs when tfm is not based on a StaticArray. It's a one-line fix, I'll do a PR shortly.

ImageTransformations.autorange(h, tfm)
ERROR: MethodError: Cannot `convert` an object of type Array{Float64,1} to an object of type Tuple
This may have arisen from a call to the constructor Tuple(...),
since type constructors fall back to convert methods.
Stacktrace:
 [1] autorange(::CartesianRange{CartesianIndex{2}}, ::CoordinateTransformations.AffineMap{Array{Float64,2},Array{Float64,1}}) at /home/cody/.julia/v0.6/ImageTransformations/src/autorange.jl:16
 [2] autorange(::OffsetArrays.OffsetArray{ColorTypes.Gray{FixedPointNumbers.Normed{UInt8,8}},2,Array{ColorTypes.Gray{FixedPointNumbers.Normed{UInt8,8}},2}}, ::CoordinateTransformations.AffineMap{Array{Float64,2},Array{Float64,1}}) at /home/cody/.julia/v0.6/ImageTransformations/src/autorange.jl:3

Some serious memory allocation and regression in Julia 0.7. Here is a MWE.

julia> img = rand(100, 100);
julia> tfm = recenter(RotMatrix(-pi/4), center(img));

In Julia 0.6,

julia> @btime warp(img, tfm);
  422.876 μs (5 allocations: 157.72 KiB)

In Julia 0.7,

julia> @btime warp(img, tfm);
  8.478 ms (261626 allocations: 9.07 MiB)

Looking at the code of warp!,

function warp!(out, img::AbstractExtrapolation, tform)
    @inbounds for I in CartesianIndices(axes(out))
        out[I] = _getindex(img, tform(SVector(I.I)))
    end
    out       <==== should not return anything
end

Obviously, warp! should not return out. But eliminating the return only helps marginally. But I cannot dig any further. Might be some problem with _getindex.

It's better to have Projective transformation to ImageTransformations.jl.

# Definition of projective transformation
"""
Projective transformation.
"""
struct Projective <: Transformation
    H::SArray{Tuple{3,3},Float64,2,9}
end
"""
Definition of projective transformation.
"""
function (p::Projective)(x)
    x_ = SA[x[1],x[2],1.0]
    x′_1, x′_2, x′_3 = p.H*x_
    return SA[x′_1/x′_3, x′_2/x′_3]
end
"""
Inverse of projective transformation.
"""
function Base.inv(p::Projective)
    Projective(inv(p.H))
end

Discussions in Slack:
https://julialang.slack.com/archives/CB1R90P8R/p1626929588093000
https://julialang.slack.com/archives/CB1R90P8R/p1626983453103200

I feel the right place to add this is in CoordinateTransformations.jl

Or we could 1) first introduce a high-level API in ImageTransformations.jl with this Projective as internal type, and then 2) migrate it to upstream CoordinateTransformations, and finally 3) use the upstream version.

(The quoted words are by @johnnychen94)

In warp.jl a docstring example for imrotate says:

# rotate with bilinear interpolation and with cropping
julia> imrotate(img, π/4, axes(img)

This works, but is confusing for two reasons. First, the option to put axes into the arguments isn't in the usage line of the docstring. Second, it fails when specifying degree and/or extrapolation other than the defaults. A more robust alternative is

imrotate(img, π/4)[axes(img)...]

Also, the references to Constant(), etc. and Flat(), etc. are unknown unless the user has imported Interpolations.jl. Maybe clarify, or just reexport that?

ImageTransformations doesn't build the transformation matrix on its own, instead, it uses CoordinateTransformations and Rotations to do it.

imrotate is an example of this, and we should probably document this so that users of ImageTransformations knows where to find it.

If I understand it right, then this package would have a lot of overlap with https://github.com/timholy/AffineTransforms.jl.

Am I correct to assume that this package is planned to be a reboot of sorts, making use of 0.5 goodies and specialized backends such as CoordinateTransformations?

Mind if we tag a new version with the current state? Could use it to make CI pass on a project I am working on and I am pressed on time a little

Happens with julia v1.0.1. Digging through Base, it looks like IdentityUnitRange was introduced in 1.1. Is my only option to update my julia install?

Stacktrace:

julia> using ImageTransformations
[ Info: Precompiling ImageTransformations [02fcd773-0e25-5acc-982a-7f6622650795]
ERROR: LoadError: LoadError: UndefVarError: IdentityUnitRange not defined
Stacktrace:
 [1] getproperty(::Module, ::Symbol) at ./sysimg.jl:13
 [2] top-level scope at none:0
 [3] include at ./boot.jl:317 [inlined]
 [4] include_relative(::Module, ::String) at ./loading.jl:1041
 [5] include at ./sysimg.jl:29 [inlined]
 [6] include(::String) at /Users/tomer/.julia/packages/ImageTransformations/rVbxR/src/ImageTransformations.jl:1
 [7] top-level scope at none:0
 [8] include at ./boot.jl:317 [inlined]
 [9] include_relative(::Module, ::String) at ./loading.jl:1041
 [10] include(::Module, ::String) at ./sysimg.jl:29
 [11] top-level scope at none:2
 [12] eval at ./boot.jl:319 [inlined]
 [13] eval(::Expr) at ./client.jl:389
 [14] top-level scope at ./none:3
in expression starting at /Users/tomer/.julia/packages/ImageTransformations/rVbxR/src/resizing.jl:158
in expression starting at /Users/tomer/.julia/packages/ImageTransformations/rVbxR/src/ImageTransformations.jl:30
ERROR: Failed to precompile ImageTransformations [02fcd773-0e25-5acc-982a-7f6622650795] to /Users/tomer/.julia/compiled/v1.0/ImageTransformations/Pfac4.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] macro expansion at ./logging.jl:313 [inlined]
 [3] compilecache(::Base.PkgId, ::String) at ./loading.jl:1187
 [4] macro expansion at ./logging.jl:311 [inlined]
 [5] _require(::Base.PkgId) at ./loading.jl:944
 [6] require(::Base.PkgId) at ./loading.jl:855
 [7] macro expansion at ./logging.jl:311 [inlined]
 [8] require(::Module, ::Symbol) at ./loading.jl:837

Right now imresize performs something that is pretty similar to simple pixel sub-sampling. Naturally the quality of the result isn't particularly breathtaking.

I open this issue to brainstorm on what would be a good way to improve the quality for downsizing and image to an arbitrary specified size.

The straight forward way of doing it would be in two steps: 1.) Filter image, 2.) sub-sample. However, this would add an additional temporary array that I suspect could probably be avoided.

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

reported by @johnnychen94 at JuliaImages/Images.jl#742

The document of imresize doesn't say sz must be Tuple type, so I just try the MATLAB way of using imresize, and it throws an unexpected StackOverflowError.

using Images, TestImages
lena = testimage("lena_gray_256")

imresize(lena, (128,128)) # work as expected
imresize(lena, [128,128]) # throw StackOverflowError

Since the introduction of _round (commit a4f2190), warp (or rather autorange) only accepts <:CoordinateTransformations.Transformation. Is there any way we can allow for more generic transformations? I, for one, use Interpolations to describe more complicated transformations (from camera calibrations), which now do not work.

Of course, one alternative would be to increase the kind of transformations CoordinateTransformations can produce.

In the docstring of imresize:

This interpolates the values at sub-pixel locations. If you are shrinking the image, you risk aliasing unless you low-pass filter img first.

Yet we don't have a good showcase to tell potential users how this can be done.

Typically, we just apply a low-pass Gaussian filter to it. However, the example in the docstring is not very illustrative as it creates a over-blurred result:

using ImageFiltering
σ = map((o,n)->0.75*o/n, size(img), sz)
kern = KernelFactors.gaussian(σ)
imgr = imresize(imfilter(img, kern, NA()), sz)

This is added to our documentation as an example, but it would be nice to have it as a standard function in ImageTransformations.

WARNING: deprecated syntax "typealias FloatLike{T<:AbstractFloat} Union{T,Gray{T}}" at /home/juliohm/.julia/v0.6/ImageTransformations/src/ImageTransformations.jl:10.
Use "FloatLike{T<:AbstractFloat} = Union{T,Gray{T}}" instead.

WARNING: deprecated syntax "typealias FloatColorant{T<:AbstractFloat} Colorant{T}" at /home/juliohm/.julia/v0.6/ImageTransformations/src/ImageTransformations.jl:11.
Use "FloatColorant{T<:AbstractFloat} = Colorant{T}" instead.

This is "simple" usage of warp that I think worth a showcase. The elastic transformation is also available in Augmentor.jl

The warped image size is vulnerable to numerical stability. For the purpose of image processing, unstable size is really a headache.

In [email protected]

julia> using Rotations

julia> tfm = RotMatrix(0.1)
2×2 RotMatrix2{Float64} with indices SOneTo(2)×SOneTo(2):
 0.995004   -0.0998334
 0.0998334   0.995004

julia> inv(tfm) * tfm
2×2 RotMatrix2{Float64} with indices SOneTo(2)×SOneTo(2):
 1.0  0.0
 0.0  1.0

In [email protected]:

julia> inv(tfm) * tfm
2×2 RotMatrix2{Float64} with indices SOneTo(2)×SOneTo(2):
 1.0          -6.47893e-18
 6.47893e-18   1.0

This breaks the current test:

img_camera = testimage("camera"); # (512, 512)
tfm = recenter(RotMatrix(-pi/8), center(img_camera))
wv3 = InvWarpedView(img_camera, inv(tfm) ∘ tfm); # (513, 512)

One strategy to work around this is to use the same discretization trick in imrotate:

(ImageTransformations) pkg> st
Project ImageTransformations v0.8.7
Status `~/Documents/Julia/ImageTransformations.jl/Project.toml`
  [13072b0f] AxisAlgorithms v1.0.0
  [c3611d14] ColorVectorSpace v0.8.7
  [150eb455] CoordinateTransformations v0.6.1
  [bbac6d45] IdentityRanges v0.3.1
  [a09fc81d] ImageCore v0.8.19
  [a98d9a8b] Interpolations v0.13.1
  [6fe1bfb0] OffsetArrays v1.4.1
  [6038ab10] Rotations v1.0.2
  [90137ffa] StaticArrays v1.0.1

cc: @c42f

It looks like the tests fail for new some reason. (I have yet to look into it)

autorange: Error During Test
  Got an exception of type MethodError outside of a @test
  MethodError: Rotations.RotMatrix{2,Float64,4}(::StaticArrays.SArray{Tuple{2,2},Float64,2,4}) is ambiguous. Candidates:
    (::Type{Rotations.RotMatrix{N,T,L}})(x::AbstractArray) where {N, T, L} in Rotations at /home/csto/.julia/v0.6/Rotations/src/core_types.jl:78
    (::Type{SA})(a::StaticArrays.StaticArray) where SA<:StaticArrays.StaticArray in StaticArrays at /home/csto/.julia/v0.6/StaticArrays/src/convert.jl:4
  Possible fix, define
    (::Type{Rotations.RotMatrix{N,T,L}})(::StaticArrays.StaticArray)
  Stacktrace:
   [1] macro expansion at /home/csto/.julia/v0.6/ImageTransformations/test/autorange.jl:108 [inlined]

I'm trying to understand and use the image transformation method described in http://juliaimages.github.io/latest/indexing.html

I'm trying to apply the same rotation as described in the above doc:

using Images, ImageView, TestImages, ImageTransformations, CoordinateTransformations
img = testimage("mandrill")
imgg = convert(Image{Gray}, img)
tfm = recenter(RotMatrix(pi/8), center(imgg))
imgrot = warp(imgg, tfm);
img0 = similar(imgrot);
fill!(img0, 0);
img0[1:512, 1:512] = img;
imgov = colorview(RGB, img0, imgrot, zeroarray)

although this works, when I try to display the image using

imshow(imgov)

I get the following error:

ERROR: BoundsError: attempt to access -78:591×-78:591 ColorView{RGB}(::ImageCore.StackedView{Float64,3,Tuple{MappedArrays.MappedArray{Float64,2,OffsetArrays.OffsetArray{ColorTypes.Gray{Float64},2,Array{ColorTypes.Gray{Float64},2}},MappedArrays.##1#3{Float64},MappedArrays.##2#4{ColorTypes.Gray{Float64}}},MappedArrays.MappedArray{Float64,2,OffsetArrays.OffsetArray{ColorTypes.Gray{Float64},2,Array{ColorTypes.Gray{Float64},2}},MappedArrays.##1#3{Float64},MappedArrays.##2#4{ColorTypes.Gray{Float64}}},ImageCore.ZeroArray{Float64,2,UnitRange{Int64}}}}) with element type ColorTypes.RGB{Float64} at index [1:670,1:670] ...

This seems to have to do with the returned OffsetArray having negative indices.

Julia version: 0.5.0
Images: 0.8.0
ImageView: 0.3.1
TestImages: 0.2.0
ImageTransformations: 0.1.0
CoordinateTransformations: 0.4.0

Thank you.

The new warp function is bit different than the old transform from AffineTransforms. Suppose I have an array and I want to translate the array by (x, y) and fill the moved out rows and columns with 0. How would I do that?

If I want to translate and rotate the array, and keep the same size as the original array and fill with 0, how should I use warp?

Thanks!

The merge of JuliaLang/METADATA.jl#11972 will break some existing files

Thank you for this awesome package!

I'm auto-tracking animal movements in videos. I'm using VideoIO.jl to extract frames from the video, resize them (uniform scale and fixing the Storage Aspect Ratio of the video), and then save a diagnostic short video of the results.

I'm using imresize to do that, but can't help thinking that there must be a way to save some of the computations if the transform is identical for each frame: the original images as well as the resized images are always the same size. Is there a way to calculate the transformation once, and then apply it multiple times to each of the frames?

b_img = ones(N0f8, (100, 100))
b_img = colorview(Gray, b_img)


mosaicview(
    b_img, 
    imrotate(b_img, π), 
    imrotate(b_img, π, axes(b_img)),
    
    imrotate(b_img, 179 / 180 * π), 
    imrotate(b_img, 180 / 180 * π), 
    imrotate(b_img, 181 / 180 * π), 

    nrow=2, ncol=3, npad=20, fillvalue=colorant"white"
)

Regardless of image size, black bars are at least two pixels in size.

Julia Version 1.7.1
Commit ac5cc99908* (2021-12-22 19:35 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      "Manjaro Linux"
  uname: Linux 5.10.84-1-MANJARO JuliaImages/Images.jl#1 SMP PREEMPT Wed Dec 8 09:50:30 UTC 2021 x86_64 unknown
  CPU: Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz: 
                 speed         user         nice          sys         idle          irq
       JuliaImages/Images.jl#1-64  1200 MHz   64345068 s      16198 s   16151708 s  1720339615 s     739238 s
       
  Memory: 991.7728500366211 GB (93890.00390625 MB free)
  Uptime: 2.83035374e6 sec
  Load Avg:  2.23  2.05  2.72
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.0 (ORCJIT, cascadelake)
Environment:
  HOME = /home/myuan
  TERM = xterm-256color
  PATH = /usr/local/sbin:/usr/local/bin:/usr/bin:/var/lib/snapd/snap/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl

(@v1.7) pkg> st
      Status `~/.julia/environments/v1.7/Project.toml`
  [c7e460c6] ArgParse v1.1.4
  [6e4b80f9] BenchmarkTools v1.2.2
  [336ed68f] CSV v0.9.11
  [82f2e89e] ClickHouse v0.2.1
  [150eb455] CoordinateTransformations v0.6.2
  [5789e2e9] FileIO v1.11.2
  [7073ff75] IJulia v1.23.2
  [82e4d734] ImageIO v0.5.9
  [6218d12a] ImageMagick v1.2.2
  [02fcd773] ImageTransformations v0.8.13
  [916415d5] Images v0.24.1
  [4138dd39] JLD v0.12.5
  [23992714] MAT v0.10.2
  [7eb4fadd] Match v1.2.0
  [b98c9c47] Pipe v1.3.0
  [91a5bcdd] Plots v1.25.3
  [438e738f] PyCall v1.92.5
  [6038ab10] Rotations v1.1.1
  [90137ffa] StaticArrays v1.3.2
  [731e570b] TiffImages v0.5.2
  [ddb6d928] YAML v0.4.7

https://juliaimages.org/ImageTransformations.jl/stable returns 404.

We can add the old documentation by adding tags with metadata such as v0.8.12+doc1.
cf. https://juliadocs.github.io/Documenter.jl/stable/man/hosting/#Documentation-Versions

I am interested to know if warp function defined in this repository work similar to the _warp_fast function available in scipy for homography ?

Is there a way to get the restrict function with faster load times?

julia> @time using ImageTransformations: restrict
  3.119409 seconds (6.69 M allocations: 376.407 MiB, 4.46% gc time)
julia> exit()
...
julia> @time using ImageTransformations: restrict
  3.212386 seconds (6.69 M allocations: 376.411 MiB, 4.51% gc time)

imrotate is a high-level interface to warp. WarpedView is a lazy version of warp, and it makes sense to also have a lazy version of imrotate built on top of WarpedView.

When tweaking the performance, the lazy version is usually a good choice because:

• it does not allocate new memory for the entire array
• it does not compute the result for all pixels (only happens when you fetch it via getindex)

The lazy version is not always the best choice because if you call getindex repeatedly, the computation happens repeatedly.

Edit:

Similarily, we can support lazy imresize.

is there something that performs this transformation?

[1 0; 0 0] -> [1 1 0 0; 1 1 0 0; 0 0 0 0; 0 0 0 0]

when I try to imresize(img, ratio=100) it seems to be interpolating but I just want to make the image bigger without degrading the pixels.

thanks

Are there any plans to add support for custom image image transformations?

I know it's just reverse(img;dim=2) in 2 different dims but would be nice to have it.

Mentioned here: https://julialang.slack.com/archives/CB1R90P8R/p1671316725407619

Since the reference test code is duplicated among three packages by now I made a little stand alone package for the reference tests. Once registered in METADATA I plan to replace the old manual code, which includes this package.

https://github.com/Evizero/ReferenceTests.jl

Unless there are objections of course. That said this will only be a test/REQUIRE dependency, and not really affect the actual package code