ValueHistories

Utility package for efficient tracking of optimization histories, training curves or other information of arbitrary types and at arbitrarily spaced sampling times

Installation

Pkg.add("ValueHistories")
using ValueHistories

For the latest developer version:

Pkg.checkout("ValueHistories")

Overview

Available history types:

• QueueUnivalueHistory
• VectorUnivalueHistory
• DynMultivalueHistory

Supported operations for univalue histories:

• push!(history, iteration, value): Appends a value to the history
• get(history): Returns all available observations as two vectors. The first vector contains the iterations and the second vector contains the values.
• enumerate(history) Returns an enumerator over the observations (as tuples)
• first(history): First stored observation (as tuple)
• last(history): Last stored observation (as tuple)
• length(history): Number of stored observations

Supported operations for multivalue histories:

• push!(history, key, iteration, value): Appends a value to the multivalue history
• get(history, key): Returns all available observations as two vectors. The first vector contains the iterations and the second vector contains the values.
• enumerate(history, key) Returns an enumerator over the observations (as tuples)
• first(history, key): First stored observation (as tuple)
• last(history, key): Last stored observation (as tuple)
• length(history, key): Number of stored observations

Tracking a single value

# Specify the type of value you wish to track
history = QueueUnivalueHistory(Float64)

for i = 1:100
  # Store some value of the specified type
  push!(history, i, i / 2)
end

# Access stored values as arrays
x, y = get(history)
@assert typeof(x) <: Vector{Int}
@assert typeof(y) <: Vector{Float64}

# You can also enumerate over the observations
for (x, y) in enumerate(history)
  @assert typeof(x) <: Int
  @assert typeof(y) <: Float64
end

Tracking multiple values dynamically

history = DynMultivalueHistory()

for i = 1:100
  # Store any kind of value without losing type stability
  # The first push! to a key defines the tracked type
  #   push!(history, key, iter, value)
  push!(history, :myval1, i, i / 2)

  # Sampling times can be arbitrarily spaced
  i % 10 == 0 && i != 20 && push!(history, :myval2, float(i), "i=$i")
end

# Access stored values as arrays
x, y = get(history, :myval1)
@assert length(x) == length(y) == 100
@assert typeof(x) <: Vector{Int}
@assert typeof(y) <: Vector{Float64}

x, y = get(history, :myval2)
@assert length(x) == length(y) == 9
@assert x == [10., 30., 40., 50., 60., 70., 80., 90., 100.]
@assert y == ["i=10", "i=30", "i=40", "i=50", "i=60", "i=70", "i=80", "i=90", "i=100"]
@assert typeof(x) <: Vector{Float64}
@assert typeof(y) <: Vector{ASCIIString}

# You can also enumerate over the observations
for (x, y) in enumerate(history, :myval2)
  @assert typeof(x) <: Float64
  @assert typeof(y) <: ASCIIString
end

Benchmarks

Compilation already taken into account. The code can be found here

Baseline: 100000 iterations that accumulates a Float64
  0.018450 seconds (498.98 k allocations: 9.140 MB, 15.75% gc time)

VectorUnivalueHistory: 100000 iterations tracking accumulator of accumulator as Float64
  0.024337 seconds (599.01 k allocations: 14.667 MB, 7.92% gc time)
VectorUnivalueHistory: Converting result into arrays
  0.000009 seconds (3 allocations: 96 bytes)

QueueUnivalueHistory: 100000 iterations tracking accumulator of accumulator as Float64
  0.020105 seconds (599.17 k allocations: 12.195 MB)
QueueUnivalueHistory: Converting result into arrays
  0.003722 seconds (100.01 k allocations: 4.578 MB, 58.66% gc time)

DynMultivalueHistory: 100000 iterations tracking accumulator as Float64 and String
  0.194958 seconds (2.10 M allocations: 70.558 MB, 22.73% gc time)
DynMultivalueHistory: Converting result into arrays
  0.110471 seconds (1.39 M allocations: 28.914 MB, 17.87% gc time)

License

This code is free to use under the terms of the MIT license.