Giter Club home page Giter Club logo

tensorflow.jl's Introduction

TensorFlow

Build Status codecov.io DOI

A wrapper around TensorFlow, a popular open source machine learning framework from Google.

⚠️ Notice ⚠️
Tensorflow.jl is in minimal maintenance mode
While it works, it is not receiving new features, and is bound to an old version, 1.13.1, of libtensorflow.
The authors recommend Flux.jl for new users.

Documentation

Other resources:

Why use TensorFlow.jl?

See a list of advantages over the Python API.

What's changed recently?

See NEWS.

Basic usage

using TensorFlow
using Test

sess = TensorFlow.Session()

x = TensorFlow.constant(Float64[1,2])
y = TensorFlow.Variable(Float64[3,4])
z = TensorFlow.placeholder(Float64)

w = exp(x + z + -y)

run(sess, TensorFlow.global_variables_initializer())
res = run(sess, w, Dict(z=>Float64[1,2]))
@test res[1]  exp(-1)

Installation

Install via

Pkg.add("TensorFlow")

To enable support for GPU usage on Linux, set an environment variable TF_USE_GPU to "1" and then rebuild the package. eg

ENV["TF_USE_GPU"] = "1"
Pkg.build("TensorFlow")

CUDA 8.0 and cudnn are required for GPU usage. If you need to use a different version of CUDA, or if you want GPU support on Mac OS X, you can compile libtensorflow from source.

Initial precompilation (eg, the first time you type using TensorFlow) can take around five minutes, so please be patient. Subsequent load times will only be a few seconds.

Installation via Docker

Simply run docker run -it malmaud/julia:tf to open a Julia REPL that already has TensorFlow installed:

julia> using TensorFlow
julia>

For a version of TensorFlow.jl that utilizes GPUs, use nvidia-docker run -it malmaud/julia:tf_gpu. Download nvidia-docker if you don't already have it.

Logistic regression example

Realistic demonstration of using variable scopes and advanced optimizers

using TensorFlow
using Distributions
using Printf

# Generate some synthetic data
x = randn(100, 50)
w = randn(50, 10)
y_prob = exp.(x*w)
y_prob ./= sum(y_prob,dims=2)

function draw(probs)
    y = zeros(size(probs))
    for i in 1:size(probs, 1)
        idx = rand(Categorical(probs[i, :]))
        y[i, idx] = 1
    end
    return y
end

y = draw(y_prob)

# Build the model
sess = Session(Graph())

X = placeholder(Float64, shape=[-1, 50])
Y_obs = placeholder(Float64, shape=[-1, 10])

variable_scope("logisitic_model"; initializer=Normal(0, .001)) do
    global W = get_variable("W", [50, 10], Float64)
    global B = get_variable("B", [10], Float64)
end

Y=nn.softmax(X*W + B)

Loss = -reduce_sum(log(Y).*Y_obs)
optimizer = train.AdamOptimizer()
minimize_op = train.minimize(optimizer, Loss)
saver = train.Saver()

# Run training
run(sess, global_variables_initializer())
checkpoint_path = mktempdir()
@info("Checkpoint files saved in $checkpoint_path")
for epoch in 1:100
    cur_loss, _ = run(sess, [Loss, minimize_op], Dict(X=>x, Y_obs=>y))
    println(@sprintf("Current loss is %.2f.", cur_loss))
    train.save(saver, sess, joinpath(checkpoint_path, "logistic"), global_step=epoch)
end

Troubleshooting

If you see issues from the ccall or python interop, try updating TensorFlow both in Julia and in the global python install:

julia> Pkg.build("TensorFlow")
$ pip install --upgrade tensorflow

Citing

If you use this software in your research, we would really appreciate if you cite us.

Malmaud, J. & White, L. (2018). TensorFlow.jl: An Idiomatic Julia Front End for TensorFlow. Journal of Open Source Software, 3(31), 1002, https://doi.org/10.21105/joss.01002

Optional: Using a custom TensorFlow binary

To build TensorFlow from source, or if you already have a TensorFlow binary that you wish to use, follow these instructions. This is recommended by Google for maximum performance, and is currently needed for Mac OS X GPU support.

For Linux users, a convenience script is included to use Docker to easily build the library. Just install docker and run julia build_libtensorflow.so from the "deps" directory of the TensorFlow.jl package. Note that this method may not link to all libraries available on the target system such as Intel MKL.

tensorflow.jl's People

Contributors

adamryczkowski avatar alexander-barth avatar andreasnoack avatar asimshankar avatar aterenin avatar baggepinnen avatar byronbest avatar colbec avatar domluna avatar femtocleaner[bot] avatar gdkrmr avatar gustafsson avatar hpoit avatar karenl7 avatar keno avatar kevin-w-li avatar kshyatt avatar lmescheder avatar malmaud avatar mikeinnes avatar oxinabox avatar pallharaldsson avatar pevnak avatar schmrlng avatar smist09 avatar soar-tm-discovery avatar staticfloat avatar tkelman avatar wookay avatar yeesian avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

tensorflow.jl's Issues

`permutedims` as a alias for `transpose`?

The way transpose work on TensorFlow is how permutedims works in normal julia.
should it thus be aliased?

tranpose, right now uses 0 based indexing,
Would a permutedims use 1 Based indexing?
Is this a convention established by argmax and indmax ?

UndefVarError: val not defined

the node_def:

julia> node_def
name: save/Const
op: Const
attr {
  key: dtype
  value {
    type: String
  }
}
attr {
  key: _output_shapes
  value {
      }
}
attr {
  key: value
  value {
    dtype: String
    shape: Int64[]
    content: Array{UInt8,1}[UInt8[0x6d,0x6f,0x64,0x65,0x6c]]
  }
}

and corresponding error message:

julia> Operation(node_def)
ERROR: UndefVarError: val not defined
 in load_proto(::TensorFlow.tensorflow.TensorProto) at /Users/yeesian/.julia/v0.5/TensorFlow/src/core.jl:522
 in TensorFlow.Operation(::TensorFlow.tensorflow.NodeDef) at /Users/yeesian/.julia/v0.5/TensorFlow/src/core.jl:656

kernel dying...

I have very little experience with tensor flow but am hoping to make a simple version of the Karpathy game eventually. However, Already at first attempt I get stuck with the kernel dying on me at the last line (julia 0.5 rc 3)

using TensorFlow
using Distributions
situationDim=10
actionDim=10
n_hidden_1=100
n_hidden_2=100


function weight_variable(shape)
    initial = map(Float64, rand(Normal(0, .001), shape...))
    return Variable(initial)
end

function bias_variable(shape)
    initial = fill(Float64(.1), shape...)
    return Variable(initial)
end

session = Session(Graph())

x = placeholder(Float64, shape=[-1, situationDim])
y = placeholder(Float64, shape=[-1, actionDim])
layer1=nn.tanh(x*weight_variable([situationDim, n_hidden_1])+bias_variable(n_hidden_1))
layer2=nn.tanh(layer1*weight_variable([n_hidden_1, n_hidden_2])+bias_variable(n_hidden_2))
y_out=nn.softmax(layer2*weight_variable([n_hidden_2, actionDim])+bias_variable(actionDim))
cross_entropy = -reduce_sum(y.*log(y_out))
train_step = train.minimize(train.AdamOptimizer(1e-4), cross_entropy)

Any suggestions would be much appreciated. If I can get some simple reinforcement algorithm working I'd be happy to add it to the examples of TensorFlow.jl

If I may also add, how do you debug tensor flow code? how can I look at how it is linked together and that dimensions are right?

UndefVarError: PyString_FromStringAndSize not defined

After a recent Pkg.update() I get the following error

ERROR: LoadError: On worker 2:
UndefVarError: PyString_FromStringAndSize not defined
 in py_bytes at /local/home/fredrikb/.julia/v0.5/TensorFlow/src/py.jl:23
 in #15 at /local/home/fredrikb/.julia/v0.5/TensorFlow/src/py.jl:30
 in py_with at /local/home/fredrikb/.julia/v0.5/TensorFlow/src/py.jl:18 [inlined]
 in make_py_graph at /local/home/fredrikb/.julia/v0.5/TensorFlow/src/py.jl:28
 in py_gradients at /local/home/fredrikb/.julia/v0.5/TensorFlow/src/py.jl:48
 in #17 at /local/home/fredrikb/.julia/v0.5/TensorFlow/src/core.jl:938
 in #625 at ./multi.jl:1421
 in run_work_thunk at ./multi.jl:1001
 in macro expansion at ./multi.jl:1421 [inlined]
 in #624 at ./event.jl:68
 in gradients(::TensorFlow.Tensor, ::Array{TensorFlow.Tensor,1}) at /local/home/fredrikb/.julia/v0.5/TensorFlow/src/core.jl:936

Search led me to JuliaPy/PyCall.jl#306
but I'm not sure how to resolve the issue :/ Any ideas?

New issues with summary_writers

I have made further experiments with the summary_writers and obtained the following error.
Every time the line write(summary_writer, summaries, epoch) is called, I get a printed output saying the file could not be opened. However, the file appears in the temporary folder. After 10k iterations of the mnist example, the program stopped with the error below.

INFO: step 10000, training accuracy 1.0, time taken: 15:59
E tensorflow/core/util/events_writer.cc:62] Could not open events file: checkpoint_path/events.out.tfevents.1474009445.billman: Not found: checkpoint_path/events.out.tfevents.1474009445.billman
E tensorflow/core/util/events_writer.cc:95] Write failed because file could not be opened.
INFO: step 10100, training accuracy 0.06, time taken: 16:8
ERROR: Tensorflow error: Status: Nan in summary histogram for: node271
     [[Node: node271 = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](node272, node37)]]

 in check_status(::TensorFlow.Status) at /local/home/fredrikb/.julia/v0.5/TensorFlow/src/core.jl:117
 in run(::TensorFlow.Session, ::Array{TensorFlow.Port,1}, ::Array{Any,1}, ::Array{TensorFlow.Port,1}, ::Array{Ptr{Void},1}) at /local/home/fredrikb/.julia/v0.5/TensorFlow/src/core.jl:802
 in run(::TensorFlow.Session, ::Array{TensorFlow.Tensor,1}, ::Dict{Any,Any}) at /local/home/fredrikb/.julia/v0.5/TensorFlow/src/core.jl:840
 in run at /local/home/fredrikb/.julia/v0.5/TensorFlow/src/core.jl:858 [inlined]
 in run(::TensorFlow.Session, ::TensorFlow.Tensor) at /local/home/fredrikb/.julia/v0.5/TensorFlow/src/core.jl:866
 in macro expansion; at ./REPL[40]:9 [inlined]
 in macro expansion; at ./util.jl:184 [inlined]
 in anonymous at ./<missing>:?
 in eval_user_input(::Any, ::Base.REPL.REPLBackend) at ./REPL.jl:64
 in macro expansion at ./REPL.jl:95 [inlined]
 in (::Base.REPL.##3#4{Base.REPL.REPLBackend})() at ./event.jl:68

I hope this information is sufficient to locate the cause.

julia> versioninfo()
Julia Version 0.5.0-rc3+49
Commit e1d2965* (2016-09-08 05:47 UTC)
Platform Info:
  System: Linux (x86_64-redhat-linux)
  CPU: Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz
  WORD_SIZE: 64
  BLAS: libopenblas (DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblasp.so.0
  LIBM: libopenlibm
  LLVM: libLLVM-3.7.1 (ORCJIT, haswell)

Pkg.installed()["TensorFlow"] => v"0.3.0+"

Error when displaying tensors in REPL

When running the basic usage example in the readme, the error below is produced.

julia> w = exp(x + z + -y)
Error showing value of type TensorFlow.Tensor:
ERROR: BoundsError: attempt to access 0-element Array{Nullable{Int64},1} at index [1]
 in getindex(::TensorFlow.ShapeInference.TensorShape, ::Int64) at /home/fredrikb/.julia/v0.5/TensorFlow/src/shape_inference.jl:29
 in get_shape(::TensorFlow.Tensor) at /home/fredrikb/.julia/v0.5/TensorFlow/src/shape_inference.jl:84
 in (::TensorFlow.ShapeInference.##15#16)(::TensorFlow.Operation) at /home/fredrikb/.julia/v0.5/TensorFlow/src/shape_inference.jl:138
 in get_shape(::TensorFlow.Tensor) at /home/fredrikb/.julia/v0.5/TensorFlow/src/shape_inference.jl:84
 in (::TensorFlow.ShapeInference.##13#14)(::TensorFlow.Operation) at /home/fredrikb/.julia/v0.5/TensorFlow/src/shape_inference.jl:132
 in get_shape(::TensorFlow.Tensor) at /home/fredrikb/.julia/v0.5/TensorFlow/src/shape_inference.jl:84
 in show(::IOContext{Base.Terminals.TTYTerminal}, ::TensorFlow.Tensor) at /home/fredrikb/.julia/v0.5/TensorFlow/src/core.jl:709
 in display(::Base.REPL.REPLDisplay{Base.REPL.LineEditREPL}, ::MIME{Symbol("text/plain")}, ::TensorFlow.Tensor) at ./REPL.jl:132
 in display(::Base.REPL.REPLDisplay{Base.REPL.LineEditREPL}, ::TensorFlow.Tensor) at ./REPL.jl:135
 in display(::TensorFlow.Tensor) at ./multimedia.jl:143
 in print_response(::Base.Terminals.TTYTerminal, ::Any, ::Void, ::Bool, ::Bool, ::Void) at ./REPL.jl:154
 in print_response(::Base.REPL.LineEditREPL, ::Any, ::Void, ::Bool, ::Bool) at ./REPL.jl:139
 in (::Base.REPL.##22#23{Bool,Base.REPL.##33#42{Base.REPL.LineEditREPL,Base.REPL.REPLHistoryProvider},Base.REPL.LineEditREPL,Base.LineEdit.Prompt})(::Base.LineEdit.MIState, ::Base.AbstractIOBuffer{Array{UInt8,1}}, ::Bool) at ./REPL.jl:652
 in (::Base.REPL.##38#47)(::Base.LineEdit.MIState, ::Base.REPL.LineEditREPL, ::Vararg{Any,N}) at ./REPL.jl:867
 in (::Base.LineEdit.##13#14{Base.REPL.##38#47,String})(::Base.LineEdit.MIState, ::Base.REPL.LineEditREPL) at ./LineEdit.jl:736
 in prompt!(::Base.Terminals.TTYTerminal, ::Base.LineEdit.ModalInterface, ::Base.LineEdit.MIState) at ./LineEdit.jl:1605
 in run_interface(::Base.Terminals.TTYTerminal, ::Base.LineEdit.ModalInterface) at ./LineEdit.jl:1574
 in run_frontend(::Base.REPL.LineEditREPL, ::Base.REPL.REPLBackendRef) at ./REPL.jl:903
 in run_repl(::Base.REPL.LineEditREPL, ::Base.##930#931) at ./REPL.jl:188
 in _start() at ./client.jl:360

Op Inv is not available in GraphDef version 20

I get this error doing Pkg.test() on both master and the latest release, after Pkg.build("TensorFlow") and pip install --upgrade tensorflow. I guess it must be due to Python side TF being out of date and putting Inv in the gradient graph, but upgrading to 0.12.1 hasn't helped – do I need to be on master or something?

libtensorflow.so issues when precompiling

I've been having trouble precompiling this package with GPU support, getting error messages like

INFO: Precompiling module TensorFlow.
WARNING: could not import Base.lastidx into LegacyStrings

signal (11): Segmentation fault
while loading /home/calle/.julia/v0.5/TensorFlow/src/TensorFlow.jl, in expression starting on line 132
unknown function (ip: 0x7f8e7e37348f)
unknown function (ip: 0x63e6278aec9fefb6)
Allocations: 5125614 (Pool: 5124555; Big: 1059); GC: 7
ERROR: Failed to precompile TensorFlow to /home/calle/.julia/lib/v0.5/TensorFlow.ji.
 in compilecache(::String) at ./loading.jl:593
 in require(::Symbol) at ./loading.jl:422

After digging and doing some research I think I might have solved it by compiling my own libtensorflow.so and replacing the one downloaded by this package during build.

For anyone else having trouble like this, my process was as follows:

  • clone the tensorflow repository https://github.com/tensorflow/tensorflow and don't forget to check out the correct tag! in my case git checkout tags/v0.11.0
  • install build dependencies as described on https://www.tensorflow.org/versions/r0.11/get_started/os_setup.html#installing-from-sources
  • run ./configure in the tensorflow repo and follow instructions. In my case I chose to configure with cuda 8.0 and compute capability 6.1.
  • build tensorflow library with the command bazel build -c opt --config=cuda tensorflow:libtensorflow_c.so
  • finally, replace $HOME/.julia/v0.5/TensorFlow/deps/usr/bin/libtensorflow_c.sowith the library you just compiled, which is located in bazel-bin/tensorflow/tensorflow_c.so

The thing is, while shipping precompiled libraries with this julia package is convenient, it is prone to give errors for some users which use incompatible hardware/software configurations. I would suggest one of two things (or both)

  • add build instructions like these in the documentation
  • automate building of different versions of the libraries and making sure the correct one will download when installing this package.

RNNCell error: no method matching cat(Operation, ...)

It seems like RNNCell may have gotten broken by the recent Tensor vs Operation changes. Defining

Base.cat(::Type{Operation}, dim, values...) = concat(dim-1, values)

doesn't seem to fix the issue, i.e. I'm trying to do something like

my_rnn_cell = nn.rnn_cell.BasicRNNCell(3)
my_rnn_cell(Variable(ones(10, 5)), Variable(ones(10, 3)))

Thanks for your help!

`ndims` not defined for `Variable`

I am not entirely sure how to use the embedding_lookup but I run into the error below

embedding = get_variable("embedding", [vocab_size, size], Float32)
inputs = nn.embedding_lookup(embedding, input_data)

Results in the error

ERROR: MethodError: no method matching ndims(::TensorFlow.Variable)
Closest candidates are:
  ndims(::Type{Char}) at char.jl:16
  ndims(::Char) at char.jl:15
  ndims(::PyCall.PyBuffer) at /local/home/fredrikb/.julia/v0.5/PyCall/src/pybuffer.jl:50
  ...
 in #embedding_lookup#21(::String, ::String, ::Bool, ::Function, ::TensorFlow.Variable, ::TensorFlow.Tensor) at /local/home/fredrikb/.julia/v0.5/TensorFlow/src/ops/nn.jl:134
 in embedding_lookup(::TensorFlow.Variable, ::TensorFlow.Tensor) at /local/home/fredrikb/.julia/v0.5/TensorFlow/src/ops/nn.jl:134
 in eval_user_input(::Any, ::Base.REPL.REPLBackend) at ./REPL.jl:64

Have I misunderstood something or is there a method definition missing?

There's also a typo gather(params, id; name=name) should be gather(params, ids; name=name) to match the function signature in embedding_lookup.

Minor(?) issue/question and broken link pointer

README: "A wrapper around Tensorflow" that link doesn't work..

Above that text "build failing". [I'm not sure, if it is related.. or a reflection on TensorFlow.jl or the Python Tensorflow package (or even the C code it's built on..).]

I would like to know if this "build failing" is a reason to stay away from TensorFlow.jl (or just a problem on master) and in general if it is as good as the Python package?Does it have the same (complete?) API? Comparable enough, while maybe tutorials you would find for Python/TensorFlow find would not work identically?

That is, I do not know much about TensorFlow (or Google's TensorUnits hardware), just that it is popular.. and is this as good a substitute? [And is Julia more-or-less 1) a good substitute from ANN-work to Python 2) and for data-science in general..? I guess you feel at least Julia valuable, I'm thinking if I can recommend Julia (over Python), if not in general then for what purposes.]

MacOS / OSX GPU support

Hi, just started taking a look at this package, looks nice, great work!

I'm on Mac (Sierra 10.12) and have managed to seemingly get TensorFlow.jl running on my GPU.

Basically, I just followed the instructions here and ran:
bazel build -c opt --config=cuda //tensorflow:libtensorflow.so
and copied the resulting .so file to
~/.julia/v0.5/TensorFlow/deps/usr/bin/libtensorflow_c.dylib.

So, am I able to upload my pre-built libtensorflow_c.dylib somewhere so others can use it? Will that work on other machines or do I need some other build flags (e.g. related to paths to python and cuda)?

If it will work:
What GPU compute capabilities should I compile with?
Any other build flags or ./configure options? I set Google Cloud Platform off and HDFS off and opencl off

Is there anything to test other than julia ~/.julia/v0.5/TensorFlow/test/runtests.jl (which had no errors btw).

Cheers,
Joel


P.S. This is how I was testing whether it used the GPU:

using TensorFlow
const tf = TensorFlow.tensorflow
sess = TensorFlow.Session(config=tf.ConfigProto(log_device_placement=true))
x = rand(100,100)
y = rand(100,100)

a = TensorFlow.constant(x)
b = TensorFlow.Variable(y)

c = a*b
run(sess, TensorFlow.initialize_all_variables())
res = run(sess, c)
Base.Test.@test res ≈ x*y

despite the fact that, even when using the default .dylib downloaded by build.jl, it shows:

I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcublas.dylib locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcudnn.dylib locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcufft.dylib locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcuda.1.dylib locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcurand.dylib locally

The next section shows that it's running on the CPU:

Device mapping: no known devices.
node: /job:localhost/replica:0/task:0/cpu:0
node/Assign: /job:localhost/replica:0/task:0/cpu:0
NoOp: /job:localhost/replica:0/task:0/cpu:0
MatMul: /job:localhost/replica:0/task:0/cpu:0
Const_2: /job:localhost/replica:0/task:0/cpu:0
Const: /job:localhost/replica:0/task:0/cpu:0

With my updated dylib it shows it's using the GPU.

LoadError: Tensorflow error: Status: NodeDef mentions attr 'T' not in Op

Using the example on the first page of the GitHub repo I get an error when running the following line:

minimize_op = train.minimize(optimizer, Loss)

LoadError: Tensorflow error: Status: NodeDef mentions attr 'T' not in Op<name=BroadcastGradientArgs; signature=s0:int32, s1:int32 -> r0:int32, r1:int32>; NodeDef: gradients/node11_grad/BroadcastGradientArgs = BroadcastGradientArgs[T=DT_INT32](gradients/node11_grad/Shape, gradients/node11_grad/Shape_1)

while loading In[5], in expression starting on line 4

in check_status(::TensorFlow.Status) at /Users/Raukhur/.julia/v0.5/TensorFlow/src/core.jl:107
in TensorFlow.Operation(::TensorFlow.NodeDescription) at /Users/Raukhur/.julia/v0.5/TensorFlow/src/core.jl:400
in TensorFlow.Operation(::TensorFlow.tensorflow.NodeDef) at /Users/Raukhur/.julia/v0.5/TensorFlow/src/core.jl:613
in extend_graph(::TensorFlow.Graph, ::Array{Any,1}) at /Users/Raukhur/.julia/v0.5/TensorFlow/src/core.jl:82
in gradients(::TensorFlow.Tensor, ::Array{Any,1}) at /Users/Raukhur/.julia/v0.5/TensorFlow/src/core.jl:1037
in compute_gradients(::TensorFlow.train.AdamOptimizer, ::TensorFlow.Tensor, ::Void) at /Users/Raukhur/.julia/v0.5/TensorFlow/src/train.jl:46
in #minimize#1(::Void, ::Void, ::String, ::Function, ::TensorFlow.train.AdamOptimizer, ::TensorFlow.Tensor) at /Users/Raukhur/.julia/v0.5/TensorFlow/src/train.jl:38
in minimize(::TensorFlow.train.AdamOptimizer, ::TensorFlow.Tensor) at /Users/Raukhur/.julia/v0.5/TensorFlow/src/train.jl:35

VAE script fails on the optimizer constructor

ERROR: LoadError: KeyError: key ::TensorFlow.TF_DataType = 101 not found
 in getindex at ./dict.jl:688 [inlined]
 in tf_to_jl_type(::TensorFlow.TF_DataType) at /Users/idunning/.julia/v0.5/TensorFlow/src/core.jl:816
 in eltype(::TensorFlow.Variable) at /Users/idunning/.julia/v0.5/TensorFlow/src/core.jl:636
 in #apply_gradients#3(::Void, ::String, ::Function, ::TensorFlow.train.GradientDescentOptimizer, ::Array{Tuple{TensorFlow.Tensor,Any},1}) at /Users/idunning/.julia/v0.5/TensorFlow/src/train.jl:67

Same script as last time basically. TF.jl master (haven't rebuilt though?)

Segfault due to empty tensor attribute

e.g.

get_shape(Tensor(Int32[]))

I narrowed this down to the TF_OperationGetAttrTensor which tries to load the empty RawTensor as the "value" attribute of the "Const" op, but I'm not sure where to go from there. Empty RawTensors seem to be created just fine and the code for setting them as an attribute of a NodeDescription seems too simple to be at fault.

pyimport("tensorflow") exception in Atom

I'm using Atom for Julia developement. Unfortunately, I seem to have an issue using TensorFlow from within Atom. When using julia and tensorflow from the terminal everything is OK. using TensorFlow in Atom however throws an exception on the line pyimport("tensorflow") with the following text

PyError (:PyImport_ImportModule) <type 'exceptions.ImportError'>
ImportError('libcudart.so.7.5: cannot open shared object file: No such file or directory',)
  File "/usr/lib/python2.7/site-packages/tensorflow/__init__.py", line 23, in <module>
    from tensorflow.python import *
  File "/usr/lib/python2.7/site-packages/tensorflow/python/__init__.py", line 49, in <module>
    from tensorflow.python import pywrap_tensorflow
  File "/usr/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 28, in <module>
    _pywrap_tensorflow = swig_import_helper()
  File "/usr/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrap_tensorflow', fp, pathname, description)

 in pyerr_check at /local/home/fredrikb/.julia/v0.5/PyCall/src/exception.jl:56 [inlined]
 in pyerr_check at /local/home/fredrikb/.julia/v0.5/PyCall/src/exception.jl:61 [inlined]
 in macro expansion at /local/home/fredrikb/.julia/v0.5/P...

For TensorFlow to work in the regular terminal, I had to set my LD_LIBRARY_PATH such that TensorFlow could find the file libcudart.so.7.5. Any ideas on whether I have to somehow do this from Atom as well, or the problem lies elsewhere?

CUDA issue

I have succesfully installed tensorflow with cuda support and it seems to work under python.
I have also run

ENV["TF_USE_GPU"] = "1"
Pkg.build("TensorFlow")

and the build succeeds. Whe executing using TensorFlow julia crashes and I get the following long error output

julia> using TensorFlow
INFO: Recompiling stale cache file /local/home/fredrikb/.julia/lib/v0.5/TensorFlow.ji for module TensorFlow.
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcurand.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcurand.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcuda.so.1 locally
F tensorflow/stream_executor/cuda/cuda_platform.cc:180] Check failed: ::perftools::gputools::port::Status::OK() == (MultiPlatformManager::RegisterPlatform(std::move(platform))) (OK vs. Internal: platform is already registered with name: "CUDA")

signal (6): Aborted
while loading no file, in expression starting on line 0
gsignal at /usr/bin/../lib64/libc.so.6 (unknown line)
abort at /usr/bin/../lib64/libc.so.6 (unknown line)
_ZN10tensorflow8internal15LogMessageFatalD1Ev at /usr/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so (unknown line)
unknown function (ip: 0x7f089ea47230)
unknown function (ip: 0x7f0b0b292739)
unknown function (ip: 0x7f0b0b29284a)
unknown function (ip: 0x7f0b0b2973a1)
unknown function (ip: 0x7f0b0b2925e3)
unknown function (ip: 0x7f0b0b2966a2)
unknown function (ip: 0x7f0b0aac9fc8)
unknown function (ip: 0x7f0b0b2925e3)
unknown function (ip: 0x7f0b0aaca630)
dlopen at /usr/bin/../lib64/libdl.so.2 (unknown line)
_PyImport_GetDynLoadFunc at /usr/lib64/libpython2.7.so (unknown line)
_PyImport_LoadDynamicModule at /usr/lib64/libpython2.7.so (unknown line)
unknown function (ip: 0x7f08f7e870d6)
PyEval_EvalFrameEx at /usr/lib64/libpython2.7.so (unknown line)
PyEval_EvalFrameEx at /usr/lib64/libpython2.7.so (unknown line)
PyEval_EvalCodeEx at /usr/lib64/libpython2.7.so (unknown line)
PyEval_EvalCode at /usr/lib64/libpython2.7.so (unknown line)
PyImport_ExecCodeModuleEx at /usr/lib64/libpython2.7.so (unknown line)
unknown function (ip: 0x7f08f7e85a91)
unknown function (ip: 0x7f08f7e8671f)
unknown function (ip: 0x7f08f7e86c47)
PyImport_ImportModuleLevel at /usr/lib64/libpython2.7.so (unknown line)
unknown function (ip: 0x7f08f7e698c7)
PyObject_Call at /usr/lib64/libpython2.7.so (unknown line)
PyEval_CallObjectWithKeywords at /usr/lib64/libpython2.7.so (unknown line)
PyEval_EvalFrameEx at /usr/lib64/libpython2.7.so (unknown line)
PyEval_EvalCodeEx at /usr/lib64/libpython2.7.so (unknown line)
PyEval_EvalCode at /usr/lib64/libpython2.7.so (unknown line)
PyImport_ExecCodeModuleEx at /usr/lib64/libpython2.7.so (unknown line)
unknown function (ip: 0x7f08f7e85a91)
unknown function (ip: 0x7f08f7e86f41)
unknown function (ip: 0x7f08f7e8671f)
unknown function (ip: 0x7f08f7e869ae)
PyImport_ImportModuleLevel at /usr/lib64/libpython2.7.so (unknown line)
unknown function (ip: 0x7f08f7e698c7)
PyObject_Call at /usr/lib64/libpython2.7.so (unknown line)
PyEval_CallObjectWithKeywords at /usr/lib64/libpython2.7.so (unknown line)
PyEval_EvalFrameEx at /usr/lib64/libpython2.7.so (unknown line)
PyEval_EvalCodeEx at /usr/lib64/libpython2.7.so (unknown line)
PyEval_EvalCode at /usr/lib64/libpython2.7.so (unknown line)
PyImport_ExecCodeModuleEx at /usr/lib64/libpython2.7.so (unknown line)
unknown function (ip: 0x7f08f7e85a91)
unknown function (ip: 0x7f08f7e86f41)
unknown function (ip: 0x7f08f7e8671f)
unknown function (ip: 0x7f08f7e869ae)
PyImport_ImportModuleLevel at /usr/lib64/libpython2.7.so (unknown line)
unknown function (ip: 0x7f08f7e698c7)
PyObject_Call at /usr/lib64/libpython2.7.so (unknown line)
unknown function (ip: 0x7f08f7dd9554)
PyObject_CallFunction at /usr/lib64/libpython2.7.so (unknown line)
PyImport_Import at /usr/lib64/libpython2.7.so (unknown line)
PyImport_ImportModule at /usr/lib64/libpython2.7.so (unknown line)
macro expansion at /local/home/fredrikb/.julia/v0.5/PyCall/src/exception.jl:78 [inlined]
pyimport at /local/home/fredrikb/.julia/v0.5/PyCall/src/PyCall.jl:387
__init__ at /local/home/fredrikb/.julia/v0.5/TensorFlow/src/TensorFlow.jl:90
unknown function (ip: 0x7f08e221b9cf)
jl_apply_generic at /usr/bin/../lib64/libjulia.so.0.5 (unknown line)
unknown function (ip: 0x7f0b0ad37949)
unknown function (ip: 0x7f0b0ad2ec9c)
jl_restore_incremental at /usr/bin/../lib64/libjulia.so.0.5 (unknown line)
unknown function (ip: 0x7f0afdda8a09)
unknown function (ip: 0x7f0afdda8f13)
unknown function (ip: 0x7f0afddaa4d6)
unknown function (ip: 0x7f0afddaaf0b)
jl_apply_generic at /usr/bin/../lib64/libjulia.so.0.5 (unknown line)
unknown function (ip: 0x7f0b0ad368e6)
jl_toplevel_eval at /usr/bin/../lib64/libjulia.so.0.5 (unknown line)
unknown function (ip: 0x7f0b0ad18f0b)
unknown function (ip: 0x7f0afdd5b889)
unknown function (ip: 0x7f0afdd5b89f)
jl_apply_generic at /usr/bin/../lib64/libjulia.so.0.5 (unknown line)
eval_user_input at ./REPL.jl:64
unknown function (ip: 0x7f08e22164e6)
jl_apply_generic at /usr/bin/../lib64/libjulia.so.0.5 (unknown line)
macro expansion at ./REPL.jl:95 [inlined]
#3 at ./event.jl:68
unknown function (ip: 0x7f08e220d8ef)
jl_apply_generic at /usr/bin/../lib64/libjulia.so.0.5 (unknown line)
unknown function (ip: 0x7f0b0ad272fe)
unknown function (ip: 0xffffffffffffffff)
Allocations: 7854361 (Pool: 7853109; Big: 1252); GC: 13
Aborted (core dumped)

any ideas?

Integrate with JuliaML

Hi @malmaud do you have time/interest to get this package working with the building blocks of JuliaML? Similar to my strategy with Plots, I think it would be a good first step to fine-tune the API of JuliaML using an existing "backend" like TensorFlow. Specifically I'd like to wrap the building of TF networks using the stuff I'm experimenting with in Transformations.jl (tom branch).

I guess you don't need to help much with the coding, but it would be good to know if I can bug you with occasional questions about the package. Thanks

Python process communication error when computing gradients

Thanks for prototyping this package! I'm getting this error trying to construct (not yet run) a reasonably simple RNN model:

LoadError: On worker 2:
PyError (:PyObject_Call) None

 in pyerr_check at /home/schmrlng/.julia/v0.5/PyCall/src/exception.jl:56 [inlined]
 in pyerr_check at /home/schmrlng/.julia/v0.5/PyCall/src/exception.jl:61 [inlined]
 in macro expansion at /home/schmrlng/.julia/v0.5/PyCall/src/exception.jl:81 [inlined]
 in #_pycall#62 at /home/schmrlng/.julia/v0.5/PyCall/src/PyCall.jl:547
 in _pycall at /home/schmrlng/.julia/v0.5/PyCall/src/PyCall.jl:535
 in #pycall#66 at /home/schmrlng/.julia/v0.5/PyCall/src/PyCall.jl:569
 in pycall at /home/schmrlng/.julia/v0.5/PyCall/src/PyCall.jl:569
 in #call#67 at /home/schmrlng/.julia/v0.5/PyCall/src/PyCall.jl:572
 in py_gradients at /home/schmrlng/.julia/v0.5/TensorFlow/src/py.jl:45
 in #65 at /home/schmrlng/.julia/v0.5/TensorFlow/src/core.jl:824
 in #625 at ./multi.jl:1421
 in run_work_thunk at ./multi.jl:1001
 in macro expansion at ./multi.jl:1421 [inlined]
 in #624 at ./event.jl:68
while loading In[23], in expression starting on line 39

 in #remotecall_fetch#606(::Array{Any,1}, ::Function, ::Function, ::Base.Worker) at ./multi.jl:1070
 in remotecall_fetch(::Function, ::Base.Worker) at ./multi.jl:1062
 in #remotecall_fetch#609(::Array{Any,1}, ::Function, ::Function, ::Int64) at ./multi.jl:1080
 in gradients(::TensorFlow.Operation, ::Array{Any,1}) at /home/schmrlng/.julia/v0.5/TensorFlow/src/core.jl:823
 in compute_gradients(::TensorFlow.train.AdamOptimizer, ::TensorFlow.Operation, ::Void) at /home/schmrlng/.julia/v0.5/TensorFlow/src/train.jl:33
 in #minimize#1(::Void, ::Void, ::String, ::Function, ::TensorFlow.train.AdamOptimizer, ::TensorFlow.Operation) at /home/schmrlng/.julia/v0.5/TensorFlow/src/train.jl:25
 in minimize(::TensorFlow.train.AdamOptimizer, ::TensorFlow.Operation) at /home/schmrlng/.julia/v0.5/TensorFlow/src/train.jl:22

In trying to construct a minimal reproduction case (minimizing the Frobenius norm of a variable), I get this different error (maybe related?):

               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.5.0-rc3+0 (2016-08-22 23:43 UTC)
 _/ |\__'_|_|_|\__'_|  |
|__/                   |  x86_64-linux-gnu

julia> using TensorFlow
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcurand.so locally

julia> Z = Variable(ones(4,4))
TensorFlow.Variable(<Operation 'node1' dtype=Float64>,<Operation 'node1/Assign' dtype=Float64>)

julia> loss = reduce_mean(Z .* Z)
<Node node4:1 dtype=Float64>

julia> optimizer = train.AdamOptimizer()
TensorFlow.train.AdamOptimizer(0.01,0.9,0.999,1.0e-8,"adam")

julia> train_op = train.minimize(optimizer, loss)
WARNING: Unrecognized attribute DstT
WARNING: Unrecognized attribute SrcT
ERROR: Tensorflow error: Status: NodeDef missing attr 'DstT' from Op<name=Cast; signature=x:SrcT -> y:DstT; attr=SrcT:type; attr=DstT:type>; NodeDef: gradients/node4_grad/Cast = Cast[SrcT=DT_INT32](gradients/node4_grad/floordiv_1)

 in check_status(::TensorFlow.Status) at /home/schmrlng/.julia/v0.5/TensorFlow/src/core.jl:118
 in TensorFlow.Operation(::TensorFlow.NodeDescription) at /home/schmrlng/.julia/v0.5/TensorFlow/src/core.jl:387
 in TensorFlow.Operation(::TensorFlow.tensorflow.NodeDef) at /home/schmrlng/.julia/v0.5/TensorFlow/src/core.jl:520
 in extend_graph(::TensorFlow.Graph, ::Array{Any,1}) at /home/schmrlng/.julia/v0.5/TensorFlow/src/core.jl:93
 in gradients(::TensorFlow.Tensor, ::Array{Any,1}) at /home/schmrlng/.julia/v0.5/TensorFlow/src/core.jl:897
 in compute_gradients(::TensorFlow.train.AdamOptimizer, ::TensorFlow.Tensor, ::Void) at /home/schmrlng/.julia/v0.5/TensorFlow/src/train.jl:33
 in #minimize#1(::Void, ::Void, ::String, ::Function, ::TensorFlow.train.AdamOptimizer, ::TensorFlow.Tensor) at /home/schmrlng/.julia/v0.5/TensorFlow/src/train.jl:25
 in minimize(::TensorFlow.train.AdamOptimizer, ::TensorFlow.Tensor) at /home/schmrlng/.julia/v0.5/TensorFlow/src/train.jl:22

julia> Z = Variable(ones(4,4))    # trying again from the start
TensorFlow.Variable(<Operation 'node7' dtype=Float64>,<Operation 'node7/Assign' dtype=Float64>)

julia> loss = reduce_mean(Z .* Z)
<Node node10:1 dtype=Float64>

julia> optimizer = train.AdamOptimizer()
TensorFlow.train.AdamOptimizer(0.01,0.9,0.999,1.0e-8,"adam")

julia> train_op = train.minimize(optimizer, loss)
ERROR: On worker 2:
MethodError: no method matching getindex(::Void, ::Symbol)
 in collect at ./array.jl:304
 in py_gradients at /home/schmrlng/.julia/v0.5/TensorFlow/src/py.jl:47
 in #59 at /home/schmrlng/.julia/v0.5/TensorFlow/src/core.jl:895
 in #625 at ./multi.jl:1421
 in run_work_thunk at ./multi.jl:1001
 in macro expansion at ./multi.jl:1421 [inlined]
 in #624 at ./event.jl:68
 in #remotecall_fetch#606(::Array{Any,1}, ::Function, ::Function, ::Base.Worker) at ./multi.jl:1070
 in remotecall_fetch(::Function, ::Base.Worker) at ./multi.jl:1062
 in #remotecall_fetch#609(::Array{Any,1}, ::Function, ::Function, ::Int64) at ./multi.jl:1080
 in gradients(::TensorFlow.Tensor, ::Array{Any,1}) at /home/schmrlng/.julia/v0.5/TensorFlow/src/core.jl:894
 in compute_gradients(::TensorFlow.train.AdamOptimizer, ::TensorFlow.Tensor, ::Void) at /home/schmrlng/.julia/v0.5/TensorFlow/src/train.jl:33
 in #minimize#1(::Void, ::Void, ::String, ::Function, ::TensorFlow.train.AdamOptimizer, ::TensorFlow.Tensor) at /home/schmrlng/.julia/v0.5/TensorFlow/src/train.jl:25
 in minimize(::TensorFlow.train.AdamOptimizer, ::TensorFlow.Tensor) at /home/schmrlng/.julia/v0.5/TensorFlow/src/train.jl:22

The corresponding python version:

Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcurand.so locally
>>> Z = tf.Variable(tf.ones((4,4)))
>>> loss = tf.reduce_mean(tf.mul(Z, Z))
>>> opt = tf.train.AdamOptimizer()
>>> train_op = opt.minimize(loss)
>>>
>>> with tf.Session() as sess:
...     sess.run(tf.initialize_all_variables())
...     for i in range(100):
...         print sess.run([loss, train_op])
...
I tensorflow/core/common_runtime/gpu/gpu_init.cc:102] Found device 0 with properties:
name: GeForce GTX 980
major: 5 minor: 2 memoryClockRate (GHz) 1.2155
pciBusID 0000:02:00.0
Total memory: 4.00GiB
Free memory: 3.91GiB
W tensorflow/stream_executor/cuda/cuda_driver.cc:572] creating context when one is currently active; existing: 0x2f23bc0
I tensorflow/core/common_runtime/gpu/gpu_init.cc:102] Found device 1 with properties:
name: GeForce GTX 980
major: 5 minor: 2 memoryClockRate (GHz) 1.367
pciBusID 0000:01:00.0
Total memory: 4.00GiB
Free memory: 3.85GiB
I tensorflow/core/common_runtime/gpu/gpu_init.cc:126] DMA: 0 1
I tensorflow/core/common_runtime/gpu/gpu_init.cc:136] 0:   Y Y
I tensorflow/core/common_runtime/gpu/gpu_init.cc:136] 1:   Y Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:839] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 980, pci bus id: 0000:02:00.0)
I tensorflow/core/common_runtime/gpu/gpu_device.cc:839] Creating TensorFlow device (/gpu:1) -> (device: 1, name: GeForce GTX 980, pci bus id: 0000:01:00.0)
[1.0, None]
[0.99800104, None]
[0.99600405, None]
...etc...

NodeDef missing attr 'shape' from Op

I'm getting the error message

julia> Operation(node)
ERROR: Tensorflow error: Status: NodeDef missing attr 'shape' from Op<name=Variable; signature= -> ref:Ref(dtype); attr=shape:shape; attr=dtype:type; attr=container:string,default=""; attr=shared_name:string,default=""; is_stateful=true>; NodeDef: hidden1_1/weights = Variable[_class=[], container="", dtype=DT_FLOAT, shared_name=""]()

 in check_status(::TensorFlow.Status) at /Users/yeesian/.julia/v0.5/TensorFlow/src/core.jl:95
 in TensorFlow.Operation(::TensorFlow.NodeDescription) at /Users/yeesian/.julia/v0.5/TensorFlow/src/core.jl:482
 in TensorFlow.Operation(::TensorFlow.tensorflow.NodeDef) at /Users/yeesian/.julia/v0.5/TensorFlow/src/core.jl:672

on the following object:

julia> dump(node)
TensorFlow.tensorflow.NodeDef
  name: String "hidden1_1/weights"
  op: String "Variable"
  input: Array{AbstractString}((0,))
  device: String ""
  attr: Dict{AbstractString,TensorFlow.tensorflow.AttrValue}
    slots: Array{UInt8}((16,)) UInt8[0x01,0x00,0x01,0x00,0x00,0x01,0x01,0x00,0x00,0x01,0x00,0x00,0x00,0x00,0x00,0x00]
    keys: Array{AbstractString}((16,))
      1: String "dtype"
      2: #undef
      3: String "shape"
      4: #undef
      5: #undef
      ...
      12: #undef
      13: #undef
      14: #undef
      15: #undef
      16: #undef
    vals: Array{TensorFlow.tensorflow.AttrValue}((16,))
      1: TensorFlow.tensorflow.AttrValue
        s: #undef
        i: Int64 0
        f: Float32 0.0
        b: Bool false
        _type: Int32 1
        shape: #undef
        tensor: #undef
        list: #undef
        placeholder: #undef
      2: #undef
      3: TensorFlow.tensorflow.AttrValue
        s: #undef
        i: Int64 0
        f: Float32 0.0
        b: Bool false
        _type: Int32 0
        shape: TensorFlow.tensorflow.TensorShapeProto
          dim: Array{TensorFlow.tensorflow.TensorShapeProto_Dim}((2,))
            1: TensorFlow.tensorflow.TensorShapeProto_Dim
              size: Int64 28
              name: String ""
            2: TensorFlow.tensorflow.TensorShapeProto_Dim
              size: Int64 128
              name: String ""
          unknown_rank: Bool false
        tensor: #undef
        list: #undef
        placeholder: #undef
      4: #undef
      5: #undef
      ...
      12: #undef
      13: #undef
      14: #undef
      15: #undef
      16: #undef
    ndel: Int64 0
    count: Int64 5
    age: UInt64 5
    idxfloor: Int64 1
    maxprobe: Int64 0

Access denied: https://storage.googleapis.com/malmaud-stuff/tensorflow_linux_v2.zip

Seems like there might be a permissions issue on that url:

~/.julia/v0.5/TensorFlow/deps/downloads> cat tensorflow.zip 
<?xml version='1.0' encoding='UTF-8'?><Error><Code>AccessDenied</Code><Message>Access denied.</Message><Details>Anonymous users does not have storage.objects.get access to object malmaud-stuff/tensorflow_linux_v2.zip.</Details></Error>

Define `size` for things with `get_shape`

get_shape seems to be TensorFlow for Base.size.
But with the ability for it to be unknown etc.

Eg

get_shape(Ts) 
Out[93]:
TensorShape[407, 16]
In [96]:

get_shape(X_ids)
Out[96]:
TensorShape[unknown]

Does it makes sense to define a overload for Base.size, that returns a tuple of Ints, or throws an exception if the size is unknown?

Cuda capability requirements

On Mint x64 I'm seeing some sort of strange loop with building TensorFlow and then not finding various things. Also, TensorFlow is working fine on this system via the python bindings.

using TensorFlow
INFO: Recompiling stale cache file /home/alex/.julia/lib/v0.5/PyCall.ji for module PyCall.
LoadError: LoadError: LoadError: could not load library "/home/alex/.julia/v0.5/TensorFlow/src/../deps/bazel-out/local_linux-opt/bin/tensorflow/libtensorflow"
libcudart.so.7.5: cannot open shared object file: No such file or directory
while loading /home/alex/.julia/v0.5/TensorFlow/src/core.jl, in expression starting on line 14
while loading /home/alex/.julia/v0.5/TensorFlow/src/TensorFlow.jl, in expression starting on line 80
while loading In[1], in expression starting on line 1

 in dlopen(::String, ::UInt32) at ./libdl.jl:90
 in include_from_node1(::String) at ./loading.jl:488 (repeats 2 times)
 in eval(::Module, ::Any) at ./boot.jl:234
 in require(::Symbol) at ./loading.jl:415

In [3]:ENV["TF_USE_GPU"] = "1"
Pkg.build("TensorFlow")
INFO: Building HttpParser
INFO: Building MbedTLS
INFO: Building Rmath
INFO: Building Blosc
INFO: Building HDF5
INFO: Building PyCall
INFO: PyCall is using /usr/bin/python (Python 2.7.6) at /usr/bin/python, libpython = libpython2.7
INFO: Building TensorFlow
INFO: Building TensorFlow.jl for use on the GPU
Archive:  /home/alex/.julia/v0.5/TensorFlow/deps/downloads/tensorflow.zip
  inflating: libc_api.so             
  inflating: libtensorflow.so        

In [4]:using Tensorflow
LoadError: ArgumentError: Module Tensorflow not found in current path.
Run `Pkg.add("Tensorflow")` to install the Tensorflow package.
while loading In[4], in expression starting on line 1

 in require(::Symbol) at ./loading.jl:365

Can't precompile on windows 64bit

Hi,

I installed the package and the build looks okay to me, but one I try to run using TensorFlow I get the following error:

julia> Pkg.build("TensorFlow")
INFO: Building Rmath
INFO: Building WinRPM
WARNING: Method definition XML_ErrorString(Any) in module LibExpat at C:\Users\BNK\.julia\v0.5\LibExpat\src\lX_common_h.jl:4 overwritten at C:\Users\BNK\.julia\v0.5\LibExpat\src\lX_common_h.jl:4.
WARNING: Method definition (::Type{WinRPM.Packages})(#T<:Union{Base.Set{LibExpat.ETree}, Array{LibExpat.ETree, 1}}) in module WinRPM at C:\Users\BNK\.julia\v0.5\WinRPM\src\WinRPM.jl:207 overwritten at C:\Users\BNK\.julia\v0.5\WinRPM\src\WinRPM.jl:209.
INFO: Downloading https://cache.julialang.org/http://download.opensuse.org/repositories/windows:/mingw:/win32/openSUSE_13.2/repodata/repomd.xml
INFO: Downloading https://cache.julialang.org/http://download.opensuse.org/repositories/windows:/mingw:/win64/openSUSE_13.2/repodata/repomd.xml
INFO: Building Blosc
INFO: Building HDF5
INFO: Updating WinRPM package list
INFO: Downloading https://cache.julialang.org/http://download.opensuse.org/repositories/windows:/mingw:/win32/openSUSE_13.2/repodata/repomd.xml
INFO: Downloading https://cache.julialang.org/http://download.opensuse.org/repositories/windows:/mingw:/win64/openSUSE_13.2/repodata/repomd.xml
INFO: Building PyCall
INFO: PyCall is using C:\Anaconda3\python.exe (Python 3.5.1) at C:\Anaconda3\python.exe, libpython = C:\Anaconda3\python35
INFO: Building HttpParser
INFO: Building MbedTLS
INFO: Building TensorFlow

julia> using TensorFlow
INFO: Precompiling module TensorFlow.
ERROR: Failed to precompile TensorFlow to C:\Users\BNK\.julia\lib\v0.5\TensorFlow.ji.
 in compilecache(::String) at .\loading.jl:593
 in require(::Symbol) at .\loading.jl:422

Is there anything I'm missing here? I'm running windows 7 64bit with AMD processor.

Thanks for the help.

concat uses python indexing (zero based) for dimension

session = Session(Graph())
a_ = placeholder(Float32, shape=[-1,1])
s_ = placeholder(Float32, shape=[-1,2])
sa = concat(1, [s_,a_]) 
run(session,sa, Dict(s_=> [1.,1.]', a_=>[1.]'))

ERROR: LoadError: Tensorflow error: Status: ConcatOp : Expected concatenating dimensions in the range [0, 2), but got 2

This code also produces an error when showing the sa-tensor in the REPL

Op type not registered 'Reciprocal'

Pkg.test("TensorFlow") error:

ERROR: LoadError: LoadError: Tensorflow error: Status: Op type not registered 'Reciprocal' while building NodeDef 'gradients/Log_2_grad/Reciprocal'

 in check_status(::TensorFlow.Status) at /usr/people/jingpeng/.julia/v0.5/TensorFlow/src/core.jl:101
 in TensorFlow.Operation(::TensorFlow.NodeDescription) at /usr/people/jingpeng/.julia/v0.5/TensorFlow/src/core.jl:488
 in TensorFlow.Operation(::TensorFlow.tensorflow.NodeDef) at /usr/people/jingpeng/.julia/v0.5/TensorFlow/src/core.jl:678
 in extend_graph(::TensorFlow.Graph, ::Array{Any,1}) at /usr/people/jingpeng/.julia/v0.5/TensorFlow/src/core.jl:79
 in gradients(::TensorFlow.Tensor, ::Array{Any,1}) at /usr/people/jingpeng/.julia/v0.5/TensorFlow/src/core.jl:951
 in compute_gradients(::TensorFlow.train.AdamOptimizer, ::TensorFlow.Tensor, ::Void) at /usr/people/jingpeng/.julia/v0.5/TensorFlow/src/train.jl:46
 in #minimize#1(::Void, ::Void, ::String, ::Function, ::TensorFlow.train.AdamOptimizer, ::TensorFlow.Tensor) at /usr/people/jingpeng/.julia/v0.5/TensorFlow/src/train.jl:38
 in minimize(::TensorFlow.train.AdamOptimizer, ::TensorFlow.Tensor) at /usr/people/jingpeng/.julia/v0.5/TensorFlow/src/train.jl:35
 in include_from_node1(::String) at ./loading.jl:488 (repeats 2 times)
 in process_options(::Base.JLOptions) at ./client.jl:262
 in _start() at ./client.jl:318
while loading /usr/people/jingpeng/.julia/v0.5/TensorFlow/test/../examples/logistic.jl, in expression starting on line 34
while loading /usr/people/jingpeng/.julia/v0.5/TensorFlow/test/runtests.jl, in expression starting on line 12
========================================[ ERROR: TensorFlow ]=========================================

failed process: Process(`/usr/people/jingpeng/lib/julia/bin/julia -Cx86-64 -J/usr/people/jingpeng/lib/julia/lib/julia/sys.so --compile=yes --depwarn=yes --check-bounds=yes --code-coverage=none --color=yes --compilecache=yes /usr/people/jingpeng/.julia/v0.5/TensorFlow/test/runtests.jl`, ProcessExited(1)) [1]

======================================================================================================
ERROR: TensorFlow had test errors
 in #test#61(::Bool, ::Function, ::Array{AbstractString,1}) at ./pkg/entry.jl:740
 in (::Base.Pkg.Entry.#kw##test)(::Array{Any,1}, ::Base.Pkg.Entry.#test, ::Array{AbstractString,1}) at ./<missing>:0
 in (::Base.Pkg.Dir.##2#3{Array{Any,1},Base.Pkg.Entry.#test,Tuple{Array{AbstractString,1}}})() at ./pkg/dir.jl:31
 in cd(::Base.Pkg.Dir.##2#3{Array{Any,1},Base.Pkg.Entry.#test,Tuple{Array{AbstractString,1}}}, ::String) at ./file.jl:59
 in #cd#1(::Array{Any,1}, ::Function, ::Function, ::Array{AbstractString,1}, ::Vararg{Array{AbstractString,1},N}) at ./pkg/dir.jl:31
 in (::Base.Pkg.Dir.#kw##cd)(::Array{Any,1}, ::Base.Pkg.Dir.#cd, ::Function, ::Array{AbstractString,1}, ::Vararg{Array{AbstractString,1},N}) at ./<missing>:0
 in #test#3(::Bool, ::Function, ::String, ::Vararg{String,N}) at ./pkg/pkg.jl:258
 in test(::String, ::Vararg{String,N}) at ./pkg/pkg.jl:258

I tried upgrading python tensorflow using pip and rebuild, but it won't work.

julia> versioninfo()
Julia Version 0.5.0
Commit 3c9d753 (2016-09-19 18:14 UTC)
Platform Info:
  System: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-5820K CPU @ 3.30GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.7.1 (ORCJIT, haswell)

Issues with summary writers

I'm having issues with the summary_write See the attached code for an example

using TensorFlow
using Distributions
include(Pkg.dir("TensorFlow","examples","mnist_loader.jl"))
loader = DataLoader()
session = Session(Graph())
function weight_variable(shape)
    initial = map(Float32, rand(Normal(0, .001), shape...))
    return Variable(initial)
end
function bias_variable(shape)
    initial = fill(Float32(.1), shape...)
    return Variable(initial)
end
function conv2d(x, W)
    nn.conv2d(x, W, [1, 1, 1, 1], "SAME")
end
function max_pool_2x2(x)
    nn.max_pool(x, [1, 2, 2, 1], [1, 2, 2, 1], "SAME")
end
x = placeholder(Float32)
y_ = placeholder(Float32)
W_conv1 = weight_variable([5, 5, 1, 32])
b_conv1 = bias_variable([32])
x_image = reshape(x, [-1, 28, 28, 1])
h_conv1 = nn.relu(conv2d(x_image, W_conv1) + b_conv1)
h_pool1 = max_pool_2x2(h_conv1)
W_conv2 = weight_variable([5, 5, 32, 64])
b_conv2 = bias_variable([64])
h_conv2 = nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
h_pool2 = max_pool_2x2(h_conv2)
W_fc1 = weight_variable([7*7*64, 1024])
b_fc1 = bias_variable([1024])
h_pool2_flat = reshape(h_pool2, [-1, 7*7*64])
h_fc1 = nn.relu(h_pool2_flat * W_fc1 + b_fc1)
keep_prob = placeholder(Float32)
h_fc1_drop = nn.dropout(h_fc1, keep_prob)
W_fc2 = weight_variable([1024, 10])
b_fc2 = bias_variable([10])
y_conv = nn.softmax(h_fc1_drop * W_fc2 + b_fc2)
cross_entropy = reduce_mean(-reduce_sum(y_.*log(y_conv), reduction_indices=[2]))
train_step = train.minimize(train.AdamOptimizer(1e-4), cross_entropy)
correct_prediction = indmax(y_conv, 2) .== indmax(y_, 2)
accuracy = reduce_mean(cast(correct_prediction, Float32)) # an operation
run(session, initialize_all_variables())
saver = train.Saver()
checkpoint_path = mktempdir()

weight_summary = histogram_summary("Parameters", W_fc2)
summary_writer = train.SummaryWriter("checkpoint_path")

function printtime(t0)
    dt = time()-t0
    "$(Int(dt÷60)):$(round(Int,dt % 60))"
end


    for epoch in 1:500 # 1000 epochs
        batch = next_batch(loader, 50)
        if epoch%50 == 0
            train_accuracy = run(session, accuracy, Dict(x=>batch[1], y_=>batch[2], keep_prob=>1.0))
            info("step $epoch, training accuracy $train_accuracy")
            train.save(saver, session, joinpath(checkpoint_path, "mnist"), global_step=epoch)
        end
        run(session, train_step, Dict(x=>batch[1], y_=>batch[2], keep_prob=>.5))
        summaries = run(session, weight_summary)
        write(summary_writer, summaries, global_step = epoch)
    end

    testx, testy = load_test_set()
    test_accuracy = run(session, accuracy, Dict(x=>testx, y_=>testy, keep_prob=>1.0))
    info("test accuracy $test_accuracy")

The errors produced are

julia> summaries = run(session, weight_summary)
"\nError showing value of type String:
ERROR: UnicodeError: invalid character index
 in next at ./strings/string.jl:92 [inlined]
 in escape_string(::IOContext{Base.Terminals.TTYTerminal}, ::String, ::String) at ./strings/io.jl:140
 in print_quoted at ./strings/io.jl:158 [inlined]
 in show at ./strings/io.jl:72 [inlined]
 in show at ./replutil.jl:4 [inlined]
 in display(::Base.REPL.REPLDisplay{Base.REPL.LineEditREPL}, ::MIME{Symbol("text/plain")}, ::String) at ./REPL.jl:132
 in display(::Base.REPL.REPLDisplay{Base.REPL.LineEditREPL}, ::String) at ./REPL.jl:135
 in display(::String) at ./multimedia.jl:143
 in print_response(::Base.Terminals.TTYTerminal, ::Any, ::Void, ::Bool, ::Bool, ::Void) at ./REPL.jl:154
 in print_response(::Base.REPL.LineEditREPL, ::Any, ::Void, ::Bool, ::Bool) at ./REPL.jl:139
 in (::Base.REPL.##22#23{Bool,Base.REPL.##33#42{Base.REPL.LineEditREPL,Base.REPL.REPLHistoryProvider},Base.REPL.LineEditREPL,Base.LineEdit.Prompt})(::Base.LineEdit.MIState, ::Base.AbstractIOBuffer{Array{UInt8,1}}, ::Bool) at ./REPL.jl:652
 in run_repl(::Base.REPL.LineEditREPL, ::Base.##930#931) at ./REPL.jl:188

and

ERROR: MethodError: no method matching write(::TensorFlow.train.SummaryWriter, ::String; global_step=1)
Closest candidates are:
  write(::TensorFlow.train.SummaryWriter, ::String) at /local/home/fredrikb/.julia/v0.5/TensorFlow/src/summary_writer.jl:46 got unsupported keyword argument "global_step"

My system info

julia> versioninfo()
Julia Version 0.5.0-rc3+0
Commit e6f843b* (2016-08-22 23:43 UTC)
Platform Info:
  System: Linux (x86_64-redhat-linux)
  CPU: Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz
  WORD_SIZE: 64
  BLAS: libopenblas (DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblasp.so.0
  LIBM: libopenlibm
  LLVM: libLLVM-3.7.1 (ORCJIT, haswell)

Difficulties with `nn.sparse_softmax_cross_entropy_with_logits`

I have been having trouble working out how to use nn.sparse_softmax_cross_entropy_with_logits.
See this StackOverflow question

I always seem to get Tensorflow error: Status: Incompatible shapes:
when I run an optimizer over it.

I feel like it would benefit from a test proving it works (and giving an example),
and maybe from some docs if it is different from the python one.

Here is a MWE (smaller than on SO):

using TensorFlow
using Distributions
sess = Session(Graph())
input  = constant([0. 2. 2.; 1. 2. 3.])

variable_scope("myscope", initializer=Normal(0, .1)) do
    global W = get_variable("weights3", [3, 3], Float64)
end

logits = input*W
labels = constant([1, 2])
costs = nn.sparse_softmax_cross_entropy_with_logits(logits, labels)
loss = reduce_mean(-costs)
optimizer = train.minimize(train.AdamOptimizer(0.1), loss)

run(sess, initialize_all_variables())
run(sess, [costs, optimizer])

Gives me:

Tensorflow error: Status: Incompatible shapes: [1,2] vs. [2,3]
	 [[Node: gradients/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits_2/Const/SparseSoftmaxCrossEntropyWithLogits_3/SparseSoftmaxCrossEntropyWithLogits_20_grad/mul = Mul[T=DT_DOUBLE, _class=[], _device="/job:localhost/replica:0/task:0/cpu:0"](gradients/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits_2/Const/SparseSoftmaxCrossEntropyWithLogits_3/SparseSoftmaxCrossEntropyWithLogits_20_grad/ExpandDims, SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits_2/Const/SparseSoftmaxCrossEntropyWithLogits_3/SparseSoftmaxCrossEntropyWithLogits_20:1)]]


 in check_status(::TensorFlow.Status) at /home/ubuntu/.julia/v0.5/TensorFlow/src/core.jl:101
 in run(::TensorFlow.Session, ::Array{TensorFlow.Port,1}, ::Array{Any,1}, ::Array{TensorFlow.Port,1}, ::Array{Ptr{Void},1}) at /home/ubuntu/.julia/v0.5/TensorFlow/src/run.jl:96
 in run(::TensorFlow.Session, ::Array{TensorFlow.Tensor,1}, ::Dict{Any,Any}) at /home/ubuntu/.julia/v0.5/TensorFlow/src/run.jl:143
 in run(::TensorFlow.Session, ::Array{TensorFlow.Tensor,1}) at /home/ubuntu/.julia/v0.5/TensorFlow/src/run.jl:169

WARNING: Unrecognized attributes

I have started receiving a large numer of warnings when using TensorFlow, see below for a small example

using TensorFlow

session = Session(Graph())
a_ = placeholder(Float32, shape=[-1,2])
s_ = placeholder(Float32, shape=[-1,4])
y_ = placeholder(Float32, shape=[-1,1])

variable_scope("Q", initializer=Distributions.Normal(0, .01)) do
    global W1 = get_variable("weights1", [4, 200], Float32)
    global W2 = get_variable("weights2", [200, 200], Float32)
    global W2a= get_variable("weights2a", [2, 200], Float32)
    global W3 = get_variable("weights3", [200, 1], Float32)
end

B1 = Variable(0.01ones(Float32,200), name="bias1")
B2 = Variable(0.01ones(Float32,200), name="bias2")
B3 = Variable(0.01ones(Float32,1  ), name="bias3")

# These lines define the network structure
l1 =         s_*W1  + B1 |> nn.relu
l2 = l1*W2 + a_*W2a + B2 |> nn.relu # Include actions only in second layer
q  =         l2*W3  + B3

sum_square = reduce_mean((y_ - q).^2)

train_step = train.minimize(train.AdamOptimizer(1e-4), sum_square)

Produces the output

julia> train_step = train.minimize(train.AdamOptimizer(1e-4), sum_square)
WARNING: Unrecognized attribute out_type
WARNING: Unrecognized attribute out_type
WARNING: Unrecognized attribute out_type
WARNING: Unrecognized attribute out_type
WARNING: Unrecognized attribute Tidx
WARNING: Unrecognized attribute Tshape
WARNING: Unrecognized attribute Tmultiples
WARNING: Unrecognized attribute out_type
WARNING: Unrecognized attribute out_type
WARNING: Unrecognized attribute Tidx
WARNING: Unrecognized attribute Tidx
WARNING: Unrecognized attribute out_type

Issues initializing variables with VAE

Here is a far-from-minimal reproducible example:
https://gist.github.com/IainNZ/541e8add942a436d813ebdfccc16b96c

I'm running on TensorFlow.jl master, TF CPU version.

ERROR: LoadError: Tensorflow error: Status: Must specify at least one target to fetch or execute.

in check_status(::TensorFlow.Status) at /Users/idunning/.julia/v0.5/TensorFlow/src/core.jl:114
in run(::TensorFlow.Session, ::Array{TensorFlow.Port,1}, ::Array{Any,1}, ::Array{TensorFlow.Port,1}, ::Array{Any,1})

`gather` returns `Variable` with unknown size

I'm running into a problem with a tensor of unknown shape after having used gather

input_data = placeholder(Int32, shape=[20, 20])
embedding = get_variable("embedding", [10000, 200], Float32)
inputs = gather(embedding, input_data)
inputs = TensorFlow.split(1, 20, inputs)
julia> get_shape(inputs)
TensorShape[unknown]

I would like to send this into nn.rnn which produces the error

julia> outputs, state = nn.rnn(cell, inputs, initial_state=initial_state)
ERROR: Shape must be inferable
 in check_shape(::TensorFlow.ShapeInference.TensorShape) at /local/home/fredrikb/.julia/v0.5/TensorFlow/src/ops/rnn_cell.jl:23
 in (::TensorFlow.nn.rnn_cell.BasicRNNCell)(::TensorFlow.Tensor, ::TensorFlow.Tensor) at /local/home/fredrikb/.julia/v0.5/TensorFlow/src/ops/rnn_cell.jl:36
 in (::TensorFlow.nn.##12#13{TensorFlow.nn.rnn_cell.BasicRNNCell})() at /local/home/fredrikb/.julia/v0.5/TensorFlow/src/ops/nn.jl:88


I might be using this the wrong way, but as I understand it this works in python.

The context is https://github.com/baggepinnen/TensorFlow.jl/blob/ptbexample/examples/ptb_word_lm.jl#L165

implicit broadcast in load_proto

load_proto(tensor::tensorflow.TensorProto) currently fails on

julia> dump(tensor)
TensorFlow.tensorflow.TensorProto
  dtype: Int32 1
  tensor_shape: TensorFlow.tensorflow.TensorShapeProto
    dim: Array{TensorFlow.tensorflow.TensorShapeProto_Dim}((2,))
      1: TensorFlow.tensorflow.TensorShapeProto_Dim
        size: Int64 100
        name: String ""
      2: TensorFlow.tensorflow.TensorShapeProto_Dim
        size: Int64 28
        name: String ""
    unknown_rank: Bool false
  version_number: Int32 0
  tensor_content: Array{UInt8}((0,)) UInt8[]
  half_val: Array{Int32}((0,)) Int32[]
  float_val: Array{Float32}((1,)) Float32[1.2]
  double_val: Array{Float64}((0,)) Float64[]
  int_val: Array{Int32}((0,)) Int32[]
  string_val: Array{Array{UInt8,1}}((0,))
  scomplex_val: Array{Float32}((0,)) Float32[]
  int64_val: Array{Int64}((0,)) Int64[]
  bool_val: Array{Bool}((0,)) Bool[]
  dcomplex_val: Array{Float64}((0,)) Float64[]

when attempting to load one of the nodedefs in an example containing the following line:

images = tf.constant(1.2, tf.float32, shape=[100, 28])

In this case, the correct behavior should be:

The argument value can be a constant value, or a list of values of type dtype. If value is a list, then the length of the list must be less than or equal to the number of elements implied by the shape argument (if specified). In the case where the list length is less than the number of elements specified by shape, the last element in the list will be used to fill the remaining entries.

Update: Similar behavior for tf.zeros, where tf.zeros([128]) produces

julia> node_def
name: hidden1_1/zeros
op: Const
attr {
  key: dtype
  value {
    type: Float32
  }
}
attr {
  key: _output_shapes
  value {
      }
}
attr {
  key: value
  value {
    dtype: Float32
    shape: [128]
    content: Float32[0.0]
  }
}

resulting in

julia> Operation(node_def)
ERROR: DimensionMismatch("new dimensions (128,) must be consistent with array size 1")
 in reshape(::Array{Float32,1}, ::Tuple{Int64}) at ./array.jl:102
 in load_proto(::TensorFlow.tensorflow.TensorProto) at /Users/yeesian/.julia/v0.5/TensorFlow/src/core.jl:527
 in TensorFlow.Operation(::TensorFlow.tensorflow.NodeDef) at /Users/yeesian/.julia/v0.5/TensorFlow/src/core.jl:656

transpose not working

I seem to have issues using transpose

julia> B = Variable(ones(Float32,2,2), name="test")
TensorFlow.Variable(<Operation 'test_5'>,<Operation 'test_5/Assign'>)

julia> TensorFlow.transpose(B)
ERROR: Tensorflow error: Status: 1 inputs specified of 2 inputs in Op while building NodeDef 'Transpose_10' using Op<name=Transpose; signature=x:T, perm:int32 -> y:T; attr=T:type>

 in check_status(::TensorFlow.Status) at /local/home/fredrikb/.julia/v0.5/TensorFlow/src/core.jl:109
 in TensorFlow.Operation(::TensorFlow.NodeDescription) at /local/home/fredrikb/.julia/v0.5/TensorFlow/src/core.jl:434
 in #transpose#117(::String, ::Function, ::TensorFlow.Variable) at /local/home/fredrikb/.julia/v0.5/TensorFlow/src/ops.jl:133
 in transpose(::TensorFlow.Variable) at /local/home/fredrikb/.julia/v0.5/TensorFlow/src/ops.jl:126
 in eval_user_input(::Any, ::Base.REPL.REPLBackend) at ./REPL.jl:64
 in macro expansion at ./REPL.jl:95 [inlined]
 in (::Base.REPL.##3#4{Base.REPL.REPLBackend})() at ./event.jl:68

also, there is a warning produced if I try to use a Julia style transpose

julia> B'
WARNING: the no-op `transpose` fallback is deprecated, and no more specific `transpose` method for 
TensorFlow.Variable exists. Consider `permutedims(x, [2, 1])` or writing a specific `transpose(x::TensorFlow.Variable)` method if appropriate.

TF_OperationGetAttrTensor

When shape inference runs on tensors I frequently get:

ccall: could not find function TF_OperationGetAttrTensor in library

For a reproduction you can simply call get_shape on x here, or step through the file line by line in Juno.

Edit: In fact, just running that file seems to cause the error. Do I need to try and get tensorflow master?

I'm also seeing some errors coming from the python process (but the error object is something like PyCall (None)) – any ideas why this might be? Could it be caused by a shape mismatch? Let me know how I can help debug.

Pkg.build() issue

I think I had installed tensorflow using pip before. Also, with brew, I no longer need to use sudo for pip. If possible, I would personally prefer to install tensorflow binaries separately, rather than going through Pkg.build(), given the number of ways one might do so. Alternately the build() could detect the existence and give an appropriate error message to uninstall.

INFO: Building TensorFlow
Archive:  /Users/viral/.julia/v0.6/TensorFlow/deps/downloads/tensorflow.zip
  inflating: libc_api.so             
  inflating: libtensorflow.so        
  inflating: tensorflow-0.10.0rc0-py2-none-any.whl  
sudo: no tty present and no askpass program specified
=============================[ ERROR: TensorFlow ]==============================

LoadError: failed process: Process(`sudo pip install tensorflow-0.10.0rc0-py2-none-any.whl`, ProcessExited(1)) [1]
while loading /Users/viral/.julia/v0.6/TensorFlow/deps/build.jl, in expression starting on line 34

================================================================================

================================[ BUILD ERRORS ]================================

WARNING: TensorFlow had build errors.

 - packages with build errors remain installed in /Users/viral/.julia/v0.6
 - build the package(s) and all dependencies with `Pkg.build("TensorFlow")`
 - build a single package by running its `deps/build.jl` script

================================================================================

Julia 0.5: Precompilation fails (Linux and Windows)

Julia version: 0.5
OS: Linux and Windows

Adding the package works. However, when Julia tries to precompile it one gets import warnings and eventually an precompilation error. So TensorFlow.jl is can not be used with the latest Julia version right now!

julia> using TensorFlow
INFO: Precompiling module TensorFlow.
WARNING: could not import Base.⊻ into TensorFlow
WARNING: could not import Base.lastidx into LegacyStrings
ERROR: Failed to precompile TensorFlow to /home/bauer/.julia/lib/v0.5/TensorFlow.ji.
 in compilecache(::String) at ./loading.jl:593
 in require(::Symbol) at ./loading.jl:422

TensorFlow 0.10 C API changes: Node -> Operation in many cases

TensorFlow in general seems to be a bit finicky about CUDA configuration, e.g., GPU on Linux isn't enabled for my system out of the box using the libraries provided at https://malmaud.github.io/files/linux/tensorflow.zip. I've produced my own libtensorflow.so and libc_api.so by following the general install-from-source instructions here plus these two commands

tensorflow> bazel build -c opt --config=cuda tensorflow:libtensorflow.so
tensorflow> bazel build -c opt --config=cuda //tensorflow/c:c_api

which works for enabling GPU computation on my system, but it seems that the TensorFlow r0.10 release has renamed a bunch of the TF_Node... calls to TF_Operation.... The diff below seems to work for me on the r0.10 pre-release branch here, but I haven't tested it on anything but the most basic models.

diff --git a/src/core.jl b/src/core.jl
index b521fac..0684003 100644
--- a/src/core.jl
+++ b/src/core.jl
@@ -352,7 +352,7 @@ type NodeDescription
     graph::Graph

     function NodeDescription(graph, op_type, node_name)
-        desc = ccall((:TF_NewNode), Ptr{Void}, (Ptr{Void}, Cstring, Cstring), graph.ptr, op_type, node_name)
+        desc = ccall((:TF_NewOperation), Ptr{Void}, (Ptr{Void}, Cstring, Cstring), graph.ptr, op_type, node_name)
         new(desc, graph)
     end

@@ -385,7 +385,7 @@ function Operation(desc::NodeDescription)
     self = Operation()
     self.filled_in = false
     status = Status()
-    ptr = ccall((:TF_FinishNode), Ptr{Void}, (Ptr{Void}, Ptr{Void}), desc.ptr, status.ptr)
+    ptr = ccall((:TF_FinishOperation), Ptr{Void}, (Ptr{Void}, Ptr{Void}), desc.ptr, status.ptr)
     check_status(status)
     self.ptr = ptr
     self.graph = Nullable(desc.graph)
@@ -543,13 +543,13 @@ end

 Returns the name of a node in the computation graph.
 """
-node_name(node::AbstractOperation) = ccall((:TF_NodeName), Cstring, (Ptr{Void},), Operation(node).ptr) |> unsafe_string
+node_name(node::AbstractOperation) = ccall((:TF_OperationName), Cstring, (Ptr{Void},), Operation(node).ptr) |> unsafe_string


 function get_attr_value_proto(node::Operation, attr_name)
     buf = Buffer()
     status = Status()
-    ccall(:TF_NodeGetAttrValueProto, Void, (Ptr{Void}, Cstring, Ptr{Void}, Ptr{Void}), node.ptr, attr_name, buf.ptr, status.ptr)
+    ccall(:TF_OperationGetAttrValueProto, Void, (Ptr{Void}, Cstring, Ptr{Void}, Ptr{Void}), node.ptr, attr_name, buf.ptr, status.ptr)
     check_status(status)
     proto = Array(buf)
     b = IOBuffer()
@@ -781,7 +781,7 @@ end
 function get_proto(node::Operation)
     output = Buffer()
     status = Status()
-    ccall(:TF_NodeToNodeDef, Void, (Ptr{Void}, Ptr{Void}, Ptr{Void}), node.ptr, output.ptr, status.ptr)
+    ccall(:TF_OperationToNodeDef, Void, (Ptr{Void}, Ptr{Void}, Ptr{Void}), node.ptr, output.ptr, status.ptr)
     check_status(status)
     convert(Array, output)
 end
@@ -872,7 +872,7 @@ Returns an operation by searching for its name in the given graph.
 """
 function get_node_by_name(graph::Graph, name::AbstractString)
     name, port = parse_port_name(name)
-    node_ptr = ccall(:TF_GraphNodeByName, Ptr{Void}, (Ptr{Void}, Cstring), graph.ptr, name)
+    node_ptr = ccall(:TF_GraphOperationByName, Ptr{Void}, (Ptr{Void}, Cstring), graph.ptr, name)
     if node_ptr == C_NULL
         return Nullable{Operation}()
     else

Thoughts on higher-level abstractions?

So I think tf.slim is becoming more canonical for TF now (I think its tf.learn now?), but I was wondering if you were planning on including that machinery here or just leaving it for other packages to implement on top of TF.jl?

Parameter and update size mismatch at the REPL

On current master, the following runs fine:

in ~/.julia/v0.5/TensorFlow/examples on master [!]
$ julia mnist_full.jl
INFO: step 1, training accuracy 0.2
INFO: step 101, training accuracy 0.86
INFO: step 201, training accuracy 0.92
INFO: step 301, training accuracy 0.9
INFO: step 401, training accuracy 0.96
INFO: step 501, training accuracy 0.96
INFO: step 601, training accuracy 1.0
INFO: step 701, training accuracy 1.0
INFO: step 801, training accuracy 1.0
INFO: step 901, training accuracy 1.0
INFO: test accuracy 0.9829

But running the same example at the REPL:

  | | |_| | | | (_| |  |  Version 0.5.0 (2016-09-19 18:14 UTC)
 _/ |\__'_|_|_|\__'_|  |  Official http://julialang.org/ release
|__/                   |  x86_64-apple-darwin13.4.0

julia> using TensorFlow

julia> using Distributions

julia> include("mnist_loader.jl")
load_test_set (generic function with 2 methods)

julia> loader = DataLoader()
DataLoader(1,[50325,19329,27586,6726,50656,29647,22463,6880,44484,14361    17765,40061,49035,4989,59588,7918,45536,26133,13169,57917])
...
julia> for i in 1:1000
           batch = next_batch(loader, 50)
           if i%100 == 1
               train_accuracy = run(session, accuracy, Dict(x=>batch[1], y_=>batch[2], keep_prob=>1.0))
               info("step $i, training accuracy $train_accuracy")
           end
           run(session, train_step, Dict(x=>batch[1], y_=>batch[2], keep_prob=>.5))
       end
INFO: step 1, training accuracy 0.14
ERROR: Tensorflow error: Status: Parameters and update must be the same size
         [[Node: node_97 = AssignSub[T=DT_FLOAT, _class=[], use_locking=false, _device="/job:localhost/replica:0/task:0/cpu:0"](node_14, Mul_63)]]

 in check_status(::TensorFlow.Status) at /Users/yeesian/.julia/v0.5/TensorFlow/src/core.jl:101
 in run(::TensorFlow.Session, ::Array{TensorFlow.Port,1}, ::Array{Any,1}, ::Array{TensorFlow.Port,1}, ::Array{Ptr{Void},1}) at /Users/yeesian/.julia/v0.5/TensorFlow/src/run.jl:21
 in run(::TensorFlow.Session, ::Array{TensorFlow.Tensor,1}, ::Dict{TensorFlow.Tensor,Any}) at /Users/yeesian/.julia/v0.5/TensorFlow/src/run.jl:115
 in run(::TensorFlow.Session, ::TensorFlow.Tensor, ::Dict{TensorFlow.Tensor,Any}) at /Users/yeesian/.julia/v0.5/TensorFlow/src/run.jl:127
 in macro expansion; at ./REPL[35]:7 [inlined]
 in anonymous at ./<missing>:?

julia>

Docs for `run` are for `Base.run`

the docs here
read:

Base.run — Function.
run(command)
Run a command object, constructed with backticks. Throws an error if anything goes wrong, including the process exiting with a non-zero status.

which is not relevant.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.