bennyfri / tfmaccpp Goto Github PK

View Code? Open in Web Editor NEW

80.0 80.0 22.0 107 KB

A few projects that demo how to develop on Mac with TF and XCode only

C++ 84.86% Python 15.14%

tfmaccpp's People

Contributors

Stargazers

Watchers

tfmaccpp's Issues

Customized freeze_saved_model.cc causing "protobuf::FatalException" While Converting Image to Tensor

I am using TensorFlow 2.4 on Ubuntu 20.04. I understand you're using TensorFlow on Mac, which might be a little different to Ubuntu system. But under the hood, they're both Linux and might not differ significantly.

I wrote my own code following the tutorial step by step and made some modifications. Everything works fine until I added the FreezeSave method. I get it compiled, but receive the following exception when running it.

I comment code backwrad line by line, and realize whenever the FreezeSavedModel is written inside the FreezeSave, the ReadTensorFromImageFile method throws the exception. It's weired because ReadTensorFromImageFile doesn't even call either the FreezeSavedModel or the FreezeSave. I tried to follow the API line by line all the way up according to the backtrace, I couldn't find the smoking gun.

I thought it was the modification I made earlier causes the issue. So I clone the code in this repository and get the same error.

These are the gdb backtrace of the exception being thrown while executing the code in this repository.

2020-12-23 09:32:00.026726: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2099940000 Hz
[libprotobuf FATAL external/com_google_protobuf/src/google/protobuf/map.h:1060] CHECK failed: it != end(): key not found: Reciprocal
terminate called after throwing an instance of 'google::protobuf::FatalException'
  what():  CHECK failed: it != end(): key not found: Reciprocal
--Type <RET> for more, q to quit, c to continue without paging--

Thread 1 "main" received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) backtrace 
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007fffe8603859 in __GI_abort () at abort.c:79
#2  0x00007fffe89d8951 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x00007fffe89e447c in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007fffe89e44e7 in std::terminate() () from /lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007fffe89e4799 in __cxa_throw () from /lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00007fffeb44aed5 in google::protobuf::internal::LogMessage::Finish() [clone .cold] () from /usr/local/lib/libtensorflow_cc.so.2
#7  0x00007ffff50353cd in tensorflow::grappler::NumOutputs(tensorflow::NodeDef const&, tensorflow::GraphDef*) () from /usr/local/lib/libtensorflow_cc.so.2
#8  0x00007ffff4bd25f6 in tensorflow::grappler::ConstantFolding::RunOptimizationPass(tensorflow::grappler::Cluster*, tensorflow::grappler::GrapplerItem*, tensorflow::GraphDef*) ()
   from /usr/local/lib/libtensorflow_cc.so.2
#9  0x00007ffff4bd3175 in tensorflow::grappler::ConstantFolding::Optimize(tensorflow::grappler::Cluster*, tensorflow::grappler::GrapplerItem const&, tensorflow::GraphDef*) ()
   from /usr/local/lib/libtensorflow_cc.so.2
#10 0x00007ffff4aa40fa in tensorflow::grappler::MetaOptimizer::RunOptimizer(tensorflow::grappler::GraphOptimizer*, tensorflow::grappler::Cluster*, tensorflow::grappler::GrapplerItem*, tensorflow::GraphDef*, tensorflow::grappler::MetaOptimizer::GraphOptimizationResult*) () from /usr/local/lib/libtensorflow_cc.so.2
--Type <RET> for more, q to quit, c to continue without paging--
#11 0x00007ffff4aa556d in tensorflow::grappler::MetaOptimizer::OptimizeGraph(tensorflow::grappler::Cluster*, tensorflow::grappler::GrapplerItem&&, tensorflow::GraphDef*) ()
   from /usr/local/lib/libtensorflow_cc.so.2
#12 0x00007ffff4aa69a9 in tensorflow::grappler::MetaOptimizer::OptimizeConsumeItem(tensorflow::grappler::Cluster*, tensorflow::grappler::GrapplerItem&&, tensorflow::GraphDef*) ()
   from /usr/local/lib/libtensorflow_cc.so.2
#13 0x00007ffff4aa88eb in tensorflow::grappler::RunMetaOptimizer(tensorflow::grappler::GrapplerItem&&, tensorflow::ConfigProto const&, tensorflow::DeviceBase*, tensorflow::grappler::Cluster*, tensorflow::GraphDef*) () from /usr/local/lib/libtensorflow_cc.so.2
#14 0x00007ffff4a95b11 in tensorflow::GraphExecutionState::OptimizeGraph(tensorflow::BuildGraphOptions const&, std::unique_ptr<tensorflow::Graph, std::default_delete<tensorflow::Graph> >*, std::unique_ptr<tensorflow::FunctionLibraryDefinition, std::default_delete<tensorflow::FunctionLibraryDefinition> >*) () from /usr/local/lib/libtensorflow_cc.so.2
#15 0x00007ffff4a9675b in tensorflow::GraphExecutionState::BuildGraph(tensorflow::BuildGraphOptions const&, std::unique_ptr<tensorflow::ClientGraph, std::default_delete<tensorflow::ClientGraph> >*) () from /usr/local/lib/libtensorflow_cc.so.2
#16 0x00007ffff4a41250 in tensorflow::DirectSession::CreateGraphs(tensorflow::BuildGraphOptions const&, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::unique_ptr<tensorflow::Graph, std::default_delete<tensorflow::Graph> >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::unique_ptr<tensorflow::Graph, std::default_delete<tensorflow::Graph> > > > >*, std::unique_ptr<tensorflow::FunctionLibraryDefinition, std::default_delete<tensorflow::FunctionLibraryDefinition> >*,--Type <RET> for more, q to quit, c to continue without paging--
 tensorflow::DirectSession::RunStateArgs*, absl::lts_2020_02_25::InlinedVector<tensorflow::DataType, 4ul, std::allocator<tensorflow::DataType> >*, absl::lts_2020_02_25::InlinedVector<tensorflow::DataType, 4ul, std::allocator<tensorflow::DataType> >*, long long*) () from /usr/local/lib/libtensorflow_cc.so.2
#17 0x00007ffff4a42b53 in tensorflow::DirectSession::CreateExecutors(tensorflow::CallableOptions const&, std::unique_ptr<tensorflow::DirectSession::ExecutorsAndKeys, std::default_delete<tensorflow::DirectSession::ExecutorsAndKeys> >*, std::unique_ptr<tensorflow::DirectSession::FunctionInfo, std::default_delete<tensorflow::DirectSession::FunctionInfo> >*, tensorflow::DirectSession::RunStateArgs*) () from /usr/local/lib/libtensorflow_cc.so.2
#18 0x00007ffff4a45418 in tensorflow::DirectSession::GetOrCreateExecutors(absl::lts_2020_02_25::Span<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const>, absl::lts_2020_02_25::Span<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const>, absl::lts_2020_02_25::Span<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const>, tensorflow::DirectSession::ExecutorsAndKeys**, tensorflow::DirectSession::RunStateArgs*) () from /usr/local/lib/libtensorflow_cc.so.2
#19 0x00007ffff4a496d6 in tensorflow::DirectSession::Run(tensorflow::RunOptions const&, std::vector<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, tensorflow::Tensor>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, tensorflow::Tensor> > > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::vector<tensorflow::Tensor, std::allocator<tensorflow::Tensor> >*, tensorflow::RunMetadata*, tensorflow::thread::ThreadPoolOptions const&) () from /usr/local/lib/libtensorflow_cc.so.2
#20 0x00007ffff4a33d08 in tensorflow::DirectSession::Run(tensorflow::RunOptions const&, std::vector<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, tenso--Type <RET> for more, q to quit, c to continue without paging--
rflow::Tensor>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, tensorflow::Tensor> > > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::vector<tensorflow::Tensor, std::allocator<tensorflow::Tensor> >*, tensorflow::RunMetadata*) () from /usr/local/lib/libtensorflow_cc.so.2
#21 0x00007fffeb81640a in tensorflow::ClientSession::Run(tensorflow::RunOptions const&, std::unordered_map<tensorflow::Output, tensorflow::Input::Initializer, tensorflow::OutputHash, std::equal_to<tensorflow::Output>, std::allocator<std::pair<tensorflow::Output const, tensorflow::Input::Initializer> > > const&, std::vector<tensorflow::Output, std::allocator<tensorflow::Output> > const&, std::vector<tensorflow::Operation, std::allocator<tensorflow::Operation> > const&, std::vector<tensorflow::Tensor, std::allocator<tensorflow::Tensor> >*, tensorflow::RunMetadata*) const ()
   from /usr/local/lib/libtensorflow_cc.so.2
#22 0x00007fffeb816547 in tensorflow::ClientSession::Run(std::unordered_map<tensorflow::Output, tensorflow::Input::Initializer, tensorflow::OutputHash, std::equal_to<tensorflow::Output>, std::allocator<std::pair<tensorflow::Output const, tensorflow::Input::Initializer> > > const&, std::vector<tensorflow::Output, std::allocator<tensorflow::Output> > const&, std::vector<tensorflow::Operation, std::allocator<tensorflow::Operation> > const&, std::vector<tensorflow::Tensor, std::allocator<tensorflow::Tensor> >*) const () from /usr/local/lib/libtensorflow_cc.so.2
#23 0x00007fffeb816788 in tensorflow::ClientSession::Run(std::unordered_map<tensorflow::Output, tensorflow::Input::Initializer, tensorflow::OutputHash, std::equal_to<tensorflow::Output>, std::allocator<std::pair<tensorflow::Output const, tensorflow::Input::Initializer> > > const&, std::vector<tensorflow::Output, std::allocator<tensorflow::Output> > const&, std::vector<tensorflow::Tensor, std::allocator<tensorflow::Tensor> >*) const () from /usr/local/lib/libtensorflow_cc.so.2
--Type <RET> for more, q to quit, c to continue without paging--
#24 0x000055555556df6a in CatDogCNN::ReadTensorFromImageFile (this=0x7fffffffd390, file_name="../../data/cats_and_dogs_small/train/cats/cat.470.jpg", outTensor=...) at ../CatDogCNN.cpp:38
#25 0x000055555556e565 in CatDogCNN::ReadFileTensors (this=0x7fffffffd390, base_folder_name="../../data/cats_and_dogs_small/train", v_folder_label=std::vector of length 2, capacity 2 = {...}, 
    file_tensors=std::vector of length 0, capacity 0) at ../CatDogCNN.cpp:60
#26 0x000055555556e972 in CatDogCNN::ReadBatches (this=0x7fffffffd390, base_folder_name="../../data/cats_and_dogs_small/train", v_folder_label=std::vector of length 2, capacity 2 = {...}, 
    batch_size=20, image_batches=std::vector of length 0, capacity 0, label_batches=std::vector of length 0, capacity 0) at ../CatDogCNN.cpp:79
#27 0x000055555556307a in main (argc=1, argv=0x7fffffffd7e8) at ../main.cpp:39

Code was obviously compiled without -Wall, -Wextra or sanitizers

This code breaks under ASan and also generates several warnings when compiling with -Wall and -Wextra. It's useful as a template but overall, is of poor quality

Error building mainV2.cpp

Hi Benny,
I tried building the mainV2.cpp file. I am getting an error which is turning out to be difficult to solve without rebuilding TensorFlow.
ERROR:
/home/rnd/Desktop/Amit/DeepLearning/TF_CC_ExampleRuns/TFMacCpp-master/mainV2/CatDogCNN.cpp:427:26: error: ‘ImageProjectiveTransform’ was not declared in this scope
shifted_images = ImageProjectiveTransform(a_root, aug_tensor_input, transforms, "BILINEAR")

When I was trying to solve this issue, it looks like the ImageProjectiveTransform is from contrib and not built in the main lib.

Did you face such an error?
I notice that you also have a contrib/image folder and there you have a BUILD file, what is the use of that file?

TensorFlow: Built from scratch, ran well for mainV1.cpp. version=1.12.0
Machine: Linux

The image_ops.h dont generated

The articles is so wonderfull.
In third article, I used your BUILD file and run
bazel build -c opt --verbose_failures //tensorflow/contrib/image:image_cc
to build the the cc_library. But I didn't found the image.ops.h in
/tensorflow/bazel-genfiles/tensorflow/contrib/image folder.

How to fix it ?

Can ouput convert to input?

In function AddConvLayer, the returned value should be Input. While tensorflow::ops::MaxPool shoudl defently return an Output value.

How is this possible?

Dropout implementation seems to have an error

I think the code is supposed to divide the remaining nodes by the keep_rate, not the drop_rate. Otherwise, on a drop_rate of 10% you multipy the remaining weights by 10 causing the network trouble in converging. Nice side effect, if you correct this: There is no need for a drop_skip anymore, as drop_rate of 0.0 has exactly the desired effect. Code (based on tf 2.7.0):

tf::Output dropOut(tf::Scope const scope,
                   tf::Input const in,
                   int64_t const inSize,
                   tf::Input const dropRate) {
  tf::Tensor dim{tf::DT_INT64, {1}};
  dim.vec<int64_t>()(0) = inSize;
  auto random{tfo::RandomUniform{scope, dim, tf::DT_DOUBLE}};
  auto const keepRate{tfo::Subtract{scope, 1.0, dropRate}};
  auto const randomKeep{tfo::Add{scope, random, keepRate}};
  auto const binaryKeep{tfo::Floor{scope, randomKeep}};
  return tfo::Multiply{scope, tfo::Div{scope, in, keepRate}, binaryKeep};
}

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.