tarrysingh / artificial-intelligence-deep-learning-machine-learning-tutorials Goto Github PK

Most of the people who start out new don't find a latest feed of community hyped resources on ML and DL topics. It would be pretty good if we add a Reddit section.
If you're fine with this suggestion I'll put up a PR with the update 😄

Will close as no description has been provided.

Originally posted by @lorenzodb1 in Yelp/detect-secrets#801 (comment)

Hello! I've found two types of performance issues in your program:

• batch() should be called before map().
• tf.Session being defined repeatedly leads to incremental overhead.

You can make your program more efficient by fixing the above two problems. Here are the tensorflow document and the Stack Overflow post to support this.

Below are detailed issues about batch() should be called before map():

• tensorflow_dl_models/official/wide_deep/wide_deep.py: dataset = dataset.batch(batch_size)(here) should be called before dataset = dataset.map(parse_csv, num_parallel_calls=5)(here).
• tensorflow_dl_models/samples/outreach/blogs/blog_estimators_dataset.py: dataset = dataset.batch(32)(here) should be called before dataset = (tf.data.TextLineDataset(file_path).skip(1).map(decode_csv))(here).
• tensorflow_dl_models/samples/outreach/blogs/blog_custom_estimators.py: .batch(32)(here) should be called before .map(decode_csv, num_parallel_calls=4)(here).
• tensorflow_dl_models/samples/core/get_started/iris_data.py: dataset = dataset.shuffle(1000).repeat().batch(batch_size)(here) should be called before dataset = dataset.map(_parse_line)(here).

Besides, you need to check the function called in map()(e.g., _parse_line called in dataset = dataset.map(_parse_line)) whether to be affected or not to make the changed code work properly. For example, if _parse_line needs data with shape (x, y, z) as its input before fix, it would require data with shape (batch_size, x, y, z).

Below are detailed issues about tf.Session being defined repeatedly:

• tensorflow_dl_models/tutorials/image/cifar10/cifar10_eval.py: with tf.Session() as sess:(here) is defined in function eval_once(here) which is repeatedly called in a loop while True:(here).
• tensorflow_dl_models/research/object_detection/eval_util.py: sess = tf.Session(master, graph=tf.get_default_graph())(here) is defined in function _run_checkpoint_once(here) which is repeatedly called in a loop while True:(here).
• tensorflow_dl_models/research/im2txt/im2txt/evaluate.py: with tf.Session() as sess:(here) is defined in function run_once(here) which is repeatedly called in a loop while True:(here).
• tensorflow_dl_models/research/capsules/experiment.py: session = tf.Session(config=tf.ConfigProto(allow_soft_placement=True))(here) is defined in function run_experiment(here) which is repeatedly called in a loop while paused < 360:(here).
• tensorflow_dl_models/research/street/python/vgsl_model.py: sess = tf.Session('')(here) is defined in a loop while True:(here).
• tensorflow_dl_models/research/skip_thoughts/skip_thoughts/track_perplexity.py: with tf.Session() as sess:(here) is defined in function run_once(here) which is repeatedly called in a loop while True:(here).
• tensorflow_dl_models/research/inception/inception/inception_eval.py: with tf.Session() as sess:(here) is defined in function _eval_once(here) which is repeatedly called in a loop while True:(here).
• tensorflow_dl_models/research/slim/datasets/download_and_convert_cifar10.py: with tf.Session('') as sess:(here) is defined in function _add_to_tfrecord(here) which is repeatedly called in a loop for i in range(_NUM_TRAIN_FILES):(here).
• deep-learning/GANs and Variational Autoencoders/BigGAN-PyTorch/scripts/tfhub/converter.py: sess = tf.Session()(here) is defined in function dump_tfhub_to_hdf5(here) and dump_tfhub_to_hdf5 is called in function convert_biggan(here) which is repeatedly called in a loop for res in RESOLUTIONS:(here).
• deep-learning/udacity-deeplearning/weight-initialization/helper.py: with tf.Session() as session:(here) is defined in function _get_loss_acc(here) which is repeatedly called in a loop for i, (weights, label) in enumerate(weight_init_list):(here).

If you define tf.Session out of the loop and pass tf.Session as a parameter to the loop, your program would be much more efficient.

Looking forward to your reply. Btw, I am very glad to create a PR to fix it if you are too busy.

the following links from
Theano to Keras Models

Not found

• http://nbviewer.jupyter.org/github/leriomaggio/deep-learning-keras-tensorflow/blob/master/1.2%20Introduction%20-%20Theano.ipynb

• http://nbviewer.jupyter.org/github/leriomaggio/deep-learning-keras-tensorflow/blob/master/1.3%20Introduction%20-%20Keras.ipynb

• http://nbviewer.jupyter.org/github/leriomaggio/deep-learning-keras-tensorflow/blob/master/1.4%20(Extra)%20A%20Simple%20Implementation%20of%20ANN%20for%20MNIST.ipynb

• http://nbviewer.jupyter.org/github/leriomaggio/deep-learning-keras-tensorflow/blob/master/2.1%20Supervised%20Learning%20-%20ConvNets.ipynb

• http://nbviewer.jupyter.org/github/leriomaggio/deep-learning-keras-tensorflow/blob/master/2.2.1%20Supervised%20Learning%20-%20ConvNet%20HandsOn%20Part%20I.ipynb

• http://nbviewer.jupyter.org/github/leriomaggio/deep-learning-keras-tensorflow/blob/master/2.2.2%20Supervised%20Learning%20-%20ConvNet%20HandsOn%20Part%20II.ipynb

• http://nbviewer.jupyter.org/github/leriomaggio/deep-learning-keras-tensorflow/blob/master/2.3%20Supervised%20Learning%20-%20Famous%20Models%20with%20Keras.ipynb

@TarrySingh Can you please go to https://travis-ci.org/TarrySingh and turn on the repo switch so that automated testing can begin?

Could you please test and post the result on Fashion-MNIST? https://github.com/zalandoresearch/fashion-mnist

It shares the same size & format as mnist, should be straightforward to integrate.

Hello,I found some performance issues.
The first one is in the definition of dataset ,tensorflow_dl_models/official/mnist/dataset.py,
tf.data.FixedLengthRecordDataset(images_file, 28 * 28, header_bytes=16).map was called without num_parallel_calls.
I think it will increase the efficiency of your program if you add this.

The same issues also exist in tf.data.FixedLengthRecordDataset(
labels_file, 1, header_bytes=8).map ,
.map(decode_csv)),
dataset = dataset.map(decode),
dataset = dataset.map(decode)
and dataset = dataset.map(_parse_line)

Here is the documemtation of tensorflow to support this thing.

The socond one is in thedefinition of Eval,tensorflow_dl_models/research/street/python/vgsl_model.py.
sess = tf.Session('') was repeatedly called and was not closed.
I think it will increase the efficiency and avoid out of memory if you close this session after using it.

The same issues also exist in sess = tf.Session()

Here are two files to support this issue,support1 and support2

Looking forward to your reply. Btw, I am very glad to create a PR to fix it if you are too busy.

I try to start my old laptop Asus for using it with Linux Mint.