Add training and inference using PyTorch and Keras

// @gautamkmr

Purpose of Jupyter book is not clear. Specifically, https://github.com/aws-samples/machine-learning-using-k8s/blob/master/jupyterhub.md needs to add an introduction on why this is needed.

This is particularly useful for users coming in new to ML from the k8s.

Consolidate https://github.com/aws-samples/machine-learning-using-k8s/blob/master/samples/mnist/inference/mxnet/README.md and https://github.com/aws-samples/machine-learning-using-k8s/blob/master/docs/mnist/inference/mxnet.md.

This is required to show that the model generated by MXNet training is used by MXNet Model Server.

TensorFlow training is giving the following error:

$ kubectl logs tensorflow
python: can't open file '/tmp/models/official/mnist/mnist.py --export_dir /model/': [Errno 2] No such file or directory

This requires a PV to be created and then the model will be generated there instead.

Running TensorFlow + Keras training on Fashion-MNIST shows the following logging statements:

Epoch 1/40
2019-04-16 20:15:01.033264: I tensorflow/core/platform/s3/aws_logging.cc:54] Found secret key
2019-04-16 20:15:01.033390: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.
2019-04-16 20:15:01.077391: I tensorflow/core/platform/s3/aws_logging.cc:54] Found secret key
2019-04-16 20:15:01.077490: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.
2019-04-16 20:15:01.083669: I tensorflow/core/platform/s3/aws_logging.cc:54] Found secret key
2019-04-16 20:15:01.083754: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.

Here is the command to create the pod:

kubectl create -f samples/mnist/training/tensorflow/mnist_train.yaml

This can be trimmed in TensorFlow setup.

@Jeffwan will look at this

Create a CDK stack with the following components:

• GPU-powered EKS cluster
• Install KubeFlow
• Create MXNet construct
• Create ML pipelines for training and prediction

Add sample to show MXNet distributed training, possibly using https://github.com/kubeflow/mxnet-operator

MXNet inference is now documented with #87. These steps are available at https://github.com/aws-samples/machine-learning-using-k8s/blob/master/docs/mnist/inference/mxnet.md.

Now, these need to be converted to run on Amazon EKS.

TensorFlow and Keras output needs to be explained

https://github.com/aws-samples/machine-learning-using-k8s/blob/master/tensorflow-keras.md

Explain the concepts of Epoch and what to look for in the output.

I think aws-samples might not be appropriate for this repo now. Since this is a full tutorial rather than some code snippets blog uses. We should move to aws-labs and rename repo to some names like "kubeflow-lab" or "kubeflow-on-eks"

Export the training model generated by MNIST and use that for inference

Create a code sample that shows using k8s for training and Fargate for inference.

Training Fashion-MNIST model using TensorFlow and Keras as explained at

Running the training gives:

Test accuracy: 0.885599970818
Traceback (most recent call last):
  File "mnist.py", line 89, in <module>
    main()
  File "mnist.py", line 86, in main
    export_model(model, args.model_export_path)
  File "mnist.py", line 69, in export_model
    outputs={t.name:t for t in model.outputs})
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/deprecation.py", line 324, in new_func
    return func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/saved_model/simple_save.py", line 83, in simple_save
    b = builder.SavedModelBuilder(export_dir)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/saved_model/builder_impl.py", line 425, in __init__
    super(SavedModelBuilder, self).__init__(export_dir=export_dir)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/saved_model/builder_impl.py", line 100, in __init__
    "directory: %s" % export_dir)
AssertionError: Export directory already exists. Please specify a different export directory: s3://eks-ml-example/mnist/export/1

This error message should not come. Instead the model should be persisted with a new version number.

@Jeffwan will look into this

Kubeflow community is growing fast. This repository covers basic tutorial for users.
I think strategy for next step is to

• bring different phases of machine learning into picture like data loading, model management and experiment, A/B testing.
• integrate with existing AWS services and simplify user's setup.

Action items could be

1. Integration with EKS component, like how to easily setup PV using CSI Plugin, Ingress Controller.
2. Integration with AWS services, like JupyterHub authentication, logs backend.
3. Integrated E2E production ready solution. Multi-tenancy support, Traffic management(Istio), metrics and log solution, CI/CD.

Once these things done, we can contribute back to community.

Currently, the MNIST training does not export the model. This needs to be exported so that it can be used during serving.

@gautamkmr will look at this

I notice some of examples are for generic purpose like how to deploy Jupyter notebook, enable TensorBoard and also serving part. On the contrary, as Kubeflow community grows fast, these components are included in the latest kubeflow suite. I want to update these tutorials in a kubeflow way and I think it will help customers better leverage entire ML pipeline on EKS.

What do you think?

Our documentation PR will be merged by end of Friday. We should change to native aws way to manage kubeflow and cluster

Show Fashion-MNIST training using TensorFlow on K8s and inference using Sagemaker.

Current TensorFlow inference takes input as mnist_input.json. This should take an image as input and then return results.

@Jeffwan will look at this

Add explanation of MXNet results at https://github.com/aws-samples/machine-learning-using-k8s/blob/master/mxnet.md

• Need to change model to dump training logs to disk
• These need to be stored in S3 bucket
• Then follow the steps at https://github.com/aws-samples/machine-learning-using-k8s/blob/master/docs/tensorboard.md to show TensorBoard visualization

This would be easier for some of the beginner users who may not use Tensorflow for computer vision or NLP.