Giter Club home page Giter Club logo

amazon-sagemaker-feature-store-in-java's Introduction

Java SDK Examples for SageMaker Feature Store

Associated Blog : https://aws.amazon.com/blogs/machine-learning/use-amazon-sagemaker-feature-store-in-a-java-environment/

Execution Steps

This example has been tested on a SageMaker Notebook Instance

  1. git clone https://github.com/aws-samples/amazon-sagemaker-feature-store-in-java.git
  2. Install maven using the below commands in a new terminal.
    • cd /opt
    • sudo wget https://apache.osuosl.org/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz
    • sudo tar xzvf apache-maven-3.6.3-bin.tar.gz
    • export PATH=/opt/apache-maven-3.6.3/bin:$PATH
  3. Back in the cloned folder, run command git fetch --all
  4. git checkout main
  5. cd Java
  6. mvn compile; mvn exec:java -Dexec.mainClass="com.example.sage.FeatureStoreAPIExample"

Java and Maven versions

openjdk 11.0.8-internal 2020-07-14
OpenJDK Runtime Environment (build 11.0.8-internal+0-adhoc..src)
OpenJDK 64-Bit Server VM (build 11.0.8-internal+0-adhoc..src, mixed mode)

Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /opt/apache-maven-3.6.3
Java version: 11.0.8-internal, vendor: N/A, runtime: /home/ec2-user/anaconda3/envs/JupyterSystemEnv
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "4.14.214-118.339.amzn1.x86_64", arch: "amd64", family: "unix"

APIs on Java SDK

AWS SageMaker SDK for Java documentation (Doc)

AWS SageMaker Feature Store SDK for Python documentation (Doc)


FeatureStoreAPIExample Code flow:

  1. Read and infer csv data
  2. Make feature definitions
  3. Make feature records
  4. Delete featureGroups that collide with current featureGroup names
  5. Create new featureGroups with current definitions
  6. Ingest(put) to featureGroups using multi-threading
  7. List feature groups
  8. Describe featureGroups and get records
  9. Delete existing records
  10. Delete created featureGroups
  11. Check Offline Store (Optional - code included)

The End-to-End Java SDK example will make use of the following SageMaker Feature Store APIs:

  • CreateFeatureGroupReques (Doc)
  • ListFeatureGroupsRequest (Doc)
  • DescribeFeatureGroupRequest (Doc)
  • DeleteFeatureGroupRequest (Doc)
  • GetRecordRequest (Doc)
  • PutRecordRequest (Doc)
  • DeleteRecordRequest (Doc)
  • CheckOfflineStorage

The FeatureStoreAPIExample example makes use of custom utility functions for Feature Group and record operations:

  • CsvIO class

    • readCSVIntoList
  • FeatureGroupOperations class

    • createFeatureGroups
    • deleteFeatureGroups
    • getAllFeatureGroups
    • describeFeatureGroups
    • getRecord
    • deleteRecord
    • runFeatureGroupGetTests
    • deleteExistingFeatureGroups
  • FeatureGroupRecordOperations class

    • getStringTimeStamp
    • buildRecord
    • makeRecordsList
    • makeColumnDefinitions
    • getDataType
  • Ingest class Extends Thread class

    • getIngestMetrics
    • ingestRecords
    • putRecordsIntoFG
    • getNumIngested
    • deepCopy
    • run
    • batchIngest
  • PerfMetrics class

    • percentile
    • startTimer
    • endTimer
    • addInterval
    • addMultiIntervals
    • printMetrics
    • getLatencies
    • getTotalTime

amazon-sagemaker-feature-store-in-java's People

Contributors

amazon-auto avatar rags1357 avatar wasabiwarrior92 avatar whosivan avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

Forkers

c24io yuandlamzn

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.