Giter Club home page Giter Club logo

s3-pit-restore's Introduction

Script for S3 Restore

This script restores several AWS-S3 folders at once using the S3-pit-restore library. For refrence, here is the official documentation of s3-pit-restore.

We must give data in the input.json file, that will be used by the script. You can run the script by simply running this command:

$ python script.js

Prerequisitex

  • Python 3
  • AWS credentials available in the environment
    • This can be accomplished in various ways:
      • Environment Variables:
        • AWS_ACCESS_KEY_ID
        • AWS_SECRET_ACCESS_KEY
        • AWS_DEFAULT_REGION
      • Your ~/.aws/ files
        • Configured with aws configure

Installing

  • clone the repository
$ git clone https://github.com/yashijain1998/s3-pit-restore.git
  • Go to the repository:
$ cd s3-pit-restore
  • Install s3-pit-restore dependency
$ pip3 install s3-pit-restore
  • Run the script:
$ python script.py

Data for input.json

We need to provide values in input.json.

{
	"dryRun": true,
	"skipDeletion": true,
	"bucket": "ms-versioning-bucket",
	"delimiter": "/class"
	"timestamp": "12-23-2021 15:25:00 +5:30",
	"items": [
		 "level1/level2-1"
	],
	"ignoreList": [
			"/group"
	]
}
  • dryRun

    • If true, it will display a list of file/folder that will be restored but will not perform any actual operations in AWS S3.
    • If false, it will restore the file/folder in AWS S3 according to the timestamp.
  • skipDeletion

    • If true, it will skip deletion of objects created after the mentioned timestamp while restoring
    • If false, it will delete the objects created after the mentioned timestamp while restoring
  • bucket - The name of the AWS S3 bucket in which the restore operation will be performed.

  • timestamp - The date and time at which the folder will be restored. The timestamp format is MM-DD-YYYY HH:MM:SS +UTC. Note: The timestamp must include the timezone offset.

  • items - list the folders which will be restored. Multiple folders/files can be passed

  • delimiter

    • Files would not be fetched from s3 if the file path had a delimiter string in it.
    • If empty string is passed, list-object-versions method works normally.
  • ignoreList

    • The files containing the ignorelist array string will be skipped from further processing in the script.
    • if empty array is passed, script will work normally.

Output

The output is logged in console and in the log file created under directory /logs

Sample Logs

With Dry Run set to True:

[Log Timestamp] [Log Level] [File Path] [Current Version] [Restore Version] [Version Timestamp] [Change Type]

2022-01-03 11:33:30,928 INFO "manpreet/file1.json" "nzstzGyhRjf8ptjEbgAvTj.pcEr3IEu_" "OVE5ZRkM4kNT9VpzpjN7PG.EUvVNRgsd" "2021-12-29 09:59:48+00:00" "modified" 
2022-01-03 11:33:31,139 INFO "manpreet/file3.json" "6rc2V8JLgz_cxOrtZd6niX4RQUBGOlWm" "MZFt6mt7Ffbx_Qho0ITQRD4lxdN8oiT_" "2021-12-29 09:59:48+00:00" "modified" 
2022-01-03 11:33:31,249 INFO "manpreet/file6.json" "nq2ofbnPcci6IqH_cWHNiJux25eY3qbF" "" "2021-12-29 11:56:15+00:00" "created" 

With Dry Run set to False:

[Log Timestamp] [Log Level] [File Path]

2022-01-03 11:49:00,293 INFO manpreet/file1.json
2022-01-03 11:49:00,381 INFO manpreet/file3.json
2022-01-03 11:49:00,439 INFO manpreet/file6.json

s3-pit-restore's People

Contributors

angeloc avatar manpreet-compro avatar kylec32 avatar susantomandal avatar timor-raiman avatar barala avatar yashijain1998 avatar arnauddury avatar josephjoice avatar mikedias avatar mrpatrick avatar jgmchan avatar lutaylor avatar stevesta avatar glennthomasau avatar

Watchers

James Cloos avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.