This script restores several AWS-S3 folders at once using the S3-pit-restore library. For refrence, here is the official documentation of s3-pit-restore.
We must give data in the input.json
file, that will be used by the script.
You can run the script by simply running this command:
$ python script.js
- Python 3
- AWS credentials available in the environment
- This can be accomplished in various ways:
- Environment Variables:
- AWS_ACCESS_KEY_ID
- AWS_SECRET_ACCESS_KEY
- AWS_DEFAULT_REGION
- Your
~/.aws/ files
- Configured with
aws configure
- Configured with
- Environment Variables:
- This can be accomplished in various ways:
- clone the repository
$ git clone https://github.com/yashijain1998/s3-pit-restore.git
- Go to the repository:
$ cd s3-pit-restore
- Install s3-pit-restore dependency
$ pip3 install s3-pit-restore
- Run the script:
$ python script.py
We need to provide values in input.json
.
{
"dryRun": true,
"skipDeletion": true,
"bucket": "ms-versioning-bucket",
"delimiter": "/class"
"timestamp": "12-23-2021 15:25:00 +5:30",
"items": [
"level1/level2-1"
],
"ignoreList": [
"/group"
]
}
-
dryRun
- If true, it will display a list of file/folder that will be restored but will not perform any actual operations in AWS S3.
- If false, it will restore the file/folder in AWS S3 according to the timestamp.
-
skipDeletion
- If true, it will skip deletion of objects created after the mentioned timestamp while restoring
- If false, it will delete the objects created after the mentioned timestamp while restoring
-
bucket - The name of the AWS S3 bucket in which the restore operation will be performed.
-
timestamp - The date and time at which the folder will be restored. The timestamp format is MM-DD-YYYY HH:MM:SS +UTC. Note: The timestamp must include the timezone offset.
-
items - list the folders which will be restored. Multiple folders/files can be passed
-
delimiter
- Files would not be fetched from s3 if the file path had a delimiter string in it.
- If empty string is passed, list-object-versions method works normally.
-
ignoreList
- The files containing the ignorelist array string will be skipped from further processing in the script.
- if empty array is passed, script will work normally.
The output is logged in console and in the log file created under directory /logs
With Dry Run set to True:
[Log Timestamp] [Log Level] [File Path] [Current Version] [Restore Version] [Version Timestamp] [Change Type]
2022-01-03 11:33:30,928 INFO "manpreet/file1.json" "nzstzGyhRjf8ptjEbgAvTj.pcEr3IEu_" "OVE5ZRkM4kNT9VpzpjN7PG.EUvVNRgsd" "2021-12-29 09:59:48+00:00" "modified"
2022-01-03 11:33:31,139 INFO "manpreet/file3.json" "6rc2V8JLgz_cxOrtZd6niX4RQUBGOlWm" "MZFt6mt7Ffbx_Qho0ITQRD4lxdN8oiT_" "2021-12-29 09:59:48+00:00" "modified"
2022-01-03 11:33:31,249 INFO "manpreet/file6.json" "nq2ofbnPcci6IqH_cWHNiJux25eY3qbF" "" "2021-12-29 11:56:15+00:00" "created"
With Dry Run set to False:
[Log Timestamp] [Log Level] [File Path]
2022-01-03 11:49:00,293 INFO manpreet/file1.json
2022-01-03 11:49:00,381 INFO manpreet/file3.json
2022-01-03 11:49:00,439 INFO manpreet/file6.json