2019-04-29 20:10:38 +01:00
|
|
|
* s3thorp
|
|
|
|
|
|
|
|
|
|
Synchronisation of files with S3 using the hash of the file contents.
|
|
|
|
|
|
|
|
|
|
Based on Alex Kudlick's JavaScript implementation [[https://github.com/akud/aws-s3-sync-by-hash][aws-s3-sync-by-hash]].
|
|
|
|
|
|
|
|
|
|
The normal ~aws s3 sync ...~ command only uses the time stamp of files
|
|
|
|
|
to decide what files need to be copied. This utility looks at the md5
|
|
|
|
|
hash of the file contents.
|
|
|
|
|
|
2019-05-10 22:44:27 +01:00
|
|
|
* Usage
|
|
|
|
|
|
|
|
|
|
#+begin_example
|
|
|
|
|
s3thorp
|
|
|
|
|
Usage: S3Thorp [options]
|
|
|
|
|
|
|
|
|
|
-s, --source <value> Source directory to sync to S3
|
|
|
|
|
-b, --bucket <value> S3 bucket name
|
|
|
|
|
-p, --prefix <value> Prefix within the S3 Bucket
|
|
|
|
|
#+end_example
|
|
|
|
|
|
2019-05-11 07:55:09 +01:00
|
|
|
* TO DO
|
2019-05-10 22:44:27 +01:00
|
|
|
|
2019-05-11 07:55:09 +01:00
|
|
|
- [x] Improve test coverage
|
2019-05-10 22:44:27 +01:00
|
|
|
- [ ] Create os-native binaries
|
2019-05-11 20:18:55 +01:00
|
|
|
- [x] Replace println with real logging
|
2019-05-10 22:44:27 +01:00
|
|
|
- [ ] Add support for logging options
|
|
|
|
|
- [ ] Add support for exclusion filters
|
2019-05-11 18:09:00 +01:00
|
|
|
- [ ] Bulk fetching of Hash values from S3
|
|
|
|
|
- [ ] ? When lastModified matches local file, skip calculating local MD5 ?
|
2019-05-10 22:44:27 +01:00
|
|
|
- [ ] Add support for multi-part uploads for large files
|
|
|
|
|
- [ ] Add support for upload progress - may only be available with
|
|
|
|
|
multi-part uploads
|
