thorp/README.org

34 lines
1 KiB
Org Mode
Raw Normal View History

2019-04-29 20:10:38 +01:00
* s3thorp
Synchronisation of files with S3 using the hash of the file contents.
Based on Alex Kudlick's JavaScript implementation [[https://github.com/akud/aws-s3-sync-by-hash][aws-s3-sync-by-hash]].
The normal ~aws s3 sync ...~ command only uses the time stamp of files
to decide what files need to be copied. This utility looks at the md5
hash of the file contents.
2019-05-10 22:44:27 +01:00
* Usage
#+begin_example
s3thorp
Usage: S3Thorp [options]
-s, --source <value> Source directory to sync to S3
-b, --bucket <value> S3 bucket name
-p, --prefix <value> Prefix within the S3 Bucket
#+end_example
2019-05-11 07:55:09 +01:00
* TO DO
2019-05-10 22:44:27 +01:00
2019-05-11 07:55:09 +01:00
- [x] Improve test coverage
2019-05-10 22:44:27 +01:00
- [ ] Create os-native binaries
- [ ] Replace println with real logging
- [ ] Add support for logging options
- [ ] Add support for exclusion filters
2019-05-11 18:09:00 +01:00
- [ ] Bulk fetching of Hash values from S3
- [ ] ? When lastModified matches local file, skip calculating local MD5 ?
2019-05-10 22:44:27 +01:00
- [ ] Add support for multi-part uploads for large files
- [ ] Add support for upload progress - may only be available with
multi-part uploads