[readme] rewritten readme
This commit is contained in:
parent
5b397ce181
commit
960e336867
1 changed files with 17 additions and 40 deletions
57
README.org
57
README.org
|
@ -8,47 +8,24 @@ The normal ~aws s3 sync ...~ command only uses the time stamp of files
|
||||||
to decide what files need to be copied. This utility looks at the md5
|
to decide what files need to be copied. This utility looks at the md5
|
||||||
hash of the file contents.
|
hash of the file contents.
|
||||||
|
|
||||||
* How does aws-s3-sync-by-hash do it?
|
* Usage
|
||||||
|
|
||||||
The following is a rough, first draft, pseudo-scala, impression of the process.
|
#+begin_example
|
||||||
|
s3thorp
|
||||||
|
Usage: S3Thorp [options]
|
||||||
|
|
||||||
** constructor
|
-s, --source <value> Source directory to sync to S3
|
||||||
|
-b, --bucket <value> S3 bucket name
|
||||||
|
-p, --prefix <value> Prefix within the S3 Bucket
|
||||||
|
#+end_example
|
||||||
|
|
||||||
val options = Load command line arguments and AWS security keys.
|
* TODO
|
||||||
|
|
||||||
** def sync(): Promise[Upload]
|
- [ ] Improve test coverage
|
||||||
|
- [ ] Create os-native binaries
|
||||||
val uploadPromise = createUploadPromise()
|
- [ ] Replace println with real logging
|
||||||
if options contains delete then createDeletePromise()
|
- [ ] Add support for logging options
|
||||||
else return uploadPromise
|
- [ ] Add support for exclusion filters
|
||||||
|
- [ ] Add support for multi-part uploads for large files
|
||||||
** def createUploadPromise(): Promise[Upload]
|
- [ ] Add support for upload progress - may only be available with
|
||||||
|
multi-part uploads
|
||||||
readdir(options(root))
|
|
||||||
loadS3MetaData
|
|
||||||
filterByHash
|
|
||||||
uploadFile
|
|
||||||
callback(file => uploadedFiles += file)
|
|
||||||
|
|
||||||
** def loadS3MetaData: Stream[S3MetaData]
|
|
||||||
|
|
||||||
HEAD(bucket, key)
|
|
||||||
map (metadata => S3MetaData(localFile, bucket, key, metadata.hash, metadata.lastModified))
|
|
||||||
|
|
||||||
** def filterByHash(p: S3MetaData => Boolean): Stream[S3MetaData]
|
|
||||||
|
|
||||||
md5File(localFile)
|
|
||||||
filter(localHash => options.force || localHash != metadataHash)
|
|
||||||
|
|
||||||
** def uploadFile(upload: Upload): IO[Unit]
|
|
||||||
|
|
||||||
S3Upload(bucket, key, localFile)
|
|
||||||
|
|
||||||
** def createDeletePromise(): Promise[Delete]
|
|
||||||
|
|
||||||
S3AllKeys(bucket, key)
|
|
||||||
filter(remoteKey => localFileExists(remoteFile).negate)
|
|
||||||
|
|
||||||
** def deleteFile(delete: Delete): IO[Unit]
|
|
||||||
|
|
||||||
S3Delete(bucket, key, remoteKey)
|
|
||||||
|
|
Loading…
Reference in a new issue