* [scalafmt] Add .scalafmt
* [sbt] Add unused import warning and make them errors
* [core] ConfigurationBuilder remove unused import
* [core] ConfigurationBuilder apply scalafmt
* [domain] reformat
* [storage-api] reformat
* [core] reformat and some refactoring
* [storage-aws] reformat
* [cli] reformat
* [core] post rebase fix up
* [storage-aws] UploaderSuite tidy up
* [domain] MD5Hash tweak
Without doing this we have a file that we don't create a valid md5
hash for. See #103
* [storage-aws] UploaderSuite remove unused import
* [storage-aws] StorageServiceSuite reformatted
* [sbt] consistent settings for all modules
Can't enable as stubbing TransferManager attempts to use the default
constructor for TransferManager.
* [storage-aws] Add AmazonTransferManager
This gives us an interface we can safely stub in unit tests without
the compiler failing the build because TransferManager's default
constructor is deprecated.
* [storage-aws] Add AmazonUpload
Prevents deprecation errors due to deprecated parameters on Transfer.
* [sbt] mode import to top of file
* [changelog] updated
* [readme] updated
* [core] ConfigQuery added sources()
* [cli] ParseArgs allow specifying multiple sources
* [domain,core,cli] Source datatype changed to Path
* [domain] Sources added to hold multiple paths in order
* [domain] Config sources change datatype to Sources
* [core] Scan sources for .thorp.config and include any sources listed
This allows the inclusion of a `.thorp.config` file in a source with a
single line `source = ....` that causes that other source to also be
synched into the same remote prefix as the current source.
* [core] ConfigurationBuilderTest add more pending tests
* [[core] ConfigurationBuilderTest rewrite using loan-pattern for fixtures
* [core] ConfigOptionTest use TemporaryFolder
* [core] ConfigOptionTest remove unused fields
* [cli] ParseArgsTest don't use get on an Option
* [core] ConfigurationBuilderTest don't use get on Either
* [core] TemporaryFolder Move import to top of file
* [core] TemporaryFolder use Try over try-finally
* [core] ConfigurationBuilderTest don't use get on Either
* [core] TemporaryFolders.withDirectory propogate errors
* [core] TemporaryFolders add writeFile and createFile
* [core] PlanBuilderTest create a plan with two sources with unique files in both
* [core] ActionGenerator only upload file by name in first source
create a plan
two sources
same filename in both
- only upload file in first source
* [domain] LastModified with no params is now()
* [core] PlanBuilderTest 2 sources w/remote only in 2nd src do nothing
* [core] PlanBuilderTest 2 sources w/remote only in 2nd src do nothing
* [domain] RemoteKey map to a file when prefix is empty
* [domain] S3ObjectData defaults to empty
* [core] KeyGenerator Avoid delimiter when empty prefix key
* [core] PlanBuilderTest when remote not in sources delete from remote
* [core] PlanBuilderTest extract helper md5Hash()
* [core] PlanBuilderTest one source a file no matching remote key
* [core] PlanBuildingTest file with matching key and hash do nothing
* [core] PlanBuilderTest file w/matching remote key and different hash
* [core] PlanBuilderTest a remote key with and without local file
* [core] DummyStorageService Use wildcards when selecting more than 6 elements
* [changelog] updated
* [core] Wrap Stream[LocalFile] as LocalFiles
* [core] LocalFiles counts files
* [core] LocalFiles sums file lengths
* [core] Restore logFileScan
* [storage-aws] Lister logs when fetching object summaries
* [storage-aws] Extract ListerLogger
* [core] Synchronise use leftMap
* [core] Syncronise extract assemblePlan
* [core] Wrap Stream[Action] in SyncPlan
* [core] Copy the file count and totalSizeBytes across to SyncPlan
* [cli] Program rename actions as syncPlan
* [cli] Program extract thorpArchive def
* [cli] Program extract createPlan def
* [cli] Program refactoring
* [cli] Program remove println showing version
* [cli] Program rename actions parameter as syncPlan
* [core] ThorpArchive add an index to each action
* [cli] Program make SyncPlan available to ThorpArchive
* [core] Pass SyncTotals to Archive
* [domain] Move SyncTotals into module
* [domain] Pass index and SyncTotals to UploadEventListener
* [domain] UploadEventLogger add file count a size progress bars
* [domain] UploadEventLogger better display stability and add file index
* [cli] Index files in correct order
* [cli] Program extends Synchronise
* [core] Rename Synchronise as PlanBuilder
* [cli] Program add test to check actions don't get reordered from plan
* [core] collect file size totals
* [domain] UploadEventLogger include percentage
* [cli] ProgramTest Use wildcards when selecting more than 6 elements
* [changelog] Updated
* [readme] Updated
* [domain] Config Add batch-mode flag
* [core] ConfigOption Add BatchMode option
* [core] ConfigQuery Add batchMode query
Also replaced verbose exists case clauses with a simple contains.
* [core] ConfigOptions added to replace Seq[ConfigOption]
* [core] Syncronise rename method to createPlan
* [cli] Program rename apply as run
* [storage-aws] S3StorageServiceBuilder stop using IO to create object
* [storage-aws] S3StorageServiceBuilder make default service lazy
* [storage-aws] Rename S3ClientCopier => Copier
* [storage-aws] Rename S3ClientDeleter => Deleter
* [storage-aws] Rename S3ClientObjectLister => Lister
* [storage-aws] Only attach upload listener when in batch mode
Only detects batch mode when selected as a command line option
* [core] Synchronise use leftMap rather than swap.map.swap
* [cli] ParseArgs add `-B` and `--batch` options to enable batch mode
* [core] ThorpArchive logs file uploaded when in batch mode
* [storage-aws] ETagGenerator add stub
* [core] MD5HashGenerator add hex and digest helpers
* [domain] MD5Hash can always provide base64 and also digest
Rather that store the base 64 digest some of the time, simply decode
it from the hex hash. The same for the binary digest.
MD5Hash is now cleaner now that it no longer has Option parameters.
* [core] MD5HashGenerator add stubs to allow reading file chunks
* [domain] MD5HashData add sub-objects
* [domain] MD5HashData move back into test where it belongs
* [sbt] add sbt-bloop plugin
* [domain] MD5HashData Add hash of big-file
* [domain] MD5HashData Add hash of big-file
* [core] MD5HashGenerator find end of chunk correctly
* [core] MD5HashGenerator offset is a Long
* [core] MD5HashGenerator don't read past the end of the file
* [storage-aws] ETagGenerator can reproduce ETags
* [storage-aws] ETagGeneratorTest added
* [storate-aws] ETagGenerator refactoring
* [storage-aws] ETageGenerator refactoring
* [core] SyncSuite remove redundant braces
* [storage-api] HashService added
* [storage-aws] S3HashService added
* [core] LocalFileStream refactoring
* [core] integrate HashService and ETagGenerator
* Optimise imports
* [domain] HexEncoder added to replace java 8 only DataTypeConverter
* [core] MD5HashGenerator refactoring
* [core] S3MetaDataEnricher refactoring
* [core] S3MetaDataEnricherSuite refactoring
* [storage-aws] ETagGeneratorTest refactoring
* [storage-aws] StorageServiceSuite refactoring
* [core] S3MetaDataEnricher refactoring
* [core] refactoring
* [storage-aws] refactoring
* [core,storage-{api,aws}] Allow Lister errors to be reported to user
* [cli,core,storage-*] Display simple error message when bucket is invalid
* [core] optimise imports
* [storate-aws] optimise imports
* [core] SyncSuite don't use get on an optional type
