thorp/build.sbt

138 lines
4.1 KiB
Text
Raw Normal View History

import sbtassembly.AssemblyPlugin.defaultShellScript
inThisBuild(List(
2019-06-30 19:00:10 +01:00
organization := "net.kemitix.thorp",
homepage := Some(url("https://github.com/kemitix/thorp")),
licenses := List("mit" -> url("https://opensource.org/licenses/MIT")),
developers := List(
Developer(
"kemitix",
"Paul Campbell",
"pcampbell@kemitix.net",
url("https://github.kemitix.net")
)
)
))
val commonSettings = Seq(
sonatypeProfileName := "net.kemitix",
scalaVersion := "2.12.8",
scalacOptions ++= Seq(
"-Ywarn-unused-import",
"-Xfatal-warnings",
"-feature",
"-deprecation",
"-unchecked",
"-language:postfixOps",
"-language:higherKinds",
"-Ypartial-unification"),
test in assembly := {}
)
Split into subprojects (#36) * [sbt] define existing single module project as legacyRoot * [sbt] add empty cli module depending on legacyRoot * [cli] move Main to cli module * [cli] move ParseArgs to cli module * [sbt] limit scope of scopt dependency to cli module * [cli] moved logging config to cli module * [cli] rename module directory * [aws-api] added empty module * [sbt] aggregate builds from cli * [aws-lib] add empty module * [core] add empty module * [sbt] add comment graphing module dependencies * [sbt] adjust module dependencies to reflect plan Include legacyRoot at the base until it can be redistributed * [legacy] make some awssdk classes non-private during this transition, these classes being private would cause problems * [aws-lib] create S3ClientBuilder This is copied from the legacy S3Client companion object * [domain] add empty module * [domain] move Bucket into module * [legacy] RemoteKey no longer has dependency on Config * [domain] move RemoteKey into module * [domain] move MD5Hash into module * [legacy] LocalFile no longer had dependency on MD5HashGenerator * [domain] move LocalFile into module * [domain] mode LastModified into module * [domain] move RemoteMetaData into module * [domain] move S3MetaData into module * [domain] move Exclude into module * [domain] move Filter into module * [domain] move KeyModified into module * [domain] move HashModified into module * [domain] RemoteKey.resolve added * [domain] add dependency on scalatest * [domain] LocalFile.resolve added * [legacy] Remove UnitTest * [legacy] optimise imports * [domain] move S3ObjectsData moved into module * [legacy] wrapper for using GeneralProgressListener * [domain] move Config into module * [sbt] move aws-api below legacyRoot in dependencies This will allow use to move S3Client into the aws-api module * [legacy] rename S3Client companion as S3ClientBuilder Preparation to move this into its own file. * Inject Logger via CLI (#34) * [S3Client] refactor defaultClient() * [S3Client] transfermanager explicitly uses the same s3client * [S3ClientPutObjectUploader] refactor putObjectRequest creation * [cli] copy in Logging trait as Logger class * [cli] Main uses Logger * [cli] simplify Logger and pass to Sync.run * [legacy] SyncLogging converted to companion * [cli] Logger info can more easily use levels again * [legacy] LocalFileStream uses injected info * [legacy] S3MetaDataEnricher remove unused Logging * [legacy] ActionGenerator remove unused Logging * [legacy] convert ActionGenerator to an object * [legacy] import log methods from SyncLogging * [legacy] move getS3Status from S3Client to S3MetaDataEnricher * [legact] convert ActionsSubmitter to an object * [legacy] convert LocalFileStream to an object * [legacy] move Action case classes inside companion * [legacy] move UploadEvent case classes inside companion and rename * [legacy] move S3Action case classes into companion * [legacy] convert Sync to an object * [cli] Logger takes verbosity level at construction No longer needs to be passed the whole Config implicitly for each info call. * [legacy] stop passing implicit Config for logging purposes Pass a more specific implicit info: Int => String => Unit instead * [legacy] remove DummyS3Client * [legacy] remove Logging * [legacy] convert MD5HashGenerator to an object * [aws-api] move S3Client into module * [legacy] convert KeyGenerator to an object * [legacy] don't use IO.unsafeRunSync directly * [legacy] refactor/rewrite Sync.run * [legacy] Rewrite sort using a for-comprehension * [legacy] Sync inline sorting * [legacy] SyncLogging rename method * [legacy] repair tests * [sbt] move core module to a dependency of legacyRoot * [sbt] add test dependencies to core module * [core] move classes into module * [aws-lib] move classes into module * [sbt] remove legacy root
2019-06-06 19:24:15 +01:00
val applicationSettings = Seq(
name := "thorp",
Split into subprojects (#36) * [sbt] define existing single module project as legacyRoot * [sbt] add empty cli module depending on legacyRoot * [cli] move Main to cli module * [cli] move ParseArgs to cli module * [sbt] limit scope of scopt dependency to cli module * [cli] moved logging config to cli module * [cli] rename module directory * [aws-api] added empty module * [sbt] aggregate builds from cli * [aws-lib] add empty module * [core] add empty module * [sbt] add comment graphing module dependencies * [sbt] adjust module dependencies to reflect plan Include legacyRoot at the base until it can be redistributed * [legacy] make some awssdk classes non-private during this transition, these classes being private would cause problems * [aws-lib] create S3ClientBuilder This is copied from the legacy S3Client companion object * [domain] add empty module * [domain] move Bucket into module * [legacy] RemoteKey no longer has dependency on Config * [domain] move RemoteKey into module * [domain] move MD5Hash into module * [legacy] LocalFile no longer had dependency on MD5HashGenerator * [domain] move LocalFile into module * [domain] mode LastModified into module * [domain] move RemoteMetaData into module * [domain] move S3MetaData into module * [domain] move Exclude into module * [domain] move Filter into module * [domain] move KeyModified into module * [domain] move HashModified into module * [domain] RemoteKey.resolve added * [domain] add dependency on scalatest * [domain] LocalFile.resolve added * [legacy] Remove UnitTest * [legacy] optimise imports * [domain] move S3ObjectsData moved into module * [legacy] wrapper for using GeneralProgressListener * [domain] move Config into module * [sbt] move aws-api below legacyRoot in dependencies This will allow use to move S3Client into the aws-api module * [legacy] rename S3Client companion as S3ClientBuilder Preparation to move this into its own file. * Inject Logger via CLI (#34) * [S3Client] refactor defaultClient() * [S3Client] transfermanager explicitly uses the same s3client * [S3ClientPutObjectUploader] refactor putObjectRequest creation * [cli] copy in Logging trait as Logger class * [cli] Main uses Logger * [cli] simplify Logger and pass to Sync.run * [legacy] SyncLogging converted to companion * [cli] Logger info can more easily use levels again * [legacy] LocalFileStream uses injected info * [legacy] S3MetaDataEnricher remove unused Logging * [legacy] ActionGenerator remove unused Logging * [legacy] convert ActionGenerator to an object * [legacy] import log methods from SyncLogging * [legacy] move getS3Status from S3Client to S3MetaDataEnricher * [legact] convert ActionsSubmitter to an object * [legacy] convert LocalFileStream to an object * [legacy] move Action case classes inside companion * [legacy] move UploadEvent case classes inside companion and rename * [legacy] move S3Action case classes into companion * [legacy] convert Sync to an object * [cli] Logger takes verbosity level at construction No longer needs to be passed the whole Config implicitly for each info call. * [legacy] stop passing implicit Config for logging purposes Pass a more specific implicit info: Int => String => Unit instead * [legacy] remove DummyS3Client * [legacy] remove Logging * [legacy] convert MD5HashGenerator to an object * [aws-api] move S3Client into module * [legacy] convert KeyGenerator to an object * [legacy] don't use IO.unsafeRunSync directly * [legacy] refactor/rewrite Sync.run * [legacy] Rewrite sort using a for-comprehension * [legacy] Sync inline sorting * [legacy] SyncLogging rename method * [legacy] repair tests * [sbt] move core module to a dependency of legacyRoot * [sbt] add test dependencies to core module * [core] move classes into module * [aws-lib] move classes into module * [sbt] remove legacy root
2019-06-06 19:24:15 +01:00
)
val testDependencies = Seq(
libraryDependencies ++= Seq(
2019-06-11 07:36:58 +01:00
"org.scalatest" %% "scalatest" % "3.0.8" % Test,
2019-06-26 22:13:34 +01:00
"org.scalamock" %% "scalamock" % "4.3.0" % Test
Split into subprojects (#36) * [sbt] define existing single module project as legacyRoot * [sbt] add empty cli module depending on legacyRoot * [cli] move Main to cli module * [cli] move ParseArgs to cli module * [sbt] limit scope of scopt dependency to cli module * [cli] moved logging config to cli module * [cli] rename module directory * [aws-api] added empty module * [sbt] aggregate builds from cli * [aws-lib] add empty module * [core] add empty module * [sbt] add comment graphing module dependencies * [sbt] adjust module dependencies to reflect plan Include legacyRoot at the base until it can be redistributed * [legacy] make some awssdk classes non-private during this transition, these classes being private would cause problems * [aws-lib] create S3ClientBuilder This is copied from the legacy S3Client companion object * [domain] add empty module * [domain] move Bucket into module * [legacy] RemoteKey no longer has dependency on Config * [domain] move RemoteKey into module * [domain] move MD5Hash into module * [legacy] LocalFile no longer had dependency on MD5HashGenerator * [domain] move LocalFile into module * [domain] mode LastModified into module * [domain] move RemoteMetaData into module * [domain] move S3MetaData into module * [domain] move Exclude into module * [domain] move Filter into module * [domain] move KeyModified into module * [domain] move HashModified into module * [domain] RemoteKey.resolve added * [domain] add dependency on scalatest * [domain] LocalFile.resolve added * [legacy] Remove UnitTest * [legacy] optimise imports * [domain] move S3ObjectsData moved into module * [legacy] wrapper for using GeneralProgressListener * [domain] move Config into module * [sbt] move aws-api below legacyRoot in dependencies This will allow use to move S3Client into the aws-api module * [legacy] rename S3Client companion as S3ClientBuilder Preparation to move this into its own file. * Inject Logger via CLI (#34) * [S3Client] refactor defaultClient() * [S3Client] transfermanager explicitly uses the same s3client * [S3ClientPutObjectUploader] refactor putObjectRequest creation * [cli] copy in Logging trait as Logger class * [cli] Main uses Logger * [cli] simplify Logger and pass to Sync.run * [legacy] SyncLogging converted to companion * [cli] Logger info can more easily use levels again * [legacy] LocalFileStream uses injected info * [legacy] S3MetaDataEnricher remove unused Logging * [legacy] ActionGenerator remove unused Logging * [legacy] convert ActionGenerator to an object * [legacy] import log methods from SyncLogging * [legacy] move getS3Status from S3Client to S3MetaDataEnricher * [legact] convert ActionsSubmitter to an object * [legacy] convert LocalFileStream to an object * [legacy] move Action case classes inside companion * [legacy] move UploadEvent case classes inside companion and rename * [legacy] move S3Action case classes into companion * [legacy] convert Sync to an object * [cli] Logger takes verbosity level at construction No longer needs to be passed the whole Config implicitly for each info call. * [legacy] stop passing implicit Config for logging purposes Pass a more specific implicit info: Int => String => Unit instead * [legacy] remove DummyS3Client * [legacy] remove Logging * [legacy] convert MD5HashGenerator to an object * [aws-api] move S3Client into module * [legacy] convert KeyGenerator to an object * [legacy] don't use IO.unsafeRunSync directly * [legacy] refactor/rewrite Sync.run * [legacy] Rewrite sort using a for-comprehension * [legacy] Sync inline sorting * [legacy] SyncLogging rename method * [legacy] repair tests * [sbt] move core module to a dependency of legacyRoot * [sbt] add test dependencies to core module * [core] move classes into module * [aws-lib] move classes into module * [sbt] remove legacy root
2019-06-06 19:24:15 +01:00
)
)
val commandLineParsing = Seq(
libraryDependencies ++= Seq(
"com.github.scopt" %% "scopt" % "4.0.0-RC2"
)
)
val awsSdkDependencies = Seq(
libraryDependencies ++= Seq(
"com.amazonaws" % "aws-java-sdk-s3" % "1.11.596",
Drop AWS SDK V2 client (#41) * [sbt] add scalamock as a test dependency * [aws-lib]SyncSuite: minor layout changes * [aws-lib]SyncSuite: remove test * [core] move SyncSuite to same module as subject it tests * [aws-lib]ThorpS3Client: remove commented lines * [aws-lib] remove PutObject versions of Uploader * [aws-lib] rename to TransferManager to remove Multi-part from name * [aws-lib]TransferManager: change logging prefix * [aws-lib] convert logging classes to objects * [aws-lib] convert ObjectLister to use V1 SDK * [aws-lib] convert Copier to use V1 SDK * [aws-lib] extract S3ObjectsBy{Hash,Key} to objects * [aws-lib]S3ClientSuite: rewrite test using mocks * [aws-lib]TransferManager rewrite using for-comprehension * [aws-lib]Copier: remote bucket name from target remote key * [aws-lib]TransferManager: refactor logging to use IO themselves * [aws-lib] Remove test class MyAmazonS3 * [aws-lib]ObjectLister: optimise imports * [aws-lib] S3ClientSuite remove commented code * [aws-lib]ThropS3ClientSuite update to V1 api * [aws-lib]S3ClientSuite: make test as pending It works okay on its own, but when run as part of a suite it fails. Will look at this again once all V2 SDK is removed. * [aws-lib] convert Deleter to use V1 SDK * [aws-lib] Client Logging remove redundant braces * [aws-lib] stop injecting the V2 SDK * [sbt] remove v2 SDK dependencies * [aws-lib] remove redundant helpers for v2 SDK * [sbt] upgrade aws jackson dependencies The jackson libraries used by AWS have security flaws, but are Java 6 compatible, which AWS want to preserve. * [aws-lib] clean up TransferManager tests
2019-06-07 21:17:14 +01:00
// override the versions AWS uses, which is they do to preserve Java 6 compatibility
"com.fasterxml.jackson.core" % "jackson-databind" % "2.9.9.2",
Drop AWS SDK V2 client (#41) * [sbt] add scalamock as a test dependency * [aws-lib]SyncSuite: minor layout changes * [aws-lib]SyncSuite: remove test * [core] move SyncSuite to same module as subject it tests * [aws-lib]ThorpS3Client: remove commented lines * [aws-lib] remove PutObject versions of Uploader * [aws-lib] rename to TransferManager to remove Multi-part from name * [aws-lib]TransferManager: change logging prefix * [aws-lib] convert logging classes to objects * [aws-lib] convert ObjectLister to use V1 SDK * [aws-lib] convert Copier to use V1 SDK * [aws-lib] extract S3ObjectsBy{Hash,Key} to objects * [aws-lib]S3ClientSuite: rewrite test using mocks * [aws-lib]TransferManager rewrite using for-comprehension * [aws-lib]Copier: remote bucket name from target remote key * [aws-lib]TransferManager: refactor logging to use IO themselves * [aws-lib] Remove test class MyAmazonS3 * [aws-lib]ObjectLister: optimise imports * [aws-lib] S3ClientSuite remove commented code * [aws-lib]ThropS3ClientSuite update to V1 api * [aws-lib]S3ClientSuite: make test as pending It works okay on its own, but when run as part of a suite it fails. Will look at this again once all V2 SDK is removed. * [aws-lib] convert Deleter to use V1 SDK * [aws-lib] Client Logging remove redundant braces * [aws-lib] stop injecting the V2 SDK * [sbt] remove v2 SDK dependencies * [aws-lib] remove redundant helpers for v2 SDK * [sbt] upgrade aws jackson dependencies The jackson libraries used by AWS have security flaws, but are Java 6 compatible, which AWS want to preserve. * [aws-lib] clean up TransferManager tests
2019-06-07 21:17:14 +01:00
"com.fasterxml.jackson.dataformat" % "jackson-dataformat-cbor" % "2.9.9"
Split into subprojects (#36) * [sbt] define existing single module project as legacyRoot * [sbt] add empty cli module depending on legacyRoot * [cli] move Main to cli module * [cli] move ParseArgs to cli module * [sbt] limit scope of scopt dependency to cli module * [cli] moved logging config to cli module * [cli] rename module directory * [aws-api] added empty module * [sbt] aggregate builds from cli * [aws-lib] add empty module * [core] add empty module * [sbt] add comment graphing module dependencies * [sbt] adjust module dependencies to reflect plan Include legacyRoot at the base until it can be redistributed * [legacy] make some awssdk classes non-private during this transition, these classes being private would cause problems * [aws-lib] create S3ClientBuilder This is copied from the legacy S3Client companion object * [domain] add empty module * [domain] move Bucket into module * [legacy] RemoteKey no longer has dependency on Config * [domain] move RemoteKey into module * [domain] move MD5Hash into module * [legacy] LocalFile no longer had dependency on MD5HashGenerator * [domain] move LocalFile into module * [domain] mode LastModified into module * [domain] move RemoteMetaData into module * [domain] move S3MetaData into module * [domain] move Exclude into module * [domain] move Filter into module * [domain] move KeyModified into module * [domain] move HashModified into module * [domain] RemoteKey.resolve added * [domain] add dependency on scalatest * [domain] LocalFile.resolve added * [legacy] Remove UnitTest * [legacy] optimise imports * [domain] move S3ObjectsData moved into module * [legacy] wrapper for using GeneralProgressListener * [domain] move Config into module * [sbt] move aws-api below legacyRoot in dependencies This will allow use to move S3Client into the aws-api module * [legacy] rename S3Client companion as S3ClientBuilder Preparation to move this into its own file. * Inject Logger via CLI (#34) * [S3Client] refactor defaultClient() * [S3Client] transfermanager explicitly uses the same s3client * [S3ClientPutObjectUploader] refactor putObjectRequest creation * [cli] copy in Logging trait as Logger class * [cli] Main uses Logger * [cli] simplify Logger and pass to Sync.run * [legacy] SyncLogging converted to companion * [cli] Logger info can more easily use levels again * [legacy] LocalFileStream uses injected info * [legacy] S3MetaDataEnricher remove unused Logging * [legacy] ActionGenerator remove unused Logging * [legacy] convert ActionGenerator to an object * [legacy] import log methods from SyncLogging * [legacy] move getS3Status from S3Client to S3MetaDataEnricher * [legact] convert ActionsSubmitter to an object * [legacy] convert LocalFileStream to an object * [legacy] move Action case classes inside companion * [legacy] move UploadEvent case classes inside companion and rename * [legacy] move S3Action case classes into companion * [legacy] convert Sync to an object * [cli] Logger takes verbosity level at construction No longer needs to be passed the whole Config implicitly for each info call. * [legacy] stop passing implicit Config for logging purposes Pass a more specific implicit info: Int => String => Unit instead * [legacy] remove DummyS3Client * [legacy] remove Logging * [legacy] convert MD5HashGenerator to an object * [aws-api] move S3Client into module * [legacy] convert KeyGenerator to an object * [legacy] don't use IO.unsafeRunSync directly * [legacy] refactor/rewrite Sync.run * [legacy] Rewrite sort using a for-comprehension * [legacy] Sync inline sorting * [legacy] SyncLogging rename method * [legacy] repair tests * [sbt] move core module to a dependency of legacyRoot * [sbt] add test dependencies to core module * [core] move classes into module * [aws-lib] move classes into module * [sbt] remove legacy root
2019-06-06 19:24:15 +01:00
)
)
val zioDependencies = Seq(
libraryDependencies ++= Seq (
"dev.zio" %% "zio" % "1.0.0-RC10-1"
)
Split into subprojects (#36) * [sbt] define existing single module project as legacyRoot * [sbt] add empty cli module depending on legacyRoot * [cli] move Main to cli module * [cli] move ParseArgs to cli module * [sbt] limit scope of scopt dependency to cli module * [cli] moved logging config to cli module * [cli] rename module directory * [aws-api] added empty module * [sbt] aggregate builds from cli * [aws-lib] add empty module * [core] add empty module * [sbt] add comment graphing module dependencies * [sbt] adjust module dependencies to reflect plan Include legacyRoot at the base until it can be redistributed * [legacy] make some awssdk classes non-private during this transition, these classes being private would cause problems * [aws-lib] create S3ClientBuilder This is copied from the legacy S3Client companion object * [domain] add empty module * [domain] move Bucket into module * [legacy] RemoteKey no longer has dependency on Config * [domain] move RemoteKey into module * [domain] move MD5Hash into module * [legacy] LocalFile no longer had dependency on MD5HashGenerator * [domain] move LocalFile into module * [domain] mode LastModified into module * [domain] move RemoteMetaData into module * [domain] move S3MetaData into module * [domain] move Exclude into module * [domain] move Filter into module * [domain] move KeyModified into module * [domain] move HashModified into module * [domain] RemoteKey.resolve added * [domain] add dependency on scalatest * [domain] LocalFile.resolve added * [legacy] Remove UnitTest * [legacy] optimise imports * [domain] move S3ObjectsData moved into module * [legacy] wrapper for using GeneralProgressListener * [domain] move Config into module * [sbt] move aws-api below legacyRoot in dependencies This will allow use to move S3Client into the aws-api module * [legacy] rename S3Client companion as S3ClientBuilder Preparation to move this into its own file. * Inject Logger via CLI (#34) * [S3Client] refactor defaultClient() * [S3Client] transfermanager explicitly uses the same s3client * [S3ClientPutObjectUploader] refactor putObjectRequest creation * [cli] copy in Logging trait as Logger class * [cli] Main uses Logger * [cli] simplify Logger and pass to Sync.run * [legacy] SyncLogging converted to companion * [cli] Logger info can more easily use levels again * [legacy] LocalFileStream uses injected info * [legacy] S3MetaDataEnricher remove unused Logging * [legacy] ActionGenerator remove unused Logging * [legacy] convert ActionGenerator to an object * [legacy] import log methods from SyncLogging * [legacy] move getS3Status from S3Client to S3MetaDataEnricher * [legact] convert ActionsSubmitter to an object * [legacy] convert LocalFileStream to an object * [legacy] move Action case classes inside companion * [legacy] move UploadEvent case classes inside companion and rename * [legacy] move S3Action case classes into companion * [legacy] convert Sync to an object * [cli] Logger takes verbosity level at construction No longer needs to be passed the whole Config implicitly for each info call. * [legacy] stop passing implicit Config for logging purposes Pass a more specific implicit info: Int => String => Unit instead * [legacy] remove DummyS3Client * [legacy] remove Logging * [legacy] convert MD5HashGenerator to an object * [aws-api] move S3Client into module * [legacy] convert KeyGenerator to an object * [legacy] don't use IO.unsafeRunSync directly * [legacy] refactor/rewrite Sync.run * [legacy] Rewrite sort using a for-comprehension * [legacy] Sync inline sorting * [legacy] SyncLogging rename method * [legacy] repair tests * [sbt] move core module to a dependency of legacyRoot * [sbt] add test dependencies to core module * [core] move classes into module * [aws-lib] move classes into module * [sbt] remove legacy root
2019-06-06 19:24:15 +01:00
)
2019-04-29 20:05:41 +01:00
// cli -> thorp-lib -> storage-aws -> core -> storage-api -> console -> domain
// core -> config -> domain
// config -> filesystem
2019-04-29 20:05:41 +01:00
lazy val thorp = (project in file("."))
.settings(commonSettings)
Restructure sync to use a State with foldLeft around actions (#74) * [changelog] updated * [cli] Program rename parameter * [core] Add AppState * [core] Synchronise rought draft replacement for Sync Uses the AppState * [core] Synchronise as sequential for-comprehensions * [core] Synchronise as nested for-comprehensions * [sbt] thorp(root) depends on cli moduke * [core] Synchronise extract methods * [core] Synchronise rewritten * [core] Synchronise generates actions * [core] Remove AppState * [core] ActionSubmitter remove unused implicit config parameter * [cli] Program rewritten to use Synchronise * [core] Synchronise useValidConfig accepts Logger implicitly * [core] Synchronise reorder methods * [core] Synchronise refactor errorMessages * [core] SyncLogging logRunStart accepts explicit parameters * [core] remove old Sync * [core] Synchronise restore logRunStart * [domain] Terminal add types to public methods and values * [domain] UploadEventLogger force flush to terminal Also make part of the progress message in green. Not flushing, by using println, cause odd behaviour. Works on normal terminal, but not great in an emacs terminal. Oh well. * [core] SyncLogging.logRunFinished remove unused parameters * [cli] Program restore final summary * [storage-aws] remove logging from module * [core] ThorpArchive replaces ActionSubmitter ActionSubmitter implementation becomes UnversionedMirrorArchive * [domain] cleaner upload progress messages * [cli] Program remove unused Logger * [cli] Program rename parameter * [core] SyncSuite use Synchronise * [sbt] Allow storage-aws to share core test classes * [domain] LocalFile stop storing a lambda The lambda breaks the equality test between LocalFile instances. * [core] MD4HashData add missing base64 digest for leafFile * [core] Synchronise drop DoNothing actions * [core] SyncSuite update tests * [sbt] aggregate modules from root module
2019-06-25 08:27:38 +01:00
.aggregate(cli, `thorp-lib`, `storage-aws`, core, `storage-api`, domain)
Split into subprojects (#36) * [sbt] define existing single module project as legacyRoot * [sbt] add empty cli module depending on legacyRoot * [cli] move Main to cli module * [cli] move ParseArgs to cli module * [sbt] limit scope of scopt dependency to cli module * [cli] moved logging config to cli module * [cli] rename module directory * [aws-api] added empty module * [sbt] aggregate builds from cli * [aws-lib] add empty module * [core] add empty module * [sbt] add comment graphing module dependencies * [sbt] adjust module dependencies to reflect plan Include legacyRoot at the base until it can be redistributed * [legacy] make some awssdk classes non-private during this transition, these classes being private would cause problems * [aws-lib] create S3ClientBuilder This is copied from the legacy S3Client companion object * [domain] add empty module * [domain] move Bucket into module * [legacy] RemoteKey no longer has dependency on Config * [domain] move RemoteKey into module * [domain] move MD5Hash into module * [legacy] LocalFile no longer had dependency on MD5HashGenerator * [domain] move LocalFile into module * [domain] mode LastModified into module * [domain] move RemoteMetaData into module * [domain] move S3MetaData into module * [domain] move Exclude into module * [domain] move Filter into module * [domain] move KeyModified into module * [domain] move HashModified into module * [domain] RemoteKey.resolve added * [domain] add dependency on scalatest * [domain] LocalFile.resolve added * [legacy] Remove UnitTest * [legacy] optimise imports * [domain] move S3ObjectsData moved into module * [legacy] wrapper for using GeneralProgressListener * [domain] move Config into module * [sbt] move aws-api below legacyRoot in dependencies This will allow use to move S3Client into the aws-api module * [legacy] rename S3Client companion as S3ClientBuilder Preparation to move this into its own file. * Inject Logger via CLI (#34) * [S3Client] refactor defaultClient() * [S3Client] transfermanager explicitly uses the same s3client * [S3ClientPutObjectUploader] refactor putObjectRequest creation * [cli] copy in Logging trait as Logger class * [cli] Main uses Logger * [cli] simplify Logger and pass to Sync.run * [legacy] SyncLogging converted to companion * [cli] Logger info can more easily use levels again * [legacy] LocalFileStream uses injected info * [legacy] S3MetaDataEnricher remove unused Logging * [legacy] ActionGenerator remove unused Logging * [legacy] convert ActionGenerator to an object * [legacy] import log methods from SyncLogging * [legacy] move getS3Status from S3Client to S3MetaDataEnricher * [legact] convert ActionsSubmitter to an object * [legacy] convert LocalFileStream to an object * [legacy] move Action case classes inside companion * [legacy] move UploadEvent case classes inside companion and rename * [legacy] move S3Action case classes into companion * [legacy] convert Sync to an object * [cli] Logger takes verbosity level at construction No longer needs to be passed the whole Config implicitly for each info call. * [legacy] stop passing implicit Config for logging purposes Pass a more specific implicit info: Int => String => Unit instead * [legacy] remove DummyS3Client * [legacy] remove Logging * [legacy] convert MD5HashGenerator to an object * [aws-api] move S3Client into module * [legacy] convert KeyGenerator to an object * [legacy] don't use IO.unsafeRunSync directly * [legacy] refactor/rewrite Sync.run * [legacy] Rewrite sort using a for-comprehension * [legacy] Sync inline sorting * [legacy] SyncLogging rename method * [legacy] repair tests * [sbt] move core module to a dependency of legacyRoot * [sbt] add test dependencies to core module * [core] move classes into module * [aws-lib] move classes into module * [sbt] remove legacy root
2019-06-06 19:24:15 +01:00
lazy val cli = (project in file("cli"))
.settings(commonSettings)
.settings(mainClass in assembly := Some("net.kemitix.thorp.cli.Main"))
Split into subprojects (#36) * [sbt] define existing single module project as legacyRoot * [sbt] add empty cli module depending on legacyRoot * [cli] move Main to cli module * [cli] move ParseArgs to cli module * [sbt] limit scope of scopt dependency to cli module * [cli] moved logging config to cli module * [cli] rename module directory * [aws-api] added empty module * [sbt] aggregate builds from cli * [aws-lib] add empty module * [core] add empty module * [sbt] add comment graphing module dependencies * [sbt] adjust module dependencies to reflect plan Include legacyRoot at the base until it can be redistributed * [legacy] make some awssdk classes non-private during this transition, these classes being private would cause problems * [aws-lib] create S3ClientBuilder This is copied from the legacy S3Client companion object * [domain] add empty module * [domain] move Bucket into module * [legacy] RemoteKey no longer has dependency on Config * [domain] move RemoteKey into module * [domain] move MD5Hash into module * [legacy] LocalFile no longer had dependency on MD5HashGenerator * [domain] move LocalFile into module * [domain] mode LastModified into module * [domain] move RemoteMetaData into module * [domain] move S3MetaData into module * [domain] move Exclude into module * [domain] move Filter into module * [domain] move KeyModified into module * [domain] move HashModified into module * [domain] RemoteKey.resolve added * [domain] add dependency on scalatest * [domain] LocalFile.resolve added * [legacy] Remove UnitTest * [legacy] optimise imports * [domain] move S3ObjectsData moved into module * [legacy] wrapper for using GeneralProgressListener * [domain] move Config into module * [sbt] move aws-api below legacyRoot in dependencies This will allow use to move S3Client into the aws-api module * [legacy] rename S3Client companion as S3ClientBuilder Preparation to move this into its own file. * Inject Logger via CLI (#34) * [S3Client] refactor defaultClient() * [S3Client] transfermanager explicitly uses the same s3client * [S3ClientPutObjectUploader] refactor putObjectRequest creation * [cli] copy in Logging trait as Logger class * [cli] Main uses Logger * [cli] simplify Logger and pass to Sync.run * [legacy] SyncLogging converted to companion * [cli] Logger info can more easily use levels again * [legacy] LocalFileStream uses injected info * [legacy] S3MetaDataEnricher remove unused Logging * [legacy] ActionGenerator remove unused Logging * [legacy] convert ActionGenerator to an object * [legacy] import log methods from SyncLogging * [legacy] move getS3Status from S3Client to S3MetaDataEnricher * [legact] convert ActionsSubmitter to an object * [legacy] convert LocalFileStream to an object * [legacy] move Action case classes inside companion * [legacy] move UploadEvent case classes inside companion and rename * [legacy] move S3Action case classes into companion * [legacy] convert Sync to an object * [cli] Logger takes verbosity level at construction No longer needs to be passed the whole Config implicitly for each info call. * [legacy] stop passing implicit Config for logging purposes Pass a more specific implicit info: Int => String => Unit instead * [legacy] remove DummyS3Client * [legacy] remove Logging * [legacy] convert MD5HashGenerator to an object * [aws-api] move S3Client into module * [legacy] convert KeyGenerator to an object * [legacy] don't use IO.unsafeRunSync directly * [legacy] refactor/rewrite Sync.run * [legacy] Rewrite sort using a for-comprehension * [legacy] Sync inline sorting * [legacy] SyncLogging rename method * [legacy] repair tests * [sbt] move core module to a dependency of legacyRoot * [sbt] add test dependencies to core module * [core] move classes into module * [aws-lib] move classes into module * [sbt] remove legacy root
2019-06-06 19:24:15 +01:00
.settings(applicationSettings)
.settings(testDependencies)
.enablePlugins(BuildInfoPlugin)
.settings(
buildInfoKeys := Seq[BuildInfoKey](name, version),
buildInfoPackage := "thorp"
)
.settings(Seq(
assemblyOption in assembly := (
assemblyOption in assembly).value
.copy(prependShellScript =
Some(defaultShellScript)),
assemblyJarName in assembly := "thorp"
))
2019-06-17 16:47:01 +01:00
.dependsOn(`thorp-lib`)
lazy val `thorp-lib` = (project in file("thorp-lib"))
.settings(commonSettings)
.settings(assemblyJarName in assembly := "thorp-lib.jar")
.dependsOn(`storage-aws`)
lazy val `storage-aws` = (project in file("storage-aws"))
.settings(commonSettings)
.settings(assemblyJarName in assembly := "storage-aws.jar")
Split into subprojects (#36) * [sbt] define existing single module project as legacyRoot * [sbt] add empty cli module depending on legacyRoot * [cli] move Main to cli module * [cli] move ParseArgs to cli module * [sbt] limit scope of scopt dependency to cli module * [cli] moved logging config to cli module * [cli] rename module directory * [aws-api] added empty module * [sbt] aggregate builds from cli * [aws-lib] add empty module * [core] add empty module * [sbt] add comment graphing module dependencies * [sbt] adjust module dependencies to reflect plan Include legacyRoot at the base until it can be redistributed * [legacy] make some awssdk classes non-private during this transition, these classes being private would cause problems * [aws-lib] create S3ClientBuilder This is copied from the legacy S3Client companion object * [domain] add empty module * [domain] move Bucket into module * [legacy] RemoteKey no longer has dependency on Config * [domain] move RemoteKey into module * [domain] move MD5Hash into module * [legacy] LocalFile no longer had dependency on MD5HashGenerator * [domain] move LocalFile into module * [domain] mode LastModified into module * [domain] move RemoteMetaData into module * [domain] move S3MetaData into module * [domain] move Exclude into module * [domain] move Filter into module * [domain] move KeyModified into module * [domain] move HashModified into module * [domain] RemoteKey.resolve added * [domain] add dependency on scalatest * [domain] LocalFile.resolve added * [legacy] Remove UnitTest * [legacy] optimise imports * [domain] move S3ObjectsData moved into module * [legacy] wrapper for using GeneralProgressListener * [domain] move Config into module * [sbt] move aws-api below legacyRoot in dependencies This will allow use to move S3Client into the aws-api module * [legacy] rename S3Client companion as S3ClientBuilder Preparation to move this into its own file. * Inject Logger via CLI (#34) * [S3Client] refactor defaultClient() * [S3Client] transfermanager explicitly uses the same s3client * [S3ClientPutObjectUploader] refactor putObjectRequest creation * [cli] copy in Logging trait as Logger class * [cli] Main uses Logger * [cli] simplify Logger and pass to Sync.run * [legacy] SyncLogging converted to companion * [cli] Logger info can more easily use levels again * [legacy] LocalFileStream uses injected info * [legacy] S3MetaDataEnricher remove unused Logging * [legacy] ActionGenerator remove unused Logging * [legacy] convert ActionGenerator to an object * [legacy] import log methods from SyncLogging * [legacy] move getS3Status from S3Client to S3MetaDataEnricher * [legact] convert ActionsSubmitter to an object * [legacy] convert LocalFileStream to an object * [legacy] move Action case classes inside companion * [legacy] move UploadEvent case classes inside companion and rename * [legacy] move S3Action case classes into companion * [legacy] convert Sync to an object * [cli] Logger takes verbosity level at construction No longer needs to be passed the whole Config implicitly for each info call. * [legacy] stop passing implicit Config for logging purposes Pass a more specific implicit info: Int => String => Unit instead * [legacy] remove DummyS3Client * [legacy] remove Logging * [legacy] convert MD5HashGenerator to an object * [aws-api] move S3Client into module * [legacy] convert KeyGenerator to an object * [legacy] don't use IO.unsafeRunSync directly * [legacy] refactor/rewrite Sync.run * [legacy] Rewrite sort using a for-comprehension * [legacy] Sync inline sorting * [legacy] SyncLogging rename method * [legacy] repair tests * [sbt] move core module to a dependency of legacyRoot * [sbt] add test dependencies to core module * [core] move classes into module * [aws-lib] move classes into module * [sbt] remove legacy root
2019-06-06 19:24:15 +01:00
.settings(awsSdkDependencies)
.settings(testDependencies)
Restructure sync to use a State with foldLeft around actions (#74) * [changelog] updated * [cli] Program rename parameter * [core] Add AppState * [core] Synchronise rought draft replacement for Sync Uses the AppState * [core] Synchronise as sequential for-comprehensions * [core] Synchronise as nested for-comprehensions * [sbt] thorp(root) depends on cli moduke * [core] Synchronise extract methods * [core] Synchronise rewritten * [core] Synchronise generates actions * [core] Remove AppState * [core] ActionSubmitter remove unused implicit config parameter * [cli] Program rewritten to use Synchronise * [core] Synchronise useValidConfig accepts Logger implicitly * [core] Synchronise reorder methods * [core] Synchronise refactor errorMessages * [core] SyncLogging logRunStart accepts explicit parameters * [core] remove old Sync * [core] Synchronise restore logRunStart * [domain] Terminal add types to public methods and values * [domain] UploadEventLogger force flush to terminal Also make part of the progress message in green. Not flushing, by using println, cause odd behaviour. Works on normal terminal, but not great in an emacs terminal. Oh well. * [core] SyncLogging.logRunFinished remove unused parameters * [cli] Program restore final summary * [storage-aws] remove logging from module * [core] ThorpArchive replaces ActionSubmitter ActionSubmitter implementation becomes UnversionedMirrorArchive * [domain] cleaner upload progress messages * [cli] Program remove unused Logger * [cli] Program rename parameter * [core] SyncSuite use Synchronise * [sbt] Allow storage-aws to share core test classes * [domain] LocalFile stop storing a lambda The lambda breaks the equality test between LocalFile instances. * [core] MD4HashData add missing base64 digest for leafFile * [core] Synchronise drop DoNothing actions * [core] SyncSuite update tests * [sbt] aggregate modules from root module
2019-06-25 08:27:38 +01:00
.dependsOn(core % "compile->compile;test->test")
Split into subprojects (#36) * [sbt] define existing single module project as legacyRoot * [sbt] add empty cli module depending on legacyRoot * [cli] move Main to cli module * [cli] move ParseArgs to cli module * [sbt] limit scope of scopt dependency to cli module * [cli] moved logging config to cli module * [cli] rename module directory * [aws-api] added empty module * [sbt] aggregate builds from cli * [aws-lib] add empty module * [core] add empty module * [sbt] add comment graphing module dependencies * [sbt] adjust module dependencies to reflect plan Include legacyRoot at the base until it can be redistributed * [legacy] make some awssdk classes non-private during this transition, these classes being private would cause problems * [aws-lib] create S3ClientBuilder This is copied from the legacy S3Client companion object * [domain] add empty module * [domain] move Bucket into module * [legacy] RemoteKey no longer has dependency on Config * [domain] move RemoteKey into module * [domain] move MD5Hash into module * [legacy] LocalFile no longer had dependency on MD5HashGenerator * [domain] move LocalFile into module * [domain] mode LastModified into module * [domain] move RemoteMetaData into module * [domain] move S3MetaData into module * [domain] move Exclude into module * [domain] move Filter into module * [domain] move KeyModified into module * [domain] move HashModified into module * [domain] RemoteKey.resolve added * [domain] add dependency on scalatest * [domain] LocalFile.resolve added * [legacy] Remove UnitTest * [legacy] optimise imports * [domain] move S3ObjectsData moved into module * [legacy] wrapper for using GeneralProgressListener * [domain] move Config into module * [sbt] move aws-api below legacyRoot in dependencies This will allow use to move S3Client into the aws-api module * [legacy] rename S3Client companion as S3ClientBuilder Preparation to move this into its own file. * Inject Logger via CLI (#34) * [S3Client] refactor defaultClient() * [S3Client] transfermanager explicitly uses the same s3client * [S3ClientPutObjectUploader] refactor putObjectRequest creation * [cli] copy in Logging trait as Logger class * [cli] Main uses Logger * [cli] simplify Logger and pass to Sync.run * [legacy] SyncLogging converted to companion * [cli] Logger info can more easily use levels again * [legacy] LocalFileStream uses injected info * [legacy] S3MetaDataEnricher remove unused Logging * [legacy] ActionGenerator remove unused Logging * [legacy] convert ActionGenerator to an object * [legacy] import log methods from SyncLogging * [legacy] move getS3Status from S3Client to S3MetaDataEnricher * [legact] convert ActionsSubmitter to an object * [legacy] convert LocalFileStream to an object * [legacy] move Action case classes inside companion * [legacy] move UploadEvent case classes inside companion and rename * [legacy] move S3Action case classes into companion * [legacy] convert Sync to an object * [cli] Logger takes verbosity level at construction No longer needs to be passed the whole Config implicitly for each info call. * [legacy] stop passing implicit Config for logging purposes Pass a more specific implicit info: Int => String => Unit instead * [legacy] remove DummyS3Client * [legacy] remove Logging * [legacy] convert MD5HashGenerator to an object * [aws-api] move S3Client into module * [legacy] convert KeyGenerator to an object * [legacy] don't use IO.unsafeRunSync directly * [legacy] refactor/rewrite Sync.run * [legacy] Rewrite sort using a for-comprehension * [legacy] Sync inline sorting * [legacy] SyncLogging rename method * [legacy] repair tests * [sbt] move core module to a dependency of legacyRoot * [sbt] add test dependencies to core module * [core] move classes into module * [aws-lib] move classes into module * [sbt] remove legacy root
2019-06-06 19:24:15 +01:00
lazy val core = (project in file("core"))
.settings(commonSettings)
.settings(assemblyJarName in assembly := "core.jar")
Split into subprojects (#36) * [sbt] define existing single module project as legacyRoot * [sbt] add empty cli module depending on legacyRoot * [cli] move Main to cli module * [cli] move ParseArgs to cli module * [sbt] limit scope of scopt dependency to cli module * [cli] moved logging config to cli module * [cli] rename module directory * [aws-api] added empty module * [sbt] aggregate builds from cli * [aws-lib] add empty module * [core] add empty module * [sbt] add comment graphing module dependencies * [sbt] adjust module dependencies to reflect plan Include legacyRoot at the base until it can be redistributed * [legacy] make some awssdk classes non-private during this transition, these classes being private would cause problems * [aws-lib] create S3ClientBuilder This is copied from the legacy S3Client companion object * [domain] add empty module * [domain] move Bucket into module * [legacy] RemoteKey no longer has dependency on Config * [domain] move RemoteKey into module * [domain] move MD5Hash into module * [legacy] LocalFile no longer had dependency on MD5HashGenerator * [domain] move LocalFile into module * [domain] mode LastModified into module * [domain] move RemoteMetaData into module * [domain] move S3MetaData into module * [domain] move Exclude into module * [domain] move Filter into module * [domain] move KeyModified into module * [domain] move HashModified into module * [domain] RemoteKey.resolve added * [domain] add dependency on scalatest * [domain] LocalFile.resolve added * [legacy] Remove UnitTest * [legacy] optimise imports * [domain] move S3ObjectsData moved into module * [legacy] wrapper for using GeneralProgressListener * [domain] move Config into module * [sbt] move aws-api below legacyRoot in dependencies This will allow use to move S3Client into the aws-api module * [legacy] rename S3Client companion as S3ClientBuilder Preparation to move this into its own file. * Inject Logger via CLI (#34) * [S3Client] refactor defaultClient() * [S3Client] transfermanager explicitly uses the same s3client * [S3ClientPutObjectUploader] refactor putObjectRequest creation * [cli] copy in Logging trait as Logger class * [cli] Main uses Logger * [cli] simplify Logger and pass to Sync.run * [legacy] SyncLogging converted to companion * [cli] Logger info can more easily use levels again * [legacy] LocalFileStream uses injected info * [legacy] S3MetaDataEnricher remove unused Logging * [legacy] ActionGenerator remove unused Logging * [legacy] convert ActionGenerator to an object * [legacy] import log methods from SyncLogging * [legacy] move getS3Status from S3Client to S3MetaDataEnricher * [legact] convert ActionsSubmitter to an object * [legacy] convert LocalFileStream to an object * [legacy] move Action case classes inside companion * [legacy] move UploadEvent case classes inside companion and rename * [legacy] move S3Action case classes into companion * [legacy] convert Sync to an object * [cli] Logger takes verbosity level at construction No longer needs to be passed the whole Config implicitly for each info call. * [legacy] stop passing implicit Config for logging purposes Pass a more specific implicit info: Int => String => Unit instead * [legacy] remove DummyS3Client * [legacy] remove Logging * [legacy] convert MD5HashGenerator to an object * [aws-api] move S3Client into module * [legacy] convert KeyGenerator to an object * [legacy] don't use IO.unsafeRunSync directly * [legacy] refactor/rewrite Sync.run * [legacy] Rewrite sort using a for-comprehension * [legacy] Sync inline sorting * [legacy] SyncLogging rename method * [legacy] repair tests * [sbt] move core module to a dependency of legacyRoot * [sbt] add test dependencies to core module * [core] move classes into module * [aws-lib] move classes into module * [sbt] remove legacy root
2019-06-06 19:24:15 +01:00
.settings(testDependencies)
.dependsOn(`storage-api`)
Use correct hash locally for comparing multi-part uploaded files (#82) * [storage-aws] ETagGenerator add stub * [core] MD5HashGenerator add hex and digest helpers * [domain] MD5Hash can always provide base64 and also digest Rather that store the base 64 digest some of the time, simply decode it from the hex hash. The same for the binary digest. MD5Hash is now cleaner now that it no longer has Option parameters. * [core] MD5HashGenerator add stubs to allow reading file chunks * [domain] MD5HashData add sub-objects * [domain] MD5HashData move back into test where it belongs * [sbt] add sbt-bloop plugin * [domain] MD5HashData Add hash of big-file * [domain] MD5HashData Add hash of big-file * [core] MD5HashGenerator find end of chunk correctly * [core] MD5HashGenerator offset is a Long * [core] MD5HashGenerator don't read past the end of the file * [storage-aws] ETagGenerator can reproduce ETags * [storage-aws] ETagGeneratorTest added * [storate-aws] ETagGenerator refactoring * [storage-aws] ETageGenerator refactoring * [core] SyncSuite remove redundant braces * [storage-api] HashService added * [storage-aws] S3HashService added * [core] LocalFileStream refactoring * [core] integrate HashService and ETagGenerator * Optimise imports * [domain] HexEncoder added to replace java 8 only DataTypeConverter * [core] MD5HashGenerator refactoring * [core] S3MetaDataEnricher refactoring * [core] S3MetaDataEnricherSuite refactoring * [storage-aws] ETagGeneratorTest refactoring * [storage-aws] StorageServiceSuite refactoring * [core] S3MetaDataEnricher refactoring * [core] refactoring * [storage-aws] refactoring
2019-06-29 19:07:51 +01:00
.dependsOn(domain % "compile->compile;test->test")
.dependsOn(config)
lazy val `storage-api` = (project in file("storage-api"))
.settings(commonSettings)
.settings(zioDependencies)
.settings(assemblyJarName in assembly := "storage-api.jar")
.dependsOn(console)
lazy val console = (project in file("console"))
.settings(commonSettings)
.settings(zioDependencies)
.settings(assemblyJarName in assembly := "console.jar")
Split into subprojects (#36) * [sbt] define existing single module project as legacyRoot * [sbt] add empty cli module depending on legacyRoot * [cli] move Main to cli module * [cli] move ParseArgs to cli module * [sbt] limit scope of scopt dependency to cli module * [cli] moved logging config to cli module * [cli] rename module directory * [aws-api] added empty module * [sbt] aggregate builds from cli * [aws-lib] add empty module * [core] add empty module * [sbt] add comment graphing module dependencies * [sbt] adjust module dependencies to reflect plan Include legacyRoot at the base until it can be redistributed * [legacy] make some awssdk classes non-private during this transition, these classes being private would cause problems * [aws-lib] create S3ClientBuilder This is copied from the legacy S3Client companion object * [domain] add empty module * [domain] move Bucket into module * [legacy] RemoteKey no longer has dependency on Config * [domain] move RemoteKey into module * [domain] move MD5Hash into module * [legacy] LocalFile no longer had dependency on MD5HashGenerator * [domain] move LocalFile into module * [domain] mode LastModified into module * [domain] move RemoteMetaData into module * [domain] move S3MetaData into module * [domain] move Exclude into module * [domain] move Filter into module * [domain] move KeyModified into module * [domain] move HashModified into module * [domain] RemoteKey.resolve added * [domain] add dependency on scalatest * [domain] LocalFile.resolve added * [legacy] Remove UnitTest * [legacy] optimise imports * [domain] move S3ObjectsData moved into module * [legacy] wrapper for using GeneralProgressListener * [domain] move Config into module * [sbt] move aws-api below legacyRoot in dependencies This will allow use to move S3Client into the aws-api module * [legacy] rename S3Client companion as S3ClientBuilder Preparation to move this into its own file. * Inject Logger via CLI (#34) * [S3Client] refactor defaultClient() * [S3Client] transfermanager explicitly uses the same s3client * [S3ClientPutObjectUploader] refactor putObjectRequest creation * [cli] copy in Logging trait as Logger class * [cli] Main uses Logger * [cli] simplify Logger and pass to Sync.run * [legacy] SyncLogging converted to companion * [cli] Logger info can more easily use levels again * [legacy] LocalFileStream uses injected info * [legacy] S3MetaDataEnricher remove unused Logging * [legacy] ActionGenerator remove unused Logging * [legacy] convert ActionGenerator to an object * [legacy] import log methods from SyncLogging * [legacy] move getS3Status from S3Client to S3MetaDataEnricher * [legact] convert ActionsSubmitter to an object * [legacy] convert LocalFileStream to an object * [legacy] move Action case classes inside companion * [legacy] move UploadEvent case classes inside companion and rename * [legacy] move S3Action case classes into companion * [legacy] convert Sync to an object * [cli] Logger takes verbosity level at construction No longer needs to be passed the whole Config implicitly for each info call. * [legacy] stop passing implicit Config for logging purposes Pass a more specific implicit info: Int => String => Unit instead * [legacy] remove DummyS3Client * [legacy] remove Logging * [legacy] convert MD5HashGenerator to an object * [aws-api] move S3Client into module * [legacy] convert KeyGenerator to an object * [legacy] don't use IO.unsafeRunSync directly * [legacy] refactor/rewrite Sync.run * [legacy] Rewrite sort using a for-comprehension * [legacy] Sync inline sorting * [legacy] SyncLogging rename method * [legacy] repair tests * [sbt] move core module to a dependency of legacyRoot * [sbt] add test dependencies to core module * [core] move classes into module * [aws-lib] move classes into module * [sbt] remove legacy root
2019-06-06 19:24:15 +01:00
.dependsOn(domain)
lazy val config = (project in file("config"))
.settings(commonSettings)
.settings(zioDependencies)
.settings(testDependencies)
.settings(commandLineParsing)
.settings(assemblyJarName in assembly := "config.jar")
.dependsOn(domain)
.dependsOn(filesystem)
lazy val filesystem = (project in file("filesystem"))
.settings(commonSettings)
.settings(zioDependencies)
.settings(testDependencies)
.settings(assemblyJarName in assembly := "filesystem.jar")
Split into subprojects (#36) * [sbt] define existing single module project as legacyRoot * [sbt] add empty cli module depending on legacyRoot * [cli] move Main to cli module * [cli] move ParseArgs to cli module * [sbt] limit scope of scopt dependency to cli module * [cli] moved logging config to cli module * [cli] rename module directory * [aws-api] added empty module * [sbt] aggregate builds from cli * [aws-lib] add empty module * [core] add empty module * [sbt] add comment graphing module dependencies * [sbt] adjust module dependencies to reflect plan Include legacyRoot at the base until it can be redistributed * [legacy] make some awssdk classes non-private during this transition, these classes being private would cause problems * [aws-lib] create S3ClientBuilder This is copied from the legacy S3Client companion object * [domain] add empty module * [domain] move Bucket into module * [legacy] RemoteKey no longer has dependency on Config * [domain] move RemoteKey into module * [domain] move MD5Hash into module * [legacy] LocalFile no longer had dependency on MD5HashGenerator * [domain] move LocalFile into module * [domain] mode LastModified into module * [domain] move RemoteMetaData into module * [domain] move S3MetaData into module * [domain] move Exclude into module * [domain] move Filter into module * [domain] move KeyModified into module * [domain] move HashModified into module * [domain] RemoteKey.resolve added * [domain] add dependency on scalatest * [domain] LocalFile.resolve added * [legacy] Remove UnitTest * [legacy] optimise imports * [domain] move S3ObjectsData moved into module * [legacy] wrapper for using GeneralProgressListener * [domain] move Config into module * [sbt] move aws-api below legacyRoot in dependencies This will allow use to move S3Client into the aws-api module * [legacy] rename S3Client companion as S3ClientBuilder Preparation to move this into its own file. * Inject Logger via CLI (#34) * [S3Client] refactor defaultClient() * [S3Client] transfermanager explicitly uses the same s3client * [S3ClientPutObjectUploader] refactor putObjectRequest creation * [cli] copy in Logging trait as Logger class * [cli] Main uses Logger * [cli] simplify Logger and pass to Sync.run * [legacy] SyncLogging converted to companion * [cli] Logger info can more easily use levels again * [legacy] LocalFileStream uses injected info * [legacy] S3MetaDataEnricher remove unused Logging * [legacy] ActionGenerator remove unused Logging * [legacy] convert ActionGenerator to an object * [legacy] import log methods from SyncLogging * [legacy] move getS3Status from S3Client to S3MetaDataEnricher * [legact] convert ActionsSubmitter to an object * [legacy] convert LocalFileStream to an object * [legacy] move Action case classes inside companion * [legacy] move UploadEvent case classes inside companion and rename * [legacy] move S3Action case classes into companion * [legacy] convert Sync to an object * [cli] Logger takes verbosity level at construction No longer needs to be passed the whole Config implicitly for each info call. * [legacy] stop passing implicit Config for logging purposes Pass a more specific implicit info: Int => String => Unit instead * [legacy] remove DummyS3Client * [legacy] remove Logging * [legacy] convert MD5HashGenerator to an object * [aws-api] move S3Client into module * [legacy] convert KeyGenerator to an object * [legacy] don't use IO.unsafeRunSync directly * [legacy] refactor/rewrite Sync.run * [legacy] Rewrite sort using a for-comprehension * [legacy] Sync inline sorting * [legacy] SyncLogging rename method * [legacy] repair tests * [sbt] move core module to a dependency of legacyRoot * [sbt] add test dependencies to core module * [core] move classes into module * [aws-lib] move classes into module * [sbt] remove legacy root
2019-06-06 19:24:15 +01:00
lazy val domain = (project in file("domain"))
.settings(commonSettings)
.settings(assemblyJarName in assembly := "domain.jar")
Split into subprojects (#36) * [sbt] define existing single module project as legacyRoot * [sbt] add empty cli module depending on legacyRoot * [cli] move Main to cli module * [cli] move ParseArgs to cli module * [sbt] limit scope of scopt dependency to cli module * [cli] moved logging config to cli module * [cli] rename module directory * [aws-api] added empty module * [sbt] aggregate builds from cli * [aws-lib] add empty module * [core] add empty module * [sbt] add comment graphing module dependencies * [sbt] adjust module dependencies to reflect plan Include legacyRoot at the base until it can be redistributed * [legacy] make some awssdk classes non-private during this transition, these classes being private would cause problems * [aws-lib] create S3ClientBuilder This is copied from the legacy S3Client companion object * [domain] add empty module * [domain] move Bucket into module * [legacy] RemoteKey no longer has dependency on Config * [domain] move RemoteKey into module * [domain] move MD5Hash into module * [legacy] LocalFile no longer had dependency on MD5HashGenerator * [domain] move LocalFile into module * [domain] mode LastModified into module * [domain] move RemoteMetaData into module * [domain] move S3MetaData into module * [domain] move Exclude into module * [domain] move Filter into module * [domain] move KeyModified into module * [domain] move HashModified into module * [domain] RemoteKey.resolve added * [domain] add dependency on scalatest * [domain] LocalFile.resolve added * [legacy] Remove UnitTest * [legacy] optimise imports * [domain] move S3ObjectsData moved into module * [legacy] wrapper for using GeneralProgressListener * [domain] move Config into module * [sbt] move aws-api below legacyRoot in dependencies This will allow use to move S3Client into the aws-api module * [legacy] rename S3Client companion as S3ClientBuilder Preparation to move this into its own file. * Inject Logger via CLI (#34) * [S3Client] refactor defaultClient() * [S3Client] transfermanager explicitly uses the same s3client * [S3ClientPutObjectUploader] refactor putObjectRequest creation * [cli] copy in Logging trait as Logger class * [cli] Main uses Logger * [cli] simplify Logger and pass to Sync.run * [legacy] SyncLogging converted to companion * [cli] Logger info can more easily use levels again * [legacy] LocalFileStream uses injected info * [legacy] S3MetaDataEnricher remove unused Logging * [legacy] ActionGenerator remove unused Logging * [legacy] convert ActionGenerator to an object * [legacy] import log methods from SyncLogging * [legacy] move getS3Status from S3Client to S3MetaDataEnricher * [legact] convert ActionsSubmitter to an object * [legacy] convert LocalFileStream to an object * [legacy] move Action case classes inside companion * [legacy] move UploadEvent case classes inside companion and rename * [legacy] move S3Action case classes into companion * [legacy] convert Sync to an object * [cli] Logger takes verbosity level at construction No longer needs to be passed the whole Config implicitly for each info call. * [legacy] stop passing implicit Config for logging purposes Pass a more specific implicit info: Int => String => Unit instead * [legacy] remove DummyS3Client * [legacy] remove Logging * [legacy] convert MD5HashGenerator to an object * [aws-api] move S3Client into module * [legacy] convert KeyGenerator to an object * [legacy] don't use IO.unsafeRunSync directly * [legacy] refactor/rewrite Sync.run * [legacy] Rewrite sort using a for-comprehension * [legacy] Sync inline sorting * [legacy] SyncLogging rename method * [legacy] repair tests * [sbt] move core module to a dependency of legacyRoot * [sbt] add test dependencies to core module * [core] move classes into module * [aws-lib] move classes into module * [sbt] remove legacy root
2019-06-06 19:24:15 +01:00
.settings(testDependencies)