Commit graph

81 commits

Author SHA1 Message Date
dependabot[bot]
c08ab5964c
Bump kemitix-aws-java-sdk-s3-wrapper from 1.11.938 to 1.11.947 (#562)
Bumps kemitix-aws-java-sdk-s3-wrapper from 1.11.938 to 1.11.947.

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-02-14 22:27:35 +00:00
dependabot[bot]
27251ec8ee
Bump kemitix-aws-java-sdk-s3-wrapper from 1.11.897 to 1.11.938 (#558)
Bumps kemitix-aws-java-sdk-s3-wrapper from 1.11.897 to 1.11.938.

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-01-26 11:23:34 +00:00
dependabot[bot]
4fd3197a93
Bump kemitix-aws-java-sdk-s3-wrapper from 1.11.893 to 1.11.897 (#549)
Bumps kemitix-aws-java-sdk-s3-wrapper from 1.11.893 to 1.11.897.

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-11-12 09:38:01 +00:00
42634f9c09
Version set to 2.0.1 (#546) 2020-11-06 18:49:20 +00:00
Snyk bot
2ce8b62a46
fix: storage-aws/pom.xml to reduce vulnerabilities (#544)
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JAVA-ORGAPACHEHTTPCOMPONENTS-1016906
2020-11-06 09:00:09 +00:00
dependabot-preview[bot]
329cb0c5df
Bump kemitix-aws-java-sdk-s3-wrapper from 1.11.856 to 1.11.859 (#527) 2020-09-14 06:50:02 +00:00
dependabot-preview[bot]
f3e6fba4e6
Bump kemitix-aws-java-sdk-s3-wrapper from 1.11.853 to 1.11.856 (#526) 2020-09-07 06:54:45 +00:00
dependabot-preview[bot]
3b567e1603
Bump kemitix-aws-java-sdk-s3-wrapper from 1.11.840 to 1.11.853 (#522) 2020-09-03 06:43:05 +00:00
dependabot-preview[bot]
020fbed6d7
Bump kemitix-aws-java-sdk-s3-wrapper from 1.11.837 to 1.11.840 (#512) 2020-08-14 08:19:31 +00:00
dependabot-preview[bot]
4de7a881db
Bump kemitix-aws-java-sdk-s3-wrapper from 1.11.834 to 1.11.837 (#511) 2020-08-12 08:29:58 +00:00
dependabot-preview[bot]
87898cc9b0
Bump kemitix-aws-java-sdk-s3-wrapper from 1.11.820 to 1.11.834 (#510) 2020-08-06 08:13:16 +00:00
dependabot-preview[bot]
58b54b170e
Bump kemitix-aws-java-sdk-s3-wrapper from 1.11.816 to 1.11.820 (#505) 2020-07-13 08:51:13 +00:00
f67e550aab
Use kemitix-aws-java-sdk-s3-wrapper (#503)
* Use kemitix-aws-java-sdk-s3-wrapper

* Update reactor graph
2020-07-11 16:06:33 +01:00
dependabot-preview[bot]
6c8b056ed9
Bump aws-java-sdk-s3 from 1.11.818 to 1.11.819 (#500) 2020-07-09 08:37:41 +00:00
dependabot-preview[bot]
d723e350eb
Bump aws-java-sdk-s3 from 1.11.817 to 1.11.818 (#499) 2020-07-08 08:26:42 +00:00
dependabot-preview[bot]
9246a98e41
Bump aws-java-sdk-s3 from 1.11.816 to 1.11.817 (#498) 2020-07-07 08:46:57 +00:00
dependabot-preview[bot]
7cb53cabb8
Bump aws-java-sdk-s3 from 1.11.815 to 1.11.816 (#496) 2020-07-03 08:22:26 +00:00
dependabot-preview[bot]
3f5be20fd1
Bump aws-java-sdk-s3 from 1.11.814 to 1.11.815 (#495) 2020-07-02 08:42:43 +00:00
dependabot-preview[bot]
a065caa36b
Bump aws-java-sdk-s3 from 1.11.813 to 1.11.814 (#494) 2020-07-01 08:44:09 +00:00
dependabot-preview[bot]
199f88fd33
Bump aws-java-sdk-s3 from 1.11.812 to 1.11.813 (#493) 2020-06-30 08:24:57 +00:00
dependabot-preview[bot]
8e218a5717
Bump aws-java-sdk-s3 from 1.11.811 to 1.11.812 (#492) 2020-06-29 08:44:04 +00:00
b960f2c8ab
Release 2.0.0 (#491)
* Version set to 2.0.0

* Remove lingering scala plugins
2020-06-27 20:09:45 +01:00
783402f745
Convert lib module to Java (#487)
* lib.FileScannerTest: convert to Java

* lib.Filters: convert to Java

* lib.FiltersTest: convert to Java

* lib.Archive: convert to Java

* lib.LocalFileSystem: convert to Java

* domain.Channel: better channel termination

* domain.Channel: improve assurance that threads die

* Ensure uisink and storage shutdown when exiting program

* domain.HashesTest: clean up

* domain.Channel.run() already shuts down channel when runner finishes

* lib.FileScannerTest: add listener before run

* domain.Channel: add lock around take and only interrupt if waiting

* .run: add intellij test run configurations

* app. update in-code version

* app: chain ui channel construction

* domain.Channel: add tracing option and simplify thread naming

* storage-aws: use default multipartUploadThreshold

* app: force exit once program is completed

* domain.channel: split up and move Channel to its own package

* lib.LocalFileSystemTest: convert to Java

* lib.SequencedAction: convert to Java

* domain.LocalFile: don’t box file length

* lib.FileScannerTest: include hashes

* lib.UnversionedMirrorArchive: convert to Java

* lib: remove scala dependencies
2020-06-27 18:00:22 +01:00
dependabot-preview[bot]
f30021873e
Bump aws-java-sdk-s3 from 1.11.810 to 1.11.811 (#486) 2020-06-26 08:29:26 +00:00
78b1bcf6ac
Convert storage-aws tests to Java (#484)
* storage-aws.ListerTest: convert to Java

* storage-aws: convert to Java
2020-06-26 08:01:29 +01:00
dependabot-preview[bot]
5661c9e7da
Bump aws-java-sdk-s3 from 1.11.809 to 1.11.810 (#482) 2020-06-25 08:41:57 +00:00
dependabot-preview[bot]
085f9f61ec
Bump jackson-dataformat-cbor from 2.11.0 to 2.11.1 (#483) 2020-06-25 08:37:26 +00:00
ebd1910cba
Java rewrite uishell module (#481)
* uishell.UploadEventListener: convert to Java

* uishell.UIShell: convert to Java

* app: shutdown storate once complete

* uishell: remove scala dependencies and plugin
2020-06-25 09:06:10 +01:00
a4bd24ebce
Remove ZIO (#479)
* [BROKEN]Remove ZIO and EIP-ZIO

* [BROKEN]uishell.UIShell.receiver: implement

* [BROKEN]domain.Channel: new implementation replacing MessageChannel

* [BROKEN] use domain.Channel

* Shutdown filescanner channel when finished

* start the uiChannel

* domain.Channel: channel completes once shutdown and queue empty

* uishell: down use eraseLineForward in batch mode

* lib: set file runner before adding listener

* uishell: don’t log do nothing events when not in batch mode

* domain.Channel: if exception in child thread then shutdown channel

* uishell: use correct line endings for showing chosen actions

* domain.Channel: don’t wait for shutdown if not running

* domain: remove legacy MessageChannel

* domain.Channel: don’t hold thread array

* lib.LocalFileSystem: restore delete scanner

* lib.LocalFileSystem: shutdown deletetion channel

* domain.Channel: improved shutdown logic

* clean up
2020-06-24 22:39:16 +01:00
dependabot-preview[bot]
ec4ed099a5
Bump aws-java-sdk-s3 from 1.11.808 to 1.11.809 (#478) 2020-06-24 09:02:10 +00:00
0ae523a1e7
Convert Storage environment to Java (#476)
* uishell.UploadProgressEvent: convert to Java

* uishell.UIEvent: convert to Java

* uishell.ProgressUI: convert to Java

* uishell.ProgressEvent: remove unused

* lib.MessageChannel: added as replacement for eip-zio version

* domain: move MessageChannel to module

* storage: convert to Java
2020-06-23 10:23:28 +01:00
dependabot-preview[bot]
97d0e0d190
Bump aws-java-sdk-s3 from 1.11.807 to 1.11.808 (#477) 2020-06-23 08:57:33 +00:00
dependabot-preview[bot]
9cd271da35
Bump aws-java-sdk-s3 from 1.11.806 to 1.11.807 (#475) 2020-06-22 09:11:22 +00:00
0104d9c08d
Post-release version set to 1.1.0-SNAPSHOT (#472) 2020-06-21 19:42:29 +01:00
53eaeeb75f
Convert console module to Java (#471)
* console.ConsoleOut: convert to Java

* console.Console: convert to Java

* console: remove scala dependencies and plugins

* console.ConsoleOut: remove lingering scala import
2020-06-21 17:46:14 +01:00
5cb929e7e4
Release 1.0.0 (#463)
Release 1.0.0
2020-06-21 07:54:43 +01:00
319c46f403
Convert to Java (domain, config, storage-aws and filesystem) (#446)
* Java rewrite domain (#438)

* domain.Bucket: convert to Java

* domain.LastModified: convert to Java

* domain.QuoteStripper: convert to Java

* domain.HexEncoder: convert to Java

* domain.MD5Hash: convert to Java

* remove unused import

* domain.RemoteKey: convert to Java

* domain.Action: convert to Java

* domain.Counters: convert to Java

* domain.HashType: convert to Java

* domain.Hashes: convert to Java

* domain.MD5HashData: convert to Java

* domain.Filter: convert to Java

* domain.LocalFile: convert to Java

* domain: make immutable field public

* domain.SizeTranslation: convert to Java

* domain.HashType: restrict access to contstructor

* domain.RemoteObjects: convert to Java

Introduce MapView and Tuple.

* domain.Sources: convert to Java

* domain.StorageEvent: convert to Java

* domain.Terminal: convert to Java

* domain => config: move SimpleLens to only module that uses it

* domain => filesystem: move TemporaryFolder

* domain.Implicits: removed

* parent: make junit, et al available

* domain: add testing dependencies

* domain.HexEncoder: convert test to Java and fix bugs

* domain.HexEncoderTest: replace with Java version

* domain.MD5HashTest: convert to Java

* domain.RemoteKeyTest: convert to Java

* domain.SizeTranslationTest: convert to Java

* domain.TerminalTest: convert to Java

* domain: remove unused dependencies

* parent: rollback zio-streams to match zio and pin them together

* storage-aws: resolve transitive dependency conflicts

* Java rewrite storage aws (#445)

* storage-aws.AmazonS3: convert to Java as AmazonS3Client

* storage-aws.S3Copier: convert to Java

* storage-aws.S3Uploader: convert to Java

* storage-aws.S3Deleter: convert to Java

* storage-aws.S3Lister: convert to Java

* filesystem: write cache data correctly (as supplied)

* domain,filesystem: fix MD5Hash generation

* filesystem: convert to Java (#450)

* remove legacy

* Rewrite config module in Java (#461)

* config.ParseConfigFile: convert to Java

* config.ParseConfigFile: convert to Java

* config.SourceConfigLoader: convert to Java

* WIP config.Configuration: convert to Java

* config.ConfigOption: convert to Java

* config.ConfigOptions: convert to Java

* config.ConfigValidation: convert to Java

* config.ConfigQuery: convert to Java

* config: move classes to correct location

* config.ConfigValidationException: convert to Java

* config.ConfigValidator: convert to Java

* config.ConfigurationBuilder: convert to Java

* config.SimpleLens: removed

* config.Config: remove environment

* config.ConfigOptionTest: convert to Java

* config.ConfigQueryTest: convert to Java

* config.ConfigurationBuilderTest: convert to Java

* config.ParseConfigFileTest: convert to Java

* config.ParseConfigLinesTest: convert to Java

* config: remove scala dependencies and plugin
2020-06-21 07:21:21 +01:00
dependabot-preview[bot]
bae324e15d
Bump aws-java-sdk-s3 from 1.11.805 to 1.11.806 (#457) 2020-06-19 08:29:46 +00:00
dependabot-preview[bot]
9e9babb72e
Bump aws-java-sdk-s3 from 1.11.804 to 1.11.805 (#454) 2020-06-18 08:32:15 +00:00
dependabot-preview[bot]
b3d0bff3fb
Bump aws-java-sdk-s3 from 1.11.803 to 1.11.804 (#452) 2020-06-17 08:38:21 +00:00
dependabot-preview[bot]
99aeb8581d
Bump aws-java-sdk-s3 from 1.11.802 to 1.11.803 (#449) 2020-06-16 08:45:19 +00:00
dependabot-preview[bot]
8b1b3a8011
Bump aws-java-sdk-s3 from 1.11.801 to 1.11.802 (#447) 2020-06-15 09:06:26 +00:00
dependabot-preview[bot]
fc00be8707
Bump scala-library from 2.13.0 to 2.13.2 (#439)
* Bump scala-library from 2.13.0 to 2.13.2

Bumps [scala-library](https://github.com/scala/scala) from 2.13.0 to 2.13.2.
- [Release notes](https://github.com/scala/scala/releases)
- [Commits](https://github.com/scala/scala/compare/v2.13.0...v2.13.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Update scala syntax from 2.13.0 to 2.13.2 (#443)

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: Paul Campbell <pcampbell@kemitix.net>
2020-06-12 17:00:02 +01:00
dependabot-preview[bot]
d229745305
Bump aws-java-sdk-s3 from 1.11.800 to 1.11.801 (#442) 2020-06-12 08:46:27 +00:00
dependabot-preview[bot]
3ccee2dcc7
Bump aws-java-sdk-s3 from 1.11.792 to 1.11.800 (#436) 2020-06-11 20:47:44 +00:00
dependabot-preview[bot]
b9f2c5685e
Bump jackson-dataformat-cbor from 2.10.4 to 2.11.0 (#432) 2020-06-11 20:40:32 +00:00
baea18f3f3
Java rewrite - step 1 - build with Maven (#431)
* Add root and parent pom.xml files

* parent: add scala-maven-plugin to pluginManagement

* fix packaging

* domain: add module

* parent,domain: add dependencies

* maven: add filesystem module

* parent: disable coverage and pitest

* domain: move classes from test to main as they are used in filesystem

* config: add module

* console: add module

* Add reactor-graph

* fix typo

* uishell: add module

* Restore explicit dependencies

Don’t depend upon transitive dependencies if we want to be able to restructure module dependencies.

* storage: add module

* lib: add module

Included adding a Resource case class in filesystem to replace one generated by a scala plugin

* storage-aws: add module

* cli: add module

* app: add module

* app: make jar runnable

* Updated build instructions in README

* Convert readme to markdown

* Add reactor graph to readme

* Switch Guthub Actions to using Maven to build
2020-06-11 21:33:28 +01:00
f35ea9795d
Create and use a cache of hashes for local files (#249)
* [domain] Define Hashes in domain package

* [filesystem] Load and parse any .thorp.cache files found

* [filesystem] Use cached file data when available and up-to-date

* [lib] FileScanner refactoring

* [filesystem] scan sub-dirs first to minimise time cache is on heap

* [filesystem] Write new cache data to temp file

* [lib] replace cache file when finished updating

* [filesystem] AppendLines to correct file with new lines

* [domain] decode HashType from String

* [filesystem] Store last modified time as epoch milliseconds

* [filesystem] parse lastmodified as a long

* [filesystem] use all hash values in cache

* [lib] FileScanner rearrange code

* [lib] Create and use a single cache file per source

* [storage-aws] Use ETag hash from cache when available

* [filesystem] Merge file data together correctly

* [filesystem] Handle exceptions thrown by Files.mode correctly

* [readme] Add section on caching

* [changelog] updated

* [changelog] add pending dependencies notes

* [lib] Filters should not name methods after their defining object

* [lib] Fix up test
2019-10-27 19:53:00 +00:00
5214bacc0b
Not wrapping exceptions thrown in waitForUploadResult (#162)
* [storage-aws] Uploader move implementation to companion

* [app] Program Refactoring

* [storage-aws] AmazonTransferManager refactoring

* [lib] UnversionedMirrorArchive refactoring

* [console] Add Console.putStr

* [uishell] UIShell show chosen actions

* [storage-aws] AmazonTransferManager try to handle errors

* [uishell] UIShell avoid line wrap with long file paths

* [storage] Log when fetching remote summaries

* Handle exceptions thrown in waitForUploadResult

* [uishell] log errors

* [console] Swap batch/non-batch error messages

* fix tests
2019-09-23 13:30:34 +01:00
a2c061d655
Upload progress via uievent (#196)
* [uishell] Fix package name

* [sbt] Update eip-zio from 0.3.1 to 0.3.2

* [sbt] add dependency on eip-zio to domain

* [uishell] move Upload*Event* to uishell

* UploadEventListener
* UploadEventLogger
* UploadProgressEvent

* [uishell] Log upload progress using UIShell

* [uishell] inline UploadEventLogger into UIShell

* [uishell] Remove println and use Console
2019-09-11 22:47:42 +01:00