24b03f959e
FIXUP upload filter
2019-05-09 17:40:05 +01:00
ebec3b1564
[s3uploader] Extract as trait
2019-05-09 17:37:06 +01:00
7d688954f7
[uploadselectionfilter] implement with tests
2019-05-09 17:34:17 +01:00
5090a78baa
[uploadselectionfilter] Extracted as trait
2019-05-09 16:55:57 +01:00
86a53bf712
[awssdk] Move AWS SDK wrapper and trait to seperate package
2019-05-09 11:50:59 +01:00
65c1915d53
[s3metadataenricher] enrich with metadata when remote doesn't exists returns file to upload
2019-05-09 08:48:46 +01:00
e75b2a4892
[s3metadataenricher] enrich with metadata when remote exists returns metadata
2019-05-09 08:45:57 +01:00
df0df49624
[s3metadataenricher] remove putStrLn
2019-05-09 08:29:18 +01:00
4e2729ae26
[reactives3client] handle key not found
2019-05-09 07:41:50 +01:00
7af4004c75
[s3client] objectHead returns an IO[Option[...]]
...
If the remote file is missing then return None.
S3MetaDataEnricher.enrichWithS3MetaData now returns an IO[Either[File,
S3MetaData]]. If objectHead returns None, the this returns the file,
otherwise, the Some[Hash, LastModified] from objectHead is used to
create the S3MetaData as before.
2019-05-09 07:11:27 +01:00
9107e6f3eb
[Reactives3client] use for-comprehension
2019-05-08 13:00:03 +01:00
837ab362c3
[Reactives3client] use S3CatsIOClient
2019-05-08 10:05:30 +01:00
f099765dd3
[sync] add test for s3client objecthead thunking
2019-05-08 08:59:29 +01:00
387904d1af
[ReactiveS3Client] Added
2019-05-08 08:37:45 +01:00
ae3243ad21
[sync] Convert to a class
2019-05-08 07:06:58 +01:00
c5044e2ff7
[s3metadataenricher] generate key for file and stub headobject
2019-05-08 07:03:11 +01:00
b87495cfb5
Replace use of Path with use of File
2019-05-07 22:36:33 +01:00
04b5d4eded
[s3metadataenricher] extend S3Client
2019-05-07 22:16:09 +01:00
c948f70b99
fix compile errors
2019-05-07 20:40:48 +01:00
a500e1f9d3
[config] Use Bucket type
2019-05-07 20:33:55 +01:00
605e62b957
[sync] add type Bucket
2019-05-07 20:32:05 +01:00
f618d73fa2
[s3client] Create triat
2019-05-07 20:30:54 +01:00
6eab5ada1e
[s3metadata] extract to file
2019-05-07 17:53:18 +01:00
9d3c124b6e
[S3MetaDataEnricher] Extract as trait
2019-05-07 17:53:18 +01:00
326fd7d4cc
[LocalFileStream] extract as trait
2019-05-07 17:53:18 +01:00
38edf68f43
[sbt] add reactive-aws-s3 1.1.3 as dependency
2019-05-07 08:58:22 +01:00
f9b3840a67
[sync] reformat to enhance ide code folding
2019-05-07 08:18:08 +01:00
21b48b1089
[sync] flesh out s3metadata fields
2019-05-07 07:23:38 +01:00
1982686e00
[sync] log files found
2019-05-06 22:37:16 +01:00
2153f109aa
[sync] sketch out rest of sync process
2019-05-06 22:09:38 +01:00
294966759c
[sync] streamDirectoryPaths (recursivly)
2019-05-06 19:37:42 +01:00
223d6b9392
[sbt] delegate cats-effect to fs2 transient version
...
Add comments and adjust order to libraryDependencies
2019-05-06 17:31:48 +01:00
afab304bae
Revert "[sbt] Bump cats-effect from 1.2.0 to 1.3.0"
...
This reverts commit 88d42e2130
2019-05-06 17:25:29 +01:00
aab2fdde3d
[sync] Extract Sync class
2019-05-06 17:20:28 +01:00
cc538b803a
[git] update gitignore
2019-05-06 17:15:43 +01:00
0011028c59
[sbt] add fs2-{core,io} 1.0.4 as dependencies
2019-05-06 17:15:43 +01:00
7f78863e8a
[main] Move prog id to program from sync
2019-05-06 17:15:43 +01:00
3dec191afa
[parseArgs] Remove version from head
2019-05-06 17:15:43 +01:00
dcda643934
[parseArgs] bucket and source are required
2019-05-06 17:15:43 +01:00
de5cd375d2
[parseArgs,Config] Move defaults to Config
2019-05-06 17:15:43 +01:00
7ce0b26d4f
[parseArgs] refactoring
2019-05-06 17:15:43 +01:00
d73f42e9ea
[parseArgs] extract to object
2019-05-06 17:15:43 +01:00
a4cf1d716b
[main] Add source directory config option
2019-05-06 17:15:43 +01:00
8a18171d7d
[main] initial parsing of command line args
2019-05-06 17:15:43 +01:00
14205f4fbf
[main] inline S3Thorp (for now)
2019-05-06 17:15:43 +01:00
b5e1036874
[main] extract IO program
2019-05-06 17:15:43 +01:00
b16862ea20
[app] S3Throp expects Config as parameter
2019-05-06 17:15:43 +01:00
d45f590c15
[config] Add stub config class
2019-05-06 17:15:43 +01:00
9fa2bfea24
[S3Thorp] Extract class
2019-05-06 17:15:43 +01:00
9f7cb86c51
[sbt] Add cats-effect recommended scalaOptions
2019-05-06 17:15:43 +01:00
