Commit graph

286 commits

Author SHA1 Message Date
218c0114c2 [syncsuite] improve tests for s3client thunk 2019-05-11 06:24:35 +01:00
960e336867 [readme] rewritten readme 2019-05-10 22:44:27 +01:00
5b397ce181 [s3client] upload returns S3's md5hash of the uploaded file 2019-05-10 22:35:01 +01:00
41e38f5cee [s3client] simple test for upload 2019-05-10 19:51:25 +01:00
bc1bffc345 [awssdk] Rewritten and simplified AWS SDK interface 2019-05-10 19:35:35 +01:00
fae876b554 [reactives3clienttest] clarify test descriptions 2019-05-10 18:31:30 +01:00
1abc34f30b [uploadselectionfilter] update call to uploadRequiredFilter 2019-05-10 18:30:52 +01:00
4c1cf89d51 Consistent logging format 2019-05-10 08:52:39 +01:00
27d1d1b99f [s3metadataenricher] remove quotes from remote hash 2019-05-10 08:39:18 +01:00
be266e0d41 [s3uploader] simplify logging 2019-05-10 08:29:59 +01:00
b61499ed8f [reactives3client] remove logging 2019-05-10 08:24:40 +01:00
c93bebb1e5 [reactives3client] handle NoSuchKeyException 2019-05-10 07:57:53 +01:00
f991c2b7b0 [s3uploadersuite] fix syntax 2019-05-10 07:56:30 +01:00
674e88d802 [reactives3clienttest] inline results 2019-05-09 23:09:47 +01:00
7593da9ce7 [reactives3client] [FAILING] when throws NoSuchKey returns None 2019-05-09 23:09:43 +01:00
1944d6620c [reactives3client] objectHead response okay then return Some 2019-05-09 22:50:15 +01:00
3eddc09a20 [catsios3client] Extract as trait 2019-05-09 21:11:46 +01:00
69029730e2 [reactives3client] Remove try with no catches 2019-05-09 18:47:52 +01:00
232ea40be6 [reactives3client] add console logging 2019-05-09 18:46:25 +01:00
e43b7dc0e3 [s3uploader] implement with (basic) test 2019-05-09 18:34:17 +01:00
befd6975fa [s3metadataenricher] rename intermediate function 2019-05-09 18:33:48 +01:00
6af5b8cafc [keygenerator] Extract as a trait 2019-05-09 18:33:29 +01:00
b348f18142 [s3client] Add upload method
Add DummyS3Client to help tests
2019-05-09 18:32:44 +01:00
e8a656ae4c [sync] rename type aliases to MD5Hash and RemoteKey 2019-05-09 18:29:42 +01:00
24b03f959e FIXUP upload filter 2019-05-09 17:40:05 +01:00
ebec3b1564 [s3uploader] Extract as trait 2019-05-09 17:37:06 +01:00
7d688954f7 [uploadselectionfilter] implement with tests 2019-05-09 17:34:17 +01:00
5090a78baa [uploadselectionfilter] Extracted as trait 2019-05-09 16:55:57 +01:00
86a53bf712 [awssdk] Move AWS SDK wrapper and trait to seperate package 2019-05-09 11:50:59 +01:00
65c1915d53 [s3metadataenricher] enrich with metadata when remote doesn't exists returns file to upload 2019-05-09 08:48:46 +01:00
e75b2a4892 [s3metadataenricher] enrich with metadata when remote exists returns metadata 2019-05-09 08:45:57 +01:00
df0df49624 [s3metadataenricher] remove putStrLn 2019-05-09 08:29:18 +01:00
4e2729ae26 [reactives3client] handle key not found 2019-05-09 07:41:50 +01:00
7af4004c75 [s3client] objectHead returns an IO[Option[...]]
If the remote file is missing then return None.

S3MetaDataEnricher.enrichWithS3MetaData now returns an IO[Either[File,
S3MetaData]]. If objectHead returns None, the this returns the file,
otherwise, the Some[Hash, LastModified] from objectHead is used to
create the S3MetaData as before.
2019-05-09 07:11:27 +01:00
9107e6f3eb [Reactives3client] use for-comprehension 2019-05-08 13:00:03 +01:00
837ab362c3 [Reactives3client] use S3CatsIOClient 2019-05-08 10:05:30 +01:00
f099765dd3 [sync] add test for s3client objecthead thunking 2019-05-08 08:59:29 +01:00
387904d1af [ReactiveS3Client] Added 2019-05-08 08:37:45 +01:00
ae3243ad21 [sync] Convert to a class 2019-05-08 07:06:58 +01:00
c5044e2ff7 [s3metadataenricher] generate key for file and stub headobject 2019-05-08 07:03:11 +01:00
b87495cfb5 Replace use of Path with use of File 2019-05-07 22:36:33 +01:00
04b5d4eded [s3metadataenricher] extend S3Client 2019-05-07 22:16:09 +01:00
c948f70b99 fix compile errors 2019-05-07 20:40:48 +01:00
a500e1f9d3 [config] Use Bucket type 2019-05-07 20:33:55 +01:00
605e62b957 [sync] add type Bucket 2019-05-07 20:32:05 +01:00
f618d73fa2 [s3client] Create triat 2019-05-07 20:30:54 +01:00
6eab5ada1e [s3metadata] extract to file 2019-05-07 17:53:18 +01:00
9d3c124b6e [S3MetaDataEnricher] Extract as trait 2019-05-07 17:53:18 +01:00
326fd7d4cc [LocalFileStream] extract as trait 2019-05-07 17:53:18 +01:00
38edf68f43 [sbt] add reactive-aws-s3 1.1.3 as dependency 2019-05-07 08:58:22 +01:00