kemitix/thorp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
thorp/CHANGELOG.org

171 lines
4 KiB
Org Mode
Raw Normal View History

[changelog] Added
2019-05-16 16:37:25 +01:00
* Changelog
All notable changes to this project will be documented in this file.
The format is based on [[https://keepachangelog.com/en/1.0.0/][Keep a Changelog]], and this project adheres to
[[https://semver.org/spec/v2.0.0.html][Semantic Versioning]].
Not reading .thorp.conf file (#110) * [core] ConfigurationBuilder read the correct source config file * [changelog] updated
2019-07-13 15:43:40 +01:00
Replace cats-effect with zio (#117) * [sbt] Add ZIO dependency to storage-api * Convert to use ZIO * [sbt] remove cats-effect dependency * [changelog] updated * [cli] Program restore actions to correct order (copy, upload, delete) * [cli] Program You should not name methods after their defining object * [core] ConfigValidationException Redundant braces after class definition * [core] LocalFileStreamSuite Usage of get on optional type. * [core] PlanBuilderTest Usage of get on optional type.
2019-07-21 21:02:04 +01:00
* [0.8.0] - 2019-??-??
Improve logging consistency (#119) * [console] create new effect MyConsole * [core] SyncLogging use ConsoleOut.ValidConfig * [storage-aws] Extract AmazonS3 trait to help testing * [console] MyConsole use UIO * [storage-aws] S3StorageServiceSuite convert to FreeSpec * [storage-aws] S3StorageServiceSuite extract S3ClientTest trait * [storage-aws] remove incomplete test UploaderSuite * [storage-aws] extract and rename AmazonS3ClientTestFixture * [storage-aws] Copier handle and log errors * [core] ThropArchive log completed uploads here * [core] ThorpArchive log copies and deletes * Improve consistency of logging * [storage-aws] CopierTest extract from S3StorageServiceSuite * [storage-aws] Copier handle hash match errors properly * [core] ThropArchive display erros in red * [core] SequencePlan extracted * [core] Clear line after deletion log * [changelog] updated * [cli] Program tidy up imports * [storage-aws] Copier replace null with Option * [storage-aws] AmazonS3 copyObject returns Some[CopyObjectResult]
2019-07-23 23:13:09 +01:00
** Added
- Log copy and delete operations (#119)
- Log errors with red label (#119)
Ensure file handles are always closed (#122) * [core] Ensure files are closed even when interrupted * [changelog] updated
2019-07-24 18:38:19 +01:00
- Ensure file handles are always closed (#122)
Improve logging consistency (#119) * [console] create new effect MyConsole * [core] SyncLogging use ConsoleOut.ValidConfig * [storage-aws] Extract AmazonS3 trait to help testing * [console] MyConsole use UIO * [storage-aws] S3StorageServiceSuite convert to FreeSpec * [storage-aws] S3StorageServiceSuite extract S3ClientTest trait * [storage-aws] remove incomplete test UploaderSuite * [storage-aws] extract and rename AmazonS3ClientTestFixture * [storage-aws] Copier handle and log errors * [core] ThropArchive log completed uploads here * [core] ThorpArchive log copies and deletes * Improve consistency of logging * [storage-aws] CopierTest extract from S3StorageServiceSuite * [storage-aws] Copier handle hash match errors properly * [core] ThropArchive display erros in red * [core] SequencePlan extracted * [core] Clear line after deletion log * [changelog] updated * [cli] Program tidy up imports * [storage-aws] Copier replace null with Option * [storage-aws] AmazonS3 copyObject returns Some[CopyObjectResult]
2019-07-23 23:13:09 +01:00
Replace cats-effect with zio (#117) * [sbt] Add ZIO dependency to storage-api * Convert to use ZIO * [sbt] remove cats-effect dependency * [changelog] updated * [cli] Program restore actions to correct order (copy, upload, delete) * [cli] Program You should not name methods after their defining object * [core] ConfigValidationException Redundant braces after class definition * [core] LocalFileStreamSuite Usage of get on optional type. * [core] PlanBuilderTest Usage of get on optional type.
2019-07-21 21:02:04 +01:00
** Changed
- Replace cats-effect with zio (#117)
Remove Monocle dependency (#121) * Replace Monocle with local SimpleLens implementation * [domain] SimpleLensTest avoid short variables and remove field imports
2019-07-24 09:40:56 +01:00
- Replace Monocle with local SimpleLens implementation (#121)
Replace cats-effect with zio (#117) * [sbt] Add ZIO dependency to storage-api * Convert to use ZIO * [sbt] remove cats-effect dependency * [changelog] updated * [cli] Program restore actions to correct order (copy, upload, delete) * [cli] Program You should not name methods after their defining object * [core] ConfigValidationException Redundant braces after class definition * [core] LocalFileStreamSuite Usage of get on optional type. * [core] PlanBuilderTest Usage of get on optional type.
2019-07-21 21:02:04 +01:00
[changelog] updated
2019-07-23 23:16:08 +01:00
** Dependencies
- Upgrade ~aws-java-sdk-s3~ from ~1.11.587~ to ~1.11.596~ (#118)
NullPointerException creating scaladoc (#115) * [domain] Drop use of Lenses macro and only create used lenses * [core] Drop use of Lenses macro and only create used lenses * [sbt] drop paradise plugin * [changelog] updated
2019-07-19 23:30:20 +01:00
* [0.7.2] - 2019-07-19
** Changed
- Apply ~scalafmt~ (#108)
- Uses Lenses (#113)
** Fixed
- Creates incorrect MD5 hash for some files (#103)
- NullPointerException creating scaladoc (#115)
[changelog] update v0.7.1 release date
2019-07-15 07:02:05 +01:00
* [0.7.1] - 2019-07-15
Not reading .thorp.conf file (#111) * [domain] Config defaults to an empty Sources list * [core] ConfigurationBuilterTest add test for parsing .thorp.conf * [core] ConfigQuery if no Sources given returns current dir * [core] rewrote config loader - Only settings from explicit sources are used * [changelog] updated * [core] Remove stray println statements * [core] SyncLogging tidy up multi-source messages
2019-07-15 07:01:06 +01:00
** Changed
- Only settings in ~.thorp.conf~ for explicit sources are used (#111)
Not reading .thorp.conf file (#110) * [core] ConfigurationBuilder read the correct source config file * [changelog] updated
2019-07-13 15:43:40 +01:00
** Fixed
Not reading .thorp.conf file (#111) * [domain] Config defaults to an empty Sources list * [core] ConfigurationBuilterTest add test for parsing .thorp.conf * [core] ConfigQuery if no Sources given returns current dir * [core] rewrote config loader - Only settings from explicit sources are used * [changelog] updated * [core] Remove stray println statements * [core] SyncLogging tidy up multi-source messages
2019-07-15 07:01:06 +01:00
- Not reading ~.thorp.conf~ file (#110)(#111)
Not reading .thorp.conf file (#110) * [core] ConfigurationBuilder read the correct source config file * [changelog] updated
2019-07-13 15:43:40 +01:00
[changelog] updated
2019-07-12 08:09:02 +01:00
* [0.7.0] - 2019-07-12
Update CHANGELOG.org Start section for 0.7.0
2019-06-30 22:05:34 +01:00
** Added
- Add a version command-line option (#99)
Add a batch mode that provides a simple log output (#85) * [changelog] Updated * [readme] Updated * [domain] Config Add batch-mode flag * [core] ConfigOption Add BatchMode option * [core] ConfigQuery Add batchMode query Also replaced verbose exists case clauses with a simple contains. * [core] ConfigOptions added to replace Seq[ConfigOption] * [core] Syncronise rename method to createPlan * [cli] Program rename apply as run * [storage-aws] S3StorageServiceBuilder stop using IO to create object * [storage-aws] S3StorageServiceBuilder make default service lazy * [storage-aws] Rename S3ClientCopier => Copier * [storage-aws] Rename S3ClientDeleter => Deleter * [storage-aws] Rename S3ClientObjectLister => Lister * [storage-aws] Only attach upload listener when in batch mode Only detects batch mode when selected as a command line option * [core] Synchronise use leftMap rather than swap.map.swap * [cli] ParseArgs add `-B` and `--batch` options to enable batch mode * [core] ThorpArchive logs file uploaded when in batch mode
2019-07-02 08:43:52 +01:00
- Add a batch mode (#85)
Display total size and progress for entire run (#94) * [changelog] updated * [core] Wrap Stream[LocalFile] as LocalFiles * [core] LocalFiles counts files * [core] LocalFiles sums file lengths * [core] Restore logFileScan * [storage-aws] Lister logs when fetching object summaries * [storage-aws] Extract ListerLogger * [core] Synchronise use leftMap * [core] Syncronise extract assemblePlan * [core] Wrap Stream[Action] in SyncPlan * [core] Copy the file count and totalSizeBytes across to SyncPlan * [cli] Program rename actions as syncPlan * [cli] Program extract thorpArchive def * [cli] Program extract createPlan def * [cli] Program refactoring * [cli] Program remove println showing version * [cli] Program rename actions parameter as syncPlan * [core] ThorpArchive add an index to each action * [cli] Program make SyncPlan available to ThorpArchive * [core] Pass SyncTotals to Archive * [domain] Move SyncTotals into module * [domain] Pass index and SyncTotals to UploadEventListener * [domain] UploadEventLogger add file count a size progress bars * [domain] UploadEventLogger better display stability and add file index * [cli] Index files in correct order * [cli] Program extends Synchronise * [core] Rename Synchronise as PlanBuilder * [cli] Program add test to check actions don't get reordered from plan * [core] collect file size totals * [domain] UploadEventLogger include percentage * [cli] ProgramTest Use wildcards when selecting more than 6 elements
2019-07-04 18:58:31 +01:00
- Display total size and progress for entire run (#94)
[changelog] fix typo
2019-07-12 08:15:45 +01:00
- Sync more than one source directory into a single bucket/prefix (#25)
Update CHANGELOG.org Start section for 0.7.0
2019-06-30 22:05:34 +01:00
[changelog] updated
2019-07-12 08:09:02 +01:00
** Dependencies
- Upgrade ~jackson-databind~ from ~2.9.9~ to ~2.9.9.1~ (#101)
- Upgrade ~aws-java-sdk-s3~ from ~1.11.580~ to ~1.11.587~ (#104)
- Upgrade ~sbt-assembly~ from ~0.14.9~ to ~0.14.10~ (#105)
Release to sonatype using correct profile name (#102) * [changelog] updated * [sbt] Specify sonatypeProfileName in commonSettings This prevents the value being overridded by the `organization` value from `inThisBuild`.
2019-07-03 13:50:28 +01:00
* [0.6.1] - 2019-07-03
** Fixed
- Release to sonatype using correct profile name (#102)
[changelog] updated
2019-06-30 15:32:25 +01:00
* [0.6.0] - 2019-06-30
Introduce backend abstraction to hide S3 (#76) Add backend abstraction to hide S3
2019-06-22 07:20:59 +01:00
** Added
- Abstraction layer encapsulating S3 as Storage (#76)
[changelog] updated
2019-06-30 15:22:11 +01:00
- Display simple error message when bucket is invalid (#81)
- Shutdown storage service once completed (#88)
- Release snapshots to sonatype from TravisCI (#92)
- Embed launch script in fat-jar (#98)
** Fixed
- Use correct hash locally for comparing multi-part uploaded files (#82)
- Handle large list of action in a stack-safe manner (#93)
** Dependencies
- Upgrade ~scalamock~ from ~4.2.0~ to ~4.3.0~ (#84)
- Upgrade ~aws-java-sdk-s3~ from ~1.11.573~ to ~1.11.580~ (#86)
Introduce backend abstraction to hide S3 (#76) Add backend abstraction to hide S3
2019-06-22 07:20:59 +01:00
Is AWS SDK calculating MD5Hash again for a local file? (#50) * [aws-lib] Uploader provide request with the already calculated md5 hash * [aws-lib] remove unused accepts method * [aws-lib] Uploader refactoring * [domain] Config remove unused threshold and max retries items * [core] Show upload errors in summary * [domain] LocalFile add helper to explicitly compare by hash value Looking to add an optional field to MD5Hash but we want to do our checks here only on the hash value, not whether a digest is available or not. * [core] Sync refactoring * [core] SyncSuite invoke subject inside it method and after declaring expectations * [core] SyncSuite use the localfile hash rather than something arbitrary * [cli] Add `--no-global` and `--no-user` options * [core] LocalFileStream refactoring * [core] SyncSuite: ignore user and global configuration files * [domain] MD5Hash now can optionally store the base64 encoded hash * [core] MD5HashGenerator pass the digest to MD5Hash * [aws-lib] Uploader use the base64 encoded hash * [changelog] updated
2019-06-21 19:20:35 +01:00
* [0.5.0] - 2019-06-21
Add a debug flag and make debug message hidden by default (#60) * [cli] add a debug flag to control logging * [core] show entering a directory as a debug message
2019-06-14 20:00:22 +01:00
** Added
Add support for global and user configuration files (#73) * [core] ConfigurationBuilder reads user and global config files * [changelog] updated * [readme] updated
2019-06-20 17:41:08 +01:00
- Add ~thorp-lib~ module (#66)
- Enable running outside of sbt (#55)
- ~-d~, ~--debug~ flag for log messages (#60)
- Read config from ~.thorp.conf~ in source directory (#71)
- Read config from ~$HOME/.config/thorp.conf~ and ~/etc/thorp.conf~
(#73)
Is AWS SDK calculating MD5Hash again for a local file? (#50) * [aws-lib] Uploader provide request with the already calculated md5 hash * [aws-lib] remove unused accepts method * [aws-lib] Uploader refactoring * [domain] Config remove unused threshold and max retries items * [core] Show upload errors in summary * [domain] LocalFile add helper to explicitly compare by hash value Looking to add an optional field to MD5Hash but we want to do our checks here only on the hash value, not whether a digest is available or not. * [core] Sync refactoring * [core] SyncSuite invoke subject inside it method and after declaring expectations * [core] SyncSuite use the localfile hash rather than something arbitrary * [cli] Add `--no-global` and `--no-user` options * [core] LocalFileStream refactoring * [core] SyncSuite: ignore user and global configuration files * [domain] MD5Hash now can optionally store the base64 encoded hash * [core] MD5HashGenerator pass the digest to MD5Hash * [aws-lib] Uploader use the base64 encoded hash * [changelog] updated
2019-06-21 19:20:35 +01:00
- Add ~--no-global~ and ~--no-user~ options (#50)
- Display any upload errors in summary report (#50)
Add a debug flag and make debug message hidden by default (#60) * [cli] add a debug flag to control logging * [core] show entering a directory as a debug message
2019-06-14 20:00:22 +01:00
Suppress "Transfer" event messages (#64) * [aws-api] remove transfer events * [changelog] updated
2019-06-14 20:18:05 +01:00
** Changed
Add support for global and user configuration files (#73) * [core] ConfigurationBuilder reads user and global config files * [changelog] updated * [readme] updated
2019-06-20 17:41:08 +01:00
- Rename project as 'thorp' (#75)
- Suppress Transfer event messages (#64)
- Better error message when source not found (#51)
- Reduced logging (#59)
Is AWS SDK calculating MD5Hash again for a local file? (#50) * [aws-lib] Uploader provide request with the already calculated md5 hash * [aws-lib] remove unused accepts method * [aws-lib] Uploader refactoring * [domain] Config remove unused threshold and max retries items * [core] Show upload errors in summary * [domain] LocalFile add helper to explicitly compare by hash value Looking to add an optional field to MD5Hash but we want to do our checks here only on the hash value, not whether a digest is available or not. * [core] Sync refactoring * [core] SyncSuite invoke subject inside it method and after declaring expectations * [core] SyncSuite use the localfile hash rather than something arbitrary * [cli] Add `--no-global` and `--no-user` options * [core] LocalFileStream refactoring * [core] SyncSuite: ignore user and global configuration files * [domain] MD5Hash now can optionally store the base64 encoded hash * [core] MD5HashGenerator pass the digest to MD5Hash * [aws-lib] Uploader use the base64 encoded hash * [changelog] updated
2019-06-21 19:20:35 +01:00
- Prevent AWS SDK from recalculating MD5 hash (#50)
Add support for global and user configuration files (#73) * [core] ConfigurationBuilder reads user and global config files * [changelog] updated * [readme] updated
2019-06-20 17:41:08 +01:00
** Fixed
- Error when calculating md5 hash for large files (#56)
Suppress "Transfer" event messages (#64) * [aws-api] remove transfer events * [changelog] updated
2019-06-14 20:18:05 +01:00
[cli] Remove verbosity flag (#63)
2019-06-14 20:21:58 +01:00
** Removed
Add support for global and user configuration files (#73) * [core] ConfigurationBuilder reads user and global config files * [changelog] updated * [readme] updated
2019-06-20 17:41:08 +01:00
- ~-v~ verbosity flag (#63)
** Dependencies
- Upgrade ~aws-java-sdk-s3~ from ~1.11.569~ to ~1.11.570~ (#57)
[cli] Remove verbosity flag (#63)
2019-06-14 20:21:58 +01:00
[changelog] Updated for 0.4.0
2019-06-11 08:01:49 +01:00
* [0.4.0] - 2019-06-11
** Added
- Multi-part uploads
- ~--include~ to select files for synchronisation
- Upload progress bar
** Changed
[changelog] updated
2019-06-30 15:22:11 +01:00
- ~--filter~ renamed to ~-exclude~
[changelog] Updated for 0.4.0
2019-06-11 08:01:49 +01:00
** Fixed
- Fetch md5 hashes for all remote objects, not just the first 1000
- Handle when a file goes away between scanning and uploading
** Dependencies
- Removed ~reactive-aws-s3-{core,cats}~ and the AWS SDK v2 that it
depended upon in favour of the AWS SDK v1
- Upgrade ~aws-java-sdk-s3~ from ~1.11.560~ to ~1.11.569~
- Upgrade ~cats-effect~ from ~1.2.0~ to ~1.3.1~
- Upgade ~scalatest~ from ~3.0.7~ to ~3.0.8~
Support multiple filters (#18) * Support multiple filters * Clean up imports * [S3ClientLogging] log the remote key value * Update changelog, readme and long arg name * [SyncSuite] update test
2019-05-23 19:35:48 +01:00
* [0.3.0] - 2019-05-23
Simple Exclusion Filter (#16) * [filter] Parse filter from command line and add to config * [filter] exclude file that match the filter
2019-05-23 09:21:09 +01:00
** Added
- Filter to exclude files
* [0.2.0] - 2019-05-22
[changelog] Added
2019-05-16 16:37:25 +01:00
** Added
- Display count of the number of files uploaded
** Changed
- Improved performance by fetching all MD5 hashes from S3 in single
request at startup
* [0.1.0] - 2019-05-13
** Added
- Initial Release
- Synchronise files with an S3 bucket, using an MD5 hash to
identify when the file has changed and needs to be uploaded
Reference in a new issue Copy permalink