kemitix/thorp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
thorp/CHANGELOG.org
Paul Campbell f35ea9795d
Create and use a cache of hashes for local files (#249) 
* [domain] Define Hashes in domain package

* [filesystem] Load and parse any .thorp.cache files found

* [filesystem] Use cached file data when available and up-to-date

* [lib] FileScanner refactoring

* [filesystem] scan sub-dirs first to minimise time cache is on heap

* [filesystem] Write new cache data to temp file

* [lib] replace cache file when finished updating

* [filesystem] AppendLines to correct file with new lines

* [domain] decode HashType from String

* [filesystem] Store last modified time as epoch milliseconds

* [filesystem] parse lastmodified as a long

* [filesystem] use all hash values in cache

* [lib] FileScanner rearrange code

* [lib] Create and use a single cache file per source

* [storage-aws] Use ETag hash from cache when available

* [filesystem] Merge file data together correctly

* [filesystem] Handle exceptions thrown by Files.mode correctly

* [readme] Add section on caching

* [changelog] updated

* [changelog] add pending dependencies notes

* [lib] Filters should not name methods after their defining object

* [lib] Fix up test
2019-10-27 19:53:00 +00:00

266 lines
6.7 KiB
Org Mode
Raw Blame History

* Changelog
All notable changes to this project will be documented in this file.
The format is based on [[https://keepachangelog.com/en/1.0.0/][Keep a Changelog]], and this project adheres to
[[https://semver.org/spec/v2.0.0.html][Semantic Versioning]].
* [1.0.0] - ???
** Added
- Create and use a cache of hashes for local files (#249)
** Dependencies
- Revert "[sbt] Rollback ~sbt-ci-release~ to ~1.3.2~ (#231)"
- Update ~sbt~ to ~1.3.3~ (#238)
* [0.10.0] - 2019-10-08
This is the last ~v0.x~ feature release. The next feature release will be ~v1.x~.
** Fixed
- Can't publish new releases to sonatype (#219)
** Added
- Accept ~parallel~ in config files (#230)
** Changed
- Reduce Progress widget from 3 lines to 2 (#221)
- Perform delete operations in parallel (#223)
** Dependencies
- Update ~sbt-bloop~ to ~1.3.3~ (#224)
- Update ~zio~, ~zio-streams~ to ~1.0.0-RC14~ (#226)
- Update ~sbt-bloop~ to ~1.3.4~ (#228)
- Rollback ~sbt-ci-release~ to ~1.3.2~ (#231)
* [0.9.1] - 2019-09-28
** Fixed
- Progress UI should be updated when upload fails (#218)
** Dependencues
- Update ~aws-java-sdk-s3~ to ~1.11.641~ (#217)
* [0.9.0] - 2019-09-27
** Added
- Support multiple parallel uploads (#188)
** Dependencues
- Update ~jackson-dataformat-cbor~ to ~2.10.0~ (#216)
- Update ~jackson-databind~ to ~2.10.0~ (#214)
- Update ~sbt-ci-release~ to ~1.4.31~ (#211)
* [0.8.1] - 2019-09-23
** Fixed
- Not wrapping exceptions thrown in waitForUploadResult (#162)
** Dependencies
- Update ~jackson-dataformat-cbor~ to ~2.9.10~ (#208)
- Update ~jackson-databind~ to ~2.9.10~ (#206)
- Update ~sbt~ to ~1.3.2~ (#207)
- Update ~zio, zio-streams~ to 1.0.0-RC13~ (#209)
- Update ~aws-java-sdk-s3~ to ~1.11.635~ (#203)
- Update ~sbt~ to ~1.3.1~ (#204)
- Update ~sbt-wartremover~ to ~2.4.3~ (#202)
- Update ~sbt-ci-release~ to ~1.3.2~ (#200)
* [0.8.0] - 2019-09-11
** Added
- Log copy and delete operations (#119)
- Log errors with red label (#119)
- Ensure file handles are always closed (#122)
- [build] Install WartRemover plugin (#150)
- [build] Add GitHub Action
- [build] Add sbt-jacoco plugin for code coverage (#184)
** Changed
- [internal] Replace cats-effect with zio (#117)(#148)(#149)(#178)
- [internal] Replace Monocle with local SimpleLens implementation (#121)
- [internal] Don't use String as key in Map for hashes (#124)
- [internal] Convert Storage to full ZIO effect module (#133)
- [internal] Convert Config to full ZIO effect module (#134)
- [internal] Create FileSystem ZIO effect module (#135)
- [internal] Merge config package object with Config object (#138)
- [internal] Merge FS object into FileSystem object (#139)
- [internal] Assorted refactoring (#129)(#128)(#137)(#141)(#142)(#143)
- [internal] case classes shouldn't be OO objects (#147)
- [internal] Restructure using EIP-ZIO channels (#183)
** Fixed
- [progressbar] Reset progress bar for each file (#145)
- Handle building large changesets better (#163)
** Dependencies
- Update ~aws-java-sdk-s3~ from ~1.11.587~ to ~1.11.610~ (#118)(#144)(#158)
- Update ~jackson-databind~ from ~2.9.9.1~ to ~2.9.9.3~ (#132)(#152)
- Update ~scalamock~ from ~4.3.0~ to ~4.4.0~ (#165)
- Upgrade to Scala 2.13.0 (#176)
- Update ~sbt~ from ~1.2.8~ to ~1.3.0~ (#180)
- Update ~sbt-ci-release~ from ~1.2.6~ to ~1.3.1~ (#186)
* [0.7.2] - 2019-07-19
** Changed
- Apply ~scalafmt~ (#108)
- Uses Lenses (#113)
** Fixed
- Creates incorrect MD5 hash for some files (#103)
- NullPointerException creating scaladoc (#115)
* [0.7.1] - 2019-07-15
** Changed
- Only settings in ~.thorp.conf~ for explicit sources are used (#111)
** Fixed
- Not reading ~.thorp.conf~ file (#110)(#111)
* [0.7.0] - 2019-07-12
** Added
- Add a version command-line option (#99)
- Add a batch mode (#85)
- Display total size and progress for entire run (#94)
- Sync more than one source directory into a single bucket/prefix (#25)
** Dependencies
- Upgrade ~jackson-databind~ from ~2.9.9~ to ~2.9.9.1~ (#101)
- Upgrade ~aws-java-sdk-s3~ from ~1.11.580~ to ~1.11.587~ (#104)
- Upgrade ~sbt-assembly~ from ~0.14.9~ to ~0.14.10~ (#105)
* [0.6.1] - 2019-07-03
** Fixed
- Release to sonatype using correct profile name (#102)
* [0.6.0] - 2019-06-30
** Added
- Abstraction layer encapsulating S3 as Storage (#76)
- Display simple error message when bucket is invalid (#81)
- Shutdown storage service once completed (#88)
- Release snapshots to sonatype from TravisCI (#92)
- Embed launch script in fat-jar (#98)
** Fixed
- Use correct hash locally for comparing multi-part uploaded files (#82)
- Handle large list of action in a stack-safe manner (#93)
** Dependencies
- Upgrade ~scalamock~ from ~4.2.0~ to ~4.3.0~ (#84)
- Upgrade ~aws-java-sdk-s3~ from ~1.11.573~ to ~1.11.580~ (#86)
* [0.5.0] - 2019-06-21
** Added
- Add ~thorp-lib~ module (#66)
- Enable running outside of sbt (#55)
- ~-d~, ~--debug~ flag for log messages (#60)
- Read config from ~.thorp.conf~ in source directory (#71)
- Read config from ~$HOME/.config/thorp.conf~ and ~/etc/thorp.conf~
(#73)
- Add ~--no-global~ and ~--no-user~ options (#50)
- Display any upload errors in summary report (#50)
** Changed
- Rename project as 'thorp' (#75)
- Suppress Transfer event messages (#64)
- Better error message when source not found (#51)
- Reduced logging (#59)
- Prevent AWS SDK from recalculating MD5 hash (#50)
** Fixed
- Error when calculating md5 hash for large files (#56)
** Removed
- ~-v~ verbosity flag (#63)
** Dependencies
- Upgrade ~aws-java-sdk-s3~ from ~1.11.569~ to ~1.11.570~ (#57)
* [0.4.0] - 2019-06-11
** Added
- Multi-part uploads
- ~--include~ to select files for synchronisation
- Upload progress bar
** Changed
- ~--filter~ renamed to ~-exclude~
** Fixed
- Fetch md5 hashes for all remote objects, not just the first 1000
- Handle when a file goes away between scanning and uploading
** Dependencies
- Removed ~reactive-aws-s3-{core,cats}~ and the AWS SDK v2 that it
depended upon in favour of the AWS SDK v1
- Upgrade ~aws-java-sdk-s3~ from ~1.11.560~ to ~1.11.569~
- Upgrade ~cats-effect~ from ~1.2.0~ to ~1.3.1~
- Upgade ~scalatest~ from ~3.0.7~ to ~3.0.8~
* [0.3.0] - 2019-05-23
** Added
- Filter to exclude files
* [0.2.0] - 2019-05-22
** Added
- Display count of the number of files uploaded
** Changed
- Improved performance by fetching all MD5 hashes from S3 in single
request at startup
* [0.1.0] - 2019-05-13
** Added
- Initial Release
- Synchronise files with an S3 bucket, using an MD5 hash to
identify when the file has changed and needs to be uploaded
Reference in a new issue View git blame Copy permalink