• s3thorp
• Usage
• Behaviour
• Creating Native Images

s3thorp

Synchronisation of files with S3 using the hash of the file contents.

Originally based on Alex Kudlick's aws-s3-sync-by-hash.

The normal aws s3 sync ... command only uses the time stamp of files to decide what files need to be copied. This utility looks at the md5 hash of the file contents.

Usage

  s3thorp
  Usage: s3thorp [options]

    -s, --source <value>             Source directory to sync to S3
    -b, --bucket <value>             S3 bucket name
    -p, --prefix <value>             Prefix within the S3 Bucket
    -f, --filters <value>[,<values>]Exclude matching paths
    -v, --verbose <value>            Verbosity level (1-5)

Behaviour

When considering a local file, the following table governs what should happen:

#	local file	remote key	hash of same key	hash of other keys	action
1	exists	exists	matches	-	do nothing
2	exists	is missing	-	matches	copy from other key
3	exists	is missing	-	no matches	upload
4	exists	exists	no match	matches	copy from other key
5	exists	exists	no match	no matches	upload
6	is missing	exists	-	-	delete

Creating Native Images

• Download and install GraalVM

  • https://github.com/oracle/graal/releases

• Install native-image using the graal updater

    gu install native-image
  

• Create native image

    native-image -cp `sbt 'export runtime:fullClasspath'|tail -n 1` \
                 -H:Name=s3thorp \
                 -H:Class=net.kemitix.s3thorp.Main \
                 --allow-incomplete-classpath \
                 --force-fallback
  

• Resulting file requires a JDK for execution