Home

Syncany User Guide Release 0.4.6-alpha Philipp

1. 8 2 User specific Configuration 73 Syncany User Guide Release 0 4 6 alpha lt status gt lt forceChecksum gt false lt forceChecksum gt lt status gt lt force gt false lt force gt lt removeOldVersions gt t rue lt removeOldVersions gt lt maxDatabaseFiles gt 15 lt maxDatabaseFiles gt lt minSecondsBetweenCleanups gt 10800 lt minSecondsBetweenCleanups gt lt clean gt lt watch gt lt folder gt lt folders gt lt users gt lt user gt lt username gt admin lt username gt lt password gt IOgotcpZzNPh lt password gt lt user gt lt users gt lt daemon gt 8 2 3 GUI Config gui xm1 The GUI config file gui xm1 is only present if the graphical user interface is installed and has been started at least once Its purpose is to store GUI specific options only All of the options are meant to be set by the GUI Options for gui xm1 Option Mand Def Description lt tray gt no DEFAUL Tray icon engine Linux only lt theme gt no DEFAUL Icon set to use for the tray lt startup gt no false Run Syncany GUI at startup user login lt notifications gt no true Show or don t show notifications The lt tray gt option defines the tray backend used to display Syncany s tray icon Possible values are DEFAULT APPINDICATOR and OSX_NOTIFICATION_CENTER If DEFAULT or no value is set Syncany will try to automat ical
2. lt config xmlns http syncany org config 1 gt lt connection type s3 gt lt accessKey gt AKIAIHIALEXANDREUI IE lt accessKey gt lt secretKey encrypted true gt af81727a87abc68afel428319fad lt secretKey gt lt bucket gt syncany demo lt bucket gt lt location gt us west 1 lt location gt lt connection gt lt config gt 6 2 2 Dropbox Plugin The Dropbox plugin plugin identifier dropbox uses a folder in your Dropbox as a storage backend Data is stored in the Syncany repository format in a dedicated Apps folder of your Dropbox The plugin authenticates against the Dropbox REST API via a OAuth 2 0 During sy init you will be asked to navigate to the Dropbox website and copy an access token from there Note Syncany will only use Dropbox as a storage backend it is not an alternative Dropbox sync client In particular you will not be able to read files synchronized with Syncany using your Dropbox web interface because Syncany files are stored in the Syncany repository format In addition to that if you run both Syncany and the Dropbox client Dropbox will regularly show notifications about Syncany originated files that have been changed Due to the fact that Dropbox cannot disable notifications for certain folders it is not practical to run both Syncany and the Dropbox client Instead we suggest to only use Dropbox as a storage backend and disable close the Dropbox client
3. 6 2 7 RAIDO Plugin The RAIDO plugin plugin identifier raid0O virtually combines two storage backends into a single storage The plugin can use any two storage plugins e g an FTP folder FTP plugin and an Amazon S3 bucket Amazon S3 plugin Unlike a RAID1 or other RAID forms it does not mirror the storage or provide protection against the failure of one backend It merely combines their disk space If one of the backends fails all repository data is lost As of today there is no RAID1 plugin but we will provide it eventually The plugin is not installed by default but it can be easily installed using the sy plugin install command For details about how to use this command refer to the command reference at sy plugin List install and remove storage backend plugins Plugin Security The RAIDO plugin uses two other storage plugins so its security directly depends on the respective plugins Please refer to their documentation for details 50 Chapter 6 Plugins Syncany User Guide Release 0 4 6 alpha Options for config xml The RAIDO plugin options are a bit different from other plugins because depending on the chosen storage plugins the sub options are different If for instance an FTP plugin is chosen as storage 1 storagel type ftp the storage options are storagel hostname storagel username and so on Plugin Option Mandatory Default Value Description storagel type yes non
4. SYNOPSIS sy cleanup o delete older than lt relative time gt f force no delete interval O no delete older than FE no temp removal lt status options gt DESCRIPTION This command performs different operations to cleanup the local database as well as the remote store It removes old versions from the local database deletes unused multichunks if possible and merges remote database files if necessary Remove old file versions File versions are deleted by two criteria The first is if it is older than 30 days configurable with o disable with 0 The second is an interval based strategy to keep the version history readable By default one version is kept per minute in the last hour one verison is kept per hour in the last three days and one version per day is kept in the last month This strategy can be disabled with I Merge remote databases The remote databases of the local client are merged together if there are more than 15 remote databases per client The purpose of this is to avoid endless amounts of small database files on the remote storage and a quicker download process for new clients In addition databases are merged whenever versions are removed This command uses the status and ls remote commands and is only executed if there are neither local nor remote changes OPTIONS Oy delete older than lt relative time gt Sets the time th
5. COPYRIGHT Syncany 0 4 4 alpha Distributed under GPLv3 Copyright c 2011 2015 Philipp C Heckel 98 Chapter 10 Appendix A Manual Pages Syncany User Guide Release 0 4 6 alpha 10 16 sy watch NAME sy watch monitor local Syncany folder and automatically sync changes SYNOPSIS sy watch i interval lt sec gt s delay lt sec gt W no watcher a announce lt host gt lt port gt N no announcements lt status options gt lt up options gt lt down options gt DESCRIPTION Automatically synchronizes the local folder with the repository The command performs the up and down command in an interval watches the file system for changes and subscribes to the Syncany pub sub server In the default configuration no options the command subscribes to the Syncany pub sub server and registers local file system watches in the locally synced folder and all of its subfolders When local events are registered the command waits a few seconds waiting for settlement and then triggers the up command After the upload has finished a message is published to the pub sub server telling other clients of this repo that there is new data Clients subscribed to the repository s channel will receive this notification and immediately perform a down command This mechanism allows instant synchronization among clients even if a dumb storage server such as FTP is used In case
6. config syncany plugins and on Windows at AppData Syncany plugins It updates plugins by removing and re installing them from the Syncany host if they are updatable Plugins are updatable if they are out of dat and are not Global plugins 94 Chapter 10 Appendix A Manual Pages Syncany User Guide Release 0 4 6 alpha It removes locally installed plugins from the user s local plugin directory Only plugins installed by the install action can be removed The plugins shipped with Syncany e g the local plugin cannot be removed ACIONS The following actions are available within the plugin command list lt args gt lt plugin id gt Lists locally installed plugins and or remotely available plugins on api syncany org If lt plugin id gt is given the result list will be shortened to the selected plugin R remote only Turns off local plugin discovery In particular the result list will not include any information about the locally installed plugins Instead only remotely available plugins will be listed Cannot be used in combination with L L local only Turns off remote plugin discovery Contrary to R the result list will only include information about the locally installed plugins and no information about remote plugins The Syncany host will not be queried Cannot be used in combination with R s snapshots Instead of listing only plugin release versions default
7. The command itself only offers the typical start stop script sub commands namely start Starts the Syncany background process daemon if it s not already running stop Stop the Syncany daemon if it is running status Displays whether the daemon is running and if it is its PID restart Restarts the daemon process by calling stop then start Linux only reload Reloads the daemon configuration daemon xm1 without restarting the daemon Linux only e force stop Kills the Syncany daemon can be used if st op doesn t respond Linux only The Syncany daemon is a user level daemon That means that it does not run as system service but rather that each user may have its own daemon process running and that the daemon config is user specific The daemon configuration can be found at sAppData Syncany daemon xml Windows or at config syncany daemon xml Linux 5 2 1 Example Starting and stopping the daemon sy daemon start Starting daemon syncanyd pid 16336 sy daemon status Checking daemon syncanyd running pid 16336 sy daemon stop Stopping daemon syncanyd 30 Chapter 5 Commands Syncany User Guide Release 0 4 6 alpha 5 3 sy init Initializing a repository This command creates a new remote repository using the specified plugin and initializes the local directory Unless I is set the command is interactive and queries the user for input Depending on the chosen plugin ch
8. The plugin is not installed by default but it can be easily installed using the sy plugin install command For details about how to use this command refer to the command reference at sy plugin List install and remove storage backend plugins Plugin Security Dropbox REST API traffic is based on HTTPS so tranport security is ensured Since Syncany itself takes care of encrypting the files before they are uploaded the confidentiality of your data is not at risk Dropbox or any third party cannot read your files even if they access the encrypted files in your Dropbox folder Options for config xm1 Plugin Option Mandatory Default Value Description accessToken yes none OAuth access token displayed on the Dropbox website path yes none Repository folder in your Dropbox Syncany app folder Example config xml lt config xmlns http syncany org config 1 gt lt connection type dropbox gt lt accessToken encrypted true gt 5379020501945a9c7e6196cbh2bcl1 lt accessToken gt lt path gt RepoWork lt path gt 46 Chapier 6 Plugins Syncany User Guide Release 0 4 6 alpha lt connection gt lt config gt 6 2 3 Flickr Plugin The Flickr plugin plugin identifier 1ickr is a bit of a fun plugin It encodes all the user data into PNG images and uploads them to a Flickr album Data of all the files in a local Syncany folder is transformed into valid square PND i
9. To shorten the syncany links using the sy init genlink short option Syn cany will send the long link to the Syncany API and request a short link from there The link will only be transferred with the short option enabled Please note that this will link while heavily encrypted contains your encrypted login credentials see syncany Links All calls to the Syncany API can be manually overridden by specifying an alternative API endpoint api endpoint The pub sub server can be overridden by the announce command line options or the corresponding configuration setting 7 6 Source Code All the cryptography related code is implemented in the org syncany crypto package Feel free to inspect the code and create a new issue if something doesn t feel right 7 7 Known Issues and Limitations e In multiple peer reviews it has been suggested to drop the cipher nesting in favor of a single cipher While there is no evidence that a nested cipher is or might be weaker than a single cipher there is very little literature about it so it is probably not a good idea See issue 59 e As of today neither the master key nor the password can be changed See issue 150 1 The Syncany server is a rented server managed by the project lead of Syncany It is located in Germany and hosted by Host Europe GmbH 7 5 Syncany Server Communication 65 Syncany User Guide Release 0 4 6 alpha 66 Chapter 7 Security CHAPTER
10. 1 1I014XYsdjHRazvUJCB4GPOSA CDhp E80 5 14 sy plugin List install and remove storage backend plugins This command performs three different actions It lists the locally installed and remotely available plugins including version information and whether plugins can be upgraded It installs new plugins from either a given URL or a local file It removes locally installed plugins from the user s local plugin directory For a detailed command reference including all command specific options please refer to the manual page of this command at sy plugin For a detailed explanations of plugins refer to the chapter Plugins 5 14 1 Example 1 List all available plugins List only released plugins no snapshots sy plugin list Id Name Local Version Remote Version Inst Upgr ftp FTP 0 1 0 alpha yes local Local 0 1 2 alpha yes s3 Amazon S3 0 1 0 alpha yes sftp SFTP 0 1 0 alpha yes webdav WebDAV 0 1 0 alpha yes List released plugins and snapshots sy plugin list snapshots Id Name Local Version Remote Version ftp FTP 0 1 0 SNAPSHOT 1404181428 git1a147 local Local 0 1 2 alpha SNAPSHOT 1404200229 gitadd2848 s3 Amazon 3 0 1 0 SNAPSHOT 1404182252 git78f0fla 0 1 0 SNAPSHOT 1404182149 gitb7b29 sftp SFTP 0 1 0 alpha SNAPSHOT 1404191549 git webdav WebDAV 0 1 0 alpha SNAPSHOT 1404200235 git79d610f 0 1 0 alpha SNAPSHOT 1404192343 git1 5 14 2 Example 2 Inst
11. 2 Code repeatd 14 07 23 18 48 19 rw r r 247367 FILE 4b66adf265 593a67cd5e 2 Code repeat dly_compilin 5 13 sy genlink Create a syncany link from an existing local folder This command creates a Syncany link syncany from an existing local folder The link can then be sent to someone else to connect to the repository Syncany links contain the connection information of the storage backend so in case of an FIP backend host user pass etc would be contained in a link If the link is shared be aware that you are giving this informa tion to the other users For a detailed command reference including all command specific options please refer to the manual page of this command at sy genlink 5 13 1 Example 1 Normal usage of genlink sy genlink To share the same repository with others you can share this link 5 13 sy genlink Create a syncany link from an existing local folder 39 Syncany User Guide Release 0 4 6 alpha syncany storage 1 1I014XYsdjHRazvUJCB4GPOSA CDhp This link is encrypted with the given password using unsecure communication WARNING The link contains the details of your repo connection which typically consist of usernames password of the connection e g FTP user pass chat e mail etc E800YNkpSCSU8Bh so you can safely share it 5 13 2 Example 2 Short output for scripts sy genlink short syncany storage
12. 58 Chapier 6 Plugins CHAPTER 7 Security Syncany takes security very seriously Our goal is to protect the confidentiality of user data stored on remote servers meaning that whatever leaves the local computer is encrypted This chapter explains the security concepts of Syncany to the interested user in a hopefully understandable way Due to the complexity of the subject however this is a rather technical chapter If you just want to use Syncany you can safely skip this chapter Contents e Security Prologue Security Assumptions Security Concepts x Encryption Scheme x HTTPS only Daemon Attack Scenarios x Attacking Data Transmissions x Provider originated Attacks x Theft of Credentials Syncany Server Communication Source Code Known Issues and Limitations 7 1 Prologue Although we take special care designing and implementing the security related parts of Syncany we are not cryp tographers We have asked multiple parties to peer review our concept and implementation and we hereby do that again Please review our security concepts and implementation If you believe that any of the concepts and or the implementation is incorrect insecure or can be attacked please let us know by creating a new issue or by contacting us directly for major issues 7 2 Security Assumptions Syncany s central security goal is to protect the user s data against attacks on confidentiality and inte
13. 8 Configuration Syncany differentiates between two types of configuration e Folder specific Configuration Each folder that is synchronized and managed by Syncany contains a syncany folder The files in this folder define the per folder config options such as the backend storage credentials or the machine name e User specific Configuration The user config defines central settings valid only for the logged in user this includes proxy configuration or the standby settings but also daemon specific configuration such as which Syncany folders are managed by the background daemon and where the web frontend and REST WS API is running Syncany stores the folder specific configuration options in syncany of the synchronized folder and other general user specific configurations such as the user and daemon config in t AppData Syncany or config syncany Contents e Configuration Folder specific Configuration x Common Settings and Storage Connection config xml x Excluding Ignoring Files and Folders syignore User specific Configuration x User Config userconfig xml x Daemon Config daemon xm1 x GUI Config gui xm1 Keys and Certificates x Private Keys keystore jks x Trusted Certificates truststore jks x Using your own Keypair and Certificate 8 1 Folder specific Configuration Whenever a new Syncany folder is initialized via sy initorsy connect Syncany creates a syncany folder in t
14. Ee a ge BAe oe T syncany Server Communication seo es 248A bed 4 eh bebe oo oe Pe ee e wR RS LOr Source Code secs s Gh ave Pow bo ee Oe eb dee bee ee we eee eb eee amp A Td Known Issues and Limitations lt 4 4 2 200 5 bia 4 eA Se REAY GES EE OA E ES Configuration 8 1 Folderspecific Comfisurationy lt a oe i Ba RG Ra eA a ae R A 8 2 User specific Configuration lt c osa se cgr 5 bea ee ea eR ee eee bee So Keys and Cerhmicates 2 ii detticg wh nadevii dear T we een a Sand FAQ Ol Gen raliquestions 2 s La e pad k oR Eee oe hee Ra oe a eee ee EA 9 2 Technial and security qUestiONS s s es sha eee A ee eee a e a 9 3 COMMON erro S eck e y Ae A a ee oe RRS RE ee a eee 10 Appendix A Manual Pages VOM ASYSYNCANY x con Gok 2 mars ie too we eae doe be Se Oe ae a oe we A os ee ae Gs 10 2 Sy Cleanup lt 2 242 24 40 846 be eB SR eA oS SE Me EA ESS Be EES SHES BESS 10 3 y Connect macmu Be eas ale Aca ects ein we amp Boe eee e Ee ae a ee Bee es 10 4 sydan 22 oe ae hehe dh Se Mate eee EG aAa Eee a bite Ale a eles eH cece S NOES ASW COW go a os we kag oO ee eo Ge ee a oe Oe Be a ceca ey bs ee Bp eo ee et 10 6 sy genin 3052 Pew ae eae be Pah ChE A GA wee oe dh Gee ea ee TOT SY AMI ee haces et 3 8 ele os oe ae Re BBO A AE ed we Sebo amp Gs bo Bw ely Sots i IOS GY lOD we be Bea ee oe Bee eA RE RA ERED owe ee MAGE ERE SS 10 9 Sy ISHeEMOle ie ares Ge eA EEE Oe REE ES Bee eR Ee Ss IO AO SIS eke RR b
15. Example config xml lt config xmlns http syncany org config 1 gt lt machineName gt PCigqLdSQiampovfBfSZZ lt machineName gt lt displayName gt pheckel lt displayName gt lt masterKey salt 51d32e99230581le2 key ba2f 6725d2ce7d90822 gt lt connection type ftp gt lt hostname gt ftp example com lt hostname gt lt username gt armin lt username gt lt password encrypted true gt 87abc68afel428319fad lt password gt lt path gt syncany repo2 lt path gt lt port gt 21 lt port gt 68 Chapter 8 Configuration Syncany User Guide Release 0 4 6 alpha lt connection gt lt cacheKeepBytes gt 524288000 lt cacheKeepBytes gt lt config gt 8 1 2 Excluding Ignoring Files and Folders syignore The syignore file allows you to ignore certain files and folders from the synchronization process It must be created manually by the user if any exclude ignore logic is desired The file resides in the root of the managed folder and is itself synchronized to other clients using Syncany Note As of today new entries in the syignore file are not picked up by Syncany files are not ignored if the to be ignored file has already been synchronized We are aware that this is not a desired behavior and are working on it to fix it The file has a simple line based structure in which each line represents a path to be ignored by Syncany The file supports the typical wildcards an
16. all database versions set this option to l s database start lt index gt Adjusts the start index of the databases to return In combination with n this option can be used for pagination If s is not given the first n databases will be returned The default for s is 0 file count lt count gt Adjusts the max number of changed files per database version to be returned and displayed by this command If this option is not set max 100 files are displayed To return all files per database version set this option to 1 X xclud mpty Excludes empty database version from the result If this option is 10 8 sy log 91 Syncany User Guide Release 0 4 6 alpha not given empty databases will be listed as empty COPYRIGHT Syncany 0 4 4 alpha Distributed under GPLv3 Copyright c 2011 2015 Philipp C Heckel 10 9 sy Is remote NAME sy ls remote list changes on remote repository SYNOPSIS sy ls remote DESCRIPTION This command compares the list of locally known remote databases with the remotely available client databases and prints new unknown files to the console This command is used by the down command to detect which remote databases to download and compare COPYRIGHT Syncany 0 4 4 alpha Distributed under GPLv3 Copyright c 2011 2015 Philipp C Heckel 10 10 sy Is NAME sy ls lists and filters
17. among clients even if a dumb storage server such as FTP is used 5 8 sy down Retrieve and apply remote changes 35 Syncany User Guide Release 0 4 6 alpha In case file system events or pub sub notifications are missed the periodic synchronization using the down and up command is implemented as a fallback Note The messages exchanged through the pub sub server do not include any confidential data They only include the repository identifier randomly generated in the init phase and a client identifier randomly generated on every restart For a detailed command reference including all command specific options please refer to the manual page of this command at sy watch 5 9 1 Example Automatically syncing a folder with sy watch The wat ch command is a blocking command That means when it is run the command will not run in the background If you desire the folder to be synced in the background use the Syncany daemon Details at Automatically syncing files cd Syncany sy watch This command blocks use the daemon if you don t want this to happen 5 10 sy cleanup Remove old versions from the local database and the repo This command performs different operations to cleanup the local database as well as the remote store It removes old versions from the local database deletes unused multichunks if possible and merges a client s own remote database files if necessary Merge
18. chained 1 n ciphers e g AES 128 and Twofish 256 e Cipher configurations IVs and salts are authenticated with an HMAC SHA256 7 3 2 HTTPS only Daemon The Syncany daemon provides an API and a web interface that can be access over HTTPS not HTTP The API is also available via secure WebSockets Keys and Certificates The keypair and certificate used for the HTTPS server is generated by Syncany upon the first startup of the daemon Syncany generates a 2048 bit RSA keypair and then uses this keypair to generate a self signed X 509v3 certificate with a validity of 5 years The certificates common name is set to the local hostname and the organization and org unit to Syncany The certificate s SHA 256 hash is signed using the RSA private key signature algorithm Certificate Data Version 3 0x2 Serial Number 1409206372293 0x1481b3ec7c5 Signature Algorithm sha256WithRSAEncryption Issuer CN localhost O Syncany OU Syncany Validity Not Before Aug 27 06 12 52 2014 GMT Not After Aug 27 06 12 52 2019 GMT Subject CN localhost O Syncany OU Syncany Subject Public Key Info Public Key Algorithm rsaEncryption Public Key 2048 bit odulus 00 a0 43 ca d6 e6 e9 70 2d ca d5 77 ad e9 3a la 50 fe Exponent 65537 0x10001 Signature Algorithm sha256WithRSAEncryption 74 7b a9 22 e3 fb 21 cf 15 3c ba 11 46 c4 7a 6c 8e 2c f4 aa cc 27 98 e7 The private key and the certificate are stored in a key tru
19. changes sy status force checksum M one thousand txt 5 5 sy status Detect and display local changes 33 Syncany User Guide Release 0 4 6 alpha 5 6 sy up Detect local changes and upload to repository This command detects changes in the local folder indexes new files and uploads changes to the remote repository If there are local changes the command determines what has changed packages these changes in new multichunks and uploads them to the remote storage alongside with a delta metadata database To determine the local changes the status command is used All options of the status command can also be used in this command If there are no local changes the up command will not upload anything no multichunks and no metadata For a detailed command reference including all command specific options please refer to the manual page of this command at sy up 5 6 1 Example 1 Basic usage index and upload locally changed files sy up A testfile txt A testfile2 txt Sync up finished 5 6 2 Example 2 Force checksum calculation per file Forcing checksum calculation means that we don t want to rely on last modified date and size sy up force checksum A testfile txt A testfile2 txt Sync up finished 5 7 sy 1ls remote Detect and display remote changes This command detects changes in the local folder indexes new files and uploads changes to the remo
20. date time at which to select the file tree e g sy 1s D20m to select the file tree 20 minutes ago or sy 1s D2014 05 02 to select the file tree at May 2 For a detailed command reference including all command specific options please refer to the manual page of this command at sy Ls 5 12 1 Example 1 Create new repository interactive mode 140628161200_IMG_3575 j lder compiling_te xml sy ls 14 07 23 22 54 07 EW r f 2174138 FILE 941494aa52 3910ca5c8a 1 14 07 23 09 08 08 rwxr xr x 4096 FOLDER 6ba412f98b 1 Code 14 07 23 22 45 58 rwxr xr x 4096 FOLDER 9027a43b2b 1 Pictures 14 07 23 22 54 07 rwxr xr x 4096 FOLDER 08319c3f16 1 Untitled Fq 14 07 23 21 10 05 rwxr xr x 4096 FOLDER 6215d124dd 1 repeatedly 14 07 23 22 12 11 rw r r 353 FILE 3a0alccbba faebf2beb1l1 1 userconfig 5 12 2 Example 2 Listing a specific subtree as per database sy ls Code 14 07 23 09 08 08 rwxr xr x 4096 FOLDER 4625720447 1 Code fanout 38 Chapter 5 Commands Syncany User Guide Release 0 4 6 alpha 5 12 3 Example 3 Listing all directories recursively S sy ls recursiv types d 14 07 23 09 08 08 rwxr xr x 4096 FOLDER 6ba412f98b 1 Code 14 07 23 09 08 08 rwxr xr x 4096 FOLDER 4625720447 1 Code fanout 14 07 23 09 08 08 rwxr xr x 4096 FOLDER 7adc2e20c5 1 Code fanout fanout 14 07 23 09 08 08 rwxr xr x 4096 FOLDER 98f8df9aec 1 Code fanout fanout debian 14 07
21. described in so called database versions see database files These database versions are somewhat similar to a commit in a version control system they describe a set of file changes Each database version is identified by a vector clock a logical clock that allows ordering of events in distributed systems Using these vector clocks Syncany knows how to order the database versions of the clients and how to resolve conflicts Whenever a client calls sy down Syncany checks for new database files i e database versions of other clients and downloads them It extracts them and orders them using the vector clocks and then compares the local file system to the result of the changes described in the database versions of the other clients These changes are then applied to the local file system 4 4 3 Conflict handling When Syncany detects that two database versions were created independently of one another i e their vector clocks are independent a conflict has occurred The conflict is resolved by simply comparing the local timestamps of the conflicting database versions to determine a winner 8 The winner s database version s are applied locally and the loser s database version s are discarded When the losing client detects that it lost it ll reconcile the database versions and re upload its changes Balasubramaniam and B C Pierce What is a file synchronizer In Proceedings of the 4th annual ACM IEEE international conferen
22. entire version history for the selected files Using group the result can be grouped by the file history identifier OPTIONS V versions Select and display the entire history of the matching files instead of only the last version Useful with group t types lt t d s gt Limits the result set to a certain file type f for files d for directories and s for symlinks Types can be combined e g sy ls tfs selects files and symlinks Default setting is tds D date lt relative date absolute date gt Selects the file tr at a certain date The date can be given as a relative date to the current time or an absolute date in form of a timestamp Absolute date format lt yy MM dd HH mm ss gt Relative date format lt value gt lt unit gt for which lt value gt may be any floating point number and lt unit gt may be any of the following s econds m inutes h ours d ays w eeks mo nths y ears Units may be shortened if they are unique Examples 5h30m or lylmo2d H file history If the option is given the lt path expression gt is interpreted as a file history identifier This option can be used to select one specific file history If H is given V is automatically switched on ry recursiv Not only selects the folder relative to the lt path expression gt but to all sub trees of it q deleted Also selects files that have been deleted from the file
23. issues or notifications In particular plugins don t have to deal with large files or resumable uploads downloads Syncany uploads files with a max size of about 4 MB and manages connections itself Plugins don t need to be bothered with any of that Syncany takes care of that and thereby keeps the reponsibility of a plugin very small Notably plugins don t have to implement any active components on the server side That means that there is no server component on the storage backend that has to deal with managing clients or connections Instead any dumb storage backend can be used to implement a plugin This obviously includes the classical storage systems such as S FTP or WebDAV but it extends to using protocols that were t really built for storage backends One could use IMAP to store files in e mails or use the Google Images Picasa API to store files in images With Syncany s minimal storage API any storage can be used 4 2 Minimizing remote disk space through deduplication Normal sync tools like rsync or Unison simply copy files from the local machine to the offsite storage Granted they transfer files pretty efficiently but they store files at the offsite storage as they store them locally Two marginally different or even identical files locally take up the same disk space as locally Syncany is different Syncany uses data deduplication to minimize the amount of disk space used on the offsite stor age Deduplication explo
24. machine readabl Only prints the link and leaves out any explanatory text Useful if the link is used in a script COPYRIGHT Syncany 0 4 4 alpha Distributed under GPLv3 Copyright c 2011 2015 Philipp C Heckel 10 6 sy genlink 89 Syncany User Guide Release 0 4 6 alpha 10 7 sy init NAME sy init intialize local directory and create remote Syncany repository SYNOPSIS sy init P plugin lt plugin gt o plugin option lt key value gt E no encryption G no compression s short t create target a advanced n add daemon password DESCRIPTION This command creates a new remote repository using the specified plugin and initializes the local directory Unless o is set the command is interactive and queries the user for input Depending on the chosen plugin chosen with P or interactively different plugin specific options are required or optional Once the init command was successfully executed the initialized local folder can be synced with the newly created repository The commands up down watch etc can be used Other clients can then be connected using the connect command OPTIONS P plugin lt plugin gt Selects a plugin to use for the repository Local files will be synced via the storage specified by this plugin Any of the following available plugins can be used SPLUGINSS O plugi
25. offsite storage instead of a provider controlled storage Syncany is the right tool for you We ve identified three major use cases that Syncany can be used for e Continuous file synchronization e Interval based or on demand backups e Simple binary compatible file versioning 1 3 What Syncany is not Because there are so many sync tools out there it s very easy to assume that Syncany is just like the others Here are a few things that Syncany is not e Syncany is not a free or pay as you go service Syncany is a tool e Syncany does not provide storage You must bring your own storage e Syncany is not a peer to peer network Data is stored centrally and you must trust the persons you share files with e Syncany has version control capabilities but it is not a fully fledged version control system like Git e Syncany is not yet usable for critical files It s alpha software Don t do it 4 Chapter 1 What is Syncany Syncany User Guide Release 0 4 6 alpha 1 4 Example Use Cases 1 4 1 Use Case 1 Friends sharing files Pim wants to share some ideas with Philipp and Philipp wants to share some pictures with Steffen Pim has a Windows server running a Samba server Windows share and Philipp has has a virtual server from a random hosting company lying around all set up with a spare SFTP account Pim s Laptop we Shared Ideas Philipp s PC Pim s a Samba Share We Shared Ideas Shared Pict
26. protocol or storage provider to store its repository files Already implemented examples include FTP SFTP Samba or Amazon S3 e Web interface plugins implement a web frontend for Syncany Contents e Plugins About Plugins x Installing and Removing Plugins x Plugin Configuration Storage Plugins x Amazon S3 Plugin Dropbox Plugin Flickr Plugin FTP Plugin Local Plugin OpenStack Swift Plugin RAIDO Plugin Samba Plugin SFTP Plugin WebDAV Plugin Graphical User Interface Plugin Web Interface Plugins x Simple Web Interface Plugin 6 1 About Plugins 6 1 1 Installing and Removing Plugins The sy plugin command offers a very easy way to list available plugins sy plugin list install new plugins sy plugin install and remove already installed plugins sy plugin remove For a detailed command reference and examples please refer to sy plugin List install and remove storage backend plugins 43 Syncany User Guide Release 0 4 6 alpha When installing new plugins or listing available plugins the plugin command accesses the Syncany Plugin Reposi tory a still very small repository in which all available Syncany plugins are registered and can be downloaded from The repository API provides information about the the newest plugins their versions and provides a download link where to get them If for instance a user wants to install the SFTP plugin calling sy plugin list wil
27. remote databases Unless M is specified the remote databases of the local client are merged together if there are more than 15 remote databases The purpose of this is to avoid endless amounts of small database files on the remote storage and a quicker download process for new clients Remove old file versions Unless V is specified file versions marked as deleted and files with as history longer than lt count gt versions will be removed from the database and the remote storage This will cleanup the local database and free up remote storage space Per default the number of available file versions per file is set to 5 This value can be overridden by setting k This command uses the status and ls remote commands and is only executed if there are neither local nor remote changes For a detailed command reference including all command specific options please refer to the manual page of this command at sy cleanup 5 10 1 Example 1 Default cleanup The default cleanup command can be run manually or triggered automatically if run in daemon mode It ll delete old multichunks shorten file histories and thereby free up space on the offsite storage sy cleanup 15 database files merged 8 multichunk s deleted on remote storage freed 12 91 MB 19 file histories shortened Cleanup successful 36 Chapter 5 Commands Syncany User Guide Release 0 4 6 alpha 5 10 2 Example 2 Shorten file h
28. simply download either the zip or the tar gz archive from the distribution site see folder releases Then extract the 10 Chapter 2 Installation Syncany User Guide Release 0 4 6 alpha archive and run Syncany and or the daemon from the bin folder This is the exact same process on every major operating system 1 Download the latest archive from the distribution site see folder releases 2 Extract the archive and run bin sy to run the Syncany command line tool If you d like to use Syncany from there but without having to always type the entire path to run it you may want to place the Syncany bin folder in the system s PATH environment variable This will let your system know where to look for the sy command It s easy to find out how to do that but just in case How to set the PATH variable on Windows e How to set the PATH variable on Mac OSX How to set the PATH variable on Linux 2 3 Installing the latest snapshot In addition to the releases we also provide snapshot versions of Syncany Snapshots are bundles that reflect the cutting edge state of development They are created for every single commit in our versioning system so updates come very frequently sometimes up to dozens of times per day Unless you d like to test a new feature before everyone else or you are asked by a developer to install that version it is highly discouraged Things might break They might be incompatible to previ
29. so Default is the central Syncany pub sub server at notify syncany org 8080 The SparkleShare pub sub server can be used interchangeably To set up your own server 10 16 sy watch 99 Syncany User Guide Release 0 4 6 alpha install a fanout instance from https github com travisghansen fanout N no announcements Disables the pub sub server entirely Syncany will not connect to the server and changes that are published to the pub sub server are not detected Instead remote changes will only be detected by the periodic synchronization loop If N is set th i interval option becomes more relevant as remote synchronization entirely relies on the interval Sag interval lt sec gt Sets the synchronization interval of the periodic down up loop to be run every lt sec gt seconds The sync loop is a fallback only and is not relevant if the pub sub server and the file system watching is enabled Default value is 120 seconds In addition to these options all arguments of the commands status ls remote up and down can be used COPYRIGHT Syncany 0 4 4 alpha Distributed under GPLv3 Copyright c 2011 2015 Philipp C Heckel 100 Chapter 10 Appendix A Manual Pages CHAPTER 11 Appendix B Videos amp Links There is quite a bit of reading material on Syncany already Check out the following links and videos Disclaimer Some of the videos and articles might be outd
30. storage Choose wisely 10 chars As soon as you ve successfully run the init command the repository is all set up You can now sync files to it using sy up and sy down and you can connect other client with the sy connect command The command will output a link that you can share with trusted friends or colleagues that enables them to connect to the same repository Note Syncany will synchronize everything in the initialized folder including sub folders If you are just starting out choose a small directory and and get a feel for the tool before synchronizing large amounts of data Do not sync your home directory 3 2 3 Syncing files Syncany repositories can be synchronized manually via the command line or automatically with the help of the background process or daemon e Syncing files manually is useful for automated backup jobs or if you want to use Syncany like a version control system e Using the automatic syncing turns Syncany into a Dropbox like tool because changes on local files are detected automatically and remote changes are synced right after they happen 3 2 4 Manually syncing files To use Syncany for manual syncing the sy status sy upand sy down commands are your best friends These commands work similar to a version control system e The sy status command shows you if you ve made local changes 18 Chapter 3 Getting Started Syncany User Guide Release 0 4 6 alpha e The sy up command
31. sy connect Connecting to an existing repository 5 sy status Detect and display local changes se s s se s a4 Saw ee eR G 5 6 sy up Detect local changes and upload to repository 5 7 sy ls remote Detect and display remote changes 5 8 sy down Retrieve and apply remote changes 5 9 sy watch Automatically synchronizes the local folder with the repo 5 10 sy cleanup Remove old versions from the local database and the repo 5 11 sy restore Restore older versions of files from the repository 5 12 sy 1s Display current and historic local database 5 13 sy genlink Create a syncany link from an existing local folder 5 14 sy plugin List install and remove storage backend plugins Plugins GIL Aout PINSIN 234 24 8 oct wa Ske os ee Ae Sete A Oe eS Ew Ae es 62 Storage PlMpINS s Beek es Eid Se ee Are ee LE Raw Bowed hb E Areal amp eae we Bele 6 3 Graphical User Interface Plugin s s pes Geb see pega Pe gama s ee GREE ee A 64 WebiInterface Plugims 24 2 4 24 h 220068 4b bai we BAe bh ee eee ba ee ewe Security TA SPROLOBWC ee bok cn tens Ae gh ee oS wh ae et AP Sok as as ths he Be eh aye A acess a Rte 7 2 S CUrtityASSUMPHONS se gk bea Roe GAO e Phe wee ee Sede ea A Tea Security CONCEPIS 442 552 S44 eee BEES Se SEE SS TA Attack Scenarios oui 6 amp are x ew YR ee SK EA Soe es
32. the current and past file tree SYNOPSIS sy ls V versions t types lt types gt D date lt date gt a recursive f full checksums g group H file history q deleted lt path expression gt DESCRIPTION This command lists and filters the file tr based on the local database The file tr selection can be performed using the following selection criteria 1 Using the lt path expression gt one can select a file pattern such as txt gt or sub tree such as subfolder only with r 2 Using r the command does not only list the folder relative to th lt path expression gt but to all sub trees of it 3 The t option limits the result set to a certain file type f for files d for directories and s for symlinks Types can be combined e g sy ls tfs selects files and symlinks 4 The D option selects the date time at which to select the file tree g sy ls D20m to select the file tr 20 minutes ago or sy ls D2014 05 02 to select the file tr at May 2 5 The H option can be used to select a specific file only If the option is given the lt path expression gt is interpreted as a file history identifier 6 The q flag will display files that have been deleted from the file system 92 Chapter 10 Appendix A Manual Pages Syncany User Guide Release 0 4 6 alpha Using the versions flag the command also displays the
33. the result list will also include daily snapshots if newer snapshots exist a api endpoint lt url gt Selects the API endpoint to query for remote plugins If not given the default endpoint URL will be used https api syncany org v3 install lt args gt lt URL gt lt file gt lt plugin id gt Installs a plugin from an arbitrary URL local file or from the available plugins on api syncany org with a plugin identifier s snapshot Installs the daily snapshot instead of the release version Only if lt plugin id gt is given Not for lt URL gt or lt file gt m minimal output Reduces the output of the command to OK or NOK instead of reporting detailed progress and download URLs update lt plugin id gt Updates the plugin with the plugin identifier lt plugin id gt or updates all updatable plugins if no identifier is given Plugins can only be updated if they are newer than the installed version and if they are User plugins and not Global plugins Plugins will be downloaded from api syncany org remove lt plugin id gt Uninstalls a plugin entirely removes the JAR file This action can only be used for plugins that were installed by the user and not for system wide plugins COPYRIGHT 10 11 sy plugin 95 Syncany User Guide Release 0 4 6 alpha Syncany 0 4 4 alpha Distributed under GPLv3 Copyright c 2011 2015 Philipp C Heckel 10 12 sy res
34. the two works synchronize and anything 9 1 11 How do you pronounce Syncany e In phonetic symbols simkni e Like in this audio file 9 1 12 SyncAny Sync Any Syncanny No Syncany 9 2 Technial and security questions 9 2 1 Why is it written in Java That s easy Because we speak Java 9 2 Technial and security questions 79 Syncany User Guide Release 0 4 6 alpha 9 2 2 Can you write a plugin for X People often ask about plugins for other storage backend examples include Hazelcast OpenStack Swift Box net Amazon Cloud Drive etc We re currently concentrating on the core functionality so plugins are not very high on the list However plugins are very easy to develop just 200 lines of code or so If you want you can of course develop the plugin yourself Plugins can be independently developed and deployed in other repos 9 2 3 What is the purpose of storing the master salt in the repo Isn t it already included in the Syncany link The master salt is stored in the repo in case we do not use or cannot use Syncany links When we simply do sy connect we don t have the salt anywhere so it must be retrieved from the server 9 2 4 What exactly is stored in the syncany file in the repo Is it needed The Syncany file stores information about the chunking mechanisms used It is currently only used to check if the password is correct see if it decrypts and deserializes correctly
35. your own storage FTP S3 WebDAV NFS Samba Windows file share e Files are encrypted before upload don t trust anybody but yourself e Files are deduplicated locally massive space savings on remote storage e Files are intelligently versioned restoring old versions or deleted files is easy Excited yet Ready to learn more Command Line Mobile Client Offsite Storage SFTP Amazon S3 3 Syncan FTP Samba CIFS Web Client pode WebDAV Daemon Sone Desktop Client Trustworthy Not trustworthy We trust our own systems Local We don t trust the cloud All machines are secure providers are assumed to be evil Syncany takes the idea of a Dropbox like continuous file synchronization tool and applies it to the real world It makes use of the storage that is available rather than being bound to a certain protocol or backend a certain provider is offering With a simple plugin based storage system Syncany can turn almost anything into a usable backend storage Right now it can use folders on an FTP or SFTP storage WebDAV or Samba shares and Amazon S3 buckets Syncany User Guide Release 0 4 6 alpha but that s just the beginning Because it s so easy to implement plugins more are definitely coming Even things like using IMAP folders or Flickr s free 1 TB of image data could be used to store your data Besides its incredible flexibilty in terms of storage Syncany is also a very privacy aware software User data never leaves t
36. 23 09 08 08 rwxXr xr x 4096 FOLDER 09fe5113f1 1 Code fanout fanout debian 14 07 23 22 45 58 rwxr xr x 4096 FOLDER 9027a43b2b 1 Pictures 14 07 23 22 54 07 rwxr xr x 4096 FOLDER 08319c3f16 1 Untitled Folder 14 07 23 21 10 05 rwxr xr x 4096 FOLDER 6215d124dd 1 repeatedly_compiling_test 14 07 23 21 10 05 rwxr xr x 4096 FOLDER fc5a5966bb 1 repeatedly_compiling_test 5 12 4 Example 4 Listing all versions of all PDF files in a certain folder sy ls versions group repeatedly_compiling_test scriptie pdf File 33b1042a91 repeatedly_compiling_test scriptie Scriptie pdf 14 07 23 10 28 25 rw r 4r 273966 FILE a1d3b30444 33b1042a91 1 repeatedly_compiling_ 14 07 23 18 48 19 rw r r 273966 FILE a1d3b30444 33b1042a91 2 Code repe atedly_compi 14 07 23 21 10 05 rw r r 273966 FILE a1d3b30444 33b1042a91 3 repeatedly_compiling_ File 593a67cd5e repeatedly_compiling_test scriptie VoorlopigeScriptie pdf 14 07 23 10 28 25 rw r r 247367 FILE 4b66adf 265 593a67cd5e 1 repeatedly_compiling_ 14 07 23 18 48 19 rw r r 247367 FILE 4b66adf265 593a67cd5e 2 Code repe atedly_compi 14 07 23 21 10 05 rw r r 247367 FILE 4b66adf 265 593a67cd5e 3 repeatedly_compiling_ 5 12 5 Example 5 Listing all PDF files at a certain time edly _compilin sy ls date 14 07 23 18 48 20 recursive pdf 14 07 23 18 48 19 rw r r 273966 FILE a1d3b30444 33b1042a91
37. 87b8d88 L S Authenticated cipher mode definitions with corresponding IVs and salts Plaintext encrypted and authenticated input for file header HMAC SHA256 with AES 128 bitin GCM mode The diagram shows an example for the default configuration For each file to be encrypted Syncany uses HKDF with SHA 256 and a random 96 bit salt to derive a 128 bit file key to use for the AES cipher 8 10 In addition to that a 128 bit random IV is created and used as input for the cipher 11 Using the 128 bit key and the 128 bit IV Syncany then encrypts plaintext multichunk file database file etc using the AES cipher in GCM mode To reduce improve encryption decryption performance Syncany re uses file keys up to 100 times meaning that up to 100 multichunks or database files are encrypted with the same key Given that the maximum file size for multichunks is about 4 MB max 400 MB might be encrypted with the same key although typically it s much less IVs are never re used The salts and IVs as well as the cipher configuration itself here AES GCM is stored in the file header of the crypto file format Since this information is required to decrypt the files they are unencrypted However to avoid an attack on the clients through header tampering the header is authenticated using an HMAC with SHA 256 using a 128 bit header key derived from the master key and a random 96 bit header salt Magic Version Header Cipher i Cipher i Au
38. Installation requirements Installing the latest release x Windows Debian Ubuntu Linux Mint Arch Linux Mac OS X Docker x Manual Installation for other operating systems Installing the latest snapshot Installing from source 2 1 Installation requirements Since Syncany heavily relies on Java you obviously need an up to date version of Java installed And that is about it No other requirements Syncany User Guide Release 0 4 6 alpha e Java JRE gt 7 See Oracle Website Downloads to install Java Be sure to load the 64 bit JVM if you intend to load the 64 bit Syncany or the gui plugin will not run e bash completion gt 2 Linux only If it s not installed by default it s definitely in the default packages 2 2 Installing the latest release 2 2 1 Windows On Windows you can either manually extract the Syncany files from the ZIP archive or use the installer The latter is obviously more comfortable but both variants have their reasons The installation using the Windows installer is easy and very similar to the installation of other applications 1 Download the latest release from the distribution site see folder releases The installer files have the exe ending You can t miss it 2 Then run the executable and follow the instructions 3 After the installation open the command prompt and type sy When you run the executable you ll see a typical installer that looks something
39. Philipp gt sy vu 1 8 alpha SNAPSHOT 1468652367 gitaf44247 EERIE 3 ON fib 1 ae 2 2 2 Debian Ubuntu Linux Mint For Debian based systems we provide an APT archive for installation via apt get as well as a way to manually download and install prebuilt deb packages see distribution site Installing via APT archive 1 Add the APT archive http archive syncany org apt release to your additional software sources this might ask you to confirm the Syncany public key as a trusted key 2 Once that is done you can now update the package archives by sudo apt get update and install Syncany with sudo apt get install syncany 3 After the installation open the terminal and type sy Again for the command line lovers sudo apt key adv keyserver hkp keyserver ubuntu com 80 recv keys A3002F0613D34268 sudo sh c sb http eas tas sudo apt get update 2 2 Installing the latest release 9 D70AEF E3F6B7 Syncany User Guide Release 0 4 6 alpha sudo apt get install syncany sy v Installing via a deb package 1 Download the latest release Debian package from the distribution site see folder releases 2 Double click the deb package and click Install or type sudo dpkg i syncany_ lt version gt deb from the command line 3 After the installation open the terminal and type sy If you re more of a command line guy simply do this wget ht
40. Syncany User Guide Release 0 4 6 alpha Philipp Heckel October 23 2015 Contents What is Syncany Va HMowdo Tuset 2 4 6205 24 208 44 ob ea 2h eed d tA bees 1 2 Whoneeds another sync tool socors ce kee HAS RR Ee Oe a t gt WhatSyncany isnot esa peme te eee aE Sees eh eee eee wee ES 14 Example Use Cases cecs tigos ponie e eA a gba EB ee we Re Se Installation 21 Installation requirements ss s e ek a BA A RR ee ee ee oe 22 installing the latest release lt x g4 s6 e B414 6 4 84 amp PRLS Bee ed DEAS Bi 23 Installing the latest snapshot sacci rs es Saad hee AE a 24 Installing tromsource 2 3 4 4 4 4 5 6 2 be LEB CRASS ed Sh ewe ed Getting Started 3 1 Using the graphical user interface GUI o e rros o eg a a e e a 3 2 Using the command line interface CLI os lt ee ca se armere etere Concepts 4 1 Abstraction for dumb storage through a minimal API osoasa aeaaaee 4 2 Minimizing remote disk space through deduplication o ooo 4 3 Privacy by design through client side encryption osoa 4 4 Trace based synchronization through vector clocks o oo 4 5 Differences and similarities to other tools sooo a 46 Further RESOUTCES cece ge p ge he a pe E aan A Se Be a i AG RR Commands ail Mesy command gea aaraa Se Be ee ae ea Be E Be ee 5 2 sy daemon Start and stop the background process daemon 3 Sy nati Imtalizp arepository 2 s255 n ee be ee eS eee NY 5 4
41. TP server using public key authentication the public key of the local machine must be present in the remote server s authorized keys use ssh copy id to copy over your public key If that is the case the password setting is interpreted as the private key s password If public key authentication is used the first time you ll connect to a server you ll be asked to verify the authenticity of the key fingerprint If you have verified the key Syncany will store the key at config syncany plugins userdata sftp known_hosts Linux or SAppData Syncany plugins userdata sftp known_hosts Windows 52 Chapter 6 Plugins Syncany User Guide Release 0 4 6 alpha SSH SFTP Confirmation The authenticity of host example com can t be established RSA key fingerprint is b0 48 b7 9d a5 56 a6 e5 5a 49 94 29 5e 73 e4 95 Are you sure you want to continue connecting Note that if public key authentication is used syncany links will not work because the private key isn t and should not be part of the link itself Syncany will generate a link but it won t work unless the the public key of the other user machine is available at the same path and was also copied to the authorized keys at the SSH SFTP server The plugin is not installed by default but it can be easily installed using the sy plugin install command For details about how to use this command refer to the command reference at sy plugin List install a
42. aDoc of all Gradle modules in the repo All results of the JUnit tests and Cobertura coverage reports are compiled to reports syncany org 102 Chapter 11 Appendix B Videos amp Links
43. all an available plugin Install plugin release version 40 Chapter 5 Commands Syncany User Guide Release 0 4 6 alpha sy plugin install webdav Plugin successfully installed from https www syncany org dist plugins releases webdav Install location home pheckel config syncany plugins syncany plugin webdav 0 1 0 alp Plugin details ID webdav Name WebDAV Version 0 1 0 alpha syncany plug ha jar Install latest snapshot of a plugin sy plugin install sftp snapshot Plugin successfully installed from https www syncany org dist plugins snapshots sftp ynceany plugit Install location home pheckel config syncany plugins syncany plugin sftp 0 1 0 alpha SNAPSHOT 140 Plugin details IDs STCP Name SFTP Version 0 1 0 alpha SNAPSHOT 1404191549 git1l0ae8b7 5 14 3 Example 3 Remove installed plugin sy plugin remove sftp Plugin successfully removed Original local was home pheckel config syncany plugins syncany plugin sftp 0 1 0 alpha SNAPSHOT 14 5 14 sy plugin List install and remove storage backend plugins 41 Syncany User Guide Release 0 4 6 alpha 42 Chapter 5 Commands CHAPTER 6 Plugins Plugins extend the Syncany functionality is different ways As of today there are two types of plugins e Storage plugins also connection transfer plugins give Syncany the ability to use a certain storage
44. anations can be queried with sy lt command gt help An extensive user guide with many examples is also available on the Syncany website https syncany org r userguide GLOBAL OPTIONS l localdir lt path gt Use lt path gt instead of the current directory as local sync folder Syncany searches for a syncany folder in the given and all parent directories d debug Sets the log level to ALL and print the log to the console Alias to loglevel ALL print h help Print this help screen and exit vV Vvv Print application version exit log lt path gt Log output to the file given by lt path gt If is given the output will be logged to STDOUT default loglevel lt level gt Change log level to lt level gt Level can be either of the following OFF SEVERE WARNING INFO FINE FINER FINEST ALL print Print the log to the console in addition to the log file AUTHORS Written by Philipp C Heckel and many others REPORTING BUGS Please report bugs to the GitHub issues page at https www github com syncany syncany issues 84 Chapter 10 Appendix A Manual Pages Syncany User Guide Release 0 4 6 alpha COPYRIGHT Syncany 0 4 4 alpha Distributed under GPLv3 Copyright c 2011 2015 Philipp C Heckel 10 2 sy cleanup NAME sy cleanup remove old versions and free remote disk space
45. at cleanup waits before deleting all versions that are older than this time Until this time deleted files can still be restored Cleanup will fully delete files that were deleted longer ago in the past than this amount of time and they will be gone permanently In addition any versions of files which are not the current version and older than this threshold will also be deleted Default is 30 days 30d Relative time format lt value gt lt unit gt for which lt value gt may be any floating point number and lt unit gt may be any of the following s econds m inutes h ours d ays w eeks mo nths y ears Units may be shortened if they are unique Examples 5h30m or lylmo2d Ey no delete interval Turns off the version removal strategy based on intervals By default the following policy is applied For the last hour the last version from each minute is kept For the last three days the last version from each hour is kept 10 2 sy cleanup 85 Syncany User Guide Release 0 4 6 alpha For the last thirty days the last version from each day is kept With this option only old file versions o will be deleted O no delete older than Turns off the removal of old versions for the command If this is set this command will not delete file versions purely because they are old T no temp removal Turns off the removal of leftover temporary files for the command If this is set this command will leave temp
46. ated Syncany moves really quickly Contents e Appendix B Videos amp Links Videos Links 11 1 Videos The following lists some random videos we made to demonstrate Syncany We are developers not directors or video editors so apologies for the quality and the lack of spoken word The two main video site profiles are the YouTube channel and the asciinema profile Bash completion Mar 15 part of this blog post Recent changes amp history browser Jan 15 36 sec History browser keeps the state browse previous versions Jan 15 36 sec History browser sneak peek Jan 15 34 sec SFTP syncing toggle notifications install plugins recent changes Jan 15 3 min Init connect and sync via graphical user interface Dec 14 3 min Flickr based storage plugin in action Dec 14 2 min Connect to WebDAV with password via GUI Dec 14 1 min GUI Demo Adding existing new folders with the Wizard Dec 14 2 min Windows Linux with GUI tray icons and conflicts Oct 14 3 min Progress bar in command line Sep 14 1 min Syncany Web Interface Preview now outdated Jun 14 2 min Trying out the newest Syncany version with Docker May 14 1 min Automatically sync folders using the S3 plugin May 14 3 min Installing and using plugins in this video SFTP May 14 1 min 101 Syncany User Guide Release 0 4 6 alpha e Installing and using
47. but it will be used in the future to store repository specific information 9 2 5 What file meta data permissions access time etc are preserved in the sync As of today for each file version we store the following metadata Version 1 2 Type FILE FOLDER SYMLINK Status NEW CHANGED Relative path e g Documents Hello txt Symlink target if type is SYMLINK e g Documents Hello orig txt Size in bytes Last modified date time second accuracy only due to file system restrictions Updated date time DOS permissions archive read only system hidden POSIX permissions rwxrwxrwx The following things are not stored e Hardlinks are NOT stored detected not efficient but data is not stored twice e POSIX uid gid are NOT stored e ACLs are NOT stored e Extended attributes are NOT stored see 392 e Access time and create time are NOT stored You can see the metadata yourself by running sy init no encryption no compression and then looking at the database files in the repository in a XML or text editor 80 Chapter 9 FAQ Syncany User Guide Release 0 4 6 alpha 9 2 6 How can edit the connection details or reconnect to a repository If your backend storage credentials change e g FTP user or password it might be necessary to reconnect a local folder to a remote repository As of today you cannot edit a connection via the Syncany commands or GUI As of now you can man
48. ce on Mobile computing and networking pages 98 108 ACM 1998 3 Kalpana Sagar and Deepak Gupta Remote file synchronization single round algorithms International Journal of Computer Applications 4 1 32 36 July 2010 Published By Foundation of Computer Science 4 Bryan O Sullivan Distributed revision control with Mercurial 2009 5 Yasushi Saito and Marc Shapiro Optimistic replication ACM Comput Surv 37 42 81 March 2005 6 Anne Marie Kermarrec Antony Rowstron Marc Shapiro and Peter Druschel The icecube approach to the reconciliation of divergent replicas In Proceedings of the twentieth annual ACM symposium on Principles of distributed computing PODC 01 pages 210 218 New York NY USA 2001 ACM 7 Benjamin C Pierce and J r me Vouillon What s in unison a formal specification and reference implementation of a file synchronizer Technical report 2004 8 The local timestamp is not used to compare which database version happened before another It is only used as a tie breaker to determine the winner between database versions 26 Chapter 4 Concepts Syncany User Guide Release 0 4 6 alpha 4 5 Differences and similarities to other tools The fundamental idea of the Syncany software architecture is a mixture between a version control system like Git SVN or Bazaar a file synchronization software like rsync or Unison and crypto software such as GPG Like in a version control system VCS Syncany keeps track of
49. ct resolve strategy if a local file does not match the expected local file as per the local database The conflict strategy describes the behavior of this command x The rename strategy automatically renames conflicting files to a conflicting file name e g Italy Philipp s conflicted copy txt x The ask strategy lets the user decide whether to keep the local file apply the remote file or create a conflicting file as above The default strategy is rename The ask strategy is currently NOT implemented COPYRIGHT Syncany 0 4 4 alpha Distributed under GPLv3 Copyright c 2011 2015 Philipp C Heckel 10 6 sy genlink NAME sy genlink generate Syncany link for initialized local directory SYNOPSIS sy genlink s short m machine readable DESCRIPTION This command creates a Syncany link syncany from an existing local folder The link can then be sent to someone else to connect to the repository Syncany links contain the connection information of the storage backend so in case of an FTP backend host user pass etc would be contained in a link If the link is shared be aware that you are giving this information to the other users OPTIONS 5 short The generated syncany link will be shortened using the Syncany link shortener service This option stores th nerypted link on the Syncany servers The option does not work if the repository is not encrypted m
50. ction gt lt config gt 6 2 5 Local Plugin The local plugin plugin identifier Local is the only built in storage plugin It provides a way to use a local folder as repository for Syncany That means that instead of connecting to a remote storage and storing the repository files remotely Syncany will use the predefined folder to store them While that sounds quite odd at first why would I want to sync to a local folder it actually makes quite a lot of sense for a few cases e Removable devices If you sync or backup to a removable device you can use the local plugin to address the target folder on that device For instance you d be specifying mnt backupdisk officeorE office as a target folder e Virtual file systems Many storage systems can already be mounted as virtual file systems NFS Samba Google Drive are just a few examples If you used a mounted folder as target you won t even need a special Samba or NFS plugin for Syncany because the local plugin can be used e Testing If you want to try out Syncany or test something the local plugin is a very simple way to do that Plugin Security Syncany assumes that the local machine is secure so if a regular local folder removable device or hard disk is used there are no security remarkds regarding this plugin If however the target repository folder points to a mounted a virtual file system it depends on the underlying protocol if how vulnerable the system is O
51. d as well as regular expression based patterns e Wildcard based exclusions matches any amount of characters including none matches exactly one character e Regular expression based exclusions Lines prefixed regex exclude files matching the given regular ex pression If C Users Steffen Syncany is the managed Syncany folder the following file if located at C Users Steffen Syncany syignore will ignore files folders ending with bak file folders named git as well as files folders matching the regular expression private 20 0 9 2 eg private 2099 or private 2000 bak igit regex private 20 0 9 2 8 2 User specific Configuration The user config defines central settings valid only for the logged in user Unlike the folder specific settings the user configuration settings apply to the entire user There are three general categories of user specific configuration files e General User Configuration userconfig xm1 Define central user specific config options such as proxy settings standby settings or other system properties e Daemon Configuration daemon xm1 Define settings specific to the Syncany background process the daemon such as which folders are managed by the daemon e GUI Configuration gui xm1 Define settings specific to the Syncany graphical user interface only present if GUI is installed The configuration can be found at 3AppData Syncany Windows and at config syncany L
52. d by the Bouncy Castle Cryptography Provider but only Twofish GCM and AES GCM both 128 bit or 256 bit are allowed by Syncany If you are considering a nested cipher setup we recommend first switching to 256 bit AES GCM 60 Chapter 7 Security Syncany User Guide Release 0 4 6 alpha Master key generation from password Key derivation from master key File encryption amp authentication Header authentication 1 User Password 6 min 10 chars 12 Plaintext ENO 5 Header HMAC multichunk or with SHA256 lt metadata 12 bytes Y y PNAC SHAT p 4 Master key 9 HkoF p 10 AES 128 Key gt 12am ee rre iM rounds 64 bytes with SHA256 16 bytes in GCM mode iabyiey Genre A Y Y 8 Rand salt for 14 Authenticated b 1s Hmac 3 Rand master AES key deriv Ciphertext SHA256 key salt 12 bytes i 64 bytes i i Y 16 Header y HMAC SHA256 Stored in repo master file Stored locally in i r O antea and locally in syncany syncany i sn ast SS los kl ght ln A Oak SS oc ok sg as TETT Header Header Authenticated Pees HET HMAC salt HMAC SHA256 ciphertext sye 12 bytes 32 bytes n bytes sy 75338979028 1177390207205 28d4ea07273c05caa064ea874c 0205 1 2706158de8 793a539a97b 71413356091300daa
53. data can be observed or even modified by the adversary because the FTP protocol does not provide communication security Similarly if the WebDAV plugin is used with a HTTP target the same attack scenario is possible However because Syncany encrypts files before upload the data being transmitted is of little to no value to the ad versary Even if the data is changed by the adversary Syncany will detect these changes because only authenticated ciphers are used meaning that data confidentiality and integrity is still ensured When using a plugin without com munication security and the adversary can modify the network data availibility might be compromised An attacker might simply read the storage access credentials and delete the entire repository If however a plugin is used that provides communicatiobn security an attacker cannot modify network transmissions and data availibility is ensured Examples for such plugins include the WebDAV plugin with a HTTPS target the Amazon S3 plugin or the SFTP plugin In summation e Syncany provides data confidentiality integrity and availability against attacks on the network if the used plugin provides communication security e Syncany provides data confidentiality integrity but not availability against attacks on the network if the used plugin does not provide communication security 7 4 2 Provider originated Attacks Syncany stores its data at a central storage By definition the provid
54. der gt lt path gt home pheckel Syncany lt path gt lt enabled gt t rue lt enabled gt lt folder gt lt folders gt lt users gt lt user gt lt username gt admin lt username gt lt password gt IOgotcpZzNPh lt password gt lt user gt lt users gt lt daemon gt Example 2 Complex daemon xm1 lt daemon xmlns http syncany org daemon 1 gt lt webServer gt lt enabled gt t rue lt enabled gt lt bindAddress gt 0 0 0 0 lt bindAddress gt lt bindPort gt 8443 lt bindPort gt lt certificateAutoGenerate gt t rue lt certificateAutoGenerate gt lt certificateCommonName gt p1atop lt certificateCommonName gt lt webServer gt lt folders gt lt folder gt lt path gt tmp repo4 lt path gt lt enabled gt t rue lt enabled gt lt watch gt lt interval gt 120000 lt interval gt lt announcements gt t rue lt announcements gt lt announcementsHost gt notify syncany org lt announcement sHost gt lt announcementsPort gt 8080 lt announcementsPort gt lt settleDelay gt 3000 lt settleDelay gt lt cleanupInterval gt 3600000 lt cleanupiInterval gt lt watcher gt t rue lt watcher gt lt up gt lt status gt lt forceChecksum gt false lt forceChecksum gt lt status gt lt forceUploadEnabled gt false lt forceUploadEnabled gt lt up gt lt down gt lt conflictStrategy gt RENAME lt conflictStrategy gt lt applyChanges gt t rue lt applyChanges gt lt down gt lt clean gt
55. dversary 64 Chapter 7 Security Syncany User Guide Release 0 4 6 alpha If only the storage credentials are retrieved by an adversary only the availability of data is at risk same scenario as above 7 5 Syncany Server Communication Syncany usually only communicates with the backend storage it is being used with However there are very few instances in which it actually calls out to the Syncany server e Listing available plugins To list available remote plugins Syncany queries the Syncany API at api syncany org Listing plugins is manually initiated via the command line sy plugin list or when opening the Preferences dialog in the GUI Downloading Installing plugins When a plugin is installed via sy plugin install lt plugin id gt or via the GUD Syncany retrieves the download location via the Syncany API and then downloads this plugin from the that location at get syncany org Checking for application updates To check for new versions of the application and or plugins Syncany will query the Syncany API This can be initiated via sy update check or via the GUI once a day e Pub sub server To quickly notify other clients that new data has been uploaded Syncany subscribes each user to a small pub sub server at notify syncany org 8080 based on Fanout The data exchanged via this pub sub server only contains the random repository identifer and is only used to trigger the other clients sync process Short links
56. e Plugin identifier of the first storage backend storagel lt opt gt yes none Plugin specific options of first plugin storagel type yes none Plugin identifier of the second storage backend storagel lt opt gt yes none Plugin specific options of second plugin Example config xml This example uses an Amazon S3 plugin and an SFTP plugin as a backend lt config xmlns http syncany org config 1 gt lt connection type raid0 gt lt storagel type s3 gt lt accessKey gt AKIAIHIALEXANDREUIIE lt accessKey gt lt secretKey gt wJalrXUtnFEMI K7MDE G bPxRfiANTHONYXZAEZ lt secretKey gt lt bucket gt syncany demo lt bucket gt lt location gt us west 1 lt location gt lt storagel gt lt storage2 type sftp gt lt hostname gt example com lt hostname gt lt username gt spikeh lt username gt lt privatekey gt none lt privatekey gt lt password encrypted true gt 0e2144feed0d93bc6e8d22da lt password gt lt path gt home spikeh SyncanyRepo lt path gt lt port gt 22 lt port gt lt storage2 gt lt connection gt lt config gt 6 2 8 Samba Plugin The Samba plugin plugin identifier samba uses a single folder on a SMB CIFS share also known as Windows Share as repository Since only a sub folder is used multiple repositories per SMB CIFS server are possible Since Microsoft Windows comes with SMB CIFS support out of the box this plugin is most useful i
57. e a we ewe ee eee ae EGA oS Ee RYE Eke MOAT sy plusin 243 444 6 ow bee hea See He awh eed 4 BOARS ea Ree TOAZ SY TESE a aa oa AS Re vente eats ec sorte aioe a BB ee cent wae a Bae De He EA eee TOTI Sy Status 5 lt 2 lt i coke we BS ce Gadd ee Mae ee BG BR Qed 4 bale Alte a elas we Hee S TOTAS UP ea te to oe eo ee ee we Ge ae ee ee a oe ble a a Oe Ee A 101S Sy updates sos 4 heed be eee ee RS ee ede eh SARE Eee CAS PA we LO VG Sy Wath 24 gk ob Pa aN Ae AREER Se ee See ee oe a eee i 11 Appendix B Videos amp Links I VideO e th Se Ee a a ee ES BE Bark ba ee PRR V2 Tanks e oe oie eS ERAGE eS EME EE ERE See hE HA ESAS Syncany User Guide Release 0 4 6 alpha This is the user guide for Syncany an open source cloud storage and filesharing application with a strong focus on security This document is targeted at end users It describes how to use Syncany and how to configure it For development and build instructions please refer to the GitHub page or the project wiki Contents Contents 1 Syncany User Guide Release 0 4 6 alpha 2 Contents CHAPTER 1 What is Syncany Syncany is an open source Dropbox like file sync and backup tool It synchronizes files and folders between computers either manually or automatically Users can define certain folders on their machine and keep them in sync with friends or colleagues So what makes Syncany awesome e Automatic continuous file synchronization e Use
58. e gt is changed and lt certificateAutoGenerate gt is set to true Syncany will re generate a new keypair certificate User Authentication lt users gt The user authentication capabilities of Syncany to the web server and REST WS API are very limited Syncany provides a simple HTTP Basic based user authentication using users defined in the daemon xm1 config file All users provided in the lt users gt option have full read write access to the Syncany web interface and REST WS API As of today there are no authorization mechanisms at all lt users gt lt user gt lt username gt P im lt username gt lt password gt IOgotcpZzNPh lt password gt lt user gt lt user gt lt username gt Philipp lt username gt lt password gt plaintextpassword lt password gt lt user gt lt users gt In the example users Pim and Philipp have the same access rights Both can access the web interface and execute any REST WS request 72 Chapter 8 Configuration Syncany User Guide Release 0 4 6 alpha Example 1 Simple daemon xm1 lt daemon xmlns http syncany org daemon 1 gt lt webServer gt lt enabled gt t rue lt enabled gt lt bindAddress gt 0 0 0 0 lt bindAddress gt lt bindPort gt 8443 lt bindPort gt lt certificateAutoGenerate gt t rue lt certificateAutoGenerate gt lt certificateCommonName gt p1atop lt certificateCommonName gt lt webServer gt lt folders gt lt fol
59. ed from the menu For De bian Ubuntu a deb package is offered for Windows a separate installer is provided 6 4 Web Interface Plugins Web Interface plugins are a way to provide a web frontend to Syncany folders managed by a Syncany daemon If a web interface plugin is installed a web based frontend will be available via the web browser Like any other plugin web interface plugins can be installed with sy plugin install and are available after restarting the Syncany daemon see details about the plugin installation at sy plugin List install and remove storage backend plugins In the default configuration the web interface is served by the internal web server at port 8443 and can be accessed at https localhost 8443 The web server settings can be changed by modifying the daemon xm1 file as described at Daemon Config daemon xml 6 4 1 Simple Web Interface Plugin Note The Simple Web Interface plugin is a proof of concept implementation It is available as a snapshot but it is currently NOT functional We are still looking for a web frontend developer to take over rewrite the web frontend Please refer to the corresponding GitHub issue 56 Chapter 6 Plugins Syncany User Guide Release 0 4 6 alpha The Simple Web Interface plugin plugin identifier simpleweb provides access to the daemon managed Syncany folders i e all folders configured in the daemon xml see Daemon Config daemon xml The web fronte
60. educed significantly if you ve uploaded an Eclipse workspace via FTP you know what I mean And secondly multichunks are compressed and encrypted before upload and thereby add another storage reduction mechanism and ensure data confidentiality and integrity for encryption details see Security Multichunks only store the raw chunk data Metadata such as as file names and permissions is stored in database files These database files are XML based delta databases each of which represents a commit describing what files were 1 Explanation of data deduplication taken from the thesis Minimizing remote storage usage and synchronization time using deduplication and multichunking Syncany as an example 24 Chapter 4 Concepts Syncany User Guide Release 0 4 6 alpha added removed or renamed They contain information about which chunks are stored in which multichunks and how to reassemble files Albums xX Repo x File Edit View Go Bookmarks Help File Edit View Go Bookmarks Help Name Name a Canada jpg v E databases Germanyjpg database BAfqbwGPpABEdfTdViyZ 000 The Netherlands jpg database BAfgbwGPpABEdfTdVlyZ 000 ts database xUCeGfdDibmyQvgSdzop 00 vi multichunks so multichunk 117f8adbScb1ad0023c6d i multichunk f8b794ccb796fdab97dedc master syncany e 3 items Free space 0O 4 items Free space 3 Local Syncany Folder Offsite Repository Syncany s
61. eeded x What file meta data permissions access time etc are preserved in the sync x How can I edit the connection details or reconnect to a repository ky kH Common errors x Error message Could not generate DH 4096 2048 bits keypair 9 1 General questions 9 1 1 What is Syncany Syncany is open source Dropbox like file sync tool It syncs files between computers either manually or automatically Users can define certain folders of their computers and keep this folder in sync with friends 77 Syncany User Guide Release 0 4 6 alpha So what make Syncany awesome e Use your own storage FTP S3 WebDAV NFS Samba Windows file share e Files are encrypted before upload don t trust anybody but yourself e Files are deduplicated locally massive space savings on remote storage What Syncany is not e A service You don t pay for it You don t get storage from us e Usable for critical files It s alpha software Don t do it e A large network of interconnected machines You can t share folders with strangers You must trust the persons you share files with e A fully fledged version control system 9 1 2 Why make something new How is it different from X If there were something like Syncany we wouldn t be developing it There are a few alternatives but they all have a different focus are not open source have no versioning cannot use arbitrary storage or don t enc
62. elease 0 4 6 alpha Armin s Laptop il Project X Syncany Daemon S3 Bucket BE Poesy L E HTTPS roj REST API MEA Fabrice s Laptop m gt Project Z Armin works on projects X and Y He uses sy connect to connect to the projects repository via the Syncany daemon Fabrice doesn t have Syncany installed but still needs to browse the files on project Z He uses the web interface to do that 6 Chapter 1 What is Syncany CHAPTER 2 Installation Syncany is written in pure Java so it available on a number of platforms To make the installation process as easy as possible we ve pre bundled distributables for most major operating systems a Mac OSX bundle is still missing see open issues Whenever we release a new version of Syncany we regenerate these bundles and publish them on the distribution site On that site you can find releases and snapshots Releases are built and published in a certain release cycle currently 2 weeks and snapshots are built with every commit Feel free to try out both but be aware that snapshots are very volatile and things might break without warning As of today we provide the following download possibilities e Windows installer exe e Ubuntu Debian APT archive or deb package e Arch Linux AUR e Mac OS X app zip package or via Homebrew e Docker Application e Manual Installation via tar gz zip Contents e Installation
63. er of that storage has complete access to the data that resides on that storage If an evil provider takes interest in that data it is very easy to gain access to it If for instance the owner on an FTP server decides to modify or delete your repository it is very easy for them to do so In fact Syncany can never provide any protection against a provider originated attack on data availibility However similarly to the above mentioned no communication security scenario Syncany still provides data confi dentiality and integrity because files are encrypted in an authenticated mode before upload A provider might be able to retrieve the encrypted files or even delete them but it won t be able to decrypt them 7 4 3 Theft of Credentials One of Syncany s assumptions is that users sharing a repository must trust each other completely The reason for that is that to access a repository credentials to the storage are shared If one of the trusted users were to be tricked into giving up the password or her laptop were to be compromised the repository password the master key or the offsite storage credentials might be in the hand of an attacker If the repository password or the master key is retrieved data confidentiality is completely breached without other users having a chance of detecting it and without a chance of changing the password Syncany can not prevent or detect if the master key or password has been stolen or was used by an a
64. erver and the local machine is not encrypted i e in this case the plugin does not provide transport security and WebDAV credentials might be read by an adversary man in the middle attack However since Syncany itself takes care of encrypting the files before they are uploaded the confidentiality of your data is not at risk Be aware that this still means that an attacker might get access to your WebDAV account and simply delete all of your files 54 Chapter 6 Plugins Syncany User Guide Release 0 4 6 alpha Options for config xml Plugin Option Mandatory Default Value Description url yes none Hostname or IP address of the WebDAV server username yes none Username of the WebDAV user basic auth password yes none Password of the WebDAV user basic auth Example config xml lt config xmlns http syncany org config 1 gt lt connection type webdav gt lt url gt https dav example com 8080 dav repol lt url gt lt username gt christof lt username gt lt password encrypted true gt 0e99b946577d26376c64b59a lt password gt lt connection gt lt config gt 6 3 Graphical User Interface Plugin The GUI plugin plugin identifier gui provides a graphical user interface for Syncany Since Syncany mainly runs in the background the user interface is very minimal It consists of a tray icon and a small wizard to initialize or connect Syncany folders It connec
65. escription hostname yes none Hostname or IP address of the Samba server username yes none Username of the Samba user password yes none Password of the samba user share yes none Name of the Samba share path no Sub path of the Samba share Example config xml This example uses the folder Repol on the Repositories share for storing the files The UNC path for this would be 192 168 1 25 Repositories Repol lt config xmlns http syncany org config 1 gt lt connection type samba gt lt hostname gt 192 168 1 25 lt hostname gt lt username gt Philipp lt username gt lt password encrypted true gt 0e99b946577d26376c64b59a lt password gt lt share gt Repositories lt share gt lt path gt Repol lt path gt lt connection gt lt config gt 6 2 9 SFTP Plugin The SFTP plugin plugin identifier sftp uses a single folder on an SSH SFTP server as repository Since only a sub folder is used multiple repositories per SFTP server are possible The plugin supports username password based authentication as well as public key based authentication e Password based authentication To use the password based auth mechanism a valid SSH user must exist Initializing a new repository or connecting to an existing one is pretty straight forward Just enter username and password leave public key related properties empty and you re good e Public key based authentication To authenticate at the SSH SF
66. face GUI or the command line client CLI While the GUI is probably easier to use the CLI offers more functionality and flexibility Contents e Getting Started Using the graphical user interface GUI x Installing a storage plugin x Initializing a new repository Using the command line interface CLI x Installing a storage plugin x Initializing a new repository x Syncing files Manually syncing files x Automatically syncing files x Connecting other clients 3 1 Using the graphical user interface GUI After installing Syncany open it from your apps menu and if all goes well you ll see a tray icon with the Syncany symbol in your notification area 13 Syncany User Guide Release 0 4 6 alpha Work Uploading 2 3 173 KB 4 Add folder Browse history Recent changes Office Syncany Team Work Preferences Report a bug Buy us a coffee Visit website Exit From this tray menu you can select a new local folder to sync change settings or install plugins Every interaction with Syncany starts from this tray icon and its menu Typically if you re new to Syncany the following steps will have to be taken 1 Select and install a storage plugin via the Preferences Plugins dialog 2 Initialize a new repository or connect to an existing repository via the Add folder wizard 3 1 1 Installing a storage plugin Syncany typically only ships with a single p
67. file Sync up finished S sy ls versions somefile 14 08 23 13311311 rw r r a 9 FILE bce8ce9cel 69101d4e4d 1 somefile 14 08 23 13 11 22 rw r r a 9 FILE 48a2c49dd8 69101d4e4d 2 somefile sy restor revision 1 69101d4e4d File restored to tmp ssh a somefile restored version 1 5 11 sy restore Restore older versions of files from the repository 37 Syncany User Guide Release 0 4 6 alpha cat somefile restored version 1 versionl 5 11 2 Example 2 Restoring a deleted file rm somefile sy up D somefile Sync up finished sy restor revision 2 69101d4e4d File restored to somefil restored version 2 cat somefile restored version 2 version2 5 12 sy 1s Display current and historic local database This command lists and filters the file tree based on the local database The file tree selection can be performed using the following selection criteria 1 Using the lt path expression gt one can select a file pattern such as txt or sub tree such as subfolder only with r 2 Using r the command does not only list the folder relative to the lt path expression gt but to all sub trees of it 3 The t option limits the result set to a certain file type f for files d for directories and s for symlinks Types can be combined e g sy ls tfs selects files and symlinks 4 The D option selects the
68. file system events or pub sub notifications are missed the periodic synchronization using the down and up command is implemented as a fallback Note The messages exchanged through the pub sub server do not include any confidential data They only include the repository identifier randomly generated in the init phase and a client identifier randomly generated on every restart OPTIONS Ss delay lt sec gt Waits for lt sec gt seconds for file system watcher to settle before starting to index newly added files If many file system actions are executed e g copying a large folder waiting a few seconds ensures that actions belonging together are uploaded in a single new database version Default value is 3 seconds W no watcher Disables folder watcher entirely Local changes in the synced folder and its subfolders will not be registered right away Instead local changes will only be detected by the periodic synchronization loop Unless other clients have also set this option changes by other clients will still be detected through the pub sub server If W is set th i interval option becomes more relevant as local synchronization entirely relies on the interval a announce lt host gt lt port gt Defines the hostname and the port of the pub sub server The pub sub server is used to notify other clients if the local client uploaded new data and to get notified if other clients did
69. grity Syncany provides data confidentiality and detects attacks on the integrity of data but does not provide mechanisms to protect against attacks on the availability of data 1 Syncany assumes the storage provider and network infrastructure cannot be trusted 59 Syncany User Guide Release 0 4 6 alpha 2 In contrast to that Syncany assumes that the local machine is secure 3 Users sharing a repository trust each other completely 7 3 Security Concepts There are two security cryptography related topics in Syncany 1 Pre upload encryption Before uploading anything to the offsite storage Syncany encrypts files locally This is done for metadata database files and for the actual data multichunk files The ciphers and modes as well as the resulting crypto file format is described in Encryption Scheme 2 Auth only and HTTPS only daemon server Accessing the daemon API or web interface is HTTPS only using either self signed or your own certificates and allows only authenticated users Details are described in HTTPS only Daemon 7 3 1 Encryption Scheme For users to share a repository with Syncany they must trust each other completely They share the pre shared repos itory password to access the Syncany repository and use the same credentials to the offsite storage Repository Initialization When a user creates a new repository using sy init Syncany asks the user for the repository password This password is used
70. gt lt tokenSecret gt bdld lt tokenSecret gt lt user gt lt id gt 585546793 N00 lt id gt lt username gt Philipp Roth lt username gt lt admin gt false lt admin gt lt pro gt false lt pro gt lt iconFarm gt 0 lt iconFarm gt lt iconServer gt 0 lt iconServer gt lt realName gt Christian Otte lt realName gt lt photosCount gt 0 lt photosCount gt lt bandwidthMax gt 0 lt bandwidthMax gt lt bandwidthUsed gt 0 lt bandwidthUsed gt 6 2 Storage Plugins 47 KETS Syncany User Guide Release 0 4 6 alpha lt filesizeMax gt 0 lt filesizeMax gt lt revContact gt false lt revContact gt lt revFriend gt false lt revFriend gt lt revFamily gt false lt revFamily gt lt user gt lt permission gt delete lt permission gt lt auth gt lt connection gt lt config gt 6 2 4 FTP Plugin The FTP plugin plugin identifier ftp uses a single folder on an FTP server as repository Since only a sub folder is used multiple repositories per FTP server are possible The plugin is not installed by default but it can be easily installed using the sy plugin install command For details about how to use this command refer to the command reference at sy plugin List install and remove storage backend plugins Plugin Security As of today the FTP plugin does not support FTPS the TLS extension for FTP That means that the FTP plugin does not provide transport security and FTP crede
71. gt yes none Human readable user name lt masterKey gt no none Master key used to derive enc keys lt cacheKeepBytes gt no 524288000 Size of cache in bytes lt connection gt yes none Key value based storage settings The lt machineName gt property represents a local machine identifier to technically identify a computer user It is purely random and carries no logic This identifier is created during folder initialization and should not be changed The lt displayName gt is the human readable user name of the user It is currently only used locally to create conflict files As of today this property is not transferred to other users but it might be in the future The property can safely be changed The lt masterKey gt represents the master key generated from the repository password It is required if the repository is encrypted and it must not be changed To learn more about the master key check out the Security chapter The lt cacheKeepBytes gt property depicts the maximum size of the local syncany cache folder in bytes default is 500 MB After each synchronization the cache is cleaned if it exceeds the keep size The cache mechanism is least recently used LRU The lt connection gt property represents the storage plugin configuration The type lt plugin id gt attribute defines the plugin used to synchronize this folder The definition of these properties is described in the corresponding plugin sub chapter
72. hat directory This folder marks the folder as a Syncany folder and contains configuration and metadata information Typically you don t have to touch any of these files but it doesn t hurt to know a little bit about them e syncany config xml main config Stores client information and the connection details e syncany syncany repo file Stores common repository information repo ID chunking options 67 Syncany User Guide Release 0 4 6 alpha e syncany master master salt file Stores the salt of the master key if repo encrypted e syncany cache local cache Cache folder to store temporarily files e syncany db local database Files stored by the local embedded HSQLDB database e syncany logs log files Stores log files created by Syncany Log files rotate automatically e syignore exclude ignore files Lists all the files to be ignored by Syncany Since only the config xmland syignore files should be changed manually the following chapters only describe these files 8 1 1 Common Settings and Storage Connection config xm1 The config xml file is initially created by the sy init or sy connect command Changing its settings is typically not necessary unless your connection storage details such as credentials or hostname change Options for config xml Option Mand Def Description lt machineName gt yes none Random local machine identifier lt displayName
73. he local machine without being encrypted Data confidentiality and user privacy are the taken very seriously 128 bit AES Twofish GCM encryption is built in by default Combining these two features makes Syncany really awesome You can use all of the free storage that the Internet gives you Google Drive OneDrive etc to share your files but you don t have to worry about your privacy Syncany takes care of encryption before files are uploaded 1 1 How do I use it As of today Syncany is available as a command line tool on all major operating systems We re working on a graphical user interface and a web frontend but until that is ready you Il have to make due with the command line You can either manually trigger the file synchronization like you do with Git rsync or Unison or you can let the Syncany background process automatically sync your files Check out the Getting Started Guide to learn more 1 2 Who needs another sync tool There are many sync tools like Syncany out there so depending on your use case Syncany might just not be for you The target audience for Syncany is not the average cloud user you know the one who puts his life in a provider s hand but rather the more privacy aware user and or users companies that want to use their own storage e If you want to share files with friends or colleagues without having to trust anyone but yourself Syncany is the right tool for you e If you want to use your own
74. he server is contacted no user query is necessary Unknown SSL TLS certificate Owner CN syncany org OU Domain Control Validated Issuer CN GlobalSign Domain Validation CA SHA256 G2 O GlobalSign nv sa C BE Serial number 1492271418628120790652059091142976109636803 Valid from Mon Apr 14 23 01 38 CEST 2014 until Wed Apr 15 23 01 38 CEST 2015 Certificate fingerprints MD5 60 FB F7 F1 E1 9E D6 74 06 41 03 01 16 D6 19 D3 SHAT DC A8 5F FA 1D 9D 92 A7 1C 8E 22 C6 43 9B 96 9E 62 13 C7 25 SHA256 84 DF 92 99 86 15 AF A6 8D EC 74 5C 13 BE 18 75 BC 08 34 Do you want to trust this certificate y n In normal situations you should not have to alter the trust store file at all You may however need to do that if you want to your own keypair and certificate See Using your own Keypair and Certificate for details 8 3 3 Using your own Keypair and Certificate Note Using your own keypair and certificate is not as easy as it should be Please let us know if you have a better way to do that 8 3 Keys and Certificates 75 Syncany User Guide Release 0 4 6 alpha To use your own keypair and certificate for Syncany s internal web server follow the following steps You can use a tool like keytool to do that 1 Set the lt certificateAutoGenerate gt option in the daemon xml to false see Web Server Configu ration lt webServer gt 2 Replace the only pri
75. icas since the last point of synchronization e In state based synchronizers 7 such as Unison or rsync this is done by comparing the file lists of all clients The result of the update detection process is a global file list created by merging the individual lists into one e In trace based synchronizers such as Syncany update detection is based on the trace log of the clients i e changes of files rather than a final file list Instead of a global file list they generate a global file history based on the individual client histories Trace based synchronizers typically compare histories and detect new file versions Update detection must additionally detect conflicting updates and determine the winner of a conflict Once the global file list history has been created the synchronizer must apply changes to the local workstation This is done in the reconciliation phase The reconciliation phase typically downloads new files deletes old files and moves renamed files Due to its versioning requirements Syncany detects updates via trace logs file histories of the individual clients Histories of the participating clients are analyzed and compared to each other based on file identifiers file versions checksums and local timestamps 4 4 2 Database versions and vector clocks Whenever a client uploads new changes by triggering sy up Syncany compares the local file tree to the local database Changes in files are packed into multichunks and
76. id having to type the credentials by using a syncany link Both methods work equally well but the syncany link method is more convenient A syncany link contains the exact same information that was initially queried by the interactive sy init command namely the plugin specific settings such as hostname user password and so on Clients that already have access to the repository can create such a link by calling sy genlink form within the Syncany folder Once a new client is connected to a repository all of the above mentioned commands can be used to either manually or automatically sync the repository 20 Chapter 3 Getting Started Syncany User Guide Release 0 4 6 alpha bin bash mkdir SharedFiles cd SharedFiles box sy connect syncany storage 1 UdMumSMFgeK6L2cLOs59kI TuaAEEUynIEtrpniZ0624HiR9fM7dTv6ctsH FsxjMWIXR9QDiALKYDONLpk98hg U3KCBQGGIRZTO 00iF5y LPEgCARFNoO85EGkQL3Qm0fKNZDsgt L3pHOGnmc4dLLsCLs7pjLfgoHLDIg63Xf nA50rIvFO03 pH20SzC397E87wPwT 9qq2ZgfNj aqZdgzPHveoCO9mUSXMYdFALYVpt2 CVcupdGwgletMaM5qSJo2Skg QAUW6QctmxbxkYvVKFUK8tycul1YU42CnOmAW S1AZ0 I vOeOW60F LygxVsbaZCzfOKDuDiRRaixGGTjQ 106200JDGrYDzFOVGuJdGO3WabYsSWanJ K7e0j JhlwO1SA i01VAroF loW8wp9TdcDoJIrZ21 hMX1Z yhF2YEoBjWdxrT j xwY j NmNgqbLZxhOVIuv2EkvaRebakCOYG10d0Msvqd10Vnte546QwRLtKN6Tk6yCP BKVLBSUcZPZVhHDf fumXcXfvUDLCjLXpGmz9LzVOsys j 9rpIwZly1Q9zapIZwafQO30W3sSDXFDSwIxsA L8 YwzLo SRLrHuCq3QO0Ku9BH6zexKuRTZcaRj Jh ij 5FaTRdbhmCXuj rIVGcjJ4X1F7V
77. indexes and uploads these changes e And the sy down command downloads and applies remote changes locally bin bash a pheckel platop sy status Germany jpg The Netherlands jpg pheckel pLatop sy up A Germany jpg A The Netherlands jpg Sync up finished pheckel pLlatop sy down A Canada jpg Sync down finished pheckel pLlatop Unlike a version control system however there is no need to first add then commit and then push changes the sy up command combines these actions into one Similarly the sy down command fetches and applies changes in a single command There are a couple of other very useful commands to manage a repository and a local Syncany folder Use sy 1s to show the current and past file tree sy restore to restore old or deleted files and sy cleanup to save space on the offsite storage by removing old file versions A full list of commands can be found in Commands 3 2 5 Automatically syncing files Note As of today the automatic synchronizaton setup is not as easy as it should be We are aware of that and are working on a solution To set up a Dropbox like folder synchronizaton for a Syncany folder the folder has to be managed by the Syncany background process also called the daemon This background process can be started with sy daemon start Once the daemon is started all registered folders are monitored for changes and remote changes are automatically applied to the local folder All of these actio
78. ing sy plugin list and then sy plugin install As of today we ve implemented a number of plugins To get a complete list check out the plugins page For this example we ll install the Amazon S3 plugin r a pheckel platop sy plugin install s53 Plugin successfully installed from https www syncany org dist plugins releases s3 syncany plugin s3 0 1 8 alpha jar Install location home pheckel config syncany plugins lib syncany plugin s3 0 1 8 alpha jar bin bash Plugin details ID s3 Name Amazon 53 Version 0 1 8 alpha pheckel pLatop 3 2 2 Initializing a new repository Once the plugin is installed in this example the Amazon S3 plugin you can set up a new repository To do that simply navigate to the folder you want to synchronize and type sy init command An interactive command line interface will ask you a couple of questions about how to connect to the offsite storage hostname credentials etc and it will ask you for a password to encrypt your data with 3 2 Using the command line interface CLI 17 Syncany User Guide Release 0 4 6 alpha bin bash pheckel pLatop sy init Choose a storage plugin Available plugins are local s3 Plugin s3 Connection details for Amazon 3 connection Access Key AKIAIEPK3MGLNGVJK3EA Secret Key not displayed Bucket Name syncanytestl Location optional default is us west 1 The password is used to encrypt data on the remote
79. inux 8 2 1 User Config userconfig xm1 The userconfig xml config file is a defines global user config settings valid only for this user but regardless of whether or not Syncany is run in daemon mode or manually The options are pretty limited are right now More config options will probably be added in future releases 8 2 User specific Configuration 69 Syncany User Guide Release 0 4 6 alpha Options for userconfig xml Option Mand Def Description lt configEncryptionKey gt yes none Encryption key to encrypt repo access credentials lt systemProperties gt no none Set any Java system properties e g proxy lt preventStandby gt no false Prevent standby shutdown during sync lt maxMemory gt no 512M Limit memory usage of Syncany The lt configEncryptionKkey gt option is used to encrypt sensitive values in the config xml file e g the S FTP WebDAV password the Amazon S3 Dropbox access token and so on The lt systemProperties gt option allows you to set Java system properties via the Syncany configuration Any of the lt property gt options will be passed to Java s System setProperty method This can be used to set proxy settings log settings and so on If the lt preventStandby gt option is set to true Syncany will make sure that your system doesn t go into standby hibernate if the synchronization process is run This option will not prevent your sy
80. irst disable the lt certificateAutoGenerate gt option and then import your keypair into keystore jks and your certificate into trust store jks Also see Keys and Certificates Option Mand Def Description lt enabled gt yes true Defines whether the web server will start lt bindAddress gt yes 127 0 0 1 Address to which the server socket is bound lt bindPort gt yes 8443 Port to which the server socket is bound lt certificateAutoGenerate gt yes true Regenerate certificate if common name changed lt certificateCommonName gt yes hostname Common name in the server certificate The lt enabled gt option can switch off the web server entirely if the option is set to false However if the webserver is disabled neither REST WS API nor the web interface are available The lt bindAddress gt and lt bindPort gt options define to which IP address i e network interface and port the server will be bound For security reasons the web interface and API is only bound to a local address 127 0 0 1 and is therefore not reachable externally To make the API and web interface publicly available set the bind address to 0 0 0 0 ora specific IP address The lt certificateAutoGenerate gt option automatically re generates a keypair and a self signed certificate for the Syncany web server using the common name CN defined in lt certificateCommonName gt Whenever lt certificateCommonNam
81. istories to one version Syncany stores multiple file versions for each file default is 5 If sy cleanup is run without any options it will still keep the last 5 versions unless keep versions N is given where N is the number of versions to keep If you ve run cleanup in the last 6 hours you ll need to also apply force echo versionl gt file sy up A file Sync up finished echo version2 gt file sy up M file Sync up finished sy cleanup keep versions 1 force 3 database files merged 1 multichunk s deleted on remote storage freed 0 00 MB 1 file histories shortened Cleanup successful 5 11 sy restore Restore older versions of files from the repository This command restores old or deleted files from the remote storage As long as a file is known to the local database and the corresponding chunks are available on the remote storage it can be restored using this command The command downloads the required chunks and assembles the file If no target revision is given with r the last version is restored To select a revision to restore the sy s command can be used For a detailed command reference including all command specific options please refer to the manual page of this command at sy restore 5 11 1 Example 1 Restoring a previous version of a file echo versionl gt somefile S sy up A somefile Sync up finished echo version2 gt somefile sy up M some
82. ith caution COPYRIGHT Syncany 0 4 4 alpha Distributed under GPLv3 Copyright c 2011 2015 Philipp C Heckel 10 11 sy plugin NAME sy plugin list install update and remove Syncany plugins SYNOPSIS sy plugin list R remote only L local only s snapshots a api endpoint lt url gt lt plugin id gt sy plugin install s snapshot m minimal output lt URL gt lt file gt lt plugin id gt sy plugin update lt plugin id gt sy plugin remove lt plugin id gt DESCRIPTION This command performs four different actions It lists the locally installed and remotely available plugins including version information and whether plugins can be upgraded The list action connects to the Syncany host to retrieve remote plugin information By default only plugin releases will be listed in the result If instead daily snapshots are desired th s option can be used It installs new plugins from either a given URL or a local file URL and local file must point to a valid Syncany plugin JAR file to be installable If lt plugin id gt is given the command will connect to the Syncany host and download the desired plugin from there If instead of the release version the daily snapshot shall be installed the s option can be used Plugins installed by the install action will be copied to the Syncany user directory On Unix based systems this directory is located at
83. its the similarities among different files to save disk space It identifies duplicate sequences of bytes chunks and only stores one copy of that sequence If a chunk appears again in the same file or in another file of the same file set deduplication only stores a reference to this chunk instead of its actual contents Depending on the amount and type of input data this technique can significantly reduce the total amount of required storage 1 1 File chunking 2 Create multichunks 3 Upload multichunks Index all files and divide Compare chunks to database Encrypt and upload multichunks them into unique chunks and combine new chunks to to offsite storage using a storage multichunks plugin 4 3 bef ot be o O J ius bs Multi FTP S3 we ae f Chunk 33 33 13 13 I 44 18 18 While this concept is typically used in backend systems of backup solutions Syncany performs deduplication on the client During the index process in sy up Syncany breaks files into chunks and compares these chunks to the database If chunks are new they are marked for upload If they are already known because they appeared in other files before they are simply referenced New chunks are packaged together into so called multichunks basically ZIP files containing chunks and uploaded to the offsite storage These multichunks exist for two reasons Firstly by combining chunks to multichunks the per file upload latency and roundtrip time is r
84. knows three versions of this file the second file will be restored Assuming that the original filename was folder file txt the target filename will be folder fil restored txt If folder does not exist it will be created sy restor revision 1 target restored file txt 3168ab663e Restores version 1 of the file with the identifier 3168ab663e to the target file restored file txt If this file exists an error will be thrown COPYRIGHT Syncany 0 4 4 alpha Distributed under GPLv3 Copyright c 2011 2015 Philipp C Heckel 96 Chapter 10 Appendix A Manual Pages Syncany User Guide Release 0 4 6 alpha 10 13 sy status NAME sy status list new and changed files in SYNOPSIS checksum D local Syncany folder sy status f fore DESCRIPTION This command compares the local file tree no delete on the disk with the local database and detects local changes Thes console Local changes ar of a file content has not changed checksum is additionally compared detected using the last This command is used by the up OPTIONS Ee force checksum and last modified date only files are modified in plac do not change decreas very often D no delet With this option this command will not If they match the local database no checksum comparison Enforces this command to compare files by checksum This option is particu
85. l show a list of locally installed plugins and remotely available plugins queried through the plugin repository API sy plugin list Id Name Local Version Remote Version Inst Upgr ftp FTP 0 1 0 alpha yes local Local 0 1 2 alpha yes s3 Amazon S3 0 1 0 alpha yes sftp SFTP 0 1 0 alpha yes webdav WebDAV 0 1 0 alpha yes Installing the plugin with sy plugin install sftp retrieves the download link again via the plugin repository API downloads the plugin JAR file verifies its checksum and installs it locally After that the plugin can be used S sy plugin install sftp Plugin successfully installed from https www syncany org dist plugins releases webdav syncany plug Install location home pheckel config syncany plugins lib syncany plugin sftp 0 1 0 alpha jar Plugins are generally installed to the user specific Syncany plugin folder On Windows this folder can be found at SAppData Syncany plugins 1lib on Linux it can be found at config syncany plugins lib Installing a plugin is nothing more than placing the plugin JAR file in that folder If you have trouble removing a plugin simply delete it manually from that folder 6 1 2 Plugin Configuration Some plugins store per user config or persist some user specific state The SFTP plugin for instance stores the known hosts file for all the known and trusted hosts Other plugins might want to store other information On Windows
86. l web server as well as by plugins that use SSL TLS e g the WebDAV plugin Both files are located at the user specific configuration directory at config syncany Linux or SAppData Syncany Windows The files are stored in the Java Key Store JKS format No password is used to protect the key trust store To analyze this file and its entries you may use the keytool util 8 3 1 Private Keys keystore jks As of today Syncany only stores one entry in the keystore jks file namely the private key part of the RSA key pair used to serve the HTTPS API and web interface As described in Web Server Configuration lt webServer gt the keypair is generated by Syncany whenever the common name in the daemon config is changed unless the lt certificateAutoGenerate gt option is disabled In normal situations you should not have to alter the key store file at all To use your own keypair and certificate please see Using your own Keypair and Certificate 8 3 2 Trusted Certificates truststore jks Syncany s user specific trust store holds trusted X 509 certificates of remote servers This trust store is mainly used by plugins that communicate via SSL TLS such as the WebDAV plugin Whenever sy connect or sy init is called with a plugin that uses TLS SSL Syncany will ask the user to confirm certificates that are not in the trust store After the user confirms these certificates are added to the trust store so that the next time t
87. larly useful if last modified date and size For large local folders the performance of this command and increase I O significantly changes are printed to the modified date and the file size the command assumes that the If f is enabled th command to detect local changes not by file size this option can tremendously list locally deleted files If command these changes will not be uploaded used with the up COPYRIGHT Syncany 0 4 4 alpha Distributed under GPLv3 Copyright c 2011 2015 Philipp C Heckel 10 14 sy up NAME sy up uploads changes in local Syncany folder to remot repository SYNOPSIS sy up R no resume lt status options gt DESCRIPTION This command detects changes in the local uploads changes to the remote repository command determines what has changed packages thes and uploads them to the remote storage alongside with a delta multichunks metadata database To determine the local changes the of the status the If there are no local changes rup status command can also be used in this command folder If there are local changes changes in new indexes new files and the command is used All options command will not upload anything 10 13 sy status 97 Syncany User Guide Release 0 4 6 alpha no multichunks and no metadata If this command is interrupted during the upload phase it will try to re
88. les will be synced via the storage specified by this plugin 86 Chapter 10 Appendix A Manual Pages Syncany User Guide Release 0 4 6 alpha O plugin option lt key value gt multiple options possible Sets a plugin specific setting in the form of a key value pair Each plugin defines different mandatory and optional settings At least all mandatory settings must be specified by this option All mandatory and optional settings can be listed using the plugin command n add daemon The initialized local folder is automatically added to the daemon configuration for automatic synchronization if this option is used password lt password gt DO NOT USE THIS OPTION Set the password to decrypt the repository This option shouldn t be used because the password might be visible to other users or be stored in history files COPYRIGHT Syncany 0 4 4 alpha Distributed under GPLv3 Copyright c 2011 2015 Philipp C Heckel 10 4 sy daemon NAME sy daemon starts and stops the background process daemon SYNOPSIS sy daemon start stop reload restart status force stop sy daemon list sy daemon add lt folder path gt sy daemon remove lt folder path gt lt folder index gt DESCRIPTION This command manages the Syncany background process aka the daemon It can start and stop the daemon display the status and reload the daemon configuration With the action
89. like that Setup Syncany Select Destination Location Where should Syncany be installed L Setup wil install Syncany into the falloning folder To continue click Next If you would like to select a different folder click Browse Browse At least 13 3 MB of free disk space is required Other than where to install Syncany the installer will only give you two additional options If you are not sure what they mean don t change them 8 Chapter 2 Installation Syncany User Guide Release 0 4 6 alpha e Add Syncany to PATH environment variable recommended If you re unsure leave this as is For Syncany to be available on the command line the command line will have to know where to look for it If this option is unchecked the sy command will not be available unless the installation path is added to the PATH environment variable e Set JAVA_HOME environment variable recommended If you re unsure leave this as is Syncany relies on the Java Runtime Environment JRE and this variable tells the Syncany commands where to look for it After the installation is complete open the command prompt by typing cmd in the Windows search box or by navi gating to Extras Command Prompt in the menu If everything goes well you ll see this after typing sy v Fa i C Windows system32 cmd exe o Microsoft Windows Version 6 1 7601 Copyright lt c gt 2669 Microsoft Corporation All rights reserved C Users
90. lly you don t need to touch any of them To see a full example including all available options see Example 2 Complex daemon xml The lt path gt option defines the local path to the Syncany folder and the lt enabled gt tag lets you enable disable folders The lt wat ch gt tag defines the behavior for the internal watch operation That includes the behavior of the index upload operation lt up gt the download apply operation lt down gt as well as the periodic cleanup mechanism lt clean gt It also includes general settings about the local file system watcher lt wat cher gt and the central Syncany pub sub server lt announcements gt 8 2 User specific Configuration 71 Syncany User Guide Release 0 4 6 alpha Web Server Configuration lt webServer gt The internal web server is used to serve the Syncany REST and WebSocket API as well as the web interface if a web interface plugin is installed Both API and web interface are HTTPS only meaning that no HTTP traffic is accepted and that all communication is encrypted The certificate used by the internal web server is automatically generated by default Syncany creates a self signed X 509 certificates based on a generated RSA 2048 bit keypair The common name CN in the certificate is the local hostname by default but can be changed if needed Note As of today providing your own certificates and keypair is possible but not as easy as it should be F
91. local files are not encrypted Syncany stores file data in multichunks but are encrypted before upload and metadata in database files On the offsite storage Syncany stores multichunks and database files side by side in a repository Besides the multichunks and database files this repository consists of the master file containing the salt for the master key and the repo file containing repository specific details Since the repository is encrypted it can only be read by another Syncany client in possession of the repository password 4 3 Privacy by design through client side encryption Unlike other sync solutions Syncany s encryption features are built in by design and enabled by default In fact not encrypting the repository doesn t bring any user advantages and only minor performance benefits Please refer to the Security chapter for details 4 4 Trace based synchronization through vector clocks The synchronization algorithm is one of Syncany s core elements Its responsibility is to detect file changes among participating workstations and to bring them to the same state 4 3 Privacy by design through client side encryption 25 Syncany User Guide Release 0 4 6 alpha 4 4 1 Trace based synchronization This particularly includes what is known by most file synchronizers as update detection and reconciliation 5 Update detection is the process of discovering where updates have been made to the separate repl
92. lugin If you d like to use other backend storages such as S FTP Amazon S3 WebDAV and such you ll have to install the corresponding plugin s To do that open the Preferences dialog from the tray menu and install the plugin on the plugins panel 14 Chapter 3 Getting Started Syncany User Guide Release 0 4 6 alpha Preferences x General Plugins yt Network es About gp Local 0 4 7 alpha Global RAIDO 0 4 0 alpha as Amazon S3 0 4 6 alpha 0 4 6 alpha 0 4 1 alpha 0 4 5 alpha 0 4 5 alpha 0 4 0 alpha Install ao Install from file To use the plugin a restart of Syncany is necessary 3 1 2 Initializing a new repository Unlike Dropbox and others Syncany does not and can t set up a default sync folder To synchronize a set of files you need to manually add a folder to Syncany and link it to a storage backend To do that click on the Add a folder menu item in the tray This will bring up a wizard dialog to guide you trough the process 3 1 Using the graphical user interface GUI 15 Syncany User Guide Release 0 4 6 alpha Syncany Add new Syncany folder Use this wizard to create a new Syncany repository or connect to an existing one Create a new online storage Choose this option if you want to create a new repository on a remote server using an FTP Samba S3 etc Connect to an existing online storage Choose this option if you wa
93. ly determine the backend to use for displaying the tray icon On Linux the Syncany GUI will use the Python based app indicator script if the Unity desktop is detected On Mac OSX it will try to use the OSX notification center The lt theme gt option controls the icon theme used by the tray icon Possible values are DEFAULT and MONOCHROME If DEFAULT is given the default color theme will be used for all operating systems except for Mac OSX On Mac OSX the MONOCHROME theme is used The lt startup gt option indicates whether the Syncany GUI is started when the user logs in i e on startup Note that changing this value in an editor does not change the autostart settings of your system Only set this value via the graphical user interface The lt notifications gt option defines if sync notifications are displayed to the user Notifications are typically displayed if new files have been added or changed Example GUI config gui xm1 lt gui gt lt tray gt DEFAULT lt tray gt lt theme gt DEFAULT lt theme gt lt startup gt t rue lt startup gt 74 Chapter 8 Configuration Syncany User Guide Release 0 4 6 alpha lt notifications gt true lt notifications gt lt gui gt 8 3 Keys and Certificates Syncany maintains its own user specific key store for private keys and trust store foreign X 509 certificates Both key store and trust store are used by the interna
94. mages that can be viewed in any image view Because Syncany repacks and encrypts data the images appear random to the human eye Note While Flickr offers 1 TB of storage for images the plugin has not extensively tested for large folders Please be aware of this The plugin authenticates against the Dropbox REST API via a OAuth 2 0 During sy init you will be asked to navigate to the Flickr website and copy an access token from there The plugin is not installed by default but it can be easily installed using the sy plugin install command For details about how to use this command refer to the command reference at sy plugin List install and remove storage backend plugins Plugin Security Flickr REST API traffic is based on HTTPS so tranport security is ensured Since Syncany itself takes care of encrypting the files before they are uploaded the confidentiality of your data is not at risk Flickr or any third party cannot read your files even if they access the encrypted files in your Flickr albums Options for config xml Plugin Option Mandatory Default Value Description auth yes none Structured authentication information album yes none Flickr album identifier Example config xml lt config gt lt connection class org syncany plugins flickr FlickrTransferSettings type f1li lt album gt 82554672157651456 lt album gt lt auth gt lt token gt 36576 lt token
95. n Windows environments Nevertheless it works equally well with the Linux implementation Samba The plugin is not installed by default but it can be easily installed using the sy plugin install command For details about how to use this command refer to the command reference at sy plugin List install and remove storage backend plugins Plugin Security The Samba plugin uses the jCIFS library for SMB CIFS Since this library only supports NT LM 0 12 which is SMBv1 the plugin currently does not encrypt the communication to the SMB CIFS server That means that the plugin does not provide transport security and credentials might be read by an adversary man in the middle attack Since Syncany itself takes care of encrypting the files before they are uploaded the 6 2 Storage Plugins 51 Syncany User Guide Release 0 4 6 alpha confidentiality of your data is not at risk However be aware that this still means that an attacker might get access to your SMB CIFS account and simply delete all of your files If the Samba plugin is used users sharing a repository typically access that repository using the same user name password combination Be aware that sharing a syncany link and the repository password with other users also means giving away these storage credentials Only share a repository with people you trust with these credentials Options for config xml Plugin Option Mandatory Default Value D
96. n option lt key value gt multiple options possible Sets a plugin specific setting in the form of a key value pair Each plugin defines different mandatory and optional settings At least all mandatory settings must be specified by this option All mandatory and optional settings can be listed using the plugin command E no encryption DO NOT USE THIS OPTION Turns off the encryption for the newly created remote repository All files are stored in plaintext No password is needed for either syncany link multichunk or metadata E G no compression Turns off Gzip compression for the newly created remote repository All files are stored in uncompressed form Can increase indexing performance but will also increase transfer times and remote storage space ty create target If not existent creates the target path on the remote storage If this option is not given the command will fail if the target folder path does not exist a advanced Runs the interactive setup in an advanced mode querying the user for more detailed encryption options In particular it is possible to select th available symmetric ciphers and modes of operation to encrypt the repository with n add daemon 90 Chapter 10 Appendix A Manual Pages Syncany User Guide Release 0 4 6 alpha The initialized local folder is automatically added to the daemon configuration for automatic s
97. nd currently implements the following functionalities e Display the file tree at different times current and past e Display file history of a file old versions e Restore old versions of a file e Download a file current or past version To install the plugin use sy plugin install simpleweb snapshot Make sure to enable the snapshot flag because there is no official release of the plugin yet As of today the web interface looks like this E lt Syncany All files in sync v aaa Menu 4 d aaa Name Version Size Last Modified Posix Perms DOS Attrs lj logs j logs 1 4 0kB 2014 08 24 21 08 01 0 CEST rwxr xr x 4 plugins U Plugins 1 4 0kB 2014 08 22 16 05 29 0 CEST rwXxr xr x a i lib __ Canada jpg 1 63 3kB 2014 08 24 02 05 48 0 CEST rws r a 4 j userdata daemon ctri 1 OB 2014 08 24 21 08 03 0 CEST rw r 1 a UJ sfp daemon pid 1 5B 2014 08 24 21 08 02 0 CEST rw r 1 a C daemon xmi 1 17KB 2014 08 24 21 07 54 0 CEST w a daemon xmi 1 14KB 2014 08 12 23 12 44 0 CEST rw t t C Germanyjpg 1 63 3kB 2014 08 24 01 30 21 0 CEST rw r 1 k keystore jks 1 21kB 2014 08 12 23 47 27 0 CEST rw r t a The Netherlands jpg 1 296kB 2014 08 24 01 18 10 0 CEST rw r r a _ truststore jks 1 3 6kB 2014 08 12 23 47 27 0 CEST rw r t a _ userconfig xml 1 414B 2014 08 18 20 55 53 0 CEST rw r 1 a 6 4 Web Interface Plugins 57 Syncany User Guide Release 0 4 6 alpha
98. nd remove storage backend plugins Plugin Security The plugin uses the JSch Java Secure Channel library All communication is SSH SFTP baed so access credentials are protected in transit Since the actual data is encrypted before upload data confidentiality is not an issue either If the SFTP plugin is used users sharing a repository typically access that repository using the same SFTP user name password combination unless public key authentication is used Be aware that sharing a syncany link and the repository password with other users also means giving away these storage credentials Only share a repository with people you trust with these credentials Options for config xml Plugin Option Mandatory Default Value Description hostname yes none Hostname or IP address of the SFTP server username yes none Username of the SFTP user privatekey yes none Private key path if public key auth is used password yes none Password of the SFTP user or priv key password path yes none Path at which to store the repository port no 22 Port of the FTP server Please note If privatekey is set to none the password is interepreted as the username s password If privatekey is set the password is interpreted as the password of the private key If the private key is not password protected leave the password empty Example config xml With username password lt config xmlns htt
99. nd will hopefully be resolved soon The plugin is not installed by default but it can be easily installed using the sy plugin install command For details about how to use this command refer to the command reference at sy plugin List install and remove storage backend plugins Plugin Security Depending on the URL configured during setup communication is either HTTP or HTTPS If HTTP is used traffic between the remote server and the local machine is not encrypted i e in this case the plugin does not provide transport security and credentials might be read by an adversary man in the middle attack However since Syncany itself takes care of encrypting the files before they are uploaded the confidentiality of your data is not at risk Be aware that this still means that an attacker might get access to your account and simply delete all of your files Options for config xml Plugin Option Mandatory Default Value Description authUrl yes none Swift API Authentication URL hittp or hitps username yes none Swift username password yes none Swift password Example config xml lt config xmlns http syncany org config 1 gt lt connection type swift gt lt authUrl gt https cloud swiftstack com auth vl 0 lt authUr1l gt lt username gt sw1f7Us3r lt username gt lt password encrypted true gt 0e2144feed0d93bc6e8d22da lt password gt lt connection gt lt config gt
100. ns happen in the background without the need for any intervention By default calling sy init orsy connect will not add the added local folder to the Syncany daemon configura tion That means that unless you call the command with the add daemon option the folder will not be daemon managed If you want to use the daemon use sy connect add daemonorsy init add daemon To register a folder manually or remove a folder from daemon management the daemon can be configured using the 3 2 Using the command line interface CLI 19 Syncany User Guide Release 0 4 6 alpha daemon config file at sAppData Syncany daemon xmlor config syncany daemon xml Assum ing that you d like home pim Syncany to be monitored and automatically synchronized simply add the folder to the daemon xm1 config file like this lt daemon xmlns http syncany org daemon 1 gt lt folders gt lt folder gt lt path gt home pim Syncany lt path gt lt enabled gt t rue lt enabled gt lt folder gt lt folders gt lt daemon gt To let the daemon know about the new folder run sy daemon restart or sy daemon reload only on Linux 3 2 6 Connecting other clients A Syncany repository can be shared among many clients There are two methods for new clients to connect to an existing repository e Use sy connect to manually enter the backend storage credentials just like with sy init e Use sy connect lt syncany 1link gt to avo
101. nt to sync your files with an existing remote storage Add an existing Syncany folder Choose this option if you want to add an existing Syncany folder to this application I htm conc Among others you will need to e select a local folder that contains your files e select a storage backend e g WebDAV Amazon S3 SFTP choose a password Once you ve gone through all of these steps you can start adding your files to the local sync folder and connect other devices to your repository Syncany will run as a daemon in the background and watch for changed added deleted files To see a screencast of the whole process check out this YouTube video Also check out our other Videos 3 2 Using the command line interface CLI The command line interface is much more powerful than the Syncany GUI To use it you can go ahead and open a terminal window and try out the sy command everything you can do with Syncany so far is implemented in this command The typical order of commands is 1 Install a storage plugin using sy plugin install lt plugin id gt 2 Initialize a new repository with sy init 3 Then sync files manually with sy up and sy down or automatically with the daemon 4 Connect and share with other clients with sy connect 16 Chapter 3 Getting Started Syncany User Guide Release 0 4 6 alpha 3 2 1 Installing a storage plugin First choose the storage backend you d like to use by do
102. nternal web server parameters lt users gt yes none Log in users for web server and API The lt folders gt option can contain multiple lt folder gt definitions each of which represent a Syncany folder managed by the daemon To add a new Syncany folder simply initialize or connect to a repository using sy init or sy connect and add the folder here Then restart the daemon Find details to this option below in Managed Folder Configuration lt folders gt The lt webServer gt option controls the internal Syncany web server bind port and address certificates The web server is used for the web interface as well as for the Syncany API Find details to this option below in Web Server Configuration lt webServer gt The lt users gt option defines the users that can access the web interface and the API Each lt user gt has full read write access to the API and all managed folders Find details to this option below in User Authentication lt users gt Managed Folder Configuration lt folders gt Note We re currently still in an alpha version of Syncany and the options inside the lt folder gt tag change more often than we desire Please forgive us for not documenting all of the options You might want to check out the daemon example xml file to see all available options The lt folders gt tag can contain multiple lt folder gt tags each of which has a vast amount of configuration options Typica
103. ntials might be read by an adversary man in the middle attack Since Syncany itself takes care of encrypting the files before they are uploaded the confidentiality of your data is not at risk However be aware that this still means that an attacker might get access to your FTP account and simply delete all of your files If the FTP plugin is used users sharing a repository typically access that repository using the same FTP user name password combination Be aware that sharing a syncany link and the repository password with other users also means giving away these storage credentials Only share a repository with people you trust with these credentials Options for config xml Plugin Option Mandatory Default Value Description hostname yes none Hostname or IP address of the FTP server username yes none Username of the FTP user password yes none Password of the FTP user path yes none Path at which to store the repository port no 21 Port of the FTP server Example config xml lt config xmlns http syncany org config 1 gt lt connection type ftp gt lt hostname gt ftp example com lt hostname gt lt username gt armin lt username gt lt password encrypted true gt 0e2144feed0d93bc6e8d22da lt password gt lt path gt syncany repo2 lt path gt lt port gt 21 lt port gt 48 Chapier 6 Plugins Syncany User Guide Release 0 4 6 alpha lt conne
104. ntral tool to perform all of the Syncany actions e g creating a new managed folder syncing files or restoring them It also offers a way start and stop the Syncany user daemon which will then run Syncany in the background Contents e Commands The sy command Sy Sy Sy Sy Sy Sy Sy Sy Sy Sy Sy Sy Sy daemon Start and stop the background process daemon init Initializing a repository connect Connecting to an existing repository status Detect and display local changes up Detect local changes and upload to repository ls remote Detect and display remote changes down Retrieve and apply remote changes watch Automatically synchronizes the local folder with the repo cleanup Remove old versions from the local database and the repo restore Restore older versions of files from the repository 1s Display current and historic local database genlink Create a syncany link from an existing local folder plugin List install and remove storage backend plugins 5 1 The sy command The sy command is not just a single command but instead offers a variety of actions to manage a Syncany folder In general it can be called by sy lt action gt args where action is any of the available sub commands and args is a list of optional global or sub command specific options Available sub commands e init Initialize the current folder as a Syncany folder e connect Connect the current folder to an existing Syncany
105. ny Links After the actual initialization the sy init command creates a so called syncany link which can be used by other users to connect to a repository This link contains the plugin credentials needed to access the repository e g FTP host user pass The link is encrypted using the same crypto format as described above except that the master salt is included and the link is base58 encoded Syncany supports two types of links 1 Encrypted links normal Links prefixed syncany storage 1 are encrypted and can be safely shared via unsecure channels 2 Plaintext links not recommended Links prefixed syncany storage 1 not encrypted are not encrypted and should never be shared via unsecure channels Encrypted links are structured like this syncany storage 1 lt master salt gt lt encrypted config gt Both lt master salt gt and lt encrypted config gt are base58 encoded The master salt is stored in plaintext and unauthenticated The encrypted config is stored in the same file format as described above i e using a nested cipher combination of AES and Twofish When a client attempts to connect to a repository using sy connect syncany storage 1 Syncany decrypts uses the master salt and the prompted password to derive a master key and then uses the master key and the IVs and salts in the encrypted config to derive the actual cipher keys These keys can then be used to decrypt the storage connection config Plainte
106. on Privacy by design through client side encryption Trace based synchronization through vector clocks Trace based synchronization Database versions and vector clocks Conflict handling Differences and similarities to other tools Further Resources 4 1 Abstraction for dumb storage through a minimal API Syncany offers quite a bit of options when it comes to where you can store your files As the Plugins page describes the backend storage options are very flexible The reason why that is possible is the storage abstraction concept of Syncany For a storage backend to be usable with Syncany a plugin for that backend only has to implement a very small set of operations This minimal storage interface comprises just five methods to manage files on the offsite storage which makes it incredibly easy to implement for developers e upload local file remote file Upload a local file to the offsite storage e download remote file local file Download a file from the offsite storage e move remote file 1 remote file 2 Move file on the offsite storage e list file type List remote files on the offsite storage e delete remote file Delete file from the offsite storage 23 Syncany User Guide Release 0 4 6 alpha Implementing these methods plus a few helper functions is enough to create a plugin Plugins don t have to deal with connection handling synchronization aspects security
107. options please refer to the manual page of this command at sy down 5 8 1 Example 1 Download and apply remote changes no conflicts S sy down A testfile txt A testfile2 txt Sync down finished ls testfile txt testfile2 txt 5 8 2 Example 2 Download and apply remote changes with conflicts echo conflicting content gt testfile txt testfile txt S sy down A testfile txt A testfile2 txt Sync down finished ls testfile2 txt testfile pheckel s conflicted copy 27 Apr 14 6 46 PM txt testfile txt 5 9 sy watch Automatically synchronizes the local folder with the repo Automatically synchronizes the local folder with the repository The command performs the up and down command in an interval watches the file system for changes and subscribes to the Syncany pub sub server In the default configuration no options the command subscribes to the Syncany pub sub server and registers local file system watches in the locally synced folder and all of its subfolders When local events are registered the command waits a few seconds waiting for settlement and then triggers the up command After the upload has finished a message is published to the pub sub server telling other clients of this repo that there is new data Clients subscribed to the repository s channel will receive this notification and immediately perform a down command This mechanism allows instant synchronization
108. orary files on the offsite storage untouched f force Forces a the cleanup even if the time between cleanups 3 hours has not passed Use this option only if a cleanup is absolutely necessary and you know what you are doing COPYRIGHT Syncany 0 4 4 alpha Distributed under GPLv3 Copyright c 2011 2015 Philipp C Heckel 10 3 sy connect NAME sy connect connect to an existing Syncany repository SYNOPSIS sy connect lt syncany link gt n add daemon password sy connect P plugin lt plugin gt o plugin option lt key value gt n add daemon password DESCRIPTION This command connects to an existing remote repository and initializes the local directory The command can be called as follows 1 Using a syncany link generated by either init or genlink the command connects to the repository given in the link If the link is encrypted the link repo password must be entered 2 If no link is given the command acts like init i e it queries the user for storage plugin and connection details of the repository to connect to Once the repository is connected the initialized local folder can be synced with the newly created repository The commands up down watch etc can be used Other clients can then be connected using the connect command OPTIONS P plugin lt plugin gt Selects a plugin to use for the repository Local fi
109. osen with P or interactively different plugin specific options are required or optional Once the init command was successfully executed the initialized local folder can be synced with the newly created repository The commands up down watch etc can be used Other clients can then be connected using the connect command For a detailed command reference including all command specific options please refer to the manual page of this command at sy init 5 3 1 Example 1 Create new repository interactive mode sy init create target Choose a storage plugin Available plugins are local s3 webdav Plugin local Connection details for Local connection Local Folder tmp x The password is used to encrypt data on the remote storage Choose wisely Password min 10 chars Confirm WARNING The password is a bit short Less than 12 chars are not future proof Are you sure you want to use it y n y Generating master key from password this might take a while Repository created and local folder initialized To share the same repository with others you can share this link syncany storage 1 y8aqJUCsxXqPtH9KuaocAKAKOOvccIUH32k tPRCineNLLc This link is encrypted with the given password so you can safely share it using unsecure communication chat e mail etc WARNING The link contains the details of your repo connection which typically consist of usernames password of
110. ous versions Or it might not work at all If you re sure you want to install a snapshot the process is very similar to the steps above The only thing that differs is the file or access channel where you get the installer package e For Windows download the latest snapshot installer from the snapshots folder e For Debian Ubuntu use the Syncany APT archive e For Arch Linux use the Git package on AUR e For Docker use the Docker snapshot repository syncany snapshot e And for other operating systems download the latest snapshot tar gz zip from the snapshots folder 2 4 Installing from source Syncany hosts its code on GitHub so if you want to compile Syncany from source you definitely can and we encourage you to do so Since this is a user manual however we don t want to go into too much detail about the build process There are details about building available on the Wiki page In short run this git clone http github com syncany syncany cd syncany gradlew installApp on Linux Mac OS gradlew installApp on Windows This compiles and installs the Syncany command line client to syncany cli build install syncany bin syncany You can run it from there 2 3 Installing the latest snapshot 11 Syncany User Guide Release 0 4 6 alpha 12 Chapter 2 Installation CHAPTER 3 Getting Started Once you ve installed Syncany you can either use the Syncany graphical user inter
111. ownload requests and the corresponding network latency by grouping these changes into bigger blocks However while rsync does that by actively gathering the file stats on the remote system Syncany only uses the downloaded metadata i e using dumb storage is possible Unlike any of the above mentioned tools Syncany is built with and around cryptography and takes confidentiality and data integrity very seriously Syncany generally assumes that everything but your local machine can be moni tored eavesdropped by others which is why it encrypts all data locally before uploading As of now Syncany only supports password based symmetric key encryption based on configurable ciphers By default it uses 128 bit AES and Twofish both in the authenticated GCM mode but basically can support anything that Java and the Bouncy Castle crypto provider have to offer 4 6 Further Resources e Master Thesis Minimizing remote storage usage and synchronization time using deduplication and multichunk ing Syncany as an example e Syncany explained idea progress development and future part 1 e Deep into the code of Syncany command line client application flow and data model part 2 4 5 Differences and similarities to other tools 27 Syncany User Guide Release 0 4 6 alpha 28 Chapter 4 Concepts CHAPTER 5 Commands The Syncany command line client CLI comes with one single main command sy or syncany This command is the ce
112. p syncany org config 1 gt lt connection type sftp gt lt hostname gt example com lt hostname gt lt username gt spikeh lt username gt lt privatekey gt none lt privatekey gt lt User password gt lt password encrypted true gt 0e2144feed0d93bc6e8d22da lt password gt lt path gt home spikeh SyncanyRepo lt path gt lt port gt 22 lt port gt lt connection gt lt config gt 6 2 Storage Plugins 53 Syncany User Guide Release 0 4 6 alpha With private key authentication lt config xmlns http syncany org config 1 gt lt connection type sftp gt lt hostname gt ftp example com lt hostname gt lt username gt armin lt username gt lt privatekey gt home localuser ssh id_rsa lt privatekey gt lt Private key password gt lt password encrypted true gt 0e2144feed0d93bc6e8d22da lt password gt lt path gt home spikeh SyncanyRepo lt path gt lt port gt 22 lt port gt lt connection gt lt config gt 6 2 10 WebDAV Plugin The WebDAV plugin plugin identifier webdav uses a single folder on a WebDAV server as repository Since only a sub folder is used multiple repositories per WebDAV server are possible The plugin supports HTTP and HTTPS connections and authenticates users via username password The HTTP and HTTPS setup are identical in terms of parameters only the URL setting differs slightly http and https However if HTTPS is u
113. plugins in this video WebDAV May 14 1 min e Automatically sync folders using the FTP plugin Apr 14 1 min e Connect to an existing repository and sync a few files using the FTP plugin Apr 14 1 min e Create a new repository and link the local folder using the FTP plugin Apr 14 1 min e Setup Amazon S3 for two users and sync two clients with Syncany Oct 13 9 min e Conflict handling on Linux using local plugin Oct 13 2 min e Checkout code compile and run two clients on Linux using FTP plugin Oct 13 14 min 11 2 Links There is quite a bit of reading material on Syncany already Check out the following links Posts and papers e Syncany The road to beta Delft Students On Software Architecture May 2015 e Blog post Syncany explained idea progress development and future part 1 Oct 2013 e Blog post Deep into the code of Syncany CLI application flow and data model part 2 Feb 2014 e Master s thesis Minimizing remote storage usage and synchronization time using deduplication and multi chunking Syncany as an example 2011 Diagrams e Diagram Syncany application flow example e Diagram Chunking framework class diagram e Diagram Storage plugins class diagram e Diagram Database class diagram e Diagram Cryptography concept Generated JavaDoc and JUnit Reports The up to date JavaDoc of the master branch is always compiled to docs syncany org javadoc It includes the Jav
114. ptions for config xml Plugin Option Mandatory Default Value Description path yes none Local folder used to store repository files to Example config xml lt config xmlns http syncany org config 1 gt lt connection type local gt lt path gt tmp tx c lt path gt lt connection gt lt config gt 6 2 6 OpenStack Swift Plugin The Swift plugin plugin identifier swift uses an OpenStack Swift container as a storage backend Data is stored within objects in the object container of a Swift Object Store The plugin authenticates against the publicly available Swift API via a authentication URL using a username and a password Swift uses HTTP or HTTPS as a method of transferring files to and from the remote server and authenticate users via username password The HTTP and HTTPS setup are identical in terms of parameters only the authentication URL setting differs slightly http and https However if HTTPS is used only server certificates signed by CAs included in the JRE JDK will be accepted e g certificates by VeriSign GlobalSign etc 6 2 Storage Plugins 49 Syncany User Guide Release 0 4 6 alpha Note At this time this plugin will not work with HTTPS based backends if the certificate is self signed or the signed by any CA not shipped with the JRE JDK In particular you will be not asked to confirm the plugin interactively manually This is a known issue a
115. r the index of the folder as printed by the list action Changes will be applied after a restart of the daemon COPYRIGHT Syncany 0 4 4 alpha Distributed under GPLv3 Copyright c 2011 2015 Philipp C Heckel 10 5 sy down NAME sy down fetch remote changes from Syncany repository and apply locally SYNOPSIS sy down C conflict strategy lt rename ask gt A no apply DESCRIPTION This command detects changes made by other clients and applies them locally If there are remote changes the command downloads the relevant metadata evaluates which multichunks are required and then downloads them It then determines what files need to be created moved changed or deleted and performs these actions if possible In some cases file conflicts may occur if the local file differs from the expected file If that happens this command can either automatically rename conflicting files and append a filename suffix or it can ask the user what to do To determine the remote changes the ls remote command is used OPTIONS A no apply 88 Chapter 10 Appendix A Manual Pages Syncany User Guide Release 0 4 6 alpha All local file system actions are skipped i e the local folder will not be changed Only the new unknown database versions will be downloaded and persisted to the database C conflict strategy lt rename ask gt Chooses the confli
116. rd this might take a while Repository connected and local folder initialized You can now use the syncany command to sync your files 5 4 2 Example 2 Connect to an existing repository interactive sy connect Choose a storage plugin Available plugins are local s3 webdav Plugin local Connection details for Local connection Local Folder tmp x Password user enters password Creating master key from password this might take a while Repository connected and local folder initialized You can now use the syncany command to sync your files 5 4 3 Example 3 Connect to an existing repository with prefilled settings sy connect plugin webdav plugin option url http dav example com repol plugin option username pheckel plugin option password lt somepass gt Password user enters password Creating master key from password this might take a while 32 Chapter 5 Commands Syncany User Guide Release 0 4 6 alpha Repository connected and local folder initialized You can now use the syncany command to sync your files 5 5 sy status Detect and display local changes This command compares the local file tree on the disk with the local database and detects local changes These changes are printed to the console Local changes are detected using the last modified date and the file size of a file If they match the local database the command as
117. repository e status List new and changed files in local Syncany folder e up Detect local changes and upload to repository e 1s remote List changes on remote repository 29 Syncany User Guide Release 0 4 6 alpha down Detect remote changes and apply locally watch Automatically synchronizes the local folder with the repo cleanup Remove old versions from the local database and the repo restore Restore the given file paths from the remote repository genlink Create a syncany link from an existing local folder ls Lists and filters the current and past file tree plugin List install and remove storage backend plugins daemon Start and stop the background process daemon Because the init and connect command initialize the current folder to a Syncany folder they cannot be exe cuted inside an already initialized Syncany folder Most of the other commands behave exactly opposite to that The commands status up ls remote down watch cleanup restore genlink and 1s can only be ex ecute inside an initialized Syncany folder The only command that doesn t care where it s executed is the plugin command For a detailed command reference including all command specific options please refer to the manual page of this command at sy syncany 5 2 sy daemon Start and stop the background process daemon This command manages the Syncany background process starts stops etc also called the Syncany daemon
118. rs work on Linux but we are also testing it on Windows We ve done short cross platform tests and it seems to work okay We ll test that even more in the future 9 1 8 What shouldn t use Syncany for Here are some things Syncany isn t good at and things you shouldn t use Syncany for e Critical files At the moment do not use Syncany for critical files The software is still alpha software e Low CPU environments Syncany uses up quite a bit of CPU power when it is indexing files So in general any situation where the client has little computation power 9 1 9 Can I use Syncany to sync two unencrypted folders Can I mirror two folders No this is not possible Syncany stores data in a packed repository form While this might seem as a disadvantage at first it makes great things possible e Save space Using the deduplication mechanisms in Syncany you can save a lot of remote storage space gt 90 space savings e File versioning Syncany versions files so if you overwrite or lose a file you can restore it very easily e Huge files Syncany splits files into chunks so it doesn t matter how large the original file is To sync two folders directly you can use rsync or Unison 9 1 10 Where does the name Syncany come from The working title was Clonebox for a while Then Philipp tried to find a name via Amazon Mechanical Turk That didn t work Philipp s girlfriend finally came up with the nice name Syncany It combines
119. rypt locally As far as we know no tool has this exact feature set Let s pick the normal contenders Dropbox Wuala SparkleShare BitTorrent Sync ownCloud rsync SyncThing And here s how to filter them out Dropbox Wuala and BitTorrent Sync are not open source ownCloud rsync and Ubuntu One don t support encryption SyncThing is bound to the peer2peer concept 9 1 3 Is there an App Web App GUI for it Yes You can install it using sy plugin install gui There is an outdated and currently broken proof of concept web interface 9 1 4 When will it be available When is the release date We don t promise anything because this is a spare time project We are slowly moving towards beta and after that it will be a lot of testing before we can remove the beta label 9 1 5 How can I help If you re a developer try out the code review it and contact us on GitHub if you want to help Check out the issues to find out more Check out the wiki page for more ways to help 9 1 6 Does it support multiple users You can connect to a single repo from many different machines but there is no role concept or access control for users All users have the same rights meaning there is no administrator and once a user is connected he or she can easily delete all repo data if they want to 78 Chapter 9 FAQ Syncany User Guide Release 0 4 6 alpha 9 1 7 Does it work on Windows Linux Mac The majority of Syncany develope
120. s list add and remove the command furthermore manages the Syncany folders controlled by the daemon Controlled folders are synced automatically when the daemon is running This daemon can be started with sy daemon start Once it is running all registered folders are monitored for changes and remote changes are automatically applied to the local folder s All of these actions happen in the background without the need for any intervention The daemon is configured using the daemon xml file at config syncany daemon xml OPTIONS start Starts the background process if it is not already running stop 10 4 sy daemon 87 Syncany User Guide Release 0 4 6 alpha Stops the background process if it is running reload Reloads the daemon configuration without restarting the proces restart Stops then starts the daemon again status Displays the status and the process ID PID of the daemon force stop Forces the process to stop Do not use this unless absolutely necessary list Lists the folders managed by the daemon add lt folder path gt Adds the given folder to the daemon configuration The added folder will be managed by the daemon after the config has been reloaded or the daemon is restarted remove lt folder path gt lt folder index gt Removes the given folder from the daemon configuration The argument can either be the full path of the folder o
121. s command refer to the command reference at sy plugin List install and remove storage backend plugins Plugin Security The plugin uses the JetS3t Amazon S3 library to access the S3 buckets All communication is HTTPS only so access credentials are protected in transit Since the actual data is encrypted before upload data confidentiality is not an issue either If the Amazon S3 plugin is used users sharing a repository typically access that repository i e the S3 bucket using the same AWS access key and secret key Be aware that sharing a syncany link and the repository password with other users also means giving away these storage credentials Only share a repository with people you trust with these credentials Options for config xml Plugin Option Mandatory Default Value Description accessKey yes none Amazon AWS access key secretKey yes none Amazon AWS secret key bucket yes none Name of the bucket to use as repository location no us west 1 Location of the bucket details see below The location of the bucket is any valid Amazon AWS location As of today valid Amazon region values are e Europe EU eu west 1 e United States us west 1 us west 2 empty string for the US Classic Region e Asia Pacific ap southeast 1 ap southeast 2 ap northeast 1 e Africa sa east 1 6 2 Storage Plugins 45 Syncany User Guide Release 0 4 6 alpha Example config xml
122. sed the first time you connect to the server during sy init or sy connect Syncany will ask you to confirm the server certificate This will happen for all certificates even if they are signed by one of the large CAs Unknown SSL TLS certificate Owner CN syncany org OU Domain Control Validated Issuer CN GlobalSign Domain Validation CA SHA256 G2 O GlobalSign nv sa C BE Serial number 1492271418628120790652059091142976109636803 Valid from Mon Apr 14 23 01 38 CEST 2014 until Wed Apr 15 23 01 38 CEST 2015 Certificate fingerprints MD5 60 FB F7 F1 E1 9E D6 74 06 41 03 01 16 D6 19 D3 SHAT DC A8 5F FA 1D 9D 92 A7 1C 8E 22 C6 43 9B 96 9E 62 13 C7 25 SHA256 84 DF 92 99 86 15 AF A6 8D EC 74 5C 13 BE 18 75 BC 08 34 Do you want to trust this certificate y n Once you ve accepted this certificate it is added to the user specific trust store at config syncany truststore jks Linux or tAppData Syncany truststore jks Win dows The plugin is not installed by default but it can be easily installed using the sy plugin install command For details about how to use this command refer to the command reference at sy plugin List install and remove storage backend plugins Plugin Security The WebDAV plugin uses the Sardine WebDAV library Depending on the URL configured during setup communi cation is either HTTP or HTTPS If HTTP is used traffic between the remote s
123. st store Using your own keypair and certificate is also possible See Keys and Certificates for details Authentication and Authorization The user authentication and authorization capabilities of Syncany to the web server and REST WS API are very lim ited Syncany provides a simple HTTP Basic based user authentication but only over HTTPS All authenticated users have complete access to the REST WS API The user configuration is done via the daemon xm1 file See User Authentication lt users gt 7 3 Security Concepts 63 Syncany User Guide Release 0 4 6 alpha 7 4 Attack Scenarios Syncany tries to prevent against a certain threat scenarios This chapter briefly shows how an adversary might try to attack Syncany In general we differentiate between attacks on the data in transit and attacks on the storage provider s side Since the local machine is assumed to be secure and data we re trying to protect is not encrypted on the local machine attacks on the local machine are disregarded 7 4 1 Attacking Data Transmissions An adversary with access to the network infrastructure e g through network monitoring or a man in the middle attack can either passively monitor the network traffic or actively modify the data being transmitted Since Syncany can be used with a many different plugins the overall security of the solution strongly depends on the storage plugin If for instance the FTP plugin is used the transmitted
124. stem from going to sleep if no upload download process is taking place Since this option might also prevent the screensaver or screen lock it is not enabled by default The lt maxMemory gt option can be used to limit Syncany s memory usage to the given value Example values are 1G 1 GB or 500M 500 MB By default Syncany will limit the max memory value to 512M 512 MB The value will be passed on to the Java Virtual Machine Xmx Useful System Properties This is a non exhaustive list of useful system properties that can be used in the above mentioned lt systemProperties gt option To add an option simply add a property tag lt property name property name gt property value lt property gt System Property Description http proxyHost Sets HTTP proxy hostname http proxyPort Sets HTTP proxy port http proxyUser Sets HTTP proxy username if proxy needs authentication http proxyPass Sets HTTP proxy password if proxy needs authentication https proxyHost Sets HTTPS proxy hostname https proxyPort Sets HTTPS proxy port https proxyUser Sets HTTPS proxy username if proxy needs authentication https proxyPass Sets HTTPS proxy password if proxy needs authentication socksProxyHost Sets SOCKS proxy hostname socksProxyPort Sets SOCKS proxy port java net socks username Sets SOCKS proxy username if proxy needs authentication java net socks password Sets SOCKS proxy pas
125. sume the upload unless R is given An interrupted upload can only be resumed if the last up failed and no down or cleanup has been done since then OPTIONS R no resum With this option up will not attempt to resume a locally stored transaction Without this option an interrupted upload will be resumed All arguments of the status command can be used COPYRIGHT Syncany 0 4 4 alpha Distributed under GPLv3 Copyright c 2011 2015 Philipp C Heckel 10 15 sy update NAME sy updat Manages application updates SYNOPSIS sy update check s snapshots a api endpoint lt url gt DESCRIPTION This command manages updates of the application It currently only performs update checks but will likely b xtended to automatically update the application The following actions exist The check action checks if a new application version is available It queries the Syncany API and outputs whether the local copy of the application is up to date If it is not it outputs the newest version and a download URL ACIONS check lt args gt Checks with the Syncany API api syncany org for a new version s snapshots Instead of checking against application release versions default the command will also include daily snapshots a api endpoint lt url gt Selects the API endpoint to query against If not given the default endpoint URL will be used https api syncany org v3
126. sumes that the content has not changed no checksum comparison If f is enabled the checksum is additionally compared This command is used by the up command to detect local changes For a detailed command reference including all command specific options please refer to the manual page of this command at sy status 5 5 1 Example 1 Basic usage display new changed file content cho new file content gt newfile txt echo changed content gt testfile txt sy status newfile txt testfile txt Z VNV 5 5 2 Example 2 Force checksum calculation per in status Forcing checksum calculation means that we don t want to rely on last modified date and size If size and last modified date are equal changes in local files will not be detected unless force checksum is set Create a fille one thousand txt containing 1000 and setting the last modified date to a specific date echo 1000 gt one thousand txt touch date Sun 27 Apr 2014 11 11 11 0200 one thousand txt sy up one thousand txt Sync up finished Pn mn Now we change the one thousand txt file but change the timestamp back to the same date as before S echo 9999 gt one thousand txt touch date Sun 27 Apr 2014 11 11 11 0200 one thousand txt As you can see below the regular sy status command does not detect the changes The command with the force checksum detects the changes sy status No local
127. sword if proxy needs authentication org syncany test tmpdir Developer property Uses the given folder for Syncany unit tests Example userconfig xml This example shows how to set the HTTP and HTTPS proxy for all HTTP HTTPS traffic by Syncany In particular this includes traffic to the Syncany Plugin API and communication by the WebDAV plugin The example furthermore shows how to enable the standby hibernate prevention lt userConfig xmlns http syncany org userconfig 1 gt lt prevent Standby gt t rue lt preventStandby gt lt systemProperties gt 70 Chapter 8 Configuration Syncany User Guide Release 0 4 6 alpha lt property name http proxyHost gt your proxy host tld lt property gt lt property name http proxyPort gt 8080 lt property gt lt property name https proxyHost gt your proxy host tld lt property gt lt property name https proxyPort gt 8080 lt property gt lt systemProperties gt lt configEncryptionkKey salt 87d32e99230581e2 key 652f6725d2ce7d90822 gt lt userConfig gt 8 2 2 Daemon Config daemon xm1 The main purpose of the daemon configuration is to tell the Syncany daemon started by sy daemon start what folders should be monitored and automatically synced whenever something changes Options for daemon xml Option Mand Def Description lt folders gt yes none Folders managed by the daemon lt webServer gt yes none I
128. sx2A Password Repository connected and local folder initialized You can now use the syncany command to sync your files sy down Sync down finished pim pimbox While the syncany link itself is encrypted and may be shared via unsecure channels sharing the link and the repository password gives users read write access to your repository and typically enables them to access the entire backend storage If for instance the repository is based on an FTP folder the syncany link contains the FTP username and password Users with access to the Syncany repository can also access the FTP storage with a regular FTP client and delete change files as they wish 3 2 Using the command line interface CLI 21 Syncany User Guide Release 0 4 6 alpha 22 Chapter 3 Getting Started CHAPTER 4 Concepts There are a few very smart concepts built into Syncany that make it so flexible and differentiate it from other solutions It s not necessary for users to understand these concepts but it helps to understand why Syncany does things the way it does them Note This chapter explains the basic concepts of Syncany to the interested user in a hopefully understandable way If you just want to use Syncany and don t care how it ticks inside you can safely skip this chapter Contents e Concepts Abstraction for dumb storage through a minimal API Minimizing remote disk space through deduplicati
129. system but are still kept in the database These files can be restored using the sy restore command By default deleted files are not displayed 9 group Only works with versions Displays the file versions grouped by file history f full checksums Displays full long checksums instead of shortened checksums lt path expression gt Selects a file pattern or sub tree of the database using substring and wildcard mechanisms The expression is applied to the relative slash separated path The only possible wildcard is equivalent If lt path expression gt does not contain a wildcard it is interpreted as prefix and extended to lt path expression gt If a wildcard is present no wildcard is appended Note The Linux shell expands the wildcard if a matching file is present Either use single quotes e g txt or use instead 10 10 sy Is 93 Syncany User Guide Release 0 4 6 alpha EXAMPLES sy ls r subfolder Selects all file entries of the current file tr in the folder subfolder including for instance subfolder some other file txt sy ls recursive types fs date 1h30m txt Selects all files and symlinks in th ntire file tree that end with txt and existed one and 30 minutes hour ago sy ls versions group recursiv Selects and displays all file versions and their file histories This selects th ntire database Use w
130. t C Users Fabrice Syncany you ll find the connection settings at C Users Fabrice Syncany syncany config xml Depending on the type of storage plugin the con tents of this file might be different See below for examples of the config xml file For other users to connect to a repository you can either provide them with these storage credentials e g FTP username password AWS credentials etc or you can share a syncany link see sy genlink Create a syncany link from an existing local folder Warning Users sharing a repository typically access that repository using the same storage credentials Be aware that sharing a syncany link and the repository password with other users also means giving away these storage credentials Only share a repository with people you trust with these credentials 6 2 1 Amazon S3 Plugin The Amazon S3 plugin plugin identifier s3 uses an Amazon S3 bucket to store the Syncany repository Amazon S3 Simple Storage Service is an online file storage web service offered by Amazon Web Services It s a pretty neat pay as you go service and works very well with Syncany If you ve never tried it you can get a free account with 5 GB of storage As of today the plugin only supports one repository per bucket It cannot use sub paths of a bucket as repository The plugin is not installed by default but it can be easily installed using the sy plugin install command For details about how to use thi
131. te repository If there are local changes the command determines what has changed packages these changes in new multichunks and uploads them to the remote storage alongside with a delta metadata database To determine the local changes the status command is used All options of the status command can also be used in this command If there are no local changes the up command will not upload anything no multichunks and no metadata For a detailed command reference including all command specific options please refer to the manual page of this command at sy s remote 5 7 1 Example Using the Is remote command sy ls remote db 2kjuahomsfgjmpft 0000000002 34 Chapter 5 Commands Syncany User Guide Release 0 4 6 alpha 5 8 sy down Retrieve and apply remote changes This command detects changes made by other clients and applies them locally If there are remote changes the command downloads the relevant metadata evaluates which multichunks are required and then downloads them It then determines what files need to be created moved changed or deleted and performs these actions if possible In some cases file conflicts may occur if the local file differs from the expected file If that happens this command can either automatically rename conflicting files and append a filename suffix or it can ask the user what to do For a detailed command reference including all command specific
132. the connection e g FTP user pass 5 3 2 Example 2 Create new repository with prefilled settings S sy init plugin s3 o accessKey AKIAJL7 o secretKey o bucket syncanytest3 o location EU 5 3 sy init Initializing a repository 31 Syncany User Guide Release 0 4 6 alpha 5 4 sy connect Connecting to an existing repository This command connects to an existing remote repository and initializes the local directory The command can be called as follows 1 Using a syncany link generated by either init or genlink the command connects to the repository given in the link If the link is encrypted the link repo password must be entered 2 If no link is given the command acts like init i e it queries the user for storage plugin and connection details of the repository to connect to Once the repository is connected the initialized local folder can be synced with the newly created repository The commands up down watch etc can be used Other clients can then be connected using the connect command For a detailed command reference including all command specific options please refer to the manual page of this command at sy connect 5 4 1 Example 1 Connect to an existing repository with a syncany link sy connect syncany storage 1 y8aqJUCsxXqPtH9KutaoAKAKOOvcc Password user enters password Creating master key from passwo
133. the files in a certain folder using metadata about these files size last modified date checksum etc It manages different versions of a file detects if a file has been moved or changed and adds a new file version if it has Like version control systems Syncany knows a concept similar to a commit i e a collection of changes the local files that are uploaded to the central repository In other ways however it is also very different In contrast to Git and its friends Syncany does not support the full range of commands that regular VCS do For instance there is no explicit branching or merging no tagging and diffing Instead Syncany has only one trunk master and auto resolves conflicts when they occur much like Dropbox does Unlike most VCS Syncany does not focus on text based files but treats all files the same large small binary text In addition Syncany is not limited to one or two transport protocols but can be easily extended to many more The similarities to file syne software are quite obvious Syncany must tackle the file synchronization problem i e the problem of keeping multiple replicas of a file set in sync Much like the widely popular rsync Syncany compares the local files to the remote copy or at least its metadata using date time size and checksums of both whole files and parts of files and then transfers only the changed parts to the remote location Like rsync Syncany tries to minimize the individual upload d
134. thenticated 4b g 1b HMAC salt rand key salt random IV ciphertext eer sever 12 bytes 12 bytes n bytes n bytes Defines ciphers to use only Plaintext encrypted and authenticated modes allowed authenticated with specified ciphers The resulting crypto file format is structured as follows e Magic identifier Used to identify Syncany encrypted files static 053790205 e Crypto format version Used to identify the crypto format version static 0x01 e Header HMAC salt Used to derive the HMAC header key with HKDF to verify the header e Cipher Count Defines the number of nested ciphers default 2 e Cipher Spec ID Identifies the algorithm and key size used for the first second cipher 7 3 Security Concepts 61 Syncany User Guide Release 0 4 6 alpha e Cipher Salt Random salt used to derive the cipher specific file key e Cipher IV Random IV used as input for the given cipher size depends on cipher spec ID e Header HMAC HMAC calculated over the cipher count and cipher specs Connecting new Clients When a user connects to an existing repository using sy connect Syncany first downloads the master file This master file contains the unencrypted master salt which in combination with the repository password can be used to derive the master key Using this master key and the salts and IVs contained in the encrypted database and multichunk files Syncany can create the file keys and thereby decrypt any file synca
135. this per user plugin configuration can be found at sAppData Syncany plugins userdata lt plugin id gt or at config syncany plugins userdata lt plugin id gt on Linux Depending on the plugin the files in this folder may differ 6 2 Storage Plugins Storage plugins are part of the core idea of Syncany They provide a simple interface to make any type of storage usable This is done by keeping all of the synchronization logic file size issues and even encryption out of the plugins Storage plugins only take care of uploading different types of files database files multichunk files and so on Once a storage plugin is installed see Installing and Removing Plugins it can be used to create a new remote repository Sy init or connect to an existing repository sy connect After you ve successfully connected a local folder to a remote repository you can synchronize files manually with sy up or sy down or configure the daemon to automatically sync the folder in the background Storage plugins typically need some connection information to connect to a remote server The FTP plugin for instance needs to know the hostname of the server its port the username password as well as a path folder where to store the repository This information the connection settings is stored within the managed Syncany folder in syncany config xml 44 Chapter 6 Plugins Syncany User Guide Release 0 4 6 alpha So if your Syncany folder is a
136. to derive a 512 bit master key using PBKDF2 with million rounds and a 512 bit random master salt 1 4 The master key is stored locally in the config xml file see Common Settings and Storage Connection config xml since the local machine is trusted it is not encrypted The master salt is stored in plaintext locally although it is not needed and on the offsite storage in the master file Since the master salt must be public for when other clients connect to the repository the master file is not encrypted Encrypting new Files Whenever new files have to be uploaded Syncany encrypts these files with 128 bit AES GCM using the crypto scheme and file format described below 8 16 File encryption happens with every file that is uploaded except the master file For instance during the file index process in sy up Syncany packages new unknown chunks into multichunks ZIP files containing many chunks and creates a new database file metadata These files are encrypted using the the following scheme In the default configuration Syncany uses one cipher However Syncany also supports nesting of ciphers In this setup the plaintext is encrypted with the first cipher and every additional cipher is applied to the ciphertext of the previous cipher This setup has relatively little research behind it but we have decided to keep it for the more paranoid users As of today Syncany can theoretically nest any authenticated cipher that is supporte
137. tore NAME sy restor restore old or deleted files SYNOPSIS sy restor e revision lt revision gt t target lt filename gt lt file identifier gt DESCRIPTION This command restores old or deleted files from the remote storage As long as a file is known to the local database and the corresponding chunks are available on the remote storage it can be restored using this command The command downloads the required chunks and assembles the fil If no target revision is given with r the last version is restored To select a revision to restore the sy ls command can be used OPTIONS E revision lt revision gt Selects a certain revision version to restore If no revision is given the last revision is used t target lt file gt Defines the target output filename to restore the file to If this option is not given the default filename is the filename of the restored fil version appended with a restored suffix All folders given in the target filename will be created lt file identifier gt Identifier of the file history as printed by the sy ls command The file identifier and a revision version number uniquely identify a single version of a file at a certain point in time The identifier can be abbreviated if it is unique in the database EXAMPLES sy restore 3168ab663e Restores the last version of the file with identifier 3168ab663e If the database
138. tps syncany org latest deb sudo dpkg i syncany latest deb sy v 2 2 0 Arch Linux Arch Linux users can use the syncany package available on the AUR to install Syncany An AUR helper like yaourt could help with this yaourt S syncany 2 2 4 Mac OS X For Mac OS X there are currently two ways of installation e Download and install the the latest app zip package including GUI e Install the Homebrew via brew install https get syncany org homebrew syncany rb no GUI Homebrew notes Since Syncany is still alpha software it cannot be installed from the official Homebrew sources Therefore we provide the needed formula on our own until we reach beta The command above installs the most recent pre release If you want to install the bleeding edge version simply append HEAD to the previous command If you like the Syncany daemon to start at system startup install the provided LaunchAgent following Homebrew instructions 2 2 5 Docker If you just want to try Syncany for a few minutes we provide it as a containerized Docker application for Syncany If you ve installed Docker already you can use the syncany release repository docker pull syncany release docker run ti syncany release syncany e52be0b2522b S sy v 0 1 8 alpha 2 2 6 Manual Installation for other operating systems If your operating system isn t listed above or if you just want to install Syncany manually for some other reason
139. ts to the background daemon and displays all daemon managed Syncany folders Folders can either be added via the GUI or by adding it to the daemon xm1 see Daemon Config daemon xml F a oe S B83 14 Nov 14 10 06 43 Dy Syncany folder Office synced 2 files added in your Syncany folder Office The tray icon indicates the progress of all daemon managed Syncany folders whether they are syncing or in sync as well as a more detailed status information in form of a status text and popup notifications 6 3 Graphical User Interface Plugin 55 Syncany User Guide Release 0 4 6 alpha Syncany Add new Syncany folder Use this wizard to create a new Syncany repository or connect to an existing one Create a new online storage Choose this option if you want to create a new repository on a remote server using an FTP Samba S3 etc Connect to an existing online storage Choose this option if you want to sync your files with an existing remote storage Watch an existing Syncany folder Choose this option if you want to add an existing Syncany folder to this application lt Previous Cancel Like other plugins the GUI can be installed via sy plugin install and then run from the command line using sy gui Even though this works the GUI plugin is not meant to be installed and run like this Instead it should be installed using the respective installers or packages That way Syncany can be start
140. ually change the syncany config xml file to achieve that There already is a ticket to do a sy reconnect command or something similar on GitHub 9 3 Common errors 9 3 1 Error message Could not generate DH 4096 2048 bits keypair If you see the error message Could not generate DH 4096 bits keypair or Could not generate DH 2048 bits keypair or a similar message during the sy init connect operation the likely cause is that Java 7 only supports DH keys up to 1024 bits due to a bug in Java 7 as indicated in this StackOverflow post and Java 8 only supports DH keys with up to 2048 bits As noted in GitHub issue 483 upgrading to Java 8 will allow DH keys with up to 2048 bits Keys with 4096 bits are not yet supported in Java 7 or 8 The only solution is to downgrade the server s TLS SSL security parameters cipher suites to accept downgrade to DH 2048 bit keys 9 3 Common errors 81 Syncany User Guide Release 0 4 6 alpha 82 Chapter 9 FAQ CHAPTER 10 Appendix A Manual Pages This appendix lists all the manual help pages available for the sy command its sub commands These help pages are available by calling sy lt action gt helporman sy lt action gt Linux only Contents e Appendix A Manual Pages sy syncany sy cleanup sy connect sy daemon sy down sy genlink sy init sy log sy ls remote syls sy plugin sy restore s
141. ures Philipp s SFTP Server Steffen s PC i Pictures Pim sets up a Syncany shared folder with Philipp using the shared Windows folder on his server as a backend storage He uses the sy init command to do that types in samba to use the Windows share plugin and chooses a strong password to encrypt the data with After the one minute setup he gets a syncany link from Syncany This link contains all the information required to access the storage Philipp can now use this and the password to access the Syncany folder using the sy connect command Since everything that lands in the cloud is encrypted with a key derived from the password Pim and Philipp don t have to worry about a nosy storage provider or other interested parties The process between Philipp and Steffen is exactly the same only that now Philipp initializes the repository on his own remote storage and that the SFTP plugin is used 1 4 2 Use Case 2 Sharing in a company Note Note that not all of of the components required for this use case have been implemented completely We are working on it though Company XYZ wants their employees to be able to share files on projects X Y and Z Since they have a Samba and an SFTP server lying around they use them for projects Y and Z The files for project X are hosted on Amazon S3 but since the data is encrypted company XYZ doesn t worry about their files 1 4 Example Use Cases 5 Syncany User Guide R
142. vate key entry and the associated certificate in the keystore jks file with your own private key and certificate The private key entry must be the only entry in the file 3 Import your own certificate in the truststore jks You don t have to remove any entries from this file After restarting the daemon Syncany should use your own certificate for the internal web server You can verify that by opening the browser and checking the certificate 76 Chapter 8 Configuration CHAPTER 9 FAQ We generally talk a lot about Syncany and whenever we explain the idea people tend to ask the same questions Here are a few of them hopefully with satisfying answers Contents e FAQ General questions x What is Syncany Why make something new How is it different from X Is there an App Web App GUI for it When will it be available When is the release date How can I help Does it support multiple users Does it work on Windows Linux Mac What shouldn t I use Syncany for Can I use Syncany to sync two unencrypted folders Can I mirror two folders Where does the name Syncany come from How do you pronounce Syncany SyncAny Sync Any Syncanny Technial and security questions x Why is it written in Java Can you write a plugin for X x What is the purpose of storing the master salt in the repo Isn t it already included in the Syncany link x What exactly is stored in the syncany file in the repo Is it n
143. xt links naturally do not contain a master salt They are structured like this syncany storage 1 not encrypted lt plaintext config gt The lt plaintext config gt is simply a base58 encoded representation of the storage connection config Warning Never share an unencrypted plaintext link over unsecure channels such as instant messengers or e mail If the link contains not encrypted it is trivial to retrieve the storage credentials from it Crypto Algorithms and Parameters This chapter sumarizes the algorithms and parameters used by Syncany This is probably a bit repetetive but maybe useful for people who don t want to read the entire text e Users of a shared folder repository share a repository password e Random values are generated using Java s default SecureRandom implementation dev urandom on Linux CryptGenRandom on Windows e The repository password is used to derive one symmetric key per cipher using PBKDF2 12 byte salt 1 million rounds e The derived symmetric key s are used to encrypt files each key is reused in max 100 files 400 MB 62 Chapter 7 Security Syncany User Guide Release 0 4 6 alpha e Cipher algorithms are configurable but not every cipher is allowed only AES and Twofish 128 256 bit only authenticated modes as of now only GCM no ECB CBC etc e Ciphers are initialized with a random initialization vector IV IVs are never reused e Multiple cipher algorithms can be nested
144. y status sy up sy update sy watch 10 1 sy syncany NAME sy secure file sync software for arbitrary storage backends SYNOPSIS sy l localdir lt path gt log lt path gt v vv loglevel lt level gt print a debug h help lt command gt lt args gt DESCRIPTION Syncany is an open source cloud storage and filesharing application It allows users to backup and share certain folders of their workstations using any kind of storage 83 Syncany User Guide Release 0 4 6 alpha Main sync commands init Initialize the current folder as a Syncany folder connect Connect the current folder to an existing Syncany repository up Detect local changes and upload to repository down Detect remote changes and apply locally Other commands status Detect and print local changes ls remote Detect and print remote changes ls List and filter the current and past file tree cleanup Remove old versions from the local database and the repo restor Restore the given file paths from the remote repository genlink Create a syncany link from an existing local folder plugin List install and remove storage backend plugins update Manage and check for application updates daemon Start and stop the background process daemon watch Automatically sync the local folder in the foreground Short command descriptions and options can be found below Detailed expl
145. ynchronization if this option is used 5 short The syncany link printed after the initialization will be shortened using the Syncany link shortener service This option stores th ncrypted link on the Syncany servers The option does not work if E is enabled password lt password gt DO NOT USE THIS OPTION Set the password used to encrypt the repository This option shouldn t be used because the password might be visible to other users or be stored in history files COPYRIGHT Syncany 0 4 4 alpha Distributed under GPLv3 Copyright c 2011 2015 Philipp C Heckel 10 8 sy log NAME sy log shows recent changes SYNOPSIS sy log x xclud mpty n database count lt count gt s database start lt index gt f file count lt count gt DESCRIPTION This command displays the recent changes to the repository grouped by the corresponding database versions By default the command will display the last 10 database versions and their associated files This default value can be changed by the n parameter The per database file count can be changed with the f option default is 100 To hide potentially empty database versions the x option can be used OPTIONS n database count lt count gt Adjusts the max number of database versions to be returned and displayed by this command If this option is not set max 10 database versions are displayed To return

Syncany User Guide Release 0.4.6-alpha Philipp

Contents

Download Pdf Manuals

Related Search

Related Contents