yuriytkach / dsync-client Goto Github PK

Currently, we register watcher recursively for all directories and when new directory is created we need to register watcher for it. On Windows the undocumented ExtendedWatchEventModifier.FILE_TREE works, so we can use it.

According to request from Dropbox:

Hello,We are reaching out to you because we have discovered an issue with the Dropbox API v2 Java SDK that your app is currently using. Although the issue does not break any existing behavior, it limits our ability to make non-breaking changes to our API endpoints going forward. For that reason, we ask that you update your app to the latest version of the SDK within the next two months. This bug was fixed specifically in v3.0.3.If you do not update SDK versions within the next two months, return types from some endpoints may fail to deserialize correctly, raising a runtime exception. We apologize for the inconvenience.

Handle upload errors correctly and restart uploads if failed at first

When you move or rename folder locally, 2 events are created: delete, then create. However, events are only for that folder. The delete is processed, and folder with all its contents is deleted from Dropbox. Then create is processed and folder is created. However, contents are lost.

Need to publish 'create' events for all folder contents recursively after we receive 'create' event for the folder.

When client start it needs to sync local directory structure with all files with the information in db. That way offline changes can be picked up and then synced to server.

When file changes locally and we try to upload it, we get exception
com.dropbox.core.v2.files.UploadErrorException: Exception in upload: {".tag":"path","reason":{".tag":"conflict","conflict":"file"},"upload_session_id":"ssss"}

Build fails on package stage because of maven-assembly plugin

System set up:

Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T17:41:47+01:00)
Java version: 1.8.0_111, vendor: Oracle Corporation
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.12.3", arch: "x86_64", family: "mac"

Maven logs:

INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 40.283 s
[INFO] Finished at: 2017-02-12T10:27:19+01:00
[INFO] Final Memory: 50M/806M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-assembly-plugin:3.0.0:single (make-assembly) on project dsync-client: Execution make-assembly of goal org.apache.maven.plugins:maven-assembly-plugin:3.0.0:single failed: group id '113584762' is too big ( > 2097151 ). Use STAR or POSIX extensions to overcome this limit -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

When you rename something in DropBox you get 2 events: delete old one and create new one. And the id remains the same. We need to combine those events and just do a rename in db and in local folder. For example:
FileData [FOLDER /Public/aaaaa, id: id:MqBCzBTzMXAAAAAAAAF6zg]
CREATE: D:\DropboxDsync\Public\aaaaa

FileData [DELETE /Public/aaaaa]
FileData [FOLDER /Public/bbbbbb, id: id:MqBCzBTzMXAAAAAAAAF6zg]
DELETED /Public/aaaaa
LOCAL_DIR /Public/bbbbbb

When you rename a file/folder on local drive you get 2 or 3 events: delete the old one, create new one, and possibly modify new one. Need to investigate the possibility to combine those events and simple do the sync the rename.

Bash script dsync-client.sh does not propagate all arguments to java application.
Steps to reproduce:

1. Build distribution archive
2. Run script without arguments $ ./dsync-client.sh
3. See results: app is starting in usual way
   22:41:26.932 [main] DEBUG: Using database at /Users/vkhvostov/.dsyncclient/dsync.db (DSyncClient, 136)
22:41:27.047 [main] DEBUG: Creating database tables (DSyncClient, 145)
22:41:27.052 [main] DEBUG: Tables created successfully (DSyncClient, 147)
22:41:27.076 [main] DEBUG: DSyncClientKeyProvider class not found (DropboxService, 147)
---------------------------
1. Go to: https://www.dropbox.com/oauth2/authorize?response_type=code&client_id=YOUR_APP_KEY
2. Click "Allow" (you might have to log in first)
3. Copy the authorization code.
4. Input code and hit Enter:
4. Run script with -h argument $ ./dsync-client.sh -h, assuming that we should get help information
5. Expected result:
   usage: java -cp "lib/*" com.yet.dsync.DSyncClient [options]
-db,--database <arg> Full path to database
-h,--help Display this help
-r,--reset Remove the database and start configuration procedure
   Actual result:
   22:44:22.488 [main] DEBUG: Using database at /Users/vkhvostov/.dsyncclient/dsync.db (DSyncClient, 136)
22:44:22.634 [main] DEBUG: DSyncClientKeyProvider class not found (DropboxService, 147)
---------------------------
1. Go to: https://www.dropbox.com/oauth2/authorize?response_type=code&client_id=YOUR_APP_KEY
2. Click "Allow" (you might have to log in first)
3. Copy the authorization code.
4. Input code and hit Enter:

Add source control analysis tools and fix found issues

Handle download errors and restart download if failed at first

We need to add upload service that will upload new files to Dropbox, create directories, delete files and folders in Dropbox. The service will be triggered from LocalFolderChange listener.

Currently, this even is just skipped. We might need to add correct handling of it, so we don't skip when files are updated, created.

Use lombok library for dto classes and for switching to SLF4j

Currently threads in upload and download services are killed after exception. That way app becomes unresponsive to changes. Need to log errors but not kill threads.

Remote sysout and add slf4j with log4j 2

If there is a new file in Dropbox and it is being downloaded by DownloadService the LocalFolderWatching picks up changes in the filesystem and reports them. We need to skip those changes, otherwise we'll try to upload files that are being downloaded.

Same thing happens backwards - when new folder is created locally and then created on dropbox, then downloadService will pick up the change and try to do the same.