Command line tool to mass upload media folders to your google photos account(s) (Mac OS / Linux)
Home Page: https://gphotosuploader.github.io/gphotos-uploader-cli/
License: MIT License
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
I tried running gphotos-uploader-cli after I changed the configurations file. after logging in on google, I'm getting failed storing token: failed storing token into keyring: exec: "dbus-launch": executable file not found in $PATH error
On my first run I received this message about quotas. I verified it is not my google storage quota so it must be an API quota. Is there a way to adjust that in the cloud API manager? If not, is there a way to rate limit the application so it doesn't exceed this quota?
2018/11/24 10:36:26 failed uploading image
--- at /home/nick/go/src/github.com/nmrshll/gphotos-uploader-cli/upload/fileUpload.go:33 (StartFileUploadWorker.func1) ---
Caused by: failed uploading image
--- at /home/nick/go/src/github.com/nmrshll/gphotos-uploader-cli/upload/fileUpload.go:53 (FileUpload.upload) ---
Caused by: failed adding media [image name].jpg
--- at /home/nick/go/src/github.com/nmrshll/google-photos-api-client-go/lib-gphotos/client.go:115 (Client.UploadFile) ---
Caused by: googleapi: Error 429: Quota exceeded for quota metric 'photoslibrary.googleapis.com/write_requests' and limit 'WritesPerMinutePerUser' of service 'photoslibrary.googleapis.com' for consumer 'project_number:367310083084'., rateLimitExceeded
I am looking at making this a docker container and am looking for the file or files it saves when you do the Oath authentication piece locally. I am wondering if I can create a container and simply copy that in to bypass having to open a browser and authenticate the application. That still might be possible in docker, but I'd rather not have to get that complicated.
Then end goal is to run this container directly on my storage to periodically check if that directory is in sync with google photos.
Any ideas?
Thanks,
Nate
I'm running gphotos-uploader-cli on a headless server and the manual oauth fails after I manually curl the reply URL from google.
2019/05/17 20:15:53 Failed to open browser, you MUST do the manual process.
2019/05/17 20:15:54 Authentication will be cancelled in 120 seconds
oauthoauthConfig.Exchange() failed with 'oauth2: cannot fetch token: 400 Bad Request
Response: {
"error": "invalid_request",
"error_description": "Missing required parameter: code"
}'
Tried installing gnome-keyring but with that did not help.
I am running gphotos-uploader-cli on MacOS. For some reason I need to do the oauth each time I run the client. Where should the program store the accepted oauth token?
First, thank you for developing this incredibly useful project.
018/09/27 09:03:22 You will now be taken to your browser for authentication
2018/09/27 09:03:23 failed authenticating new client
Not sure how to go past this, is there a way?
I upload from a Docker image from my Synology, which means that there are @eaDir directories all over the place. Sigh. But the files in there are filtered out as not images, that shouldn’t matter, but it does.
Here’s a log from a session:
2019/06/30 01:16:52 Entering Directory: /photos/2019/0616 Washer shelving
2019/06/30 01:16:53 CCVA5551.JPG uploaded successfully as xxxxxxxxxx
2019/06/30 01:16:53 Marked as uploaded: /photos/2019/0612 xxxxxxxxxx
2019/06/30 01:16:53 IMG_6132.MOV uploaded successfully as xxxxxxxxxx
2019/06/30 01:16:53 Marked as uploaded: /photos/2019/0609 xxxxxxxxxx
2019/06/30 01:16:54 using album: 0616 Washer shelving / Id: xxxxxxxxxx
2019/06/30 01:16:54 not a media file: /photos/2019/0616 Washer shelving/.DS_Store: skipping file...
2019/06/30 01:16:54 Entering Directory: /photos/2019/0616 Washer shelving/@eaDir
2019/06/30 01:16:55 xxxxxxxxxx uploaded successfully as xxxxxxxxxx
2019/06/30 01:16:55 Marked as uploaded: /photos/2019/0614 xxxxxxxxxx
2019/06/30 01:16:58 using album: @eaDir / Id: xxxxxxxxxx
2019/06/30 01:16:58 not a media file: /photos/2019/0616 Washer shelving/@eaDir/.DS_Store@SynoResource: skipping file...
2019/06/30 01:16:58 not a media file: /photos/2019/0616 Washer shelving/IMG_6144.AAE: skipping file...
2019/06/30 01:16:58 Uploading IMG_6144.JPG
2019/06/30 01:16:58 Uploading IMG_6145.JPG
2019/06/30 01:16:58 Uploading IMG_6146.JPG
2019/06/30 01:16:58 Uploading IMG_E6144.JPG
(Some items are redacted.)
There is a folder, /photos/2019/0616 Washer shelving that has four .jpg files in it. It also has a @eaDir directory in it. What seems to have happened is that the uploader created an album named 0616 Washer shelving, then one named @eaDir, and then managed to confusedly drop the four images that were in the parent directory into the album named after the child directory.
I was able to catch this because this was a small batch, but this is something that should be fixed.
(FYI, I'm using rfgamaral/docker-gphotos-uploader who wraps this.)
This has been working great for me, but when I tried today I got:
2019/06/15 21:44:28 Uploading HQXS2530.JPG
2019/06/15 21:44:30 failed uploading image
--- at /go/src/github.com/nmrshll/gphotos-uploader-cli/upload/fileUpload.go:33 (StartFileUploadWorker.func1) ---
Caused by: failed uploading image
--- at /go/src/github.com/nmrshll/gphotos-uploader-cli/upload/fileUpload.go:53 (FileUpload.upload) ---
Caused by: status message should be OK, found: Success
--- at /go/src/github.com/nmrshll/google-photos-api-client-go/lib-gphotos/client.go:147 (Client.UploadFile) ---
Is there a recent change to the Photos API?
master fails to build with the following output:
go get -u github.com/nmrshll/gphotos-uploader-cli
# github.com/nmrshll/gphotos-uploader-cli/upload
../Documents/go/src/github.com/nmrshll/gphotos-uploader-cli/upload/fileUpload.go:44:41: fileUpload.gphotosClient.GetOrCreateAlbumByName undefined (type "github.com/nmrshll/google-photos-api-client-go/lib-gphotos".Client has no field or method GetOrCreateAlbumByName)
../Documents/go/src/github.com/nmrshll/gphotos-uploader-cli/upload/fileUpload.go:51:63: too many arguments in call to fileUpload.gphotosClient.UploadFile
have (string, []string...)
want (string)Running inside FreeBSD jail w/ Go version 1.11.4
./gphotos-uploader-cli
[INFO] Config file found. Loading...
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x7ac988]
goroutine 1 [running]:
github.com/zalando/go-keyring.Get(0x912495, 0x1c, 0xc000022700, 0x17, 0x6, 0x800cc54d8, 0x453990, 0x800cbbb00)
/root/go/src/github.com/zalando/go-keyring/keyring.go:32 +0x28
github.com/nmrshll/gphotos-uploader-cli/datastore/tokenstore.RetrieveToken(0xc000022700, 0x17, 0x1000000000008, 0x18, 0x18)
/root/go/src/github.com/nmrshll/gphotos-uploader-cli/datastore/tokenstore/tokenstore.go:37 +0x4e
github.com/nmrshll/gphotos-uploader-cli/upload.authenticate(0xc00014e120, 0xc00014e120, 0xc0000c9be0, 0x40db78)
/root/go/src/github.com/nmrshll/gphotos-uploader-cli/upload/folderUploadJob.go:61 +0x51
github.com/nmrshll/gphotos-uploader-cli/upload.NewFolderUploadJob(0xc000154180, 0xc000152060, 0xc000020ea0)
/root/go/src/github.com/nmrshll/gphotos-uploader-cli/upload/folderUploadJob.go:38 +0x62
main.startUploader(0xc000130780, 0xcd04c0, 0x0, 0x0)
/root/go/src/github.com/nmrshll/gphotos-uploader-cli/cmd/gphotos-uploader-cli/main.go:37 +0x20a
github.com/spf13/cobra.(*Command).execute(0xc000130780, 0xc00001e1b0, 0x0, 0x0, 0xc000130780, 0xc00001e1b0)
/root/go/src/github.com/spf13/cobra/command.go:766 +0x2cc
github.com/spf13/cobra.(*Command).ExecuteC(0xc000130780, 0x2, 0x0, 0xc000130c80)
/root/go/src/github.com/spf13/cobra/command.go:852 +0x2fd
github.com/spf13/cobra.(*Command).Execute(0xc000130780, 0xc0000cdf68, 0x1)
/root/go/src/github.com/spf13/cobra/command.go:800 +0x2b
main.main()
/root/go/src/github.com/nmrshll/gphotos-uploader-cli/cmd/gphotos-uploader-cli/main.go:76 +0x16a
Succeeded on serveral movies but then crashed with an OOM error on a 1.789GB file. I know it has succeeded doing a 1.505GB file previously, but it's possible there is a leak somewhere that after uploding several movies it accumulates.
runtime stack:
runtime.throw(0x945e85, 0x16)
/usr/local/go/src/runtime/panic.go:608 +0x72
runtime.sysMap(0xc160000000, 0xec000000, 0xd32478)
/usr/local/go/src/runtime/mem_linux.go:156 +0xc7
runtime.(*mheap).sysAlloc(0xd19120, 0xec000000, 0x7f2a49927aa0, 0x7f2a276bad38)
/usr/local/go/src/runtime/malloc.go:619 +0x1c7
runtime.(*mheap).grow(0xd19120, 0x75520, 0x0)
/usr/local/go/src/runtime/mheap.go:920 +0x42
runtime.(*mheap).allocSpanLocked(0xd19120, 0x75520, 0xd32488, 0x20300000000000)
/usr/local/go/src/runtime/mheap.go:848 +0x337
runtime.(*mheap).alloc_m(0xd19120, 0x75520, 0xffffffffffff0101, 0x7f2a276bae18)
/usr/local/go/src/runtime/mheap.go:692 +0x119
runtime.(*mheap).alloc.func1()
/usr/local/go/src/runtime/mheap.go:759 +0x4c
runtime.(*mheap).alloc(0xd19120, 0x75520, 0x7f2a49010101, 0x7f2a49916240)
/usr/local/go/src/runtime/mheap.go:758 +0x8a
runtime.largeAlloc(0xeaa3eeb8, 0x8a0101, 0x8ab6a0)
/usr/local/go/src/runtime/malloc.go:1019 +0x97
runtime.mallocgc.func1()
/usr/local/go/src/runtime/malloc.go:914 +0x46
runtime.systemstack(0x0)
/usr/local/go/src/runtime/asm_amd64.s:351 +0x66
runtime.mstart()
/usr/local/go/src/runtime/proc.go:1229
goroutine 1 [running]:
runtime.systemstack_switch()
/usr/local/go/src/runtime/asm_amd64.s:311 fp=0xc001c55218 sp=0xc001c55210 pc=0x456d40
runtime.mallocgc(0xeaa3eeb8, 0x883ae0, 0x40d601, 0xc001c552f0)
/usr/local/go/src/runtime/malloc.go:913 +0x896 fp=0xc001c552b8 sp=0xc001c55218 pc=0x40dc06
runtime.makeslice(0x883ae0, 0xeaa3eeb8, 0xeaa3eeb8, 0x8, 0x68, 0xc001c6e2a0)
/usr/local/go/src/runtime/slice.go:70 +0x77 fp=0xc001c552e8 sp=0xc001c552b8 pc=0x442257
bytes.makeSlice(0xeaa3eeb8, 0x0, 0x0, 0x0)
/usr/local/go/src/bytes/buffer.go:231 +0x6d fp=0xc001c55328 sp=0xc001c552e8 pc=0x4d8ced
bytes.(*Buffer).grow(0xc001c6e2a0, 0xeaa3eeb8, 0xc001c553a8)
/usr/local/go/src/bytes/buffer.go:144 +0x15a fp=0xc001c55378 sp=0xc001c55328 pc=0x4d865a
bytes.(*Buffer).Grow(0xc001c6e2a0, 0xeaa3eeb8)
/usr/local/go/src/bytes/buffer.go:163 +0x3a fp=0xc001c553a0 sp=0xc001c55378 pc=0x4d880a
io/ioutil.readAll(0x9c0060, 0xc000166410, 0xeaa3eeb8, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/io/ioutil/ioutil.go:34 +0x93 fp=0xc001c553e8 sp=0xc001c553a0 pc=0x50b933
io/ioutil.ReadFile(0xc0019fc7e0, 0x2a, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/io/ioutil/ioutil.go:73 +0xd3 fp=0xc001c55440 sp=0xc001c553e8 pc=0x50bb43
github.com/nmrshll/gphotos-uploader-cli/utils/filesystem.BufferFromFile(0xc0019fc7e0, 0x2a, 0x2b, 0xc001c55518, 0x434efc, 0xc000000480, 0x300000002)
/home/nick/go/src/github.com/nmrshll/gphotos-uploader-cli/utils/filesystem/filesystem.go:45 +0x6e fp=0xc001c554c8 sp=0xc001c55440 pc=0x70e10e
github.com/nmrshll/gphotos-uploader-cli/fileshandling.IsImage(0xc0019fc7e0, 0x2a, 0x0)
/home/nick/go/src/github.com/nmrshll/gphotos-uploader-cli/fileshandling/file.go:28 +0x50 fp=0xc001c556c0 sp=0xc001c554c8 pc=0x79d630
github.com/nmrshll/gphotos-uploader-cli/fileshandling.IsMedia(0xc0019fc7e0, 0x2a, 0xc0019fc801)
/home/nick/go/src/github.com/nmrshll/gphotos-uploader-cli/fileshandling/file.go:50 +0x35 fp=0xc001c556e8 sp=0xc001c556c0 pc=0x79de65
github.com/nmrshll/gphotos-uploader-cli/upload.(*FolderUploadJob).Upload.func1(0xc0019fc7e0, 0x2a, 0x9c6360, 0xc0001701a0, 0x0, 0x0, 0xc, 0xc001c55828)
/home/nick/go/src/github.com/nmrshll/gphotos-uploader-cli/upload/folderUploadJob.go:111 +0xc1 fp=0xc001c557a0 sp=0xc001c556e8 pc=0x7d8f11
path/filepath.walk(0xc0019fc7e0, 0x2a, 0x9c6360, 0xc0001701a0, 0xc000594d30, 0x0, 0x0)
/usr/local/go/src/path/filepath/path.go:358 +0x41c fp=0xc001c55878 sp=0xc001c557a0 pc=0x50b0ec
path/filepath.walk(0xc001ac8080, 0x1d, 0x9c6360, 0xc00008d1e0, 0xc000594d30, 0x0, 0x0)
/usr/local/go/src/path/filepath/path.go:382 +0x2fe fp=0xc001c55950 sp=0xc001c55878 pc=0x50afce
path/filepath.walk(0xc001ae0580, 0x1a, 0x9c6360, 0xc000170820, 0xc000594d30, 0x0, 0x0)
/usr/local/go/src/path/filepath/path.go:382 +0x2fe fp=0xc001c55a28 sp=0xc001c55950 pc=0x50afce
path/filepath.walk(0xc000026680, 0x15, 0x9c6360, 0xc0005965b0, 0xc000594d30, 0x0, 0x10)
/usr/local/go/src/path/filepath/path.go:382 +0x2fe fp=0xc001c55b00 sp=0xc001c55a28 pc=0x50afce
path/filepath.Walk(0xc000026680, 0x15, 0xc000594d30, 0x15, 0x0)
/usr/local/go/src/path/filepath/path.go:404 +0x105 fp=0xc001c55b60 sp=0xc001c55b00 pc=0x50b225
github.com/nmrshll/gphotos-uploader-cli/upload.(*FolderUploadJob).Upload(0xc00000c020, 0xc00014c060, 0xc00000c020)
/home/nick/go/src/github.com/nmrshll/gphotos-uploader-cli/upload/folderUploadJob.go:103 +0xea fp=0xc001c55bf0 sp=0xc001c55b60 pc=0x7d8baa
main.startUploader(0xc0000b7900, 0xd309b8, 0x0, 0x0)
/home/nick/go/src/github.com/nmrshll/gphotos-uploader-cli/cmd/gphotos-uploader-cli/main.go:38 +0x218 fp=0xc001c55ce0 sp=0xc001c55bf0 pc=0x832308
github.com/spf13/cobra.(*Command).execute(0xc0000b7900, 0xc00001e1b0, 0x0, 0x0, 0xc0000b7900, 0xc00001e1b0)
/home/nick/go/src/github.com/spf13/cobra/command.go:766 +0x2cc fp=0xc001c55dd0 sp=0xc001c55ce0 pc=0x82a28c
github.com/spf13/cobra.(*Command).ExecuteC(0xc0000b7900, 0x2, 0x0, 0xc00012e000)
/home/nick/go/src/github.com/spf13/cobra/command.go:852 +0x2fd fp=0xc001c55f10 sp=0xc001c55dd0 pc=0x82ae2d
github.com/spf13/cobra.(*Command).Execute(0xc0000b7900, 0xc0000cff68, 0x1)
/home/nick/go/src/github.com/spf13/cobra/command.go:800 +0x2b fp=0xc001c55f40 sp=0xc001c55f10 pc=0x82ab0b
main.main()
/home/nick/go/src/github.com/nmrshll/gphotos-uploader-cli/cmd/gphotos-uploader-cli/main.go:71 +0x16a fp=0xc001c55f98 sp=0xc001c55f40 pc=0x8325da
runtime.main()
/usr/local/go/src/runtime/proc.go:201 +0x207 fp=0xc001c55fe0 sp=0xc001c55f98 pc=0x42e197
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1333 +0x1 fp=0xc001c55fe8 sp=0xc001c55fe0 pc=0x458e21
goroutine 5 [select]:
github.com/syndtr/goleveldb/leveldb/util.(*BufferPool).drain(0xc000140000)
/home/nick/go/src/github.com/syndtr/goleveldb/leveldb/util/buffer_pool.go:206 +0x12a
created by github.com/syndtr/goleveldb/leveldb/util.NewBufferPool
/home/nick/go/src/github.com/syndtr/goleveldb/leveldb/util/buffer_pool.go:237 +0x177
goroutine 21 [select, 52 minutes]:
github.com/syndtr/goleveldb/leveldb.(*DB).compactionError(0xc00009ab60)
/home/nick/go/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:90 +0xd3
created by github.com/syndtr/goleveldb/leveldb.openDB
/home/nick/go/src/github.com/syndtr/goleveldb/leveldb/db.go:142 +0x40c
goroutine 22 [select]:
github.com/syndtr/goleveldb/leveldb.(*DB).mpoolDrain(0xc00009ab60)
/home/nick/go/src/github.com/syndtr/goleveldb/leveldb/db_state.go:101 +0xe7
created by github.com/syndtr/goleveldb/leveldb.openDB
/home/nick/go/src/github.com/syndtr/goleveldb/leveldb/db.go:143 +0x42e
goroutine 23 [select, 52 minutes]:
github.com/syndtr/goleveldb/leveldb.(*DB).tCompaction(0xc00009ab60)
/home/nick/go/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:834 +0x331
created by github.com/syndtr/goleveldb/leveldb.openDB
/home/nick/go/src/github.com/syndtr/goleveldb/leveldb/db.go:149 +0x58c
goroutine 24 [select, 52 minutes]:
github.com/syndtr/goleveldb/leveldb.(*DB).mCompaction(0xc00009ab60)
/home/nick/go/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:762 +0x12e
created by github.com/syndtr/goleveldb/leveldb.openDB
/home/nick/go/src/github.com/syndtr/goleveldb/leveldb/db.go:150 +0x5ae
goroutine 25 [running]:
goroutine running on other thread; stack unavailable
created by github.com/nmrshll/gphotos-uploader-cli/upload.StartFileUploadWorker
/home/nick/go/src/github.com/nmrshll/gphotos-uploader-cli/upload/fileUpload.go:29 +0x87
goroutine 26 [chan receive, 52 minutes]:
github.com/nmrshll/gphotos-uploader-cli/fileshandling.StartDeletionsWorker.func1(0xc00014c058)
/home/nick/go/src/github.com/nmrshll/gphotos-uploader-cli/fileshandling/image.go:41 +0xa2
created by github.com/nmrshll/gphotos-uploader-cli/fileshandling.StartDeletionsWorker
/home/nick/go/src/github.com/nmrshll/gphotos-uploader-cli/fileshandling/image.go:40 +0x87
goroutine 27 [IO wait, 52 minutes]:
internal/poll.runtime_pollWait(0x7f2a4991af00, 0x72, 0xc0001b2860)
/usr/local/go/src/runtime/netpoll.go:173 +0x66
internal/poll.(*pollDesc).wait(0xc00013e198, 0x72, 0xc000136000, 0x1000, 0x0)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9a
internal/poll.(*pollDesc).waitRead(0xc00013e198, 0xc0001b2800, 0x10, 0x10)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
internal/poll.(*FD).ReadMsg(0xc00013e180, 0xc0001b2860, 0x10, 0x10, 0xc000136020, 0x1000, 0x1000, 0x0, 0x0, 0x0, ...)
/usr/local/go/src/internal/poll/fd_unix.go:243 +0x200
net.(*netFD).readMsg(0xc00013e180, 0xc0001b2860, 0x10, 0x10, 0xc000136020, 0x1000, 0x1000, 0xc0000abdc0, 0x7bf202, 0x8a2820, ...)
/usr/local/go/src/net/fd_unix.go:214 +0x90
net.(*UnixConn).readMsg(0xc00014c068, 0xc0001b2860, 0x10, 0x10, 0xc000136020, 0x1000, 0x1000, 0x97, 0xc000149600, 0xc0001b2758, ...)
/usr/local/go/src/net/unixsock_posix.go:115 +0x91
net.(*UnixConn).ReadMsgUnix(0xc00014c068, 0xc0001b2860, 0x10, 0x10, 0xc000136020, 0x1000, 0x1000, 0x7, 0xcdcf53, 0x203000, ...)
/usr/local/go/src/net/unixsock.go:140 +0xa8
github.com/godbus/dbus.(*oobReader).Read(0xc000136000, 0xc0001b2860, 0x10, 0x10, 0x18, 0xc000149980, 0x0)
/home/nick/go/src/github.com/godbus/dbus/transport_unix.go:21 +0x8f
io.ReadAtLeast(0x9bf7c0, 0xc000136000, 0xc0001b2860, 0x10, 0x10, 0x10, 0x8795e0, 0xc000566a01, 0xc000149980)
/usr/local/go/src/io/io.go:310 +0x88
io.ReadFull(0x9bf7c0, 0xc000136000, 0xc0001b2860, 0x10, 0x10, 0xc00011c3d0, 0x0, 0x0)
/usr/local/go/src/io/io.go:329 +0x58
github.com/godbus/dbus.(*unixTransport).ReadMessage(0xc00000c0a0, 0xc000000007, 0x7, 0xc0001c2301)
/home/nick/go/src/github.com/godbus/dbus/transport_unix.go:91 +0x10f
github.com/godbus/dbus.(*Conn).inWorker(0xc00010a120)
/home/nick/go/src/github.com/godbus/dbus/conn.go:292 +0x37
created by github.com/godbus/dbus.(*Conn).Auth
/home/nick/go/src/github.com/godbus/dbus/auth.go:118 +0x76f
goroutine 1448 [IO wait]:
internal/poll.runtime_pollWait(0x7f2a4991ad60, 0x72, 0xc001c57870)
/usr/local/go/src/runtime/netpoll.go:173 +0x66
internal/poll.(*pollDesc).wait(0xc00013ec18, 0x72, 0xffffffffffffff00, 0x9c0f20, 0xcd76b8)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9a
internal/poll.(*pollDesc).waitRead(0xc00013ec18, 0xc000144000, 0x1000, 0x1000)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
internal/poll.(*FD).Read(0xc00013ec00, 0xc000144000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/internal/poll/fd_unix.go:169 +0x179
net.(*netFD).Read(0xc00013ec00, 0xc000144000, 0x1000, 0x1000, 0x40a4cb, 0xc000010000, 0x8bc4c0)
/usr/local/go/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc00000e008, 0xc000144000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:177 +0x68
crypto/tls.(*block).readFromUntil(0xc001a0cde0, 0x9bfd00, 0xc00000e008, 0x5, 0xc00000e008, 0x490fa2)
/usr/local/go/src/crypto/tls/conn.go:492 +0x89
crypto/tls.(*Conn).readRecord(0xc000564e00, 0x95f317, 0xc000564f20, 0x10)
/usr/local/go/src/crypto/tls/conn.go:593 +0xdd
crypto/tls.(*Conn).Read(0xc000564e00, 0xc001a10000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/crypto/tls/conn.go:1145 +0xf1
bufio.(*Reader).Read(0xc0015702a0, 0xc001abe3b8, 0x9, 0x9, 0xc001c57c50, 0xc001c57ca8, 0x60c91d)
/usr/local/go/src/bufio/bufio.go:216 +0x22f
io.ReadAtLeast(0x9bf460, 0xc0015702a0, 0xc001abe3b8, 0x9, 0x9, 0x9, 0xc001c57cf0, 0x680858, 0xc000564e00)
/usr/local/go/src/io/io.go:310 +0x88
io.ReadFull(0x9bf460, 0xc0015702a0, 0xc001abe3b8, 0x9, 0x9, 0x6cdeb4, 0x7f2a499354b8, 0xc000564e00)
/usr/local/go/src/io/io.go:329 +0x58
net/http.http2readFrameHeader(0xc001abe3b8, 0x9, 0x9, 0x9bf460, 0xc0015702a0, 0x0, 0xc000000000, 0x42be11, 0x95f3c0)
/usr/local/go/src/net/http/h2_bundle.go:1545 +0x7b
net/http.(*http2Framer).ReadFrame(0xc001abe380, 0xc001bc6080, 0x0, 0x0, 0x0)
/usr/local/go/src/net/http/h2_bundle.go:1803 +0xa3
net/http.(*http2clientConnReadLoop).run(0xc001c57fb8, 0x95ed40, 0xc001b667b8)
/usr/local/go/src/net/http/h2_bundle.go:8261 +0x9e
net/http.(*http2ClientConn).readLoop(0xc00170e380)
/usr/local/go/src/net/http/h2_bundle.go:8189 +0x76
created by net/http.(*http2Transport).newClientConn
/usr/local/go/src/net/http/h2_bundle.go:7264 +0x636
exit status 2
Installation via go get -u github.com/nmrshll/gphotos-uploader-cli/cmd/gphotos-uploader-cli fails with
package google.golang.org/api/photoslibrary/v1: cannot find package "google.golang.org/api/photoslibrary/v1"
Indeed, this package is not in the list if you go to https://godoc.org/google.golang.org/api -- has it been removed?
This is the current README.md:
But github.com/nmrshll/gphotos-uploader-cli/cmd/gphotos-uploader-cli doesn't exist anymore.
Shouldn't the command now be:
$ go get -u github.com/nmrshll/gphotos-uploader-cli
?
Hi guys!
I've opened this issue to discuss if we should remove keychain integration to store the Google Photos token, and store it in LevelDB instead.
Historically gphotos-uploader-cli used gnome-keyring to store this token in an efficient and secure way. @nmrshll decided to do it this way (I'm guessing) because he handn't any other DB available. But now we can use the LevelDB database (needed for tracking uploaded items) to store the token.
PROS:
github.com/99designs/keyring dependency.CONS:
What do you think? Should we remove keychain integration?
GIF files are uploaded but never deleted from local disk
Mint, go version go1.6.2 linux/amd64
$ go get -u github.com/nmrshll/gphotos-uploader-cli
package math/bits: unrecognized import path "math/bits" (import path does not begin with hostname)
package context: unrecognized import path "context" (import path does not begin with hostname)
When I upload a photo to Google Photos with drag-and-drop, the EXIF text becomes a caption of the photo. When I do the same with the uploader tool, it shows file path instead of the caption.
I 'd like to help with the client side compression, to allow users to upload unlimited photos.
Have you already started working on it?
To add this feature we would definitely need an option in the configuration of the cli to allow the users to enable/disable the compression, but where do you think it's better to implement the compression? in the cli o directly on the library?
Does this preserve the photo taken dates or does the Google photos date become the uploaded date?
(FYI, I'm using rfgamaral/docker-gphotos-uploader who wraps this.)
I have ARW files and gphotos-uploader-cli is saying:
not a supported image or video: /photos/2012/0703 Scandinavia-Copenhagen to Gothenburg/DSC02006.ARW: skipping file...
etc.
But if I drop those ARW files onto the web UI they upload. Can gphotos-uploader-cli be fixed to upload anything that Google Photos allows?
Thanks for a great project!
Since Google Photos supports a range of raw files, it would be great if the uploader supported them too.
Using @rfgamaral's docker implementation from issue 15, the uploader skips my raw files. I assume it is a limitation with this repo.
Hi
I want to use this on Ubuntu server (non Desktop GUI) and when i run ./gphotos-uploader-cli this is the error i get, i was wondering is there a way to use this on terminal version ? or how about opening it on desktop and copying the authentication files to certain folder ?
2019/02/15 10:05:07 Need to log login into account .......
2019/02/15 10:05:08 You will now be taken to your browser for authentication
2019/02/15 10:05:10 failed authenticating new client: all options failed with errors:
--- at /root/go/src/github.com/nmrshll/google-photos-api-client-go/noserver-gphotos/no-server.go:37 (NewClient) ---
Caused by: 1 error occurred:
* failed authenticating user
--- at /root/go/src/github.com/nmrshll/google-photos-api-client-go/noserver-gphotos/no-server.go:70 (AuthenticateUser.func1) ---
Caused by: failed opening browser window
--- at /root/go/src/github.com/nmrshll/oauth2-noserver/oauth2ns.go:90 (AuthenticateUser) ---
Caused by: exit status 3
After setting up the config file, i run the program and it gives me the page to confirm granting access. I do that, it reports successfully completed, but in the terminal this happens:
hoyd@e5400:~$ gphotos-uploader-cli
2019/02/25 09:48:30 Need to log login into account [email protected]
2019/02/25 09:48:31 You will now be taken to your browser for authentication or open the url below in a browser.
...
2019/02/25 09:48:36 Shutting down server...
Server gracefully stopped
2019/02/25 09:48:36 failed storing token: failed storing token into keyring: failed to unlock correct collection '/org/freedesktop/secrets/aliases/default'
taken from config file
I tried to upload a bunch of photos and did set album
makeAlbums: {
enabled: true
use: folderNames
}
which resulted in one album per photo instead of per folder. This is with an installation as of today (2018-10-22) direct from Github with go, running Debian/sid.
I am not able to install since of the missing package from above.
Trying to do:
go get -u -> fails with package google.golang.org/api/photoslibrary/v1: cannot find package "google.golang.org/api/photoslibrary/v1" in any of:
make build
Why do you attempt to screen what's uploaded? I can imagine that in a scenario where the images are mixed in with lots of other file types, sure, but in the case where I'm passing in a photos folder full of only photos, I'd really like an option to bypass all filetype comparing and just try to upload everything, and if the server fails to take it, make a note that it was uploaded and don't try again.
IE: can I leave the programme running and if a new photo is put in folder can it auto upload it?
Maybe i'm wrong but the uploader does not delete photos from remote if they do not exist in local, does not it?
The uploader could check if a remote file exists locally and if it does not: delete it. Otherwise I have a bunch of photos remotely that i uploaded sometime and i don't want to delete it manually.
Hi,
I've tried to run this tool following the instructions, but It's not working:
$ go get -u github.com/nmrshll/gphotos-uploader-cli
package github.com/nmrshll/gphotos-uploader-cli: no Go files in /home/paco/go/src/github.com/nmrshll/gphotos-uploader-cli
and trying to use the make command, dependencies are not meet (they are not installed).
Can you provide me more detailed documentation?
Thanks
I saw the following build error.
$ go get -u github.com/nmrshll/gphotos-uploader-cli/cmd/gphotos-uploader-cli
# github.com/nmrshll/gphotos-uploader-cli/fileshandling
go/src/github.com/nmrshll/gphotos-uploader-cli/fileshandling/file.go:31:14: invalid operation: kind != "gopkg.in/h2non/filetype.v1/types".Unknown (mismatched types "github.com/h2non/filetype/types".Type and "gopkg.in/h2non/filetype.v1/types".Type)
So, I installed v0.1.11 instead.
Following the README, I tried this:
go get -u github.com/nmrshll/gphotos-uploader-cli/cmd/gphotos-uploader-cli
That fails with:
# github.com/steakknife/hamming
../../go/src/github.com/steakknife/hamming/popcnt_amd64.s:11:10: fatal error: textflag.h: No such file or directory
#include "textflag.h"
^~~~~~~~~~~~
compilation terminated.
I'm trying to update my Docker container with the latest version (v0.3.0) but now I'm getting this during the authentication process:
Enter passphrase to unlock /root/.gphotos-uploader-cli:
I get this very time I call gphotos-uploader-cli, the command to actually start uploading and it is requiring me to press return to continue. Which is a blocker for an automated system, such as my Docker image, to use gphotos-uploader-cli.
@pacoorozco Any idea how this was introduced?
It would be nice if we could specify exclude patterns per job that allow excluding files with certain names or files in certain folders.
Use-Case: The folder I am trying to upload contains thumbnails in a .thumbs-db folder which should not be uploaded to Google Photos.
Super simple, but I can't even tell if the functionality is there...
Can videos be set to delete after upload? I have delete after upload set to true, but the CLI output just says it's skipping the file because it's not an image.
after go get -u github.com/nmrshll/gphotos-uploader-cli/cmd/gphotos-uploader-cli if I type gphotos-uploader-cli I have the following message:
gphotos-uploader-cli : The term 'gphotos-uploader-cli' is not recognized as the name of a cmdlet, function, script
file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct
and try again.
At line:1 char:1
+ gphotos-uploader-cli
+ ~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (gphotos-uploader-cli:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
How can I fix this so I can use the program I just install?
When I try to upload my photos I get this error:
--- at /Users/david/go/src/github.com/nmrshll/google-photos-api-client-go/lib-gphotos/client.go:115 (Client.UploadFile) ---
Caused by: googleapi: Error 400: No permission to add media items to this album., badRequest
Before this error was thrown gphotos-uploader-cli was able to upload quite a few files to the given album
This appears when uploading:
2019/07/16 20:05:45 not a media file: /home/kentabussi/Documents/Photos/P6130019.MOV: skipping file...
Is google putting the smack down on this uploader, or is this just me?
2019/05/26 01:14:55 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:15:05 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:15:16 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:15:26 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:15:36 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:15:46 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:15:56 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:16:06 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:16:16 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:16:26 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:16:36 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:16:46 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:16:56 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:17:06 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:17:17 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:17:27 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:17:37 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:17:47 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:17:57 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:18:07 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:18:17 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:18:27 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:18:37 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:18:47 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:18:57 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:19:07 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:19:17 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:19:27 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:19:37 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:19:47 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:19:57 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:20:08 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:20:18 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:20:28 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:20:38 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:20:48 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:20:58 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:21:08 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:21:18 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:21:28 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:21:38 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:21:48 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:21:58 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:22:08 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:22:18 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:22:28 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:22:38 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:22:48 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:22:59 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:23:09 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:23:19 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:23:29 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:23:39 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:23:49 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:23:59 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:24:09 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:24:19 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:24:29 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:24:39 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:24:49 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:24:59 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:25:09 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:25:19 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:25:30 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:25:40 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:25:50 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:26:00 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:26:10 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:26:20 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:26:30 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:26:40 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:26:50 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:27:00 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:27:10 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:27:20 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:27:30 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:27:40 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:27:50 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:28:00 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:28:10 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:28:20 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:28:31 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:28:41 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:28:51 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:29:01 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:29:11 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:29:21 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:29:31 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:29:41 Rate limit reached, sleeping for 10 seconds...
2019/05/26 01:29:51 Rate limit reached, sleeping for 10 seconds...
Error: failed storing token: failed storing token into keyring: failed to unlock correct collection '/org/freedesktop/secrets/aliases/default'
The cause seems to be that gnome-keyring doesn't work on my system.
Looking at #15 there appears to be some work on using a different keystore if dbus isn't running. dbus is running here, so maybe some way to force that might be the answer?
Otherwise, it looks like https://github.com/99designs/keyring supports more backends, including encrypted files, and might be more flexible? Also may resolve #51 given that it also supports wincred.
Set up was a bit rough.
First, after installation with go get -u github.com/nmrshll/gphotos-uploader-cli, neither gphotos-uploader-cli nor gphotos-uploader was found; it did not install globally. I had to use ~/go/bin/gphotos-uploader-cli.
Second, it said it generated a config file at ~/.config/gphotos-uploader-cli/config.hjson, but it did not. I had to copy the example and create it manually.
Besides that, setup went quite smoothly and I am up and running, uploading my automated (5sec interval) screenshots to an alternative account 😄
Repro:
gphotos-uploader-cli should not be halting if it tried to upload a file that was rejected due to it being an unsupported format. It should mark that in the database and move on.
According to https://github.com/nmrshll/gphotos-uploader-cli/blob/master/.docs/installation.md
gphotos-uploader-cli can be installed via snapcraft but:
sudo snap install gphotos-uploader-cli
error: snap "gphotos-uploader-cli" not foundHi, I tried running this tool on Windows 10 64bit, but I'm running into issues with the configuration file.It seems that characters like : are not allowed, but are necessary on Windows to give an absoulte Path to a Folder.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.