Giter Club home page Giter Club logo

google-photos-api-client-go's Introduction

Google Photos API client for Go

Go Reference Go Report Card codebeat badge codecov GitHub release GitHub

This package provides a client for using the Google Photos API in Go. Uses the original photoslibrary package, that was provided by Google and, now it's archived here.

The package offers access to these Google Photos services:

  • Albums is a service to manage albums.
  • MediaItems is a service to manage media items (Photos and Videos).
  • Uploader is a service to upload items.

This project will maintain compatibility with the last three major published published versions of Go.

Installation

google-photos-api-client-go is compatible with modern Go releases in module mode. You can install it using:

$ go get github.com/gphotosuploader/google-photos-api-client-go/v3

Usage

import gphotos "github.com/gphotosuploader/google-photos-api-client-go/v3"

Construct a new Google Photos client, then use the various services on the client to access different parts of the Google Photos API.

For example:

// httpClient has been previously authenticated using oAuth authenticated
client := gphotos.NewClient(httpClient)

// list all albums for the authenticated user
albums, err := client.Albums.List(context.Background())

The services of a client divide the API into logical chunks and correspond to the structure of the Google Photos API documentation at https://developers.google.com/photos/library/reference/rest.

NOTE: Using the context package, one can easily pass cancelation signals and deadlines to various services of the client for handling a request. In case there is no context available, then context.Background() can be used as a starting point.

Authentication

The gphotos library does not directly handle authentication. Instead, when creating a new client, pass a net/http.Client that can handle authentication for you. The easiest and recommended way to do this is using the oauth2 library, but you can always use any other library that provides a net/http.Client.

Access to the API requires OAuth 2.0 client credentials from a Google developers project. This project must have the Library API enabled as described here.

import (
    "golang.org/x/oauth2"

    gphotos "github.com/gphotosuploader/google-photos-api-client-go/v3"
)

func main() {
    ctx := context.Background()
    oc := oauth2Config := oauth2.Config{
        ClientID:     "... your application Client ID ...",
        ClientSecret: "... your application Client Secret ...",
        // ...
    }
    tc := oc.Client(ctx, "... your user Oauth Token ...")
    
    client := gphotos.NewClient(tc)
    
    // list all albums for the authenticated user
    albums, err := client.Albums.List(ctx)
}

Note that when using an authenticated Client, all calls made by the client will include the specified OAuth token. Therefore, authenticated clients should almost never be shared between different users.

See the oauth2 docs for complete instructions on using that library.

Features

Retries following best practices

Albums service

Media Items service

  • Offers an independent albums.Service implementing the Google Photos MediaItems API.
  • The client accepts a customized media items service using client.MediaItems.

Uploader

  • Offers two uploaders implementing the Google Photos Uploads API.
    • uploader.SimpleUploader is a simple HTTP uploader.
    • uploader.ResumableUploader is an uploader implementing resumable uploads. It could be used for large files, like videos. See documentation.
  • The client accepts a customized media items service using client.Uploader.

Limitations

Only images and videos can be uploaded. If you attempt to upload non-videos or images or formats that Google Photos doesn't understand, Google Photos will give an error when creating media item.

Photo storage and quality

All media items uploaded to Google Photos using the API are stored in full resolution at original quality. They count toward the user’s storage. The API does not offer a way to upload in "high quality" mode.

Duplicates

If you upload the same image twice, with the same binary content, then Google Photos will deduplicate it. However, it will retain the filename from the first upload which may be confusing. In practice, this shouldn't cause too many problems.

Albums

Note that you can only add media items that have been uploaded by this application to albums that this application has created, see here why.

Rate Limiting

Google Photos imposes a rate limit on all API clients. The quota limit for requests to the Library API is 10,000 requests per project per day. The quota limit for requests to access media bytes (by loading a photo or video from a base URL) is 75,000 requests per project per day.

Used by

google-photos-api-client-go's People

Contributors

arran4 avatar dependabot[bot] avatar mlbright avatar nhorvath avatar nmrshll avatar pacoorozco avatar pdecat avatar timjonischkat avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

google-photos-api-client-go's Issues

Panic using v1.1.1 (due to Mutex)

./gphotos-uploader-cli                                   
Enter passphrase to unlock /home/paco/.config/gphotos-uploader-cli: 
2019/10/12 18:26:27 Token expiration: 2019-10-12 19:24:50.99081676 +0200 CEST
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x46f55d]

goroutine 1 [running]:
sync.(*Mutex).Lock(0x0)
        /snap/go/4520/src/sync/mutex.go:74 +0x2d
github.com/gphotosuploader/google-photos-api-client-go/lib-gphotos.(*Client).GetOrCreateAlbumByName(0xc000093500, 0xc00002b0a5, 0x7, 0x0, 0x0, 0x0)
        /home/paco/src/go/pkg/mod/github.com/gphotosuploader/[email protected]/lib-gphotos/albums.go:68 +0x68
github.com/gphotosuploader/gphotos-uploader-cli/upload.getGooglePhotosAlbumID(0xc00002b0a5, 0x7, 0xc000093500, 0x7, 0x0)
        /home/paco/src/gphotos-uploader-cli/upload/folderUploadJob.go:134 +0x4f
github.com/gphotosuploader/gphotos-uploader-cli/upload.(*Job).ScanFolder.func1(0xc00002b080, 0x39, 0xa2ba40, 0xc0001451e0, 0x0, 0x0, 0x4c8fda, 0xc0001451e0)
        /home/paco/src/gphotos-uploader-cli/upload/folderUploadJob.go:104 +0x2d4
path/filepath.walk(0xc00002b080, 0x39, 0xa2ba40, 0xc0001451e0, 0xc000093560, 0x0, 0x0)
        /snap/go/4520/src/path/filepath/path.go:358 +0x426
path/filepath.walk(0xc00002c450, 0x2c, 0xa2ba40, 0xc000145040, 0xc000093560, 0x0, 0x0)
        /snap/go/4520/src/path/filepath/path.go:382 +0x300
path/filepath.walk(0xc00002c270, 0x24, 0xa2ba40, 0xc000144ea0, 0xc000093560, 0x0, 0x30)
        /snap/go/4520/src/path/filepath/path.go:382 +0x300
path/filepath.Walk(0xc00002c270, 0x24, 0xc000093560, 0x0, 0x11)
        /snap/go/4520/src/path/filepath/path.go:404 +0xff
github.com/gphotosuploader/gphotos-uploader-cli/upload.(*Job).ScanFolder(0xc000093530, 0xc000030a80, 0xc000010090, 0x0)
        /home/paco/src/gphotos-uploader-cli/upload/folderUploadJob.go:67 +0x11d
github.com/gphotosuploader/gphotos-uploader-cli/cmd.startUploader(0xd9fde0, 0xdc2768, 0x0, 0x0)
        /home/paco/src/gphotos-uploader-cli/cmd/root.go:125 +0x96e
github.com/spf13/cobra.(*Command).execute(0xd9fde0, 0xc000090190, 0x0, 0x0, 0xd9fde0, 0xc000090190)
        /home/paco/src/go/pkg/mod/github.com/spf13/[email protected]/command.go:830 +0x2ae
github.com/spf13/cobra.(*Command).ExecuteC(0xd9fde0, 0xc0000d1f68, 0x85f90e, 0xd9fde0)
        /home/paco/src/go/pkg/mod/github.com/spf13/[email protected]/command.go:914 +0x2fc
github.com/spf13/cobra.(*Command).Execute(...)
        /home/paco/src/go/pkg/mod/github.com/spf13/[email protected]/command.go:864
github.com/gphotosuploader/gphotos-uploader-cli/cmd.Execute()
        /home/paco/src/gphotos-uploader-cli/cmd/root.go:43 +0x32
main.main()
        /home/paco/src/gphotos-uploader-cli/main.go:8 +0x20

Race condition in GetOrCreateAlbumByName()

When more than one worker is calling GetOrCreateAlbumByName() a race condition may appear. While a thread is looking for an album, another thread coulbe creating it. The first thread ends without finding it and try to create it, but the second thread already created it.

Google Photos doesn't disallow having several albums with the same name, so this needs to be handled in our side.

A sync.RWLock could be a good idea.

See gphotosuploader/gphotos-uploader-cli#135 for more details. Thanks to @mjhatcher

Split lib-gphotos and noserver-gphotos

We have two different modules sharing the same gphotos package name. That's a bit confusing, so we should address it differently to use different packages.

This is a breaking change and should be part of new v2 release

repository.Create does not return HTTP status code of the operation on failure

gphotos-uploader-cli does not respect Google Photos quote limit because google-photos-api-client-go album Repository.Create does not return anything useful in the error message that would indicate that the limit has been reached.
All it returns is

(*url.Error)(0xc00072cc30)(Post "https://photoslibrary.googleapis.com/v1/albums?alt=json": POST https://photoslibrary.googleapis.com/v1/albums?alt=json giving up after 5 attempt(s))

Informing the user there has been 5 attempts, but the HTTP code there is no where to be found.
To solve this issue in gphotos-uploader-cli one can attempt to write something ugly like this

album, err := service.Create(ctx, title)
	if err != nil {
		if urlerr, ok := err.(*url.Error); ok {
			if  strings.ToUpper(urlerr.Op) == http.MethodPost && urlerr.URL == "https://photoslibrary.googleapis.com/v1/albums?alt=json" {
				wrappedErr := urlerr.Unwrap()
				if strings.HasPrefix(wrappedErr.Error(), "POST https://photoslibrary.googleapis.com/v1/albums?alt=json giving up after ") {
					return "", fmt.Errorf("quota limit reached for %w, check https://console.cloud.google.com/apis/api/photoslibrary.googleapis.com/quotas", err)
				}
			}
		}
		return "", err
	}

But since it does not know what the original problem was, it could mislead. So essentially there is no way to fix it in gphotos-uploader-cli It has to be fixed here, in this project.

Method to list MediaItems in an Album with pagination

Describe the bug
Currently it seems that MediaItems.ListByAlbums only returns the first page of results. For albums with more photos, only the latest 25 are returned in the []mediaItems slice.

To Reproduce
Steps to reproduce the behavior:

        gpc, err := gphotos.NewClient(client)
        ta, err := gpc.Albums.GetById(ctx, "<AlbumId>")
        if err != nil {
                log.Fatal(err)
        }
        taid := ta.ID
        log.Println("Album Count is: " + ta.MediaItemsCount)
        mic,err := strconv.Atoi(ta.MediaItemsCount)
        if err != nil {
                log.Fatal(err)
        }
        cmi := rand.Intn(mic - 1)
        log.Println("Chosen Media Index is: " + strconv.Itoa(cmi))

        am, err := gpc.MediaItems.ListByAlbum(ctx, taid)
        log.Println("Number of Media Items Returned is: " + strconv.Itoa(len(am)))

Output:

2022/01/04 09:36:45 Album Count is: 11558
2022/01/04 09:36:45 Chosen Media Index is: 11346
2022/01/04 09:36:46 Number of Media Items Returned is: 25

Expected behavior
Paging should be supported so that all media items can be retrieved.

Additional context
I'm writing a go program that will randomly choose a photo from a given (large) album.

The upload file is throwing an bad request

Hi ,

I need some help from you guys, probably this is not a bug and and I'm doing something wrong, but I'm getting always bad request when trying to upload an photo.

I've investigate a bit the code and something raise my attention. In bellow function we are putting nil on body request, is this the expected value or should it be the return of item.Open() ?

func (u BasicUploader) prepareUploadRequest(item uploader.UploadItem) (*http.Request, error) {
	_, size, err := item.Open()
	if err != nil {
		return nil, err
	}

	req, err := http.NewRequest("POST", u.url, nil)

Describe the bug

The upload file is throwing an bad request (http 400)

To Reproduce

Sample code for basic upload:

...

srv, err := gphotos.NewClient(client)

...

s, err := srv.Uploader.UploadFile(ctx, "img1.jpg")

I'm receiving the following error:

got 400 Bad Request: {
"code": 3,
"message": "Payload must not be empty"
}

Expected behavior

Get the upload token if the photo was successful uploaded.

Disable DEBUG log on each request

Is your feature request related to a problem? Please describe.
This package writes a log line to stdErr for each request. There is not way to disable it.

Describe the solution you'd like
Disable DEBUG log by default.

Describe alternatives you've considered
Add a new option to enable/disable this DEBUG log

v3.x: feedback requested

v3.0.0 is almost ready

We're almost ready to release a new major version v3.x. πŸ₯³

There are plenty of changes on it, most of them are non-backwards compatible, and we would like to let you provide feedback before releasing the final version. πŸ‘‰πŸΌ Take a look to the CHANGELOG on the 3.x branch. πŸ‘ˆπŸΌ

Please review the latest v3.x tag and use it in your code. πŸ™πŸΌ

// New module path
import gphotos "github.com/gphotosuploader/google-photos-api-client-go/v3"

Feel free to add new issues to address bugs or feature requests and share the link here.

Do you have any feedback?

Why PhotosLibraryAlbumsRepository ListAll function use excludeNonAppCreatedData ?

Describe the bug
Why PhotosLibraryAlbumsRepository ListAll function use excludeNonAppCreatedData ?

Expected behavior
Maybe use ListAll to get all albums and another function like ListNotAppCreateAlbum to get albums that not created by this app.
Or pass option to ListAll function, let user can enable the exclude option or not.

Screen Shot 2022-01-10 at 5 52 42 PM

AlbumByName checks only partial Album List

The method AlbumByName is getting Google Photos album list and checking if there is an album with the specified name. For this purpose is using album.list, but based on the documentation it's only checking first 20 albums, so It's possible that some albums are not checked, causing albums with the same name are created or an error is thrown.

check for permission errors etc

Various places in https://github.com/gphotosuploader/google-photos-api-client-go/blob/master/lib-gphotos/uploads.go call .Do to send a request. They check the err returned but that err is unset if the http request executes correctly but returns a non-200 status.

Something needs to also check res.StatusCode

https://golang.org/pkg/net/http/#Response

Otherwise you get mysterious failures with no useful error output. In my case it's an authentication error.

Sorry not in a good position to supply a patch but it seems like it should be fairly trivial.

Remove cache from this module

Is your feature request related to a problem? Please describe.
The v2.x of the module was offering an Album's cache to reduce the number of calls to the Google Photos API.

Example: When you were trying to GetByTitle several times, we were expecting to reuse previous List calls to eventually get the album without sending a new request.

Over the time, the Album's cache has become ineffective, and it's strongly recommend to implement this kind of cache on the client side. The client has more context to set what to cache and how long. The gphotos-uploader-cli is already implementing it this way, and reducing the amount of request by 20x.

Describe the solution you'd like
Remove the CachedAlbumsService and create a raw service wrapping the Google Photos client.

The contract should be kept, so backwards compatibility is expected.

Describe alternatives you've considered
Offer both, the CachedAlbumsService and the new one.

Additional context
Given the change, a major version should be published after this.

Panic is produced when adding media items to an album

Describe the bug
The bug was reported on gphotosuploader/gphotos-uploader-cli#262. In some cases (I don't know when) Google Photos response include no MediaItem. The code panics when it tries to access to a nil MediaItem.

To Reproduce
When creating a media item the lines:

       mediaItemsResult := make([]MediaItem, len(result.NewMediaItemResults))
	for i, res := range result.NewMediaItemResults {
		m := res.MediaItem
		mediaItemsResult[i] = r.convertPhotosLibraryMediaItemToMediaItem(m)
	}

If result.NewMediaItemResults has some MediaItem pointing to nil, the code fails when it tries to access to it.

Expected behavior
It should not panic.

Additional context
The Google Photos API documentation is not saying anything about MediaItem pointing to nil on successful responses..

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.