grahamearley / firestoregoogleappsscript Goto Github PK

Distinct from #11, has anyone started on a type/interface file for FirestoreApp and friends?
(in the style of the Google App Script one)

That seems to be the only way to get intellisense/autocomplete to work in an IDE.

If I update/establish a date field using the Google.Cloud.Firestore.V1Beta1 (1.0.0-beta12), under C#, in the Firestore DB Console, the date field displays just fine (as valid) and as timestamp as expected. When reading it back with the GAS library, it is not recognized as valid... displaying it back with GAS from retrieving a document shows as 'Wed Dec 31 18:00:00 GMT-06:00 1969' even though the console displays the date as a valid timestamp type (example, '7 November 2018 at 14:14:19 UTC-6' shown in the console, not recognized properly with GAS library)... If you have any ideas, or something I may have missed, let me know, thank you! [dates created/updated w/GAS library in my testing, no issues]
Both of the following work fine for updating the date in C#, but either way is an issue reading back with GAS library...
`Dictionary<FieldPath, object> updates = new Dictionary<FieldPath, object>
{
{ new FieldPath("Status"), "TEST" },
{ new FieldPath("ItemNo"), spID },
{ new FieldPath("SPDate"), Timestamp.FromDateTime(dUTC) }
};

Dictionary<FieldPath, object> updates = new Dictionary<FieldPath, object>
{
{ new FieldPath("Status"), "TEST" },
{ new FieldPath("ItemNo"), spID },
{ new FieldPath("SPDate"), dUTC }
};`
Thank you,

• Jerry

the link in the readme to GSA is a dead link: https://console.developers.google.com/permissions/serviceaccounts

I think this may be the new link: https://console.developers.google.com/projectselector/iam-admin/serviceaccounts?supportedpurview=project&project=&folder=&organizationId=
(found on this Google OAuth page: https://developers.google.com/identity/protocols/OAuth2ServiceAccount)

can't do step #1 on readme.md:
Quick start
Creating a service account

1. Open the Google Service Accounts page by clicking here.

Thanks for building this. It's exactly what I needed and nice and simple to use. Great job.

The issue:
I can use createDocument with a custom document name that includes the "|" character
The document appears in the Firebase dashboard

When I use getDocument later, using the exact collection/name format, the response is the same as if the document doesn't exist

Replicated the above without "|", and getDocument was successful

Very minor issue, but perhaps illegal characters could be added to the docs?
It's bad form on my part to use that char, I was just using it while testing for visual separation
I haven't tested if the limitation extends to Collection names

from the docs..
const data = { "name": "test!", "check": true }
firestore.createDocument("FirstCollection/FirstDocument", data);
const allDocumentsWithTest = firestore.query("FirstCollection").where("name", "==", "Test1").execute();

returns no records in v18
returns a record in v17, but only if the strings match case (test! vs Test!)

// this works in both versions
firestore.query("FirstCollection").where("check","==",true).execute();

Hi grahamearley

I like the project yet it seems to be standing still, so I forked it. I was meaning to basically make the same thing but you made life easy. Are you still planning to work on this?

My goal is to make a full Firebase wrapper which works like the NodeJS FirebaseAPI you use in a cloud function.

e.g. firestore.collection('foo').doc('bar').get();

Another goal of mine is to make it all in Typescript, so I converted all your code to typescript and refactored with some of my code style. It is mainly your code though just TS and moved around, hope you don't mind.

Another goal is to get a cool CI pipeline going with Google Cloudbuild. Maybe some tests too if I'm feeling motivated enough.

Just to note in your code within this repo has comments which don't align with the code it describes. For example in the Request.js it says the method function will return this FirestoreRequest to be chained but I don't see how it is possible? This doesn't make sense to me, am I missing something?

Awesome project and thanks a lot. Would love to have your help if you're interested. If you really want I could just PR my code to you and take it from there.

Lastly as a fair warning, as of writing this issue on 9/25/2019, I have not tested my code yet, just made the conversion to TS. As it is mainly what you wrote it should work pretty soon after some debugging.

kferrone/FirebaseGoogleAppsScript

The project key is deprecated can you share the script id instead?

https://firebase.google.com/docs/firestore/manage-data/add-data#update_fields_in_nested_objects

does this work with this libary?
haven't been able to do successfully.
Updating only particular value in an object without affecting anything else.

Please make getPage method in Reads.js public so all pages can be fetched.

Is there a way to check if a document exists?

These types return null, when they could either return the string or actual referenced object (perhaps making that an option if it's not circular?).

I am using Version 15 of the script.

Here's my Firebase Test Scenario:

Running firebase.getDocuments('test'); and logging the output gave me:

[18-05-14 20:09:21:391 CDT] [{createTime=2018-05-15T01:01:30.788343Z, name=projects/myprojectname/databases/(default)/documents/test/example, updateTime=2018-05-15T01:01:55.783332Z, fields={dateVal=null, stringVal=I am a string, refVal=null}}]

Apps Script now supports source control and deployment from the command line. Can we migrate this project to the new CLASP format to enable this?

https://developers.google.com/apps-script/guides/clasp

Is the onSnapshot implemented for realtime updates?

When I create or update a document, the trigger function associated with it is not called. Please fix this.

A number of functions call get_, which in turn calls getPage_ which already does checkForError_ internally before returning the response up the call stack.

The following functions call get_ then call checkForError_ again, duplicating efforts:

• getDocumentResponsesFromCollection_

  FirestoreGoogleAppsScript/Read.js

  Lines 49 to 51 in 81989e0

  	function getDocumentResponsesFromCollection_(pathToCollection, authToken, projectId) {
  	const initialResponse = get_(pathToCollection, authToken, projectId);
  	checkForError_(initialResponse);

• getDocument_

  FirestoreGoogleAppsScript/Read.js

  Lines 128 to 130 in 81989e0

  	function getDocument_(path, authToken, projectId) {
  	const doc = get_(path, authToken, projectId);
  	checkForError_(doc);

I'm getting the above error where trying to query into a nested collection. Anyone else getting this error and how to resolve?

var runningProcesses = firestore.query("collection/doument/collection").where("key1","==","value").execute();
Logger.log(runningProcesses.length);

Currently updateDocument behaves exactly like createDocument, and will always overwrite the document if it exists, and create a new one if it doesn't.

Adding this functionality will allow users to update certain fields and ignore others. A true "patch".

Details can be found in the docs: https://cloud.google.com/firestore/docs/reference/rest/v1beta1/projects.databases.documents/patch#query-parameters

When calling updateDocument with mask=true and a big set of fields to update, i got

Error in firestore.updateDocument: {"message":"Limit Exceeded: URLFetch URL Length.","name":"Exception","fileName":"Util (FirestoreApp)","lineNumber":25,"stack":"\tat Util (FirestoreApp):25 (fetchObject_)\n\tat Request (FirestoreApp):46\n\tat Request (FirestoreApp):98\n\tat Write (FirestoreApp):48 (updateDocument_)\n\tat Firestore (FirestoreApp):87\n}

There is an hard limit in URLFetch about url lenght:
https://issuetracker.google.com/issues/35892557
(This feature request is pending since 2012)

Inspecting the problem I noticed that fields to be masked in the update are added as request params in Write.js:

Most probably my request exceeds the hard limit (in my use case, I need to update 60 over 63 fields in my doc, so the resulting url contains tens of fields).

I'm using version 22.

I think it's really important to return the document id as field when getting all the documents from a collection.

Hi,

when I get a document I have to do this to get just the id...

var result = firestore.createDocument("users", obj)
var id = result.name.replace("projects/wibble-wobble/databases/(default)/documents/users/", "")

... is the attribute lurking in the result anywhere that I can't see?

Thank you...

See title.

Is it possible to get the "fields" property without the Firebase types but a JSON object that can be directly updated and passed to updateDocument ?

Thanks

Hi there.

I have tried this library this is the result I get:

{
"error": {
"code": 403,
"message": "Missing or insufficient permissions.",
"status": "PERMISSION_DENIED"
}
}

I have picked apart the code and found the a bearer token is generated as expected.

Might I ask if we up date the docs with example rules for the Firestore DB that except a bearer token for Authentication. I have tried many different rule sets but none of them seem to match with this approach to authentication.

I looks like, at the moment, if I want to write many records at once, I have to either do a series of individual document writes. Or I have to write all the records to a single doc. It would be great to be able to create and write out to many docs simultaneously and atomically.

Hi @grahamearley ,

Thank you for making this library. I'm working on a GAS project and I'm glad that there is Firestore now (I was puzzled when I had to choose between MongoDB and Firebase Realtime Database - MongoDB is not realtime, no GAS support, while Firebase Realtime Database is totally sucked)

I checked your document but I can't find how to listen for realtime updates https://firebase.google.com/docs/firestore/query-data/listen

How can I do it with this library? Or if this library can't handle it yet, is there any way to do it manually with GAS?

Thank you.

• Read.getPage_ has mis-ordered jsdoc params 'request' and 'pageToken'
• Utils.isNumberNaN has type set as param name (ie '{value}' instead of 'value')
• Write.updateDocument_ is missing param name ('mask'), so it uses next word 'if' instead

The library currently only supports strings, numbers, nulls, lists, and maps. There are other data types possible (like timestamps or references to other documents). Implement these!

See here for where this should be implemented.

Firestore timestampValue field type uses a timestamp in RFC3339 UTC "Zulu" format. App Script Date object is unable to properly parse that format, resulting in an 'Invalid date'.

Maybe do a custom parsing in FirestoreDocument.js#L92 before returning?

First of all, I'd like to say a big thank to you for this awesome repos, it save me a lot of time.
I made a EER diagram visualization UI to visualize the table list and its dependency.
As I can see, Resource --> Advanced Google Service... does not support FireStore currently.
Google hasn't provided us a Firestore SDK for Google App Script.
I guess that using the authentication method in this repos is the only way.
.
I don't know this can be an issue or not, just place my concern here.
But expose private key is very dangerous. Is there another way to avoid placing the private key in the *.gs files?

Hi there,
I tried to add your library to my script and this is what I get:

"You do not have access to library FirestoreApp, used by your script, or it has been deleted."

Would be able to check that?

Cheers!

else if (isNaN(value)) { operator = 'nan' }

always runs even if the value is not a number

When the key is entered from - say - a googlesheet, it is then not parsed correctly, as it still contains the \n in the string.
Maybe the library could automatically parse it so, for instance:

• any \n are replaced correctly
• anything before and after the key ("-----BEGIN PRIVATE KEY" and "END PRIVATE KEY-----\n" is automatically discarded

I didn't find this feature on the project Readme, but I think it would be great to have it in the library as well. For what I'm building I need this feature

To reduce code multiplicity, we should create a base function with common code from calling the API to reduce chances of error if the calls need to be updated.

Functions that are currently making these calls:

• Read.getPage_

  FirestoreGoogleAppsScript/Read.js

  Lines 19 to 35 in 81989e0

  	function getPage_(path, projectId, authToken, pageToken) {
  	var baseUrl = "https://firestore.googleapis.com/v1beta1/projects/" + projectId + "/databases/(default)/documents/" + path;
  	const options = {
  	'muteHttpExceptions': true,
  	'headers': {'content-type': 'application/json', 'Authorization': 'Bearer ' + authToken}
  	};
  	if (pageToken) {
  	baseUrl += "?pageToken=" + pageToken;
  	options['pageToken'] = pageToken;
  	}
  	var responseObj = getObjectFromResponse_(UrlFetchApp.fetch(baseUrl, options));
  	checkForError_(responseObj);
  	return responseObj;
  	}

• Delete.deleteDocument_

  FirestoreGoogleAppsScript/Delete.js

  Lines 10 to 22 in 81989e0

  	function deleteDocument_(pathToDocument, authToken, projectId) {
  	const baseUrl = "https://firestore.googleapis.com/v1beta1/projects/" + projectId + "/databases/(default)/documents/" + pathToDocument;
  	const options = {
  	'method': 'delete',
  	'muteHttpExceptions': true,
  	'headers': {'content-type': 'application/json', 'Authorization': 'Bearer ' + authToken}
  	};
  	var responseObj = getObjectFromResponse_(UrlFetchApp.fetch(baseUrl, options));
  	checkForError_(responseObj);
  	return responseObj;
  	}

• Write.createDocumentWithId_

  FirestoreGoogleAppsScript/Write.js

  Lines 11 to 33 in 81989e0

  	function createDocumentWithId_(path, documentId, fields, authToken, projectId) {
  	const firestoreObject = createFirestoreDocument_(fields);
  	const pathWithNoTrailingSlash = removeTrailingSlash_(path);
  	var baseUrl = "https://firestore.googleapis.com/v1beta1/projects/" + projectId + "/databases/(default)/documents/" + pathWithNoTrailingSlash;
  	if (documentId) {
  	baseUrl += "?documentId=" + documentId;
  	}
  	const options = {
  	'method': 'post',
  	'muteHttpExceptions': true,
  	'payload': JSON.stringify(firestoreObject),
  	'headers': {'content-type': 'application/json', 'Authorization': 'Bearer ' + authToken}
  	};
  	const response = UrlFetchApp.fetch(baseUrl, options);
  	const responseObj = getObjectFromResponse_(response);
  	checkForError_(responseObj);
  	return responseObj;
  	}

• Write.updateDocument_

  FirestoreGoogleAppsScript/Write.js

  Lines 57 to 73 in 81989e0

  	function updateDocument_(path, fields, authToken, projectId) {
  	const firestoreObject = createFirestoreDocument_(fields);
  	const baseUrl = "https://firestore.googleapis.com/v1beta1/projects/" + projectId + "/databases/(default)/documents/" + path;
  	const options = {
  	'method': 'patch',
  	'muteHttpExceptions': true,
  	'payload': JSON.stringify(firestoreObject),
  	'headers': {'content-type': 'application/json', 'Authorization': 'Bearer ' + authToken}
  	};
  	const response = UrlFetchApp.fetch(baseUrl, options);
  	const responseObj = getObjectFromResponse_(response);
  	checkForError_(responseObj);
  	return responseObj;
  	}

• The upcoming Query.query_ https://github.com/thesandlord/FirestoreGoogleAppsScript/blob/f861f08daec96bbbc069c56ba882c3e6858dbfff/Query.js#L31-L51

I've tried...

Currently, every time you call a library function that writes or reads to Firestore, a new auth token is generated.

It would be better to have an object that you authenticate once and then can make multiple read/write calls to (e.g. if you are writing documents in a loop). We can leave the current read/write methods in the API, and just add this object as an additional feature (it can make calls to the current read/write methods).

I have several GAS Apps in production that use the FirebaseApp library and I would like to start using this library so that I can make use of the new Firestore Database. I just wanted to ask if anyone else is using this library in production and what their experience is? Is this library production ready?

It would be great if we could have the GitHub repo filled with the 15 (soon to be 16) versions that have been released as tagged versions.

@grahamearley please make this possible so anyone can checkout previous versions as necessary from the times they were released!

Thanks!

It seems like Google Apps Script doesn't provide autocomplete for methods that are nested within an object. Now that all methods are nested within the Firestore object, we should document the available methods in this repo (in the wiki?).

This operator is missing.

https://firebase.google.com/docs/firestore/reference/rest/v1beta1/StructuredQuery#Operator_1

When calling for the doc fields using getDocument, the datatype "Timestamp" from firebase returns null , so theres no way of knowing what the original value was before writing it back to firebase with update document. Let me know if there is a work around. Thanks!

I detailed my issue in this OS question.

I followed the instructions on readme and created a database, service account, and wrote the app script to create document {"name": "test!"}. However, the function returned the following error:

Error: Missing or insufficient permissions. (line 33, file "Util", project "FirestoreApp")

As far as I am aware, I have set the right permissions in my database's rules as well as the correct rights in my service account. Please advise on how to resolve this issue.

Missing runQuery functionality to execute a query on the Firestore database.

Currently, if you want to get a list of all documents (as JSON objects) from a collection, you have to

1. Call getDocumentIds(pathToCollection, email, key, projectId) to get the IDs for the collection
2. Loop through these IDs and combine the pathToCollection with the ID to make the document path.
3. Call getDocumentFields(path, email, key, projectId) for each document ID / path to get a JSON object for that document.

This should be combined into a single method.

Thanks for the awesome code @grahamearley

Can we add support for undefined fields in the query?

This is how it will be called:

FirestoreApp.getFirestore(SERVICE_ACCOUNT, PRIVATE_KEY, PROJECT_ID) .query("/path/to/collection").where("field", "==", undefined).execute()

Currently, the get method takes a path parameter and returns the JSON response for whatever is at that path — a document or a collection.

This should be made into two methods: getDocument and getCollection (which should include a list of the documents in the collection).

I have nested collections inside firestore.
Both "getDocument()" and "getDocuments()" do not seem working...

Is nested collection supported by this library?
Years-year-months-month-days-date-entries-entry
getDocuments('years') //return empty array
getDocument('years/year') //generates error saying "Document .... not found"

It working fine with
collections-document-entries

Thank you,

Until Yesterday, the library works perfect, but today I got an error when connecting to FirestoreApp.

In GAS script, these following functions calling can cause error.

Logger.log(FirestoreApp)
Logger.log(FirestoreApp.getFirestore(config))

Firstly, I doubt connection error when authenticating with firebase, but it wasn't. It might be some changes in firebase API, though I could not specify the causes.

Can this project be used to sync continuously between hundreds of Google spreadhseets and Firestore.
I would like to use Firestore as a backend for my data and Spreadsheets to collaborate and visualize and edit the data.

Does it make sense?