Comments (12)
On App Engine and Compute Engine, a request is required to fetch the project/app name. It will make the existing codebase complicated to maintain if we do this lazily before request time.
Two options:
-
emit a 'ready' event:
var ds = datastore.dataset(config.appengine(context)); ds.on('ready', function() { /* do stuff here */ });
-
make config creation async:
config.appengine(function(conf) { var ds = datastore.dataset(conf); });
I'd prefer we follow the first option for all. Any ideas?
from google-cloud-node.
+1 for first option
from google-cloud-node.
First looks good, but not useful. You basically want to get done with configuration for once and reuse it for multiple services.
config.compute(function() {
// start server and don't care about config anymore
gcloud.datastore.dataset.get(...)
});
This approach is not ideal if you want to authorise for multiple accounts/projects within a single process. We need to avoid global vars and need to pass some configuration object to service constructors which is likely to confuse a user.
config.compute(function(err, conf) {
gcloud.datastore.dataset(conf).get(...);
});
IMHO, requiring multiple account support is not edgy. For cases where user is migrating data from one project to another, or for running backup jobs, it's a necessity.
from google-cloud-node.
Just so I can get my head around this, can we update to show a "before and after" type view?
It seems that something like config.compute(...)
would lead to different code depending on the environment (correct me if I'm misunderstanding...). I'd like to avoid that if possible.
Using an example, let's say I wanted to get something from GCD. The code I'd like to write is something along the lines of...
var gcloud = require('gcloud'),
datastore = gcloud.datastore;
gcloud.authenticate();
entity = datastore.get(...);
That authentication line would ideally do a few things:
- Since we know this is running server-side, we should focus on service accounts (I don't think 3-party auth will be very common).
- If the user is explicit, try what they intended, raise an exception otherwise:
- Specific configuration passed in as code (ie,
gcloud.authenticate(<something here>)
) - Environment variables overridden (ie,
DATASTORE_AUTH_JSON
or something)
- Specific configuration passed in as code (ie,
- If the user is not explicit, look at the environment, and try our best to authenticate. Raise an exception only after none of the options work.
- Try GCE with the appropriate scopes (userinfo and datastore in this case)
- Try GAE production
- Try GAE local dev_appserver
- Raise an exception with the log of all things not working.
- Regardless of whether the user is explicit or implicit, I'd like to have introspection methods to see how exactly we're authenticating. That is:
gcloud.get_authentication_method()
- Logging when we first authenticate of what we tried, what failed, and why.
from google-cloud-node.
It seems that something like config.compute(...) would lead to different code depending on the environment (correct me if I'm misunderstanding...). I'd like to avoid that if possible.
True, but only for the configuration step. It's not ideal but falling back to other options (GCE and GAE) confuses people who has never heard of GCE's metadata server. The final fallback option returns the final error message (DNS error for http://metadata). Therefore, I thought user should be explicit about the environment if they are a GCE or GAE user.
Auth layer should also resolve the project ID if it's running on GCE, that's why we can't have gcloud.authenticate()
synchronous. It has to look like:
gcloud.authenticate(opt_explicitConf, function(err) {
// start writing code here
entity = datastore.get(...);
});
(We can't lazily retrieve project ID during request time. If we do, codebase becomes unnecessarily complicated.)
The flow should be very similar to what you've said above. I'm adding a few other details to the 3rd step.
GAE local server is detectable by an env flag. We will be able to determine if it's local GAE.
i. If GAE local flag is set, use local server and don't authenticate.
ii. Else, try GCE with scopes (GAE prod = GCE).
iii. Return error if nothing has worked.
from google-cloud-node.
I've read through this a few times, and I'm still trying to understand the wires between our code and the various engines, but I thought I'd make a suggestion that hopefully isn't entirely useless, using @jgeewax's code example as a basis.
While you're still considering APIs to go with, I just wanted to mention the Promise system can sometimes ease the pain of having "this before this before this" conditions.
The object passed to gcloud.authenticate
can include or omit various properties that are relevant for the intended environment, which would define an implicit connection type that authenticate
can use however it needs. The chained success
and error
functions are only executed after gcloud.authenticate
has triggered them, based on the results of the operations, handing them whatever bits of data it wants.
Here would be an example of creating a server that listens for a ping to api/users/:id
, which then responds with the json object dump from the datastore.
./index.js
var gcloud = require('gcloud');
var createServer = require('./server');
gcloud
.authenticate({ /* ... */ })
.then(createServer, function handleError() { /* ... */ });
./server.js
var http = require('http');
var dataset;
module.exports = function (gcloud) {
dataset = new gcloud.datastore.dataset({ projectId: 'MyApp' });
createServer();
};
function createServer() {
http.createServer(function (req, res) {
if (req.url.indexOf('/api/user') === 0) {
getUser(req.params.id, res);
return;
}
});
}
function getUser(id, res) {
dataset.get(['Users', id], function (err, res) {
res.end(JSON.stringify(res.data));
});
}
This is obviously a made-up scenario, but if we can integrate promises in a similar way, the consumer gets the benefit of DI -- so their server.js
(or whatever makes sense for their use) module doesn't have to care about striking a connection, and can just proceed knowing it has everything it needs. Just a nice perk 👍
Hopefully this applies at least in some small way. If you see that I'm not grasping something, please let me know!
from google-cloud-node.
I am hugely in favor of something like this...
gcloud.authenticate({...}).then(function() {
...
})
would be really nice.
from google-cloud-node.
Updated to use then
👍
from google-cloud-node.
We don't support promises, I see no reason why we should support promises during auth.
gcloud.authenticate({ /* ... */ }, createServer);
is much slicker and not giving a bad impression that gcloud has promises support.
from google-cloud-node.
gcloud.authenticate({ /* ... */ }, createServer);
works just as well for me.
from google-cloud-node.
I've slept over this issue and decided to close it (for a while). We should always require users to set a project ID regardless of which context they run gcloud (at least, in the scope of M1 release). Use a JSON key file for service account authorization, otherwise we'll fallback to metadata server to retrieve an access token if no key file is set.
There is no significant value added if we resolve their project IDs, yet users will need to handle gcloud.authenticate
.
Currently, this is how we do auth:
// works everywhere
var ds = gcloud.datastore.dataset({
projectId: 'bamboo-45-lol',
keyFilename: '/path/to/keyfile.json'
});
ds.get(...)
// works on GCE
var ds = gcloud.datastore.dataset({
projectId: 'bamboo-45-lol'
});
ds.get(...)
// works on GAE
var ds = glcoud.datastore.dataset({
projectId: <appEngineAppName>
});
ds.get(...)
// i may add GAE local server support
// if process.env.SERVER_SOFTWARE === 'Development'
I want to focus on GAE's Node runtime this week to see if we have any problems. Maybe we should include gcloud in their runtime module and doesn't require GAE specific bits in this one.
// cc @rrch
from google-cloud-node.
App Engine exposes project ID as GAE_LONG_APP_ID
env variable. We can go from there, we don't need to ask metadata server.
from google-cloud-node.
Related Issues (20)
- Documentation links in @googlemaps/places don't work HOT 2
- How to pull stats from a google cloud tasks queue that has finished?
- Permission Error When Executing Vertex AI Fine Tuning Job with Node.js HOT 2
- @google-cloud/procurement is missing most API methods
- Breaking change in minor release of @google-cloud/secret-manager 5.4.0 HOT 2
- Google documentation headline missing arguments HOT 1
- @google-cloud/secret-manager SecretManagerServiceClient.secretVersionPath throws a type error HOT 3
- [Dialogflow] [Reminder] Start using "locations/global" resource location identifier for Dialogflow ES and CX
- Stackdriver Monitoring v4.0.0 raising errors when try to write TimeSeries
- Warning: a recent release failed HOT 1
- [Google Maps Places API] - Text Search missing next_page_token in response HOT 4
- Score is always 0.8999999761581421, even for automated tests in @google-cloud/recaptcha-enterprise HOT 5
- storage-control: Add HNS folders samples HOT 1
- d HOT 1
- Deadline Exceeded (30s) for Vertex AI Image Generation Predict Endpoint / Need Configurable Setting HOT 3
- Configuration error for release-please
- storage-control: Samples for Managed Folders HOT 2
- How to enable server certificate verification on the Nodejs clients like MetricServiceClient/KeyManagementServiceClient for Mutual TLS ? HOT 4
- Warning: a recent release failed
- Speech: Unhandled exception when reusing a closed client HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from google-cloud-node.