agrimatics / activemodel-datastore Goto Github PK

We need the ability to exclude indexes on individual entity properties. For example, we can't currently create a string property that exceeds 1500 bytes.

The gcloud client supports this:

entity.exclude_from_indexes! :content, true # This allows a string up to 1,048,487 bytes.

I can't seem to figure out if it's supported to use embedded entities? In case it is, how would the DSL for assigning such a property on a record look like in an example?

We need to review if we should still be retrying the save, update and destroy methods.

Also review if it is still necessary on any Lookup or RunQuery. The GAX client does provide a retry mechanism as defined here.

For a datastore Lookup, since it has a retry_codes_name defined of "idempotent", if an UNAVAILABLE error was raised, it would retry using the default params.
For a datastore Commit, since it has a retry_codes_name defined of "non_idempotent", any error raised would not be retried.

It provides an exponential back-off for the desired error codes with a max overall timeout (so it won't retry forever) but also allows a force retry after a specific time.

"initial_rpc_timeout_millis": 5000
(the initial timeout parameter to the request)
Times out if a response is not initially received within 5 sec.

"initial_retry_delay_millis": 500
(the initial delay time, in milliseconds, between the completion of the first failed request and the initiation of the first retrying request)
Waits 0.5s and automatically retries the request.

"rpc_timeout_multiplier": 1.0
(the multiplier by which to increase the timeout parameter between failed requests)
Would wait 5 seconds after the retry before timing out.

"retry_delay_multiplier": 2.0
(the multiplier by which to increase the delay time between the completion of failed requests, and the initiation of the subsequent retrying request)
The request will keep being retried with an exponential back-off (0.50, 1, 2) waiting 5 seconds for each request until it succeeds or finally:

"total_timeout_millis": 17000
(the total time, in milliseconds, starting from when the initial request is sent, after which an error will be returned, regardless of the retrying attempts made meanwhile)

I was wondering if there is a built in way to do this, and/or what the best/proper way would be. Would love it if there was syntax like:

validates :email, uniqueness: true

We currently support the parent_key_id attribute as an integer id. We should add support for also using a parent key string name. Something like id_or_name.

Example:

def build_model(entity)
  model_entity = new
  model_entity.id = entity.key.id unless entity.key.id.nil?
  model_entity.id = entity.key.name unless entity.key.name.nil?
  if entity.key.parent.present?
    model_entity.parent_key_id = entity.key.parent.id if entity.key.parent.id.present?
    model_entity.parent_key_id = entity.key.parent.name if entity.key.parent.name.present?
  end
  model_entity
end

Cloud Datastore supports multi-tenancy through entity namespaces. This allows for logical separation of content: in our case, customer content. Currently, entities created using Active Model Datastore are created in the default namespace exclusively.

See the following for more information:

Multitenancy
https://cloud.google.com/datastore/docs/concepts/multitenancy

Entities, Properties, and Keys
https://cloud.google.com/datastore/docs/concepts/entities

I would like to be able to change the entity name in datastore
similar to table_name in active record.

Google even provides it: https://hub.docker.com/r/google/cloud-sdk/

Would that be something that was interesting to setup? Running tests on CI would be fairly easy using that.

Example:
query.select(*options[:select]) if options[:select]

See here:
https://cloud.google.com/datastore/docs/tools/indexconfig

GAE Production authentication appears to be overridden with hardcoded emulator settings. The initialization code in lib/active_model/datastore/connection.rb overrides the GCLOUD_PROJECT environment variable with the test datastore project and sets DATASTORE_EMULATOR_HOST regardless of the existing environment. The latter forces emulator usage.

module CloudDatastore
  if defined?(Rails) == 'constant'
    if Rails.env.development?
      ENV['DATASTORE_EMULATOR_HOST'] = 'localhost:8180'
      ENV['GCLOUD_PROJECT'] = 'local-datastore'
    elsif Rails.env.test?
      ENV['DATASTORE_EMULATOR_HOST'] = 'localhost:8181'
      ENV['GCLOUD_PROJECT'] = 'test-datastore'
    elsif ENV['SERVICE_ACCOUNT_PRIVATE_KEY'].present? &&
          ENV['SERVICE_ACCOUNT_CLIENT_EMAIL'].present?
      ENV['GCLOUD_KEYFILE_JSON'] = '{"private_key": "' + ENV['SERVICE_ACCOUNT_PRIVATE_KEY'] + '",
      "client_email": "' + ENV['SERVICE_ACCOUNT_CLIENT_EMAIL'] + '"}'
    end
  else
    ENV['DATASTORE_EMULATOR_HOST'] = 'localhost:8181'
    ENV['GCLOUD_PROJECT'] = 'test-datastore'
  end

  def self.dataset
    @dataset ||= Google::Cloud.datastore(ENV['GCLOUD_PROJECT'])
  end
...

Maybe I'm missing something but it seems the model should not override the environment and simply allow Google::Cloud.datastore to initialize using the existing environment per the Google Cloud Client Library convention.

See: https://olivierlacan.com/posts/changelogs-on-rubygems-org/

I noticed that the dirty tracking "is to be documented" in the readme. I played around with it and noticed to my surprise that changes are not wiped after a successful save/save! is it intentional that one would have to manually call reload! on the objects to reset the change tracking after a save to data store?