Consider caching of modelInfo for each type about storm HOT 9 CLOSED

Thanks for looking into it, I appreciate the effort.

from storm.

I like our fist profile pics :-) Thinking men, heavy heads that need support ...

from storm.

I agree, it would be great to find a way to cache modelInfo by currently i don't see how we could cache fields. If you have an idea, feel free to suggest

from storm.

I don't have a lot of experience with Go reflect, but it seems that the metadata (e.g. StructField) needs to be separated from the Value. So you would build a model about the metadata of a struct type (and cache it in a map[reflect.Type]model, e.g. what field is the PK, what fields are indexed, unique constraints, etc. and keep just the field name (and perhaps type if needed). Then when an instance is saved, you pull the actual field values from the reflect.Value using Value.FieldByName()
Due to so much refelection also happening when you do the gob/json encoding, this may not buy us that much, hard to tell without benchmarking it.

from storm.

Caching those values would mean saving them in a bucket, which means that we need to decode them every time with implies reflection...

from storm.

We don't persist them, we just cache them for the duration of the process, in memory. Reflection to cache the meta-data is done only once per type and cached in a map[reflect.Type]model. May need to be smart about T and *T, but that's just an impl detail.

from storm.

@bits01 @asdine I agree. #50 is now moot. More than one way to skin a cat.
Happy to see reflection was dropped. In go its a heavy hit, whereas in other language now where near as heavy at runtime. So great to see this i feel.

That generic lib is not used by pretty much anyone it seems, based on godocs cross references.
There are other libs for doing codegen like this for go to replace generics, that support more type structure out there. But none of them actually advertise directly implementing generics, but rather just doing code gen.

This one is very well used:
https://clipperhouse.github.io/gen/

Its at V4 and been used a fair bit in anger..

Have used it on many project, and might fit the bill for Storm..
Also it implements many expressions / predicates that are very SQL like.

Are you still considering using goa as a the master meta data store to drive things, because its possible that it could be used to drive this Storm layer ?

from storm.

@bits01 I have tried to cache modelInfo but the cache has to be updated on every new structure, which takes more time than using extract directly, approximately twice the time for twice the memory.
The idea is good but the execution is difficult and in our case doesn't work unfortunately

I am closing for now, please don't hesitate opening the issue if you have a better idea.

from storm.

from storm.