Consider using a DSL that generates code instead of reflection about storm HOT 14 CLOSED

As discussed here, i believe this should be something outside of Storm, in another repo, the same way as gorma is for gorm

from storm.

But this is not about a query language, it's about saving/loading objects via generated code rather than reflection. I don't see how a DSL on top of storm would avoid reflection when saving/loading objects if the core storm does reflection anyway, it would just be a completely different project that completely avoids reflection.
I'm suggesting that the storm implementation changes from using reflection to using generated code instead.

from storm.

Let's say i have this simple file:

package main

import (
    "log"
    "time"

    "github.com/asdine/storm"
)

type User struct {
    ID          int    `storm:"id"`
    Name        string `storm:"index"`
}

func main() {
    db, err := storm.Open("my.db", storm.AutoIncrement())
    if err != nil {
        log.Fatal("db open:", err)
    }
    defer db.Close()

    user := User{Name: "John"}

    if err = db.Save(user); err != nil {
        log.Fatal("save user: ", err)
    }
}

How would using generated code instead of reflection work?

from storm.

A DSL may not help much if #51 is implemented as something like the stdlib gob or json encoding will still end up doing reflection to serialize the object. Generated code would help if storm had a default encoder that used generated code instead of reflection (generated code can just access the fields of struct instances directly and turn them into a []byte).

Depending on the application this may or may not make a difference.

from storm.

What would be the generated encoder code for the example above?

from storm.

You would not start with the User struct, but with the DSL which would generate the User struct with a custom encoder/decoder code that doesn't use reflection. The advantage I think is that you could customize the User struct so that you can still have volatile exported fields that are not persisted for example, an ignore tag if you will, which is not easily possible today as the gob encoder for example serializes all exported fields unless you provide a custom GobEncode/Decode impl.
The indexes, constraints, validators would also be specified in the DSL which is more powerful than tags. But I suppose at this point it just becomes a completely different project :-)

from storm.

At this stage i am with @bits01 on this one :)

TO help this discussion have a look at this. Its all code gen, and zero reflection.
https://github.com/goadesign/goa-cellar/blob/master/design/api_definition.go

https://goa.design/
The docs are excellent too.

I have used goa on 3 projects now.

Its also possible to make a DSL and codegen, after the refection based approach is cooked.

TO extend from what was already said about code gen and reuse:

• code gen the JS Or gopherjs code
• code gne the CLI and bash completion (https://github.com/goadesign/goagenfish)

from storm.

What would be the generated encoder code for the example above?

But none of you did answer my simple question.

from storm.

i dont get why an encoder comes into it.

from storm.

The encoder doesn't really matter as long as it doesn't use reflection and it's just efficient generated code.

from storm.

I get the idea but i still can't visualize how the end user would use it. If you have a realistic (even if not perfect) example (code or cli) please post one

from storm.

Let's say the developer needs a User struct. You do not start by writing the struct type. Instead, you start with the DSL (let's assume goa style). In the DSL itself you define your User type, what attributes it has, which is the PK, indexes, ignored attributes that don't get persisted, etc. Then you run goagen gen with the custom storm generator which goes and creates the User struct Go code with fields and indexes based on the DSL and with custom Encode()/Decode() methods that an efficiently serialize/deserialize to/from []byte without using reflection. You never edit User by hand, you alway modify the DSL and re-generate the code as the User definition evolves. The User type is then available to the rest of the code to use normally, it's just not written by the developer, it's generated code.

Have a look at the goa and gorma examples. Just throwing some ideas out there. Nothing wrong with the current storm implementation, it's just an alternate approach for applications that may need high performance.

https://github.com/goadesign/goa
https://github.com/goadesign/gorma

Note that goa is extensible, custom DSLs can be added. I currently use it in a project to generate a REST API and it would be useful to also use it for the persistence/model layer (that's the idea behind gorma but for "traditional" RDBMS/SQL).

from storm.

Thank you all for your advices, a new issue #72 has been dedicated to think about additional methods to avoid reflection and to help code generation. Any help would be appreciated

from storm.

Closing this issue, please reopen if you have any suggestion

from storm.