switchupcb / copygen Goto Github PK

Problem

Using types that are cyclic (A contains A, A contains B contains A, etc) can result in multiple fields being created at the parser level. These are matched accordingly which results in duplicate assignments. We can use the field cache to ensure cyclic types are not searched at an infinite level.

Fix

The parser will use a field-level cache during field search to ensure cyclic types are recursed. When deepcopy is supported, we can generate a for-loop to assign properties (until one of them is nil) as opposed to stopping at the cyclic type (in a shallow copy). In this way, the user will be provided an easy way to manage how cyclic types are dealt with; in addition to templates.

Copygen allows developers to define types, structs, and more in its setup.go file and has no issues copying them to the output; except for comments. The Go AST makes it "very difficult to update the AST and retain correct comment place (from 2017)" and is not likely to change any time soon. As a result, Copygen sometimes generates free-floating comments (i.e multi.go). Copygen's comments are handled in its Parser. Feel free to use any necessary libraries or rewriting of the comment-parsing mechanism to fix this issue.

Problem

Lines that call x.Obj().Pkg()... on certain types with no package will result in a crash since their go/types Pkg is nil. In addition, types with no package have . appended in the generated parameter list.

Fix

setFieldImportAndPackage can be modified to accept the types.Package and check if a package is set accordingly. ParameterName can be modified to support type fields with no package. The generate template can be refactored to support casting.

Problem

Specifying functions in non-alphabetical order can result in comment-options being mismatched to function options.

Reason

The go/types Interface.Method() function returns the "i'th method of interface t ... ordered by their unique Id.". In contrast, Copygen option-comments are parsed by AST in order of declaration. As a result, assigning using i (where i is 0) results in the comment for the first declared function being assigned to the first alphabetically ordered function, and so on and so forth for subsequent positions (1,2,3,...). These are NOT necessarily equivalent which results in the issue where a function is assigned a mismatched option.

A side effect of this behavior is that Copygen always outputs functions in alphabetical order.

Solution

There are two possible solutions:

1. Do NOT use alphabetical order (which entails finding a different method to iterate over go/types interface methods). This will change the Copygen output to be non-alphabetical as opposed to the non-intended alphabetical order.
2. Keep alphabetical order and re-work comments to be assigned alphabetically. This will maintain the current alphabetically-ordered function output; albeit unintended.

Implementing 1 entails setting i to the declared position and modifying the order of access to the go/types Copygen interface. Implementing 2 entails setting i to the "unique" (alphabetical) position and modifying the order of access to the ast Copygen Interface.

Problem

Using a setup file with third party import can result in failure to identify the import a type is located in. This happens because — unlike the actual loading of the package — we assume a few rules regarding imports and aliases. This works for the stdlib and straightforward imports, but not imports which include versioned folders such as .../v4 whose package names aren't actually v4 and don't contain an alias. Instead, a method using the packages module should be implemented (as it is how Go loads modules).

Fix

When we attempt to locate the file a type is declared in, we only have a types definition (i.e log.Logger) and the imports of the setup file. Using the packages method will eliminate import edge-cases by using a single source of truth. This will ensure errors only occur when people use packages incorrectly and reduce maintenance. The tradeoff is a slight decrease in performance due to the packages module.

1. Load the files in the setup file's imports using packags.Load().
2. Iterate over each file's AST (Decls) until a type's declaration is found.

Changes

This will eliminate the need for astLocateImports and the package load in execute (of the parser) as well as the need to identify any imprts. Instead, the parser's execute can accept an *ast.File and typename to deduce all necessary field information.

Reused types are already cached (per function). If the process of loading packages is slow, we can implement a cache which keeps track of the files loaded per import. Each cache can be implemented at the parser-level. The package loader can be implemented using the definition's package, but this can vary for the same package (where a custom subfield of a custom type has no definition). This is solved by using the same cache files if we used a package definition for the last type and the current type has no definition but is a custom type.

like this
type Source struct {
Name string
Children []*Source
}

type Dest struct {
Name string
Children []*Dest
}

Problem

Copygen assumes that the user outputs to a package that does not contain imports in the setup file. As a result, using the following example results in invalid output.

// Package requests contains the setup information for copygen generated code.
package requests

import (
	"github.com/switchupcb/disgo/wrapper/requests" // setup file imports package it will output to
	"github.com/switchupcb/disgo/wrapper/requests/responses"
	"github.com/switchupcb/disgo/wrapper/resources"
)

// Copygen defines the functions that will be generated.
type Copygen interface {
        // setup file uses package for field, but in the output this package isn't necessary
	SendGetGlobalApplicationCommands(*requests.GetGlobalApplicationCommands) ([]*resources.ApplicationCommand, error)
}

Fix

Allow the user to do this (extreme edge case) by modifying the field's package - at some point in the parser - based on the setup file's imports. We already substitute package names with aliases so we only have to modify how the aliasImportMap is handled. Basically this issue is solved by detecting when an import uses a package that the setup file is in, then not assigning packages to the field's of the objects defined in the setup file AND the same package-import.

Version 0.4 will require the following changes:

• refactor text/template to match equivalent output to generate.go,
• fix free floating comments (i.e multi).
• exclude example packages from build (go get and go install).
• Fix github actions Go modules caching.

Status
Not Started.

Hey there! There is a failure when using aliased imports for certain model packages (see below). Removing the alias removes the shown error as well, albeit the conversion still doesn't work or me 😅 .

I believe we make sure aliased imports work correctly, since many services have the same package name for their models

Setup

YML

generated:
  setup: ./models/copygen.go
  output: ./models/mappers_gen.go

Go

THIS DOESN'T WORK

package models

import usermodels "github.com/..blahblah../users/models"

type Copygen interface {
	ModelsToDomain(usermodels.User) User
}

THIS DOES

package models

import "github.com/..blahblah../users/models"

type Copygen interface {
	ModelsToDomain(models.User) User
}

Output

No output

Error

an error occurred while parsing the types of function "ModelsToDomain".
an error occurred while searching for the top-level Field "User" of package "usermodels".
the type declaration for the Field "User" of package "usermodels" could not be found in the AST.
Is the imported package up to date?

Generation

paste any generated code here.

Environment

Operating System: OSX
Copygen Version: 0.2.1

go install: github.com/switchupcb/copygen@latest (in github.com/switchupcb/[email protected]):
The go.mod file for the module providing named packages contains one or
more replace directives. It must not contain directives that would cause
it to be interpreted differently than if it were the main module.

Caching

go/types string provides a struct in the following format: struct{ID int; Name string; Password string; Email string}. The issue here is that a user can define two types (in different packages) with the exact same name and fields (i.e Account) and the string will resolve to the same go/types string. In addition, each go/types type is a different object, regardless of its definition.

Problem

In most practical cases, this isn't an issue. A user will likely copy two types with the same name, but different fields. However, when this is not the case, checking for cyclic activity doesn't work. This means a user can run into stackoverflow while parsing a struct (interface and/or func) that is defined with similar fields (in a single setup file) regardless of its name or package.

Fix

There are two issues we must solve whenever we cache fields.

1. A unique key per type definition (using information from go/types).

2. Keeping the cache valid: For example, setting the cache with a field, then filtering that field for depth will result in the field being modified and no longer valid.

1

Adding the go/types package to the cache's key will prevent issues where similar structs or interfaces result in an invalid cache hit. However, this does not prevent two types in the same package (and import) from incorrect caching (with the same fields). In order to correctly cache a field, we need the actual name, package, and import of the struct, interface, or name, provided by *types.Named.

In contrast, basic and simple composite types do not inherently have names; only when they are struct fields. As a result, basic and simple composite type go/types String()s can be used to provide cached fields - that should be cached without a name, parent, or package - and have these values (including variable name) explicitly set in structs. This is done by moving the responsibility for the assignment of variable name and field name to the fieldParser struct.

Cache needs to be checked in case Named and when go/types string isn't a struct or interface. This is done by implementing the caching method explained in the previous paragraph.

if type.String() != struct {
    // check cache of type string
    // set name, variable name using fieldParser
}

// in the Named case
if struct or interface {
    // check cache of type string + name + importpkg
    // or in case of cache fail
    // return underlying with set name, variablename of fieldParser
}

A named func is not considered a type, and thus invalid.

The final issue lies with cyclic fields, where a field must be parsed before it can be deepcopied to the cache, but waiting to do this means inevitably waiting on the other cyclic field, which continues until a stack overflow. An intuitive flow of the program (for cyclic support) works like this:

// check for cached field and return
// otherwise parse field
// store field prior to parse
    // parse subfield
    // calls main parse which will
        // a. Find cyclic in cache and return pointer
        // b. Parse a new field
// pointer field is eventually completed; which also updates the cyclic pointer

If we want to use field caching for performance, we require:

// check for cached field and return
// otherwise parse field
    // parse subfield
    // calls main parse which will
            // a. Find cyclic in cache and return a pointer (that is eventually deepcopied).
            // b. Parse a new field
// store completed field as a deepcopy

In the intuitive flow, assigning a cyclic type is deceiving: Simply point the cyclic type to the cached field, right? Wrong. Using multiple of the same cyclic type means you will assign multiple unassociated fields to a single cyclic type, which complicates the matcher when AllFields is inevitably called OR when a user selects the parent of a cyclic type expecting a unique field, but receiving the universal cyclic type.

As a result... We are forced to use field caching assignment. The complex part of this method is the following: When you store a deepcopy (of an incomplete field), you will receive a deepcopy of an incomplete field back. As a result, we must set cache (and cyclic) hits to a pointer that is eventually deepcopied. This entails checking the cache up front, then defering the assignment of a pointer (to a deepcopied field).

// check for cached field
    // defer assignment of new pointer after cached field is stored.

// otherwise parse field
    // parse subfield
    // calls main parse which will
            // a. Find cyclic in cache and return a pointer (that is eventually deepcopied).
            // b. Parse a new field

// store completed field as a deepcopy
// deferred statement is called and pointer is assigned with a deepcopy

You can't really defer an assignment so... But before we even figure that out, there is one other issue. Deepcopying a cyclic field's fields implies that it's field's eventually point to a cyclic field that... This issue is already solved and implemented in other methods.

How do we defer an assignment to a pointer?

You can't. However, we can manage the "immutable" cache (discussed in the next section). Instead of checking the cache for a valid field, we can always build a field from the cache (using an additional field to identify cyclic types). Then, when we get a deepcopy of the field, it will always be complete. Essentially, we always return a deepcopy of the top-level typefield (using cached subfields to speed up the process).

// if field is cached, create deepcopy in cache, point it, return it, then name the copy
// else; create incomplete entry
    // parse field pointer (from cache)
        // parse subfield
        // calls main parse which will
            // b. Parse a new field
            // a. Find type in cache 
                // point subfield to main field
                // name accordingly
                // substitute if cyclic with incomplete entry

    // return deepcopy of cached field
    // which is named accordingly

FlowExample(string) account

1: [cached string], deepcopied string
2:
account [incomplete cached]
    deepcopied string [to cache]
    [cached int], deepcopied int [to cache]
    cyclic ->
3:
account [incomplete cached]
    deepcopied string [to cache]
    deepcopied int [to cache]
    cyclic [incomplete cached]
        deepcopied string [to cache]
        deepcopied int [to cache]
        account deepcopied with Fields[0], renamed, fields redirected to account.Fields, an parent to cyclic
4:
account [incomplete cached]
    deepcopied string
    deepcopied int
    cyclic [complete cached], deepcopied
        string
        int
        account
5:
Account is cached, then deepcopy (of entire tree) is returned as a typefield. 

The cache has a size of 13 fields: 1 string, 1 int, 1 cyclic (with 2 fields), 1 real account (with a cyclic deepcopy +4, a fake account, and 2 fields). A total of 6 fields return from the parse when the account is deepcopied or encountered again.

The size of a field is 192 bytes so 1 GB of RAM (1073741824 bytes or ~1 billion bytes) holds up to 5.6 million total fields or ~932067 account parses after the first one.

2

Instead of caching the field directly, we must maintain a deepcopy of the field for re-assignment of other fields. There are many ways to do this but it's often complex. One way to simplify this is by setting options at the function level (when all field's have been parsed) instead of the field level.

Using an "immutable" cache allows us to always stores or get a deepcopy of a field (and it's fields). In this way, we can return a deepcopy of a field (and set it's parent explicitly) on each cache hit and modify that field without creating other implications in the program.

Tests

Add a duplicate cyclic type in cyclic to test the "intuitive logic". Programmatic run of tests using a global cache that is not reset ensures everything works correctly.

I am able to use map for mapping a field within model to struct inside domain directly but doing the same via a convert does not work.

I also tried using a pointer to the field (as argument to the convert function) but it is still of no use.

On a side note, I am using this as an alternative to mapstruct in Java (https://mapstruct.org/)

Domains&Models

type SubA struct {
 SomeString1 string
 SomeInteger2 int64
}

type Domain struct {
 Sub *SubA
}


type Model {
 SomeString1 string
 SomeString2 string
}

YML

generated:
  setup: ./mapping.go
  output: ../gen/mapper.gen.go

Setup Go

type Copygen interface {
	// map Model.SomeString1 Domain.Sub.SomeString1
	Mapper(model *Model) *Domain
}


// convert Model.SomeString2 Domain.Sub.SomeInteger2
func StringToInt64(s string) int64 {
	v, err := strconv.Atoi(s)
	if err != nil {
		v = 1
	}
	return int64(v)
}

Output

Generation

package copygen

import (
	"strconv"
)

func StringToInt64(s string) int64 {
	v, err := strconv.Atoi(s)
	if err != nil {
		v = 1
	}
	return int64(v)
}

// Mapper copies a *Model to a *Domain.
func MapAvailableIndexToIndexCard(tS *Domain, fS *Model) {
	// *Domain fields
	tS.Sub.SomeString1 = fS.SomeString1
}

Environment

Operating System: MacOS
Copygen Version: v0.4.0

Please provide the following information.

Setup

YML

generated:
  setup: ./setup.go
  output: ../copygen.go

Go

// setup.go

package setup

import (
	"github.com/switchupcb/copygen/examples/oboe/domain"
	"github.com/switchupcb/copygen/examples/oboe/model"
)

type Copygen interface {
	ModelsToDomain(model.Adgroup, model.Campaign) domain.Adgroup
}

// domain.go 

package domain

type Adgroup struct {
	ID   int64
	Name string
	Planning Planning
	Promotion Promotion
	ResourcePos ResourcePos
}

type Promotion struct {
	PromotionType int32
}

type Planning struct {
	PlanningID string
}

type ResourcePos struct {
	PlacementIDs []string
}

// model.go

package model

type Adgroup struct {
	ID           int64
	Name         string
	PlanningID   string
	PlacementIDs []string
}

type Campaign struct {
	PromotionType int32
}

Output

Error

Generation

// Code generated by github.com/switchupcb/copygen
// DO NOT EDIT.

package setup

import (
	"github.com/switchupcb/copygen/examples/oboe/domain"
	"github.com/switchupcb/copygen/examples/oboe/model"
)

// ModelsToDomain copies a Adgroup, Campaign to a Adgroup.
func ModelsToDomain(tA domain.Adgroup, fA model.Adgroup, fC model.Campaign) {
	// Adgroup fields
	tA.ResourcePos.PlacementIDs = fA.PlacementIDs
	tA.ResourcePos.PlacementIDs = fA.PlacementIDs
	tA.Promotion.PromotionType = fC.PromotionType
	tA.ResourcePos.PlacementIDs = fA.PlacementIDs
	tA.ResourcePos.PlacementIDs = fA.PlacementIDs
	tA.Promotion.PromotionType = fC.PromotionType
	tA.Planning.PlanningID = fA.PlanningID
	tA.Name = fA.Name
	tA.ID = fA.ID

}

Environment

windows: windows
Copygen Version: v0.2.4

Please provide the following information.

Setup

YML

# Define where the code will be generated.
generated:
  setup: ./setup.go
  output: ./copygen.go

  # Define the optional custom templates used to generate the file (.go, .tmpl supported).
  # template: ./generate.go
# Define custom options (which are passed to generator options) for customization.
# custom:
#   option: The possibilities are endless.

Go

package copygen

import (
	"demo/admin/adapters/reposit/dal"
	"demo/admin/domain"
)

type Copygen interface {
	Basic(user *domain.AgrUser) *dal.SysUser
}

Output

Error

the "type Copygen interface" could not be found in the setup file

Generation

Environment

Operating System: windows 10
Copygen Version: 0.4.0
Golang Version : 1.19.4

Please update examples and documentation

I like the concept of copygen, where in theory the generated could approach ~20x faster than a json.Marshal -> protojson.Unmarshal sandwich (and infinitely faster than jinzhu/copier).

However, when using on structs that aren't flat with concrete builtins, things seem to go a little sideways.

Is there something I'm missing on how to handle non-concrete builtin types?

Setup

By default, Created will be ignored because its name/type combo don't match between the structs, which makes sense.

However, with tag .* json (which is required in my case) or adding a type override, copygen performs unexpectedly:

YML

generated:
  setup: ./gen.go
  output: ./generated/generated.go

Go

package copygen

import (
	"time"

	"google.golang.org/protobuf/runtime/protoimpl"
	"google.golang.org/protobuf/types/known/timestamppb"
)

type Copygen interface {
	EntityToPB(E *Entity) *EntityPB
	EntityToPBCreated(E *Entity, Created *timestamppb.Timestamp) *EntityPB
	// tag .* json
	EntityToPBTag(E *Entity) *EntityPB
	// tag .* json
	EntityToPBCreatedTag(E *Entity, Created *timestamppb.Timestamp) *EntityPB
}

type Entity struct {
	Created *time.Time `json:"created,omitempty"`
	Name    string     `json:"name,omitempty"`
}

type EntityPB struct {
	state         protoimpl.MessageState
	sizeCache     protoimpl.SizeCache
	unknownFields protoimpl.UnknownFields

	Created *timestamppb.Timestamp `protobuf:"bytes,2,opt,name=created,proto3,oneof" json:"created,omitempty"`
	Name    string                 `protobuf:"bytes,4,opt,name=name,proto3" json:"name,omitempty"`
}

Generation (gofmted for better readability)

// Code generated by github.com/switchupcb/copygen
// DO NOT EDIT.

package copygen

import (
	"time"

	"google.golang.org/protobuf/runtime/protoimpl"
	"google.golang.org/protobuf/types/known/timestamppb"
)

type EntityPB struct {
	state         protoimpl.MessageState
	sizeCache     protoimpl.SizeCache
	unknownFields protoimpl.UnknownFields

	Created *timestamppb.Timestamp `protobuf:"bytes,2,opt,name=created,proto3,oneof" json:"created,omitempty"`
	Name    string                 `protobuf:"bytes,4,opt,name=name,proto3" json:"name,omitempty"`
}
type Entity struct {
	Created *time.Time `json:"created,omitempty"`
	Name    string     `json:"name,omitempty"`
}

// EntityToPB copies a *Entity to a *EntityPB.
func EntityToPB(tE *EntityPB, fE *Entity) {
	// *EntityPB fields
	tE.Name = fE.Name
}

// EntityToPBCreated copies a *Entity, *timestamppb.Timestamp to a *EntityPB.
func EntityToPBCreated(tE *EntityPB, fE *Entity, fT *timestamppb.Timestamp) {
	// *EntityPB fields
	tE.state = fT.state
	tE.sizeCache = fT.sizeCache
	tE.unknownFields = fT.unknownFields
	tE.Created = fT
	tE.Name = fE.Name
}

// EntityToPBTag copies a *Entity to a *EntityPB.
func EntityToPBTag(tE *EntityPB, fE *Entity) {
	// *EntityPB fields
	tE.Created = tE.Name = fE.Name
}

// EntityToPBCreatedTag copies a *Entity, *timestamppb.Timestamp to a *EntityPB.
func EntityToPBCreatedTag(tE *EntityPB, fE *Entity, fT *timestamppb.Timestamp) {
	// *EntityPB fields
	tE.Created.Seconds = fT.Seconds
	tE.Created.Nanos = fT.Nanos
	tE.Created = tE.Name = fE.Name
}

Errors

	tE.state = fT.state

Copying private member fields isn't going to end well. ;-)

	tE.Created = tE.Name = fE.Name
// ...
	tE.Created = tE.Name = fE.Name

It seems like custom fields confuses the internal type-matching logic.

Operating System: darwin/arm64 or linux/amd64
Copygen Version: latest == v0.4.0

Problem

Using FullNameWithoutContainer("") on a type defined in a package external to setup with a container adds the import to its output.

Fix

Add complex tests to multi and fix the FullNameWithoutContainer("") method to account for the definition that go/types provides, or fix the parsed definition.

Reproduce:

add

type StructType struct {
    A string
}

to examples/automatch/domain/domain.go and

type StructType struct {
    B string
}

examples/automatch/model/model.go
than add field
FieldTypeA StructType
to both files.
After codegen you will get

	tA.FieldTypeA = fA.FieldTypeA
	tA.FieldTypeA = fA.FieldTypeA

1. you will get two same lines
2. types are not compatible.

Version 0.5 will require the following changes.

• implement #33
• Options, Matcher, Generator: cast option for direct type conversions (as opposed to a convert function)

The following changes are dependent on a third-party.

• #30

Status
Started on 3/24/23.
Paused on 3/25/23.
Resumed on 3/26/23.
Paused on 3/27/23.
Resumed on 3/29/23.
Paused on 3/29/23 (Housing Issues).

ETA
This target is under development.

I consider myself to be an experienced Go developer and have used various codegen tools in the past including SQLBoiler. I've been grappling with the issue of mapping data between different representations (structs) at different levels of an application, and came across both goverter and copygen.

Copygen seems like it's solving the problem I want to solve, the way I want to solve it: CLI tool to generate zero-overhead mapping code between structs. Therefore I'm quite interested.

I read through the README and I have some points of feedback and questions. They may or may not be palatable and I acknowledge I'm spouting off some opinions 10 minutes after finding your project, but first impressions from would-be users can have a certain value so please take it in that light.

1. Looking at the README overall, I think a more simple onramp would be valuable. If you look at goverter it starts with a straightforward 1-to-1 mapping with identical field names and types, and the examples get more complex from there. In the copygen README, you're starting off with a 2-to-1 struct mapping, so off the bat I'm spending brain cycles trying to figure out how the made-up model objects are supposed to relate to one another, rather than just grokking the tool. I suggest cutting back the gradient so you start with an absolute rock-bottom example and take it up one piece at a time.
2. Moving on, the README states that setup involves a .go file and a .yml file. That seems like perhaps excessive moving parts and setup for the most basic scenarios. The most vital component of the .yml file seems to be specifying input and output file names, but sane defaults could be used to allow things to "just work" with only the .go file. Perhaps it looks for copygen.go and copygen.yml by default, and generates copygen.gen.go? Thoughts?
3. Moving on to the setup.go section, it makes sense until the statement Copygen uses no allocation with pointers which means fields are assigned to objects passed as parameters. This is confusing because the signature you show just above it is ModelsToDomain(models.Account, models.User) *domain.Account which I gather is creating a domain.Account and returning a pointer to it; it shouldn't be assigning to the objects that are passed as parameters. I am probably misunderstanding; a clarification or better wording would be valuable.
4. On the options header:
   • I would suggest changing away from a table to a format that doesn't introduce line breaks, since said line breaks make it more difficult to visually parse the examples.
   • Suggest always showing fully concrete examples before showing examples that use regexes.
   • Instead of just saying map from to I would suggest map <from> <to> or similar, so it's easy to see what's literal and what's meant to be variable.
   • Suggest showing actual code examples for each of these so you can see how they work in practice.
5. In the output of the first example, it seems like it copied the Itoa function. If I already wrote the function in one .go file, it doesn't seem like I'd need it defined again. I suppose if you're using a setup.go that's in one directory and putting your copygen.go in another directory, yes you'd need to move the bits over. But why not put it all in the same place and avoid that complexity?
6. The Customization section is, at the risk of sounding whiny and negative, a little mind-boggling. I don't understand the use cases behind Custom Types or Templates. Okay, I see that you need to use Templates to add an error-return although that seems a little extreme, I believe such is available out of the box with goverter.
7. Rather than jumping into deep customization, let's say I want to do a 1-to-1 struct mapping where a couple of the fields have different names and a couple have different types (int vs. int64 or int vs. string, let's say). I want to leverage copygen's automatic mapping for the most part and just provide a couple options, arguments, utility functions, what have you, to deal with the special cases. An example of this would be awesome.
8. There's not really enough data given under Matcher, apparently there's manual methods and there's automatic methods. Examples and comparisons would be great.
9. The depth example is hard to parse, I see that it's supposed to illustrate field depth but the indentation is a little confusing. I would suggest some curly braces so you get better delineation of where the sub-structs start and end. And then, actual code examples that perform mapping using the detailed example of depth that was just provided.

Apologies if any of this comes across as harsh or entitled, it's not meant that way. I'm very interested in using a tool like copygen for my projects, and this one seems overall more powerful and flexible than goverter, but I find myself right away bouncing off of some confusions and perceived complexity, and I wanted to give some feedback on this.

Problem

Separating files of a .go template file results in an error if a declaration cannot be found.

Fix

Determine how to interpret the entire package before calling func Generate(gen *models.Generator) (string, error).

Problem

[]*int Fail.

Setup

Fix subsequent collections in types and add tests for collections with random pointers in them (*[]int is tested but not the above).

Raise an error in case a field is not matched.

Let's say that there are missing fields in target that were not mapped, there should be an option that raises an error on missing mapping.

Refer #41.
Goverter does this out of the box. It also has an option for ignoring non-exported fields, so it helps prevent false positives especially when mapping protos

Problem

The Field.IsInterface() function doesn't work for named interfaces. This occurs because — while parsing — named field's are deepcopied, then have their definition's replaced. Such that a string comparison of that definition (i.e NamedInterface[9:] == "interface") would return false. This function used to be resolved by checking a .Collection field (of models.Field), but that field was removed due to being unnecessary... until now.

Solution

Collection

The previous .Collection field specifies which type of collection (f.IsPointer() || (f.IsArray() || f.IsSlice() || f.IsMap() || f.IsChan()) || f.IsFunc() || f.IsInterface()) the field is. This field was removed in a previous version because it was being used to account for the actual type definition of the field (i.e *Account where Definition == "Account" and Collection == *). Not only did this contribute to confusion, but it also made generating type-to-type code harder.

Adding back the Collection field would not cause confusion anymore because it would not be used for any other portion of the code. However, expanding the purpose of this field to represent an Underlying type of the field may be more useful. Currently, there is no way to determine the underlying type of a field. As an example, a field id Number will have a Name=="id" and Definition=="Number"; Number is an alias to a string. However, there is currently no way to find more information about the underlying field string that Number represents.

Underlying

An Underlying field of a models.Field would represent the actual underlying type (in a models.Field object) if it exists. There is a distinction between the go/types Underlying() method and models.Field Underlying() method. In go/types, the underlying function represents a unique instance of a go/type, and can represent go/types that have underlying go/types (such that a "Named' type is nothing more than a go/types.Named type). In contrast, the models.Field underlying field represents a default type (in colloquial terms), which likely shares an instance with other default underlying types.

To be specific, models.Field objects typically represent the copygen definition of a field: These objects are created in a context which always gives them a Name (either in the Copygen Interface function parameters func(to string) from string or as a field of other models.Field structs, etc). In contrast, the model.Field present in the Field.Underlying field would represent actual default types (basic types, map, struct, interface, category.go) which a user (developer) can use to determine more information about a field's Go Type.

The objects contained in a .Underlying models.Field field should represent the same object when it represents the same Go type. As an example, id Number and word Word models.Field objects represent two different models.Field. However, since the underlying type of each models.Field.Definition is an alias (Number, Word) to a string, the Field.Underlying of each models.Field should point to the same object (which is a models.Field representation of a string).

Thus, it would be expected for users who — for whatever reason — modify the Underlying field of a models.Field to also modify the .Underlying field of every other models.Field that references it. Otherwise, the user can always use field.Deepcopy() prior to modifying that field, if that is ever necessary.

Version 0.3 will include the following changes:

• new method of Parsing
• Weird Edge Case Fix when using a lot of Conversion
• go/template template support from #9
• Ability to match by tags
• Ability to invoke automatch using automatch option.
• Getting closer to supporting third party module imports in interpreted templates.
• Logo

BONUS

• integration tests
• programmatic usage
• parse every Go type including basic, array, slice, map, chan, interface, and func types (but not generate)

Status
Started on April 5, 2022.
Expectation of completion on April 12, 2022 (as of April 10, 2022).
Packaging attempted on April 12, 2022.
Completed April 13, 2022.

In the generated code below, mapping directly between types works correctly, but for the slice types it doesn't output the full import path.

Setup

YML

generated:
  setup: ./setup.go
  output: ../copygen.go

Go

package copygen

import (
	"local/test/domain"
	"local/test/models"
)

type Copygen interface {
	MyModelToMyDomain(src *models.MyModel) *domain.MyDomain
	MyModelsToMyDomains(src []*models.MyModel) []*domain.MyDomain
}

Output

Generation

// Code generated by github.com/switchupcb/copygen
// DO NOT EDIT.

package copygen

import (
	"local/test/domain"
	"local/test/models"
)

// MyModelToMyDomain copies a *models.MyModel to a *domain.MyDomain.
func MyModelToMyDomain(tM *domain.MyDomain, fM *models.MyModel) {
	// *domain.MyDomain fields
	tM.Foo = fM.Foo
}

// MyModelsToMyDomains copies a []*MyModel to a []*MyDomain.
func MyModelsToMyDomains(tM []*MyDomain, fM []*MyModel) {    // <------ Error here: "undeclared name: MyDomain" (should be: domain.MyDomain)
	// []*MyDomain fields
}

Environment

Operating System: macOS
Copygen Version: latest