dave / dst Goto Github PK
View Code? Open in Web Editor NEWDecorated Syntax Tree - manipulate Go source with perfect fidelity.
License: Other
Decorated Syntax Tree - manipulate Go source with perfect fidelity.
License: Other
This is problematic because only *ast.File
stores comments. If we convert a dst.Expr
to an ast.Expr
, there's nowhere to put the comments so they will be lost. Workarounds? For Decl
perhaps we could create an empty File
with one Decl
, but what for Stmt
and Expr
?
The docs should be easier to find. Would you accept to add a badge or a link in the README?
I'm trying to load a package and walk the resulting tree. I'm noticing that all of the dst.Ident
Path
values are set to an empty string if the type is in the package being loaded but the Path
values are set correctly if the type refers to another package. I pasted a simple program below that loads the net/http
package and finds the CookieJar
interface for which it prints an identity of just CookieJar
(I expected net/http.CookieJar
). It also prints the identity of the type of the first parameter to the SetCookies
method as net/url.URL
.
Is there some way to have the path info filled in?
package main
import (
"github.com/dave/dst"
"github.com/dave/dst/decorator"
"golang.org/x/tools/go/packages"
)
func main() {
pkgs, err := decorator.Load(&packages.Config{
Mode: packages.NeedName |
packages.NeedFiles |
packages.NeedCompiledGoFiles |
packages.NeedImports |
packages.NeedTypes |
packages.NeedTypesSizes |
packages.NeedSyntax |
packages.NeedTypesInfo,
}, "net/http")
if err != nil {
panic(err)
}
for _, pkg := range pkgs {
for _, file := range pkg.Syntax {
for _, decl := range file.Decls {
if genDecl, ok := decl.(*dst.GenDecl); ok {
for _, spec := range genDecl.Specs {
if typeSpec, ok := spec.(*dst.TypeSpec); ok {
if typeSpec.Name.Name == "CookieJar" {
println("CookieJar ident:", typeSpec.Name.String())
for _, method := range typeSpec.Type.(*dst.InterfaceType).Methods.List {
if method.Names[0].Name == "SetCookies" {
param1Type := method.Type.(*dst.FuncType).Params.List[0].Type.(*dst.StarExpr)
println(" Param1 StarExpr X ident:", param1Type.X.(*dst.Ident).String())
}
}
}
}
}
}
}
}
}
}
Output:
CookieJar ident: CookieJar
Param1 StarExpr X ident: net/url.URL
What do we do with the current Doc
(Field, File, FuncDecl, GenDecl, ImportSpec, TypeSpec, ValueSpec) and Comment
(Field, ImportSpec, TypeSpec, ValueSpec) fields?
Is there any alternative to go/format
's format.Node
function? I couldn't find any. Any help is greatly appreciated, thank you!
Thanks for starting this package, the problem it solves is one I encountered a very long time ago!
I just started off with a very simple test:
package b
func B() {
println("B")
}
package a
import "github.com/pwaller/example/testdata/a/b"
func A() {
b.B()
}
With the following program I get unexpected output:
package main
import (
"golang.org/x/tools/go/packages"
"github.com/dave/dst/decorator"
)
func main() {
pkgs, err := decorator.Load(&packages.Config{Mode: packages.LoadAllSyntax}, ".")
if err != nil {
panic(err)
}
for _, p := range pkgs {
for _, s := range p.Syntax {
decorator.Print(s)
}
}
}
Here is the output:
package a
import "github.com/pwaller/example/testdata/a/b"
func A() {
B()
}
I expected that I should see a call to b.B()
, not B()
, so it seems the package qualifier is lost.
This is using github.com/dave/dst v0.23.0
.
I'm a bit surprised by the bug, I would have thought this case should work - have I done something wrong? Thanks!
Hi dave~
package main
// Typo is blabla...
// more comments here, lines not fixed...
// ...
// finally, comments are done...
type Demo struct {
}
func main() {
println(a, b, c)
}
In the above example, does dst
support explicitly change the comment attached to Demo
structure ? Like change the word Typo
to Demo
.
I'm trying to generate some code using dst package.
Part of the code involves multi-line string literal.
I'd like to keep the code prettier by propagating the indentation level into the string literal
... // node :=
&dst.CallExpr{
Fun: &dst.SelectorExpr{
X: &dst.Ident{Name: "mypackage"},
Sel: &dst.Ident{Name: "MyFunc"},
},
Args: []dst.Expr{
&dst.BasicLit{
Kind: token.STRING,
Value: fmt.Sprintf("`\n%v\n`", "multi\nline\nliteral"),
},
},
},
The generated code will look like
func GeneratedFunc() {
node := mypackage.MyFunc(`
multi
line
literal
`) // <-- This non-indent looks rather bothering
}
Is it possible that I can make the string literal indent aligned with the caller, or somehow retrieve the 'indentation level' via the dst package, so that I can manually Tweak the literal? e.g.
func GeneratedFunc() {
node := mypackage.MyFunc(`
multi
line
literal
`)
}
Hello,
This might be a real edge case, but just wanting to share.
When for example, missing a package main
you will get a panic:
panic: runtime error: invalid memory address or nil pointer dereference
func TestDecorator(t *testing.T) {
code := `import "github.com/MyPackage/Name/package1"
func main() {
fmt.Println("Hello!")
}`
decorator.Parse(code)
}
I think earlier versions did not fail this test, and feel like it should gracefully return an error.
Hi! I'm very new to your project, and I'm still learning on it.
When I use Load function from your project, it only allows packages.LoadAllSyntax or packages.LoadSyntax which is deprecated in ast.
https://pkg.go.dev/golang.org/x/tools/go/packages#pkg-constants
Adding or removing nodes that contain Qualified Imports (SelectorExpr) from the tree can leave the import block out of sync. There are several problems:
Do we make dst
more opinionated? Or create another package to handle this? I think this is a common enough problem that dst
should handle it internally.
Solutions:
Perhaps add Path string
to dst.Ident
. Decorator will convert all qualified ident ast.SelectorExpr
to dst.Ident
with Path
set to the the package path. Restorer will convert idents that need a selector into ast.SelectorExpr
. All dst.Ident
will have Path
set - even local? That would ensure nodes are portable - e.g. could be copied to a new package. But would this be the preferred behaviour? Perhaps not - copying nodes that expect local variables would probably expect the local variables to stay local in the new package. Perhaps local idents simply use blank Path
.
For this to work, both Decorator and Restorer will have to be able to reliably determine the mapping between package name and package path. Perhaps we can have a Resolver
interface that could be plugged in as the developer prefers the context.
loader
package.The restorer will have to scan the tree and reconstruct the imports block, making sure any existing decorations aren't broken.
Thanks for the very useful dst package, I have used it a few times for rewriting code with great success! :)
Now that the Go language proposal golang/go#43651 has been accepted, I was wondering what your thoughts are on updating the dst package when generics land?
Do you plan to make the required changes so that dst can read, modify and write code that uses generics? (I’m assuming changes will be required.)
Thanks,
Nodes can't be re-used in the tree because restoreNode
both restores the node (creates an ast node) and also applies the node (creates the comments, increments the cursor). When a duplicate node is detected, it exits early so the apply is skipped.
We should separate these so restoring the node can be performed before applying. That way if a duplicate node is detected, the restore function will exit early, but the apply function will run.
This will enable us to simplify restoreObject
. All the nodes can be restored without being applied, which will mean the restore step won't need to be deferred until after the file is restored.
I plan to add DecorationDecl
and perhaps DecorationStmt
that hold nothing apart from a set of decorations.
Pros?
These comments aren't attached to nodes, so it makes sense to have them as their own node.
Cons?
The dst
package becomes more opinionated - I'd prefer it to stay as close to go/ast
as possible so it's more familiar.
I've assumed that the format package will handle all indenting, so dst
doesn't bother reconstructing the byte offsets for indent characters. However, there seems to be one exception:
switch a {
case 1:
// Comment about case 1
case 2:
}
switch a {
case 1:
// Comment about case 2
case 2:
}
Both are valid and will not be modified by gofmt. Since the AST nodes created by the Restorer have no indenting, the format
package will always choose the second version.
How to fix this?
Are these needed? I've ignored adding spaces between tokens, and it seems to render fine - but is this always the case?
package main
import (
"github.com/dave/dst/decorator"
"github.com/dave/dst/decorator/resolver/goast"
"github.com/dave/dst/decorator/resolver/guess"
"go/token"
)
func main() {
code := `package main
import (
"github.com/opentracing/opentracing-go"
)
var a opentracing.Tracer
`
dec := decorator.NewDecoratorWithImports(token.NewFileSet(), "main", goast.New())
f, err := dec.Parse(code)
if err != nil {
panic(err)
}
res := decorator.NewRestorerWithImports("main", guess.New())
if err := res.Print(f); err != nil {
panic(err)
}
}
get:
package main
var a opentracing.Tracer
expect:
package main
import (
"github.com/opentracing/opentracing-go"
)
var a opentracing.Tracer
Hello dave, I've got one question about the rewrite API for you:)
I think the Apply(root dst.Node, pre, post ApplyFunc) (result dst.Node)
API is not as useful as it can be, one will have to do many ast node cast to do useful things, which essentially duplicates the work done in ast.Walk
etc.
The most tricky part when rewrite is finding and replacing all its references, not just renaming the declare side, is that right? IMHO a useful API should consider references automatically.
UPDATE
I find dst can automatically handle references for renaming import
, but not type
, var
, const
etc.
Would be nice to have support for go's line directives.
The simplest way I believe would be to avoid indenting line comments Decorations starting with //line
Thank you!
How should we handle these?
I can't seem to figure out how to make an empty struct without a new line. I've tried the struct decorator and the field list decorator. It feels like I should be altering the field decorator but there are no fields. I'm fighting with a new linter that is removing the empty line from empty structs and I don't want generated code to be affected by the linter. Here's what I get:
type MyEmptyStruct struct {
}
Here's what I want:
type MyEmptyStruct struct {}
Thanks in advance!
Comments that are placed inside empty function, loop or if statement bodies are not found in the respective node's decorations. I would expect them to be placed somewhere in the e.g. dst.FuncDeclDecorations
of the function or similarly the dst.ForStmtDecorations
of the loop declaration. This snipped demonstrates the issue:
package main
func main() {
}
func empty() {
// inside empty function, not found
}
func foo() {
i := 1
for i < 2 {
// inside empty for statement, not found
}
// after empty for statement, found and associated with for statement
if i == 3 {
// inside empty if statement, not found
}
i = 4
}
And here you can find code to reproduce the issue: https://play.golang.org/p/Yg9OOkoP6IF [Disclaimer: sometimes the playground needs multiple tries to import all packages, so if you encounter a timeout running go build
error, just execute the code again.]
I tried using dst
in order to fix the issue golang/go#39753 I had with go/ast
. As I am a fairly new user of this module, I might also have overlooked something - Please let me know if this is the case and how I could get the correct comment associations :) Thanks!
$ go version go version go1.14.2 darwin/amd64
go env
Output$ go env GO111MODULE="" GOARCH="amd64" GOBIN="" GOCACHE="/Users/.../Library/Caches/go-build" GOENV="/Users/.../Library/Application Support/go/env" GOEXE="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="darwin" GOINSECURE="" GONOPROXY="" GONOSUMDB="" GOOS="darwin" GOPATH="/Users/.../go" GOPRIVATE="" GOPROXY="https://proxy.golang.org,direct" GOROOT="/usr/local/go" GOSUMDB="sum.golang.org" GOTMPDIR="" GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64" GCCGO="gccgo" AR="ar" CC="clang" CXX="clang++" CGO_ENABLED="1" GOMOD="/Users/.../go.mod" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/yb/hqncynqs0b5_3hyxcjpdsyr00000gr/T/go-build447564702=/tmp/go-build -gno-record-gcc-switches -fno-common"
it is hard to move the existed code based on go/ast
to dst
.
Hi,
I wrote code with the standard ast, astutil and inspector packages, which gets some code from a file and adds some new blocks to a specific position, plus some imports.
Worked well, I had only one problem, wasn't able to figure out how to add a newline before the inserted block. :D
That's how I found your library.
Currently, I have a working code, which is:
package main
import (
"go/ast"
"go/format"
"go/parser"
"go/token"
"os"
"strings"
"golang.org/x/tools/go/ast/astutil"
"github.com/dave/dst"
"github.com/dave/dst/decorator"
"github.com/dave/dst/dstutil"
"golang.org/x/tools/go/ast/inspector"
)
func main() {
code := `package main
import "github.com/prometheus/client_golang/prometheus/promhttp"
func main() {
c.echoEngine.GET("metrics", echo.WrapHandler(promhttp.Handler()))
}
`
newFile, err := decorator.Parse(code)
if err != nil {
panic(err)
}
var newBlockStmt *dst.BlockStmt
for _, d := range newFile.Decls {
f, ok := d.(*dst.FuncDecl)
if !ok {
continue
}
if f.Name.Name == "main" {
newBlockStmt = f.Body
}
}
fset := token.NewFileSet()
origFile, err := parser.ParseFile(fset, "controller.go.source", nil, 0)
if err != nil {
panic(err)
}
blockStmt, exprStmt := healthCheckParentNodes(origFile)
dec := decorator.NewDecorator(fset)
dstFile, err := dec.DecorateFile(origFile)
if err != nil {
panic(err)
}
dstBlockStmt := dec.Dst.Nodes[blockStmt].(*dst.BlockStmt)
dstExprStmt := dec.Dst.Nodes[exprStmt].(*dst.ExprStmt)
dstutil.Apply(dstBlockStmt, func(c *dstutil.Cursor) bool {
if c.Node() != dstExprStmt {
return true
}
for i := len(newBlockStmt.List) - 1; i >= 0; i-- {
c.InsertAfter(newBlockStmt.List[i])
}
return false
}, nil)
newSource, err := os.Create("controller_new.go.source")
if err != nil {
panic(err)
}
defer newSource.Close()
fset, origFile, err = decorator.RestoreFile(dstFile)
if err != nil {
panic(err)
}
for _, i := range newFile.Imports {
astutil.AddImport(fset, origFile, strings.Trim(i.Path.Value, `"`))
}
if err := format.Node(newSource, fset, origFile); err != nil {
panic(err)
}
}
func healthCheckParentNodes(f *ast.File) (blockStmt ast.Node, exprStmt ast.Node) {
types := []ast.Node{new(ast.BasicLit)}
ins := inspector.New([]*ast.File{f})
ins.WithStack(types, func(n ast.Node, push bool, stack []ast.Node) (prune bool) {
if !push || blockStmt != nil {
return false
}
lit := n.(*ast.BasicLit)
if !strings.Contains(lit.Value, "healthcheck") {
return true
}
for i := len(stack) - 1; i >= 0; i-- {
if s, ok := stack[i].(*ast.ExprStmt); ok {
exprStmt = s
continue
}
if s, ok := stack[i].(*ast.BlockStmt); ok {
blockStmt = s
break
}
}
return false
})
return blockStmt, exprStmt
}
The input file looks like this:
package rest
import (
"net/http"
"net/http/pprof"
"os"
"runtime"
"github.com/labstack/echo"
)
type controller struct {
echoEngine *echo.Echo
}
func NewController(echoEngine *echo.Echo) *controller {
return &controller{echoEngine: echoEngine}
}
func (c *controller) Start() {
if os.Getenv("DEBUG") == "true" {
runtime.SetBlockProfileRate(1)
runtime.SetMutexProfileFraction(5)
c.echoEngine.GET("/debug/pprof/*", echo.WrapHandler(http.HandlerFunc(pprof.Index)))
c.echoEngine.GET("/debug/pprof/cmdline", echo.WrapHandler(http.HandlerFunc(pprof.Cmdline)))
c.echoEngine.GET("/debug/pprof/profile", echo.WrapHandler(http.HandlerFunc(pprof.Profile)))
c.echoEngine.GET("/debug/pprof/symbol", echo.WrapHandler(http.HandlerFunc(pprof.Symbol)))
c.echoEngine.GET("/debug/pprof/trace", echo.WrapHandler(http.HandlerFunc(pprof.Trace)))
}
c.echoEngine.Add(http.MethodGet, "/healthcheck", func(eCtx echo.Context) error {
return eCtx.String(http.StatusOK, "ok")
})
c.echoEngine.Add(http.MethodGet, "/something-else", func(eCtx echo.Context) error {
return eCtx.String(http.StatusOK, "ok")
})
}
The code is way too complicated and using both the ast and dst packages, plus the inspector, so I have some questions.
First, I wasn't able to figure out how to add the import from the new code snippet to the old one. Checked the readme about resolvers, but it feels way too complicated and mainly developed for automatic import management. I just wanted to add an import manually without any magic, like the astutil.AddImport
.
Tried to do something like dstFile.Imports = append(dstFile.Imports, newFile.Imports...)
, but when printed out the source code with decorator.Print(dstFile)
, the import was missing.
Is it possible to do this with dst?
Second thing, that there is a decorator.Decorate
function, which returns a dst.Node
, first I used this to add the newline, but wasn't able to figure out how to convert back to an ast.Node
, I only saw a method to restore a whole ast.file.
Is it possible? Would this work? I mean, would that keep the newline in the ast.Node
too?
Third, I wasn't able to figure out how to replace the inspector WithStack
method with dst related code, because I don't know how to build a stack.
I can't see any method which tells about parent nodes during Inspect or Walk, so I don't know how the current node relates to the previous one.
Is it possible with dst?
It's a little long, sorry, checked the documentation, but didn't find the answers.
`
func TestPosition(t *testing.T) {
path := "../internal/transports/bulletin/router_bulletin.go"
//===============================================================
// 1: Use ast directly
astFset := token.NewFileSet()
astFile, err := parser.ParseFile(astFset, path, nil, parser.ParseComments)
if err != nil {
log.Fatalln(err)
}
fmt.Print(astFile.Decls[3].Pos(), ",")
fmt.Println(astFile.Decls[3].End())
//===============================================================
// 2: parse to ast, then use decorator
dec := decorator.NewDecorator(astFset) //包装器
dstFile, err := dec.DecorateFile(astFile) //对astFile进行包装
if err != nil {
log.Fatalln(err)
}
fmt.Print(dec.Ast.Nodes[dstFile.Decls[3]].Pos(), ",")
fmt.Println(dec.Ast.Nodes[dstFile.Decls[3]].End())
//===============================================================
// 3: parse to dst, then use restorer
dstFset := token.NewFileSet()
file, err := decorator.ParseFile(dstFset, path, nil, parser.ParseComments)
if err != nil {
log.Fatalln(err)
}
restorer := decorator.NewRestorer()
_, err = restorer.RestoreFile(file)
if err != nil {
log.Fatalln(err)
}
a := restorer.Ast.Nodes[file.Decls[3]]
fmt.Print(a.Pos(), ",")
fmt.Println(a.End())
}`
Result:
1942,2357
1942,2357
1845,2210 //wrong
Is this normal?
Some comments are mis-aligned after format.Node
... They get the correct alignment after a second run through gofmt. I suspect we need to add spacing for separator characters?
Hi dave~
ast.MergePackageFiles
can be used to merge multiple files, but the resultant file will have multiple import declarations which is very ugly and maybe not even valid syntax, like this:
package demo
import "fmt"
func hi() {
fmt.Println("hi")
}
import (
"fmt"
"math"
)
func hello() {
fmt.Println(math.Abs(-1))
}
Does dst support merging multiple import declarations into one while reserving comments?
I'm trying to generate code similar to this:
type something struct {
here int
}
func newSomething() *something {
return &something{
here: 42,
}
}
I've got the struct in place and a start on the func but the body of the func is giving me an error:
panic: duplicate node: &dst.Ident{Name:"something", Obj:(*dst.Object)(nil), Path:"", Decs:dst.IdentDecorations{NodeDecs:dst.NodeDecs{Before:0, Start:dst.Decorations(nil), End:dst.Decorations(nil), After:0}, X:dst.Decorations(nil)}}
goroutine 1 [running]:
github.com/dave/dst/decorator.(*FileRestorer).restoreNode(0xc000626e10, 0x1457e60, 0xc000625a80, 0x13dd1ff, 0xc, 0x13db130, 0x4, 0x13db0c0, 0x4, 0x0, ...)
/Users/dschultz/workspace/go/pkg/mod/github.com/dave/[email protected]/decorator/restorer-generated.go:15 +0x14356
github.com/dave/dst/decorator.(*FileRestorer).restoreNode(0xc000626e10, 0x1457c80, 0xc000559c20, 0x13dc48c, 0x9, 0x13dacc7, 0x1, 0x13db0c0, 0x4, 0x0, ...)
/Users/dschultz/workspace/go/pkg/mod/github.com/dave/[email protected]/decorator/restorer-generated.go:459 +0x7fe
github.com/dave/dst/decorator.(*FileRestorer).restoreNode(0xc000626e10, 0x1458140, 0xc000619730, 0x13dc8fe, 0xa, 0x13dbaef, 0x7, 0x13db0c0, 0x4, 0x0, ...)
/Users/dschultz/workspace/go/pkg/mod/github.com/dave/[email protected]/decorator/restorer-generated.go:1839 +0x1b0d
I dumped the DST of a go file similar to what I want to create:
1280 . . . Body: *dst.BlockStmt {
1281 . . . . List: []dst.Stmt (len = 1) {
1282 . . . . . 0: *dst.ReturnStmt {
1283 . . . . . . Results: []dst.Expr (len = 1) {
1284 . . . . . . . 0: *dst.UnaryExpr {
1285 . . . . . . . . Op: &
1286 . . . . . . . . X: *dst.CompositeLit {
1287 . . . . . . . . . Type: *dst.Ident {
1288 . . . . . . . . . . Name: "something"
1289 . . . . . . . . . . Obj: *(obj @ 50)
1290 . . . . . . . . . . Path: ""
My code is creating an identical structure but it looks like the restorer code thinks that something
is duplicate when defined on line 1288 above. Also not sure what to do about the Op
field. I'm setting it to token.AND
but not sure if that's right.
Any help is greatly appreciated!
Right now we have very fine-grained low-level control of the imports block. It would be better to have some convenience functions so that adding a qualified identifier automatically adds the import spec. This is problematic because we don't know the mapping between package name and package path.
I have no need for import management, and wanted to use Load to get a DST of a package; but there is no way to configure the load and opt-out. My (messy) workaround was to use packages
to load the package, then decorate each *ast.File
syntax separately, and pass around the associated decorator.
Perhaps a custom Load config is needed, that might also address #53?
I want to do the following to my Go project
How to do this with dst ?
Investigating...
This code is a bit of a mess. I will remove the fragger
type and combine with Decorator
This is very similar to #9 but a more general case.
A statement that flows onto multiple lines will be indented. This means that comments that follow will be able to have two indent levels:
package a
const a = 1 +
1
// a1
// a2
const b = 1
const c = 1 +
1
// d1
// d2
const d = 1
This is not a small change, and I will be hiking in Patagonia from December 2021 until April or May 2022 without my laptop, so I won't be able to get this done until I get back.
When I decorate an AST, and then walk the DST, I encouter a *dst.FuncType
node, but there is no mapping back to the corresponding *ast.FuncType
node.
In decorator/decorator-node-generated.go
, it appears that the mapping is not made in the *ast.FuncDecl
case.
could you please help me to figure out one thing?
in the docs here
Line 24 in e3c2080
// All nodes contain position information marking the beginning of
// the corresponding source text segment; it is accessible via the
// Pos accessor method. Nodes may contain additional position info
// for language constructs where comments may be found between parts
// of the construct (typically any larger, parenthesized subpart).
// That position information is needed to properly position comments
// when printing the construct.
but at the same time the interface and it's implementations do not contain this method at all.
I have a doubt if dst nodes contain positions at all
If there're no positions provided, what approach would you suggest in implementing them?
The code that generates the conversion functions should be simpler. FragmentInfo
should contain all the info needed to do the generation so the generation functions don't need to use go/types
. In addition we should generate a file with all the FragmentInfo objects as literals, to serve as documentation for how the generation code works.
Currently each Comment is added to it's own CommentGroup. This renders ok, but it would be more correct to group sequences of Comments into CommentGroups.
Hi there! Thank you so much for this awesome library!
I want to write a small refactoring tool. The idea is to load a package, move some things around using dst
, and then write the changes back to disk. I am able to load my package, do changes and print the result of each file to stdout
with code like this:
func featureRequest() {
dir := "/tmp/t"
pkgs, err := decorator.Load(&packages.Config{Dir: dir, Mode: packages.LoadSyntax}, "root")
if err != nil {
panic(err)
}
p := pkgs[0]
fmt.Printf("parsed %d files\n", len(p.Syntax))
for _, f := range p.Syntax {
b := f.Decls[0].(*dst.FuncDecl).Body
b.List = append(b.List, &dst.ExprStmt{
X: &dst.CallExpr{
Fun: &dst.Ident{Path: "fmt", Name: "Println"},
Args: []dst.Expr{
&dst.BasicLit{Kind: token.STRING, Value: strconv.Quote("Hello, World!")},
},
},
})
fmt.Println("=====================================")
r := decorator.NewRestorerWithImports("root", gopackages.New(dir))
if err := r.Print(f); err != nil {
panic(err)
}
}
}
This successfully loads the files, applies changes and resolves the imports (great!). Here's the output of calling the function above (I have these files in my /tmp/t
directory: main.go
, sum.go
and go.mod
):
$ go run ./cmd/refactorer/main.go
parsed 2 files
=====================================
package main
import "fmt"
func sum(int x, int y) {
_ = x + y
fmt.Println("Hello, World!")
}
=====================================
package main
import "fmt"
func main() { fmt.Println("Hello, World!") }
However, as far as I could see, the paths of .go
files that were loaded are not exposed anywhere. This means at the moment I can't write the changes back to disk because I don't know which parsed file inside p.Syntax
corresponds to which path.
It would be awesome if we could expose the file names in the struct returned by decorator.Load
, perhaps by converting Syntax
:
Line 96 in ce1c8af
Line 674 in ce1c8af
Hi there, can dst parse a single go file with it's all imported dependencies to a AST tree? I noticed "NewDecoratorWithImports" may help but when I dig into it, seems like not meet my expectation. I have an example below:
File request.go:
import Team type Request struct { team Team
json:"team" }
File team.go
type Request struct { name string
json:"name" }
What I'd like to have is to get the whole AST tree of Request with all the information of Team. Thank you.
This is minor, but curious whether you'd take a patch to grant users of the library control over import grouping.
Some organizations (including my own) tend to group imports beyond stdlib/non-stdlib. I've seen two permutations at different orgs:
goimports
supports the former via its -local
flag, such that:
import (
"net/http"
"github.com/public/pkg"
"github.com/private/pkg"
)
when processed via goimports -local github.com/private
will result in:
import (
"net/http"
"github.com/public/pkg"
"github.com/private/pkg"
)
Further, goimports does not remove pre-existing groups.
I'd like dst's import management to provide the ability to retain the original grouping of imports - though this does introduce the problem of needing to find an appropriate place to insert any new imports that aren't stdlib.
Alternately, perhaps the best bet is to avoid using an import-aware FileRestorer and implement similar logic by hand.
When dst
automatically manages the imports block, the imports are internally keyed by path. This incorrectly assumes that only one import for each package path is possible. Although very rare, it is possible to add the same package multiple times with different aliases.
See: golang/go@3409ce3
In TestLoadStdLibAll
we test that the entire standard library is able to be parsed and rebuilt including management of the imports block. This test fails for the two files server.go
and request.go
in net/http
because of this.
In order to handle this gracefully would involve a major rewrite of the import management code which is the most complex part of dst
. For now I've skipped the TestLoadStdLibAll
for these two files.
Re: @dmitshur
Perhaps we should stop generating the code and hand-craft the conversion functions. It would make handling special cases simpler... and the code generation scripts aren't simple to follow.
Decoration points that come after lists of nodes are not needed - there will always have a node to attach to, and usually this will be a better attachment point.
Unfortunately we can't remove the End
decorations that come directly after lists - this would interfere with the detection of hanging indents (#18).
We should remove:
Field.Names
CallExpr.Args
AssignStmt.Lhs
CaseClause.List
ValueSpec.Names
Two colons may be present if Slice3 == true
even if Max == nil
?
No arrow when Dir == 0
Semicolon length = 1/0 when ???
When Type == nil
, then .(type)
is rendered
"case" token changes to "default" if Comm == nil
"range" token rendered before X?
"func" token from Type rendered before "Recv" and "Name"
my code:
dec := decorator.NewDecoratorWithImports(token.NewFileSet(), "main", goast.New())
f, err := dec.Parse("package test\nfunc init() {}")
if err != nil {
panic(err)
}
b := f.Decls[0].(*dst.FuncDecl).Body
b.List = append(b.List, &dst.ExprStmt{
X: &dst.CallExpr{
Fun: &dst.SelectorExpr{
X: &dst.Ident{
Name: "Gin",
},
Sel: &dst.Ident{
Name: "GET",
},
},
Args: []dst.Expr{
&dst.BasicLit{
Kind: token.STRING,
Value: strconv.Quote("url"),
},
&dst.SelectorExpr{
X: &dst.Ident{
Name: "packageName",
Path: "module/packageName",
},
Sel: &dst.Ident{
Name: "functionName",
},
},
},
},
})
res := decorator.NewRestorerWithImports("main", guess.New())
res.Print(f)
result:
package test
import "module/packageName"
func init() { Gin.Get("url", packageName.packageName.functionName) }
====
I want Gin.Get("url", packageName.functionName)
import is not work
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.