bazel run //:gazelle fails on https://github.com/harness/harness-core about rules_jvm HOT 16 CLOSED

#35 should fix this :)

from rules_jvm.

It looks like we have a bug where if java files are present with no package, we crash. We should fix this up, but in the mean time, it looks like product/ci/addon/testintelligence/java/testdata/pkg_detection/NoPackage.java is the problematic file - if you delete it, hopefully you should see everything else working.

from rules_jvm.

I tried running gazelle in your repo, but ran into this issue:

ERROR: error loading package '': in /Users/dwagnerhall/src/github.com/harness/harness-core/tools/bazel/macros.bzl: in /.../src/github.com/harness/harness-core/tools/bazel/GenTestRules.bzl: cannot load '//:test-util.bzl': no such file

Do you know how I'd find test-util.bzl and why it's missing?

from rules_jvm.

Thanks for the quick response Daniel. test-util.bzl seems to be written inside ./.github/workflows/go_build.yaml -- probably happens as part of the build.

Meanwhile, I tried after deleting the NoPackage.java and got the following error. Btw, how did you find NoPackage.java was the issue, can I run in some debug mode with more logs?

rting local Bazel server and connecting to it...
INFO: Invocation ID: fd24e425-79b4-4f54-9c2c-82f4d5b55aa7
INFO: Analyzed target //:gazelle (318 packages loaded, 18371 targets configured).
INFO: Found 1 target...
Target //:gazelle up-to-date:
/private/var/tmp/_bazel_gauravnanda/31c0a55cc70a91108079d63a7aab4a62/execroot/harness_monorepo/bazel-out/darwin_arm64-fastbuild/bin/gazelle-runner.bash
/private/var/tmp/_bazel_gauravnanda/31c0a55cc70a91108079d63a7aab4a62/execroot/harness_monorepo/bazel-out/darwin_arm64-fastbuild/bin/gazelle
INFO: Elapsed time: 9.599s, Critical Path: 0.50s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
INFO GrpcServer - Set of classes has no package
panic: runtime error: index out of range [0] with length 0

goroutine 4244 [running]:
github.com/bazel-contrib/rules_jvm/java/gazelle/private/java.NewImport({0x1400021d320, 0x15})
external/contrib_rules_jvm/java/gazelle/private/java/import.go:19 +0x2b8
main.filterImports(0x140003c7490, {0x14000299b30, 0x5, 0x5})
external/contrib_rules_jvm/java/gazelle/private/javaparser/cmd/javaparser-wrapper/main.go:276 +0xbc
main.(*server).ParsePackage(0x140000b8200, {0x10319cc00, 0x140003fa450}, 0x140003f0be0)
external/contrib_rules_jvm/java/gazelle/private/javaparser/cmd/javaparser-wrapper/main.go:212 +0x124
github.com/bazel-contrib/rules_jvm/java/gazelle/private/javaparser/proto/gazelle/java/javaparser/v0.JavaParser_ParsePackage_Handler.func1({0x10319cc00, 0x140003fa450}, {0x103153020, 0x140003f0be0})
bazel-out/host/bin/external/contrib_rules_jvm/java/gazelle/private/javaparser/proto/gazelle/java/javaparser/v0/gazelle_java_build_v0_go_proto/github.com/bazel-contrib/rules_jvm/java/gazelle/private/javaparser/proto/gazelle/java/javaparser/v0/javaparser.pb.go:319 +0x7c
main.activityTrackerUSI.func1({0x10319cc00, 0x140003fa450}, {0x103153020, 0x140003f0be0}, 0x1400023d1e0, 0x1400027b7e8)
external/contrib_rules_jvm/java/gazelle/private/javaparser/cmd/javaparser-wrapper/main.go:182 +0x54
github.com/bazel-contrib/rules_jvm/java/gazelle/private/javaparser/proto/gazelle/java/javaparser/v0.JavaParser_ParsePackage_Handler({0x10310fb40, 0x140000b8200}, {0x10319cc00, 0x140003fa450}, 0x140006864e0, 0x140000a6660)
bazel-out/host/bin/external/contrib_rules_jvm/java/gazelle/private/javaparser/proto/gazelle/java/javaparser/v0/gazelle_java_build_v0_go_proto/github.com/bazel-contrib/rules_jvm/java/gazelle/private/javaparser/proto/gazelle/java/javaparser/v0/javaparser.pb.go:321 +0x14c
google.golang.org/grpc.(*Server).processUnaryRPC(0x140000ea000, {0x1031a3ac0, 0x140003b4000}, 0x140003b9320, 0x140000a0630, 0x103470b70, 0x0)
external/org_golang_google_grpc/server.go:1282 +0xc3c
google.golang.org/grpc.(*Server).handleStream(0x140000ea000, {0x1031a3ac0, 0x140003b4000}, 0x140003b9320, 0x0)
external/org_golang_google_grpc/server.go:1619 +0xa54
google.golang.org/grpc.(*Server).serveStreams.func1.2(0x14000390040, 0x140000ea000, {0x1031a3ac0, 0x140003b4000}, 0x140003b9320)
external/org_golang_google_grpc/server.go:921 +0x94
created by google.golang.org/grpc.(*Server).serveStreams.func1
external/org_golang_google_grpc/server.go:919 +0x1f0
panic: rpc error: code = Unavailable desc = error reading from server: EOF

goroutine 1 [running]:
github.com/bazel-contrib/rules_jvm/java/gazelle.javaLang.GenerateRules({{0x1012b1478, 0x14000138280}, {0x1012b14b0, 0x1400011bdc0}, 0x1400055a000, {{0x1012ad3e8, 0x1400011bda0}, 0x1, {0x0, 0x0}, ...}, ...}, ...)
external/contrib_rules_jvm/java/gazelle/generate.go:113 +0x1dd8
main.runFixUpdate.func1({0x140038c6540, 0x55}, {0x14005558ba0, 0x29}, 0x140055493f0, 0x1, 0x0, {0x0, 0x0, 0x0}, ...)
external/bazel_gazelle/cmd/gazelle/fix-update.go:294 +0xc0c
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x14005ea5b80, {0x140038c6540, 0x55}, {0x14005558ba0, 0x29}, 0x1)
external/bazel_gazelle/walk/walk.go:179 +0x500
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x14005ea5ad0, {0x140051b8190, 0x50}, {0x14005069620, 0x24}, 0x1)
external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x14005ea5970, {0x140051b8000, 0x48}, {0x1400507a560, 0x1c}, 0x1)
external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x14005ea5810, {0x14004cb3c20, 0x45}, {0x1400507a4a0, 0x19}, 0x1)
external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x14005ea5600, {0x14005af8540, 0x40}, {0x14003eecbe8, 0x14}, 0x1)
external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x14005ea54a0, {0x14005af8400, 0x3b}, {0x14004494b40, 0xf}, 0x1)
external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x14000170fd0, {0x140059fba40, 0x37}, {0x14004494880, 0xb}, 0x1)
external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x14000170fd0, {0x140001862d0, 0x2b}, {0x0, 0x0}, 0x0)
external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk(0x14000170fd0, {0x14000534180, 0x7, 0xc}, {0x1400011be70, 0x1, 0x1}, 0x0, 0x14000435968)
external/bazel_gazelle/walk/walk.go:182 +0x198
main.runFixUpdate({0x1400004210a, 0x2b}, 0x0, {0x14000140010, 0x0, 0x0})
external/bazel_gazelle/cmd/gazelle/fix-update.go:271 +0x5a0
main.run({0x1400004210a, 0x2b}, {0x14000140010, 0x1, 0x1})
external/bazel_gazelle/cmd/gazelle/gazelle.go:95 +0x29c
main.main()
external/bazel_gazelle/cmd/gazelle/gazelle.go:72 +0x140

from rules_jvm.

Oh, it looks like there are a few more:

% comm -23 <(fd '\.java$' | sort) <(fd '\.java$' | xargs grep -l '^package ' | sort)
127-cd-nextgen-entities/src/main/java/io/harness/ng/core/environment/beans/EnvironmentRequest.java
360-cg-manager/src/main/java/software/wings/scheduler/LdapGroupSyncJobHandler.java
400-rest/src/main/java/io/harness/event/timeseries/processor/instanceeventprocessor/instanceaggregator/MonthlyAggregator.java
400-rest/src/main/java/io/harness/event/timeseries/processor/instanceeventprocessor/instanceaggregator/WeeklyAggregator.java
400-rest/src/main/java/software/wings/beans/loginSettings/PasswordSource.java
900-git-sync-sdk/src/main/java/io/harness/gitsync/interceptor/GitSyncBranchThreadLocal.java
930-delegate-tasks/src/main/java/software/wings/beans/appmanifest/HelmChart.java
930-delegate-tasks/src/main/java/software/wings/beans/appmanifest/ManifestFileDTO.java
930-delegate-tasks/src/main/java/software/wings/beans/artifact/ArtifactFile.java
930-delegate-tasks/src/main/java/software/wings/beans/container/HelmChartSpecificationDTO.java
930-delegate-tasks/src/main/java/software/wings/common/BuildDetailsComparatorAscending.java
950-delegate-tasks-beans/src/main/java/io/harness/delegate/task/jira/response/JiraTaskNGResponse.java
product/ci/addon/testintelligence/java/testdata/pkg_detection/NoPackage.java

from rules_jvm.

The last error I pasted actually seems different from the first one. Regardless, I deleted all the files with no package, but still get the same error.

~/source-code/harness-core $ bazel run //:gazelle
INFO: Invocation ID: a83fbdb9-339a-4db0-90e1-ec5642d5300b
INFO: Analyzed target //:gazelle (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //:gazelle up-to-date:
/private/var/tmp/_bazel_gauravnanda/31c0a55cc70a91108079d63a7aab4a62/execroot/harness_monorepo/bazel-out/darwin_arm64-fastbuild/bin/gazelle-runner.bash
/private/var/tmp/_bazel_gauravnanda/31c0a55cc70a91108079d63a7aab4a62/execroot/harness_monorepo/bazel-out/darwin_arm64-fastbuild/bin/gazelle
INFO: Elapsed time: 0.347s, Critical Path: 0.03s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
panic: runtime error: index out of range [0] with length 0

goroutine 4244 [running]:
github.com/bazel-contrib/rules_jvm/java/gazelle/private/java.NewImport({0x14000481ef0, 0x15})
external/contrib_rules_jvm/java/gazelle/private/java/import.go:19 +0x2b8
main.filterImports(0x1400023d030, {0x140003b6230, 0x5, 0x5})
external/contrib_rules_jvm/java/gazelle/private/javaparser/cmd/javaparser-wrapper/main.go:276 +0xbc
main.(*server).ParsePackage(0x14000268f00, {0x100bd0c00, 0x140003ae0f0}, 0x140003b6000)
external/contrib_rules_jvm/java/gazelle/private/javaparser/cmd/javaparser-wrapper/main.go:212 +0x124
github.com/bazel-contrib/rules_jvm/java/gazelle/private/javaparser/proto/gazelle/java/javaparser/v0.JavaParser_ParsePackage_Handler.func1({0x100bd0c00, 0x140003ae0f0}, {0x100b87020, 0x140003b6000})
bazel-out/host/bin/external/contrib_rules_jvm/java/gazelle/private/javaparser/proto/gazelle/java/javaparser/v0/gazelle_java_build_v0_go_proto/github.com/bazel-contrib/rules_jvm/java/gazelle/private/javaparser/proto/gazelle/java/javaparser/v0/javaparser.pb.go:319 +0x7c
main.activityTrackerUSI.func1({0x100bd0c00, 0x140003ae0f0}, {0x100b87020, 0x140003b6000}, 0x140001d4de0, 0x140003a6900)
external/contrib_rules_jvm/java/gazelle/private/javaparser/cmd/javaparser-wrapper/main.go:182 +0x54
github.com/bazel-contrib/rules_jvm/java/gazelle/private/javaparser/proto/gazelle/java/javaparser/v0.JavaParser_ParsePackage_Handler({0x100b43b40, 0x14000268f00}, {0x100bd0c00, 0x140003ae0f0}, 0x14000607e60, 0x140002a7bf0)
bazel-out/host/bin/external/contrib_rules_jvm/java/gazelle/private/javaparser/proto/gazelle/java/javaparser/v0/gazelle_java_build_v0_go_proto/github.com/bazel-contrib/rules_jvm/java/gazelle/private/javaparser/proto/gazelle/java/javaparser/v0/javaparser.pb.go:321 +0x14c
google.golang.org/grpc.(*Server).processUnaryRPC(0x140001f5180, {0x100bd7ac0, 0x14000452000}, 0x140003b4000, 0x1400023efc0, 0x100ea4b70, 0x0)
external/org_golang_google_grpc/server.go:1282 +0xc3c
google.golang.org/grpc.(*Server).handleStream(0x140001f5180, {0x100bd7ac0, 0x14000452000}, 0x140003b4000, 0x0)
external/org_golang_google_grpc/server.go:1619 +0xa54
google.golang.org/grpc.(*Server).serveStreams.func1.2(0x140000a0060, 0x140001f5180, {0x100bd7ac0, 0x14000452000}, 0x140003b4000)
external/org_golang_google_grpc/server.go:921 +0x94
created by google.golang.org/grpc.(*Server).serveStreams.func1
external/org_golang_google_grpc/server.go:919 +0x1f0
panic: rpc error: code = Unavailable desc = error reading from server: EOF

goroutine 1 [running]:
github.com/bazel-contrib/rules_jvm/java/gazelle.javaLang.GenerateRules({{0x1054f1478, 0x14000010288}, {0x1054f14b0, 0x1400019beb0}, 0x1400015c000, {{0x1054ed3e8, 0x1400019be90}, 0x1, {0x0, 0x0}, ...}, ...}, ...)
external/contrib_rules_jvm/java/gazelle/generate.go:113 +0x1dd8
main.runFixUpdate.func1({0x14003bb5da0, 0x55}, {0x14005489b90, 0x29}, 0x140054e71e0, 0x1, 0x0, {0x0, 0x0, 0x0}, ...)
external/bazel_gazelle/cmd/gazelle/fix-update.go:294 +0xc0c
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x1400516e420, {0x14003bb5da0, 0x55}, {0x14005489b90, 0x29}, 0x1)
external/bazel_gazelle/walk/walk.go:179 +0x500
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x1400516e370, {0x140048dfd60, 0x50}, {0x1400515c990, 0x24}, 0x1)
external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x1400516e2c0, {0x140048dfb80, 0x48}, {0x140050625a0, 0x1c}, 0x1)
external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x1400516e210, {0x140048df900, 0x45}, {0x140050624e0, 0x19}, 0x1)
external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x1400516e160, {0x14005870000, 0x40}, {0x14005970360, 0x14}, 0x1)
external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x1400516e0b0, {0x1400597bec0, 0x3b}, {0x140042e2d30, 0xf}, 0x1)
external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x140001f1080, {0x1400597b200, 0x37}, {0x140042e2ab0, 0xb}, 0x1)
external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x140001f1080, {0x14000036210, 0x2b}, {0x0, 0x0}, 0x0)
external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk(0x140001f1080, {0x14000136180, 0x7, 0xc}, {0x1400019bf60, 0x1, 0x1}, 0x0, 0x140004b5968)
external/bazel_gazelle/walk/walk.go:182 +0x198
main.runFixUpdate({0x1400004210a, 0x2b}, 0x0, {0x140001c0010, 0x0, 0x0})
external/bazel_gazelle/cmd/gazelle/fix-update.go:271 +0x5a0
main.run({0x1400004210a, 0x2b}, {0x140001c0010, 0x1, 0x1})
external/bazel_gazelle/cmd/gazelle/gazelle.go:95 +0x29c
main.main()
external/bazel_gazelle/cmd/gazelle/gazelle.go:72 +0x140

from rules_jvm.

I've been looking at this; it looks like the javaparser library we use parses uses of java.beans.Transient as java.beans..Transient which is an invalid package name and causes issues. I'll keep debugging...

This is specifically in 980-commons/src/main/java/io/harness/yaml/YamlRepresenter.java which you could try ignoring / commenting out some of the contents of.

from rules_jvm.

Thanks for the help, Daniel.

I re-ran after that fix and now the out of index error is back again.

Double checked that comm -23 <(fd '\.java$' | sort) <(fd '\.java$' | xargs grep -l '^package ' | sort) -- returns empty for me.

Were you able to see a successful run on your side?

~/source-code/harness-core $ bazel run //:gazelle
Starting local Bazel server and connecting to it...
INFO: Invocation ID: 65cd16d1-542d-41a1-b58d-bd417c375b78
INFO: Analyzed target //:gazelle (318 packages loaded, 18371 targets configured).
INFO: Found 1 target...
Target //:gazelle up-to-date:
  /private/var/tmp/_bazel_gauravnanda/31c0a55cc70a91108079d63a7aab4a62/execroot/harness_monorepo/bazel-out/darwin_arm64-fastbuild/bin/gazelle-runner.bash
  /private/var/tmp/_bazel_gauravnanda/31c0a55cc70a91108079d63a7aab4a62/execroot/harness_monorepo/bazel-out/darwin_arm64-fastbuild/bin/gazelle
INFO: Elapsed time: 10.711s, Critical Path: 0.59s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
panic: runtime error: index out of range [0] with length 0

goroutine 4229 [running]:
github.com/bazel-contrib/rules_jvm/java/gazelle/private/java.NewImport({0x14000028ae0, 0x15})
        external/contrib_rules_jvm/java/gazelle/private/java/import.go:19 +0x2b8
main.filterImports(0x140002361c0, {0x140000e6e60, 0x5, 0x5})
        external/contrib_rules_jvm/java/gazelle/private/javaparser/cmd/javaparser-wrapper/main.go:276 +0xbc
main.(*server).ParsePackage(0x14000260300, {0x1053e8c00, 0x140005b5050}, 0x140005a2b40)
        external/contrib_rules_jvm/java/gazelle/private/javaparser/cmd/javaparser-wrapper/main.go:212 +0x124
github.com/bazel-contrib/rules_jvm/java/gazelle/private/javaparser/proto/gazelle/java/javaparser/v0._JavaParser_ParsePackage_Handler.func1({0x1053e8c00, 0x140005b5050}, {0x10539f020, 0x140005a2b40})
        bazel-out/host/bin/external/contrib_rules_jvm/java/gazelle/private/javaparser/proto/gazelle/java/javaparser/v0/gazelle_java_build_v0_go_proto_/github.com/bazel-contrib/rules_jvm/java/gazelle/private/javaparser/proto/gazelle/java/javaparser/v0/javaparser.pb.go:319 +0x7c
main.activityTrackerUSI.func1({0x1053e8c00, 0x140005b5050}, {0x10539f020, 0x140005a2b40}, 0x140001cc6c0, 0x140001aec18)
        external/contrib_rules_jvm/java/gazelle/private/javaparser/cmd/javaparser-wrapper/main.go:182 +0x54
github.com/bazel-contrib/rules_jvm/java/gazelle/private/javaparser/proto/gazelle/java/javaparser/v0._JavaParser_ParsePackage_Handler({0x10535bb40, 0x14000260300}, {0x1053e8c00, 0x140005b5050}, 0x140004f08a0, 0x140004022e0)
        bazel-out/host/bin/external/contrib_rules_jvm/java/gazelle/private/javaparser/proto/gazelle/java/javaparser/v0/gazelle_java_build_v0_go_proto_/github.com/bazel-contrib/rules_jvm/java/gazelle/private/javaparser/proto/gazelle/java/javaparser/v0/javaparser.pb.go:321 +0x14c
google.golang.org/grpc.(*Server).processUnaryRPC(0x140001fb180, {0x1053efac0, 0x1400025cd00}, 0x140005a6a20, 0x14000238cf0, 0x1056bcb70, 0x0)
        external/org_golang_google_grpc/server.go:1282 +0xc3c
google.golang.org/grpc.(*Server).handleStream(0x140001fb180, {0x1053efac0, 0x1400025cd00}, 0x140005a6a20, 0x0)
        external/org_golang_google_grpc/server.go:1619 +0xa54
google.golang.org/grpc.(*Server).serveStreams.func1.2(0x1400019d120, 0x140001fb180, {0x1053efac0, 0x1400025cd00}, 0x140005a6a20)
        external/org_golang_google_grpc/server.go:921 +0x94
created by google.golang.org/grpc.(*Server).serveStreams.func1
        external/org_golang_google_grpc/server.go:919 +0x1f0
panic: rpc error: code = Unavailable desc = error reading from server: EOF

goroutine 1 [running]:
github.com/bazel-contrib/rules_jvm/java/gazelle.javaLang.GenerateRules({{0x1058fd478, 0x14000120250}, {0x1058fd4b0, 0x14000140d80}, 0x14000692000, {{0x1058f93e8, 0x14000140d60}, 0x1, {0x0, 0x0}, ...}, ...}, ...)
        external/contrib_rules_jvm/java/gazelle/generate.go:113 +0x1dd8
main.runFixUpdate.func1({0x14005c2bd40, 0x55}, {0x140062bd3e0, 0x29}, 0x140062ab600, 0x1, 0x0, {0x0, 0x0, 0x0}, ...)
        external/bazel_gazelle/cmd/gazelle/fix-update.go:294 +0xc0c
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x140060ce0b0, {0x14005c2bd40, 0x55}, {0x140062bd3e0, 0x29}, 0x1)
        external/bazel_gazelle/walk/walk.go:179 +0x500
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x140060ce000, {0x140055720a0, 0x50}, {0x14005f7ff50, 0x24}, 0x1)
        external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x14005a5fef0, {0x140054bbe50, 0x48}, {0x1400609e600, 0x1c}, 0x1)
        external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x14005a5fe40, {0x140054bbbd0, 0x45}, {0x1400609e540, 0x19}, 0x1)
        external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x14005a5fd90, {0x140060bc700, 0x40}, {0x140060ac3f0, 0x14}, 0x1)
        external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x14005a5fce0, {0x140060bc600, 0x3b}, {0x140058a7600, 0xf}, 0x1)
        external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x14000164580, {0x14005a65dc0, 0x37}, {0x140058a7350, 0xb}, 0x1)
        external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x14000164580, {0x14000036390, 0x2b}, {0x0, 0x0}, 0x0)
        external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk(0x14000164580, {0x140002fc180, 0x7, 0xc}, {0x14000140e30, 0x1, 0x1}, 0x0, 0x14000435968)
        external/bazel_gazelle/walk/walk.go:182 +0x198
main.runFixUpdate({0x1400004210a, 0x2b}, 0x0, {0x14000032050, 0x0, 0x0})
        external/bazel_gazelle/cmd/gazelle/fix-update.go:271 +0x5a0
main.run({0x1400004210a, 0x2b}, {0x14000032050, 0x1, 0x1})
        external/bazel_gazelle/cmd/gazelle/gazelle.go:95 +0x29c
main.main()
        external/bazel_gazelle/cmd/gazelle/gazelle.go:72 +0x140

from rules_jvm.

I have been building on develop rather than your branch and I managed to get a successful run (though some packages took >30 seconds to parse, so I bumped the 30 second timeout in java/gazelle/private/javaparser/cmd/javaparser-wrapper/main.go to 300 just to see things run - this is something we also will need to address in this codebase) - I'll try running in your branch now and see what happens...

from rules_jvm.

Oh thats great that you could get it working !!

However, I tried with the develop branch too today ( https://github.com/harness/harness-core/pull/34386/files#diff-7fc57714ef13c3325ce2a1130202edced92fcccc0c6db34a72f7b57f60d552a3 has the changes), but strangely got the same error. I did remove all the empty packages too. Can you please look at my changes or share your branch where you got it working?

panic: runtime error: index out of range [0] with length 0

goroutine 4388 [running]:
github.com/bazel-contrib/rules_jvm/java/gazelle/private/java.NewImport({0x140000ba198, 0x15})
external/contrib_rules_jvm/java/gazelle/private/java/import.go:19 +0x2b8
main.filterImports(0x140001cc620, {0x1400067e370, 0x5, 0x5})
external/contrib_rules_jvm/java/gazelle/private/javaparser/cmd/javaparser-wrapper/main.go:276 +0xbc
main.(*server).ParsePackage(0x14000120c00, {0x1006bcc00, 0x140000f9da0}, 0x140000f6c30)
external/contrib_rules_jvm/java/gazelle/private/javaparser/cmd/javaparser-wrapper/main.go:212 +0x124
github.com/bazel-contrib/rules_jvm/java/gazelle/private/javaparser/proto/gazelle/java/javaparser/v0.JavaParser_ParsePackage_Handler.func1({0x1006bcc00, 0x140000f9da0}, {0x100673020, 0x140000f6c30})
bazel-out/host/bin/external/contrib_rules_jvm/java/gazelle/private/javaparser/proto/gazelle/java/javaparser/v0/gazelle_java_build_v0_go_proto/github.com/bazel-contrib/rules_jvm/java/gazelle/private/javaparser/proto/gazelle/java/javaparser/v0/javaparser.pb.go:319 +0x7c
main.activityTrackerUSI.func1({0x1006bcc00, 0x140000f9da0}, {0x100673020, 0x140000f6c30}, 0x14000381500, 0x1400020e5b8)
external/contrib_rules_jvm/java/gazelle/private/javaparser/cmd/javaparser-wrapper/main.go:182 +0x54
github.com/bazel-contrib/rules_jvm/java/gazelle/private/javaparser/proto/gazelle/java/javaparser/v0.JavaParser_ParsePackage_Handler({0x10062fb40, 0x14000120c00}, {0x1006bcc00, 0x140000f9da0}, 0x14000100a20, 0x140000719c0)
bazel-out/host/bin/external/contrib_rules_jvm/java/gazelle/private/javaparser/proto/gazelle/java/javaparser/v0/gazelle_java_build_v0_go_proto/github.com/bazel-contrib/rules_jvm/java/gazelle/private/javaparser/proto/gazelle/java/javaparser/v0/javaparser.pb.go:321 +0x14c
google.golang.org/grpc.(*Server).processUnaryRPC(0x1400036e000, {0x1006c3ac0, 0x140003781a0}, 0x140002125a0, 0x1400034a7b0, 0x100990b70, 0x0)
external/org_golang_google_grpc/server.go:1282 +0xc3c
google.golang.org/grpc.(*Server).handleStream(0x1400036e000, {0x1006c3ac0, 0x140003781a0}, 0x140002125a0, 0x0)
external/org_golang_google_grpc/server.go:1619 +0xa54
google.golang.org/grpc.(*Server).serveStreams.func1.2(0x1400002da90, 0x1400036e000, {0x1006c3ac0, 0x140003781a0}, 0x140002125a0)
external/org_golang_google_grpc/server.go:921 +0x94
created by google.golang.org/grpc.(*Server).serveStreams.func1
external/org_golang_google_grpc/server.go:919 +0x1f0
panic: rpc error: code = Unavailable desc = error reading from server: EOF

goroutine 1 [running]:
github.com/bazel-contrib/rules_jvm/java/gazelle.javaLang.GenerateRules({{0x105491478, 0x14000010288}, {0x1054914b0, 0x1400019bf60}, 0x1400014c000, {{0x10548d3e8, 0x1400019bf40}, 0x1, {0x0, 0x0}, ...}, ...}, ...)
external/contrib_rules_jvm/java/gazelle/generate.go:113 +0x1dd8
main.runFixUpdate.func1({0x14000f4ccc0, 0x55}, {0x140061c8360, 0x29}, 0x14005899550, 0x1, 0x0, {0x0, 0x0, 0x0}, ...)
external/bazel_gazelle/cmd/gazelle/fix-update.go:294 +0xc0c
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x1400519edc0, {0x14000f4ccc0, 0x55}, {0x140061c8360, 0x29}, 0x1)
external/bazel_gazelle/walk/walk.go:179 +0x500
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x1400519ed10, {0x14001520230, 0x50}, {0x140054b0120, 0x24}, 0x1)
external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x1400519ea50, {0x140047079a0, 0x48}, {0x14003c7e820, 0x1c}, 0x1)
external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x1400519e630, {0x140047072c0, 0x45}, {0x14004aedac0, 0x19}, 0x1)
external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x14005085ef0, {0x14004d8b5c0, 0x40}, {0x1400649c828, 0x14}, 0x1)
external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x140050858c0, {0x14004d8b400, 0x3b}, {0x1400477f760, 0xf}, 0x1)
external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x140001f1080, {0x14004a11e00, 0x37}, {0x1400477f580, 0xb}, 0x1)
external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk.func1(0x140001f1080, {0x14000038270, 0x2b}, {0x0, 0x0}, 0x0)
external/bazel_gazelle/walk/walk.go:172 +0x694
github.com/bazelbuild/bazel-gazelle/walk.Walk(0x140001f1080, {0x14000126180, 0x7, 0xc}, {0x14000170010, 0x1, 0x1}, 0x0, 0x1400091f968)
external/bazel_gazelle/walk/walk.go:182 +0x198
main.runFixUpdate({0x140000460ba, 0x2b}, 0x0, {0x140001c0010, 0x0, 0x0})
external/bazel_gazelle/cmd/gazelle/fix-update.go:271 +0x5a0
main.run({0x140000460ba, 0x2b}, {0x140001c0010, 0x1, 0x1})
external/bazel_gazelle/cmd/gazelle/gazelle.go:95 +0x29c
main.main()
external/bazel_gazelle/cmd/gazelle/gazelle.go:72 +0x140

from rules_jvm.

@illicitonion - Can you please share your branch where you were able to successfully run it? -- want to check if I have something missing in my setup.

from rules_jvm.

I've pushed a branch to https://github.com/illicitonion/harness-core/tree/gazelle-java - PTAL!

from rules_jvm.

Thanks Daniel, I was able to get it running too. This would be extremely useful for us, managing dependencies manually is next to impossible!

A couple of follow up questions:

1. Is there a way to restrict gazelle execution on a sub-directory. It would be great if we could run this on every pre-submit!
2. I saw you used "https://github.com/bazel-contrib/rules_jvm/archive/ed3619f54e7fb19948928fe8516617aecd88a58c.zip" -- do we plan to have stable releases going forward, so that we can keep getting bug fixes?
3. During the execution, I saw a bunch of errors (pasted below), are these errors or warnings?

11:10AM ERR external/contrib_rules_jvm/java/gazelle/resolve.go:184 > convertImport found MULTIPLE results in rule index pkg=io.harness.checks targets=["@harness_monorepo//tools/cereal-killer/src/main/java/io/harness/checks","@harness_monorepo//tools/checkstyle/src/main/java/io/harness/checks"]
11:10AM ERR external/contrib_rules_jvm/java/gazelle/resolve.go:184 > convertImport found MULTIPLE results in rule index pkg=io.harness.checks targets=["@harness_monorepo//tools/cereal-killer/src/main/java/io/harness/checks","@harness_monorepo//tools/checkstyle/src/main/java/io/harness/checks"]
11:10AM ERR external/contrib_rules_jvm/java/gazelle/resolve.go:184 > convertImport found MULTIPLE results in rule index pkg=io.harness.checks targets=["@harness_monorepo//tools/cereal-killer/src/main/java/io/harness/checks","@harness_monorepo//tools/checkstyle/src/main/java/io/harness/checks"]
11:10AM ERR external/contrib_rules_jvm/java/gazelle/resolve.go:184 > convertImport found MULTIPLE results in rule index pkg=io.harness.checks targets=["@harness_monorepo//tools/cereal-killer/src/main/java/io/harness/checks","@harness_monorepo//tools/checkstyle/src/main/java/io/harness/checks"]
11:10AM ERR external/contrib_rules_jvm/java/gazelle/resolve.go:184 > convertImport found MULTIPLE results in rule index pkg=io.harness.checks targets=["@harness_monorepo//tools/cereal-killer/src/main/java/io/harness/checks","@harness_monorepo//tools/checkstyle/src/main/java/io/harness/checks"]
11:10AM ERR external/contrib_rules_jvm/java/gazelle/resolve.go:184 > convertImport found MULTIPLE results in rule index pkg=io.harness.checks targets=["@harness_monorepo//tools/cereal-killer/src/main/java/io/harness/checks","@harness_monorepo//tools/checkstyle/src/main/java/io/harness/checks"]
11:10AM ERR external/contrib_rules_jvm/java/gazelle/resolve.go:184 > convertImport found MULTIPLE results in rule index pkg=io.harness.checks targets=["@harness_monorepo//tools/cereal-killer/src/main/java/io/harness/checks","@harness_monorepo//tools/checkstyle/src/main/java/io/harness/checks"]

from rules_jvm.

• Is there a way to restrict gazelle execution on a sub-directory. It would be great if we could run this on every pre-submit!

You can add comments to BUILD.bazel files of the form # gazelle:ignore path to not run Gazelle for certain directories, e.g. # gazelle:ignore 400-rest would cause gazelle to ignore that directory completely (i.e. neither try to parse it, nor try to generate BUILD files for it) - so you could list all of your exclusions. Otherwise, I think it'd be a feature request for https://github.com/bazelbuild/bazel-gazelle to be more configurable.

• I saw you used "https://github.com/bazel-contrib/rules_jvm/archive/ed3619f54e7fb19948928fe8516617aecd88a58c.zip" -- do we plan to have stable releases going forward, so that we can keep getting bug fixes?

There are a few releases in https://github.com/bazel-contrib/rules_jvm/tags - I'm not sure what the release policy is for this repo specifically, but there's at least capability to perform releases :)

During the execution, I saw a bunch of errors (pasted below), are these errors or warnings?

I also see them! These are warnings, but they're pointing to problems you probably care about. The way gazelle for java works is that it tracks each Java package based on package directives in Java files, and then when it sees a symbol being used (e.g. imported) it finds its package, then looks up in its index from parsing to see which BUILD target generates that package.

These warnings are saying "multiple directories are outputting classes into the same package" - this means that gazelle doesn't know which directory a dependency should be added on (so it won't generate any dependency at all), but also is a problem because newer versions of Java support java modules, which don't allow different projects to define classes in the same package.

When rolling out java-gazelle in our internal repos, we've been either moving files to new packages where this happens, or moving around files so that the package is defined by one unique directory on the filesystem, but I realise this is a non-trivial amount of work in such a large project!

from rules_jvm.

Thanks Daniel for the detailed answers! You are right that moving files around would be a non-trivial amount of work for sure.

Otherwise, do you think that it is possible to switch tracking from Package level index to the complete class path?

from rules_jvm.

Also, one more question -- how does the tool ignore the test directories? Generally the test directories are different but they have the same package name.

from rules_jvm.