Under go 1.23, golangci-lint uses all available memory #4909

jaevans · 2024-08-14T13:16:54Z

Welcome

Yes, I'm using a binary release within 2 latest releases. Only such installations are supported.
Yes, I've searched similar issues on GitHub and didn't find any.
Yes, I've read the typecheck section of the FAQ.
Yes, I've tried with the standalone linter if available (e.g., gocritic, go vet, etc.).
I agree to follow this project's Code of Conduct

Description of the problem

Updating the toolchain line to 1.23.0 in go.mod causes the linter to run out of memory. I've duplicated this on both linux x86_64 and darwin arm64. I haven't yet identified which linter is causing the problem, but it's one of the default set

Version of golangci-lint

$ golangci-lint --version
golangci-lint has version 1.59.1 built with go1.22.4 from 1a55854 on 2024-06-08T22:05:49Z

Configuration

No configuration file or command line arguments passed.

Go environment

$ go version && go env
go version go1.23.0 darwin/arm64
GO111MODULE=''
GOARCH='arm64'
GOBIN=''
GOCACHE='/Users/jaevans/Library/Caches/go-build'
GOENV='/Users/jaevans/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/jaevans/go/pkg/mod'
GONOPROXY='gitlab.sicura.dev/*'
GONOSUMDB='gitlab.sicura.dev/*'
GOOS='darwin'
GOPATH='/Users/jaevans/go'
GOPRIVATE='gitlab.sicura.dev/*'
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/Users/jaevans/go/pkg/mod/golang.org/[email protected]'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/Users/jaevans/go/pkg/mod/golang.org/[email protected]/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.23.0'
GODEBUG=''
GOTELEMETRY='local'
GOTELEMETRYDIR='/Users/jaevans/Library/Application Support/go/telemetry'
GCCGO='gccgo'
GOARM64='v8.0'
AR='ar'
CC='clang'
CXX='clang++'
CGO_ENABLED='1'
GOMOD='/Users/jaevans/git/test-app/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/f3/1lg41dsx5cqc1vnt1x37x6_40000gn/T/go-build785670845=/tmp/go-build -gno-record-gcc-switches -fno-common'

Verbose output of running

$ golangci-lint cache clean
$ golangci-lint run -v
INFO golangci-lint has version 1.59.1 built with go1.22.4 from 1a55854 on 2024-06-08T22:05:49Z
INFO [config_reader] Config search paths: [./ /Users/jaevans/git/test-appller /Users/jaevans/git/saas /Users/jaevans/git /Users/jaevans /Users /]
INFO [lintersdb] Active 6 linters: [errcheck gosimple govet ineffassign staticcheck unused]
INFO [loader] Go packages loading at mode 575 (imports|name|types_sizes|compiled_files|deps|exports_file|files) took 428.207ms
INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 1.086542ms
^C (after memory usage grew above 32Gig

A minimal reproducible example or link to a public repository

semvertool (golangci-lint-issue branch)

Validation

Yes, I've included all information above (version, config, etc.).

Supporter

I am a sponsor/backer through GitHub or OpenCollective

The text was updated successfully, but these errors were encountered:

boring-cyborg · 2024-08-14T13:16:57Z

Hey, thank you for opening your first Issue ! 🙂 If you would like to contribute we have a guide for contributors.

ldez · 2024-08-14T13:19:24Z

You are using golangci-lint v1.59.1 (built with go1.22.4) to analyze go1.23 code, this cannot work.

You should use golangci-lint v1.60.0 (built with go1.23)

Related to #4837

Duplicate of #4874, #4908

scvna · 2024-08-14T14:12:59Z

@ldez FYI you're likely to continue to see reports like this come in from homebrew users. golangci-lint v1.60.1 on homebrew was built with go1.22.6

golangci-lint has version 1.60.1 built with go1.22.6 from 3298c10 on 2024-08-14T01:09:25Z

Building from source on go 1.23.0 works as expected.

ldez · 2024-08-14T14:21:12Z

I know, it's always the same thing when a new version of Go is released: homebrew uses the previous version of Go.

skaji · 2024-08-15T00:31:51Z

How about upgrading the Go version to 1.23.0 in the go.mod file?
https://github.com/golangci/golangci-lint/blob/master/go.mod#L3

Then homebrew cannot build golangci-lint on go 1.22, I think.

ldez · 2024-08-15T00:41:23Z

It's not possible because golangci-lint needs to be compiled with go1.22 and go1.23.

The go version inside the go.mod is a hard requirement.
The users that use the "tools pattern" need to compile with go1.22.

skaji · 2024-08-15T00:43:51Z

The users that use the "tools pattern" need to compile with go1.22.

I see.

This also required upgrading golangci-lint due to golangci/golangci-lint#4909

mskonovalov · 2024-08-20T01:05:49Z

Sorry, is it the same thing with nix?
I switched to go 1.23 and golangci-lint 1.60.1 but symptoms are the same :(

ldez · 2024-08-20T03:43:21Z

is it the same thing with nix?

It depends on your system, and how you build golangci-lint.

If golangci-lint v1.60.1 is compiled by go1.23, there is no problem.

mskonovalov · 2024-08-20T04:04:28Z

Yeah, I see. Thanks
I guess golangci-lint v1.60.1 for nix may be built with go 1.22.
https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/tools/golangci-lint/default.nix
Although I'm not sure how to approach this.
@ldez do you think it might be beneficial to have 2 binaries: 1.60.1-go122 and 1.60.1-go123?

ldez · 2024-08-20T04:07:28Z

We don't need 2 binaries because a binary compiled with go1.23 works with go1.22.
We, officially, only compile with go1.23 to support both.

Before each release of Go, I create an issue to explain that, the issue about Go1.23 is still pinned.

To see the version used to compile: golangci-lint version

golangci-lint has version 1.60.1 built with go1.23.0 from 3298c104 on 2024-08-14T01:15:05Z

We are not responsible for package systems that wrongly compile golangci-lint.
We only support official binaries.

mskonovalov · 2024-08-20T04:21:26Z

thanks for the explanation 👍
nix definitely has the wrong one :(

pugnascotia · 2024-09-09T15:43:26Z

Is there a regression here? I downloaded a fresh binary on Mac ARM and I see:

golangci-lint has version v1.60.3 built with go1.22.7 from (unknown, modified: ?, mod sum: "h1:l38A5de24ZeDlcFF+EB7m3W5joPD99/hS5SIHJPyZa0=") on (unknown)

ldez · 2024-09-09T16:18:16Z

No regression, you should use go1.23 to compile golangci-lint if you want to analyze go1.23 code.

pugnascotia · 2024-09-09T18:47:54Z

Huh. So I was installing with go install ..., but despite using 1.23.0 to do this, it still fetched a binary for 1.22.*. Weird.

ldez · 2024-09-09T19:32:04Z

go install doesn't fetch binaries, it builds the binary from sources, so you are using go1.22 to compile.

ingwarsw · 2024-11-06T09:24:18Z

You are using golangci-lint v1.59.1 (built with go1.22.4) to analyze go1.23 code, this cannot work.

You should use golangci-lint v1.60.0 (built with go1.23)

@ldez while I agree with your answer that it cannot work.
But it should either fail, or do whatever it could, but not eat all the memory on the world.

If you know its not supported then the best way would be to fail with message.
I was compiled with 1.22 support and your trying to force me to work with 1.23, go away and update me plisss

ldez · 2024-11-06T10:26:02Z

Since v1.60.3 (2024-08-23), there is a log https://golangci-lint.run/product/changelog/#v1603

$ ./golangci-lint run
Error: can't load config: the Go language version (go1.22) used to build golangci-lint is lower than the targeted Go version (1.23)
Failed executing command with error: can't load config: the Go language version (go1.22) used to build golangci-lint is lower than the targeted Go version (1.23)

Related to #4938

ingwarsw · 2024-11-07T09:00:33Z

I didnt know that..
But I think it dont work in all cases..

My tests all using same linter.

golangci-lint has version v1.61.0 built with go1.22.1 from (unknown, modified: ?, mod sum: "h1:VvbOLaRVWmyxCnUIMTbf1kDsaJbTzH20FAMXTAlQGu8=") on (unknown)

go version go1.22.1 darwin/arm64 + go.mod -> go 1.22.0/no toolchain
works as expected
go version go1.23.1 darwin/arm64 + go.mod -> go 1.22.0/no toolchain
Eats all the memory and hangs forever
go version go1.22.1 darwin/arm64 + go.mod -> go 1.23.0/no toolchain
Throws error "Error: can't load config: the Go language version (go1.22) used to build golangci-lint is lower than the targeted Go version (1.23.0)" which is expected
go version go1.23.1 darwin/arm64 + go.mod -> go 1.23.0/no toolchain
Throws error "Error: can't load config: the Go language version (go1.22) used to build golangci-lint is lower than the targeted Go version (1.23.0)" which is expected
go version go1.22.1 darwin/arm64 + go.mod -> go 1.22.0/toolchain go1.23.1
Eats all the memory and hangs forever
go version go1.23.1 darwin/arm64 + go.mod -> go 1.22.0/toolchain go1.23.1
Eats all the memory and hangs forever

So seems like its not even issue with toolchain but the check only checks the go version in the go.mod file and dont check actually used go runtime version.

ldez · 2024-11-07T10:56:50Z

build of golangci-lint with go1.22

$ export GOTOOLCHAIN=local

$ go version
go version go1.22.9 linux/amd64

$ ./golangci-lint version
golangci-lint has version (devel) built with go1.22.9 from (a1d6c560de1a193a0c68ffed68cd5928ef39e884, modified: false, mod sum: "") on 2024-09-09T14:33:19Z

go1.22.x + go.mod 1.22.0 -> OK

$ export GOTOOLCHAIN=local
$ go version
go version go1.22.9 linux/amd64

$ golangci-lint cache clean

$ ./golangci-lint run

go1.23.x + go.mod 1.22.0 -> OK

$ export GOTOOLCHAIN=local
$ go version
go version go1.23.1 linux/amd64

$ golangci-lint cache clean

$ ./golangci-lint run                          
../../../../../usr/lib/go/src/slices/iter.go:50:17: cannot range over seq (variable of type iter.Seq[E]) (typecheck)
        for v := range seq {
                       ^
../../../../../usr/lib/go/src/os/stat_linux.go:47:57: not enough arguments in call to time.Unix
        have (unknown type)
        want (int64, int64) (typecheck)
        return time.Unix(fi.Sys().(*syscall.Stat_t).Atim.Unix())

$ export GOTOOLCHAIN=local
$ go version
go version go1.23.1 linux/amd64

$ export GOTOOLCHAIN=auto
$ go version               
go version go1.23.1 linux/amd64

$ golangci-lint cache clean

$ ./golangci-lint run                          
../../../../../usr/lib/go/src/slices/iter.go:50:17: cannot range over seq (variable of type iter.Seq[E]) (typecheck)
        for v := range seq {
                       ^
../../../../../usr/lib/go/src/os/stat_linux.go:47:57: not enough arguments in call to time.Unix
        have (unknown type)
        want (int64, int64) (typecheck)
        return time.Unix(fi.Sys().(*syscall.Stat_t).Atim.Unix())

go1.22.x + go.mod 1.23.0 -> OK

$ export GOTOOLCHAIN=local
$ go version
go version go1.22.9 linux/amd64

$ golangci-lint cache clean

$ ./golangci-lint run 
ERRO Running error: context loading failed: failed to load packages: failed to load packages: failed to load with go/packages: err: exit status 1: stderr: go: go.mod requires go >= 1.23.0 (running go 1.22.9; GOTOOLCHAIN=local)

$ export GOTOOLCHAIN=local
$ go version
go version go1.22.9 linux/amd64

$ export GOTOOLCHAIN=auto
$ go version               
go version go1.23.0 linux/amd64

$ golangci-lint cache clean

$ ./golangci-lint run
Error: can't load config: the Go language version (go1.22) used to build golangci-lint is lower than the targeted Go version (1.23.0)
Failed executing command with error: can't load config: the Go language version (go1.22) used to build golangci-lint is lower than the targeted Go version (1.23.0)

go1.23.x + go.mod 1.23.0 -> OK

$ export GOTOOLCHAIN=local
$ go version                  
go version go1.23.1 linux/amd64

$ golangci-lint cache clean
$ ./golangci-lint run   
Error: can't load config: the Go language version (go1.22) used to build golangci-lint is lower than the targeted Go version (1.23.0)
Failed executing command with error: can't load config: the Go language version (go1.22) used to build golangci-lint is lower than the targeted Go version (1.23.0)

go1.22.x + go.mod 1.22.0, toolchain go1.23.1 -> OK

$ export GOTOOLCHAIN=local
$ go version               
go version go1.22.9 linux/amd64

$ golangci-lint cache clean

$ ./golangci-lint run

go1.23.x + go.mod 1.22.0, toolchain go1.23.1 -> OK

$ export GOTOOLCHAIN=local

$ go version                  
go version go1.23.1 linux/amd64

$ golangci-lint cache clean

$ ./golangci-lint run 
../../../../../usr/lib/go/src/slices/iter.go:50:17: cannot range over seq (variable of type iter.Seq[E]) (typecheck)
        for v := range seq {
                       ^
../../../../../usr/lib/go/src/os/stat_linux.go:47:57: not enough arguments in call to time.Unix
        have (unknown type)
        want (int64, int64) (typecheck)
        return time.Unix(fi.Sys().(*syscall.Stat_t).Atim.Unix())

ingwarsw · 2024-11-07T13:12:26Z

I was able to create repro case..
Its not just about the versions but the code itself.

I found its breaking for github.com/grpc-ecosystem/grpc-gateway/v2 package.

You can found all repro files under https://github.com/kentik/.github/pull/1/files

https://github.com/kentik/.github/actions/runs/11723589591/job/32655587129 <- killed by GH after 2m when toolchain is set
https://github.com/kentik/.github/actions/runs/11723611754/job/32655657846 <- passed without toolchain

The line thats makes if fail or pass

I have also testd without toolchain but with installing go 1.23 and it fails as well see https://github.com/kentik/.github/actions/runs/11723770608/job/32656157617

ldez · 2024-11-07T13:27:17Z

I have no problem even with toolchain.

OK

$ docker run --rm -v $(pwd):/app -w /app -it golang:1.22.1-alpine sh  
/app # go install github.com/golangci/golangci-lint/cmd/[email protected]
...
/app # golangci-lint version
golangci-lint has version v1.61.0 built with go1.22.1 from (unknown, modified: ?, mod sum: "h1:VvbOLaRVWmyxCnUIMTbf1kDsaJbTzH20FAMXTAlQGu8=") on (unknown)
/app # go version
go version go1.22.1 linux/amd64
/app # golangci-lint run

Details

module github.com/golangci/sandbox

go 1.22.0

toolchain go1.23.1

require github.com/grpc-ecosystem/grpc-gateway/v2 v2.23.0

require (
	golang.org/x/net v0.28.0 // indirect
	golang.org/x/sys v0.24.0 // indirect
	golang.org/x/text v0.19.0 // indirect
	google.golang.org/genproto/googleapis/api v0.0.0-20241021214115-324edc3d5d38 // indirect
	google.golang.org/genproto/googleapis/rpc v0.0.0-20241021214115-324edc3d5d38 // indirect
	google.golang.org/grpc v1.67.1 // indirect
	google.golang.org/protobuf v1.35.1 // indirect
)

ingwarsw · 2024-11-07T13:36:09Z

You have the code on GH. and you see its failing on GH..
I cannot do much more..
Try with my code..

ingwarsw · 2024-11-07T13:43:08Z

But your not doing actually golangci-lint run
And your setting GOTOOLCHAIN to lower than go version and should be opposite.

ldez · 2024-11-07T13:44:00Z

https://github.com/ldez/cuddly-funicular/actions/runs/11724207066

ldez/cuddly-funicular@4a5aced

And your setting GOTOOLCHAIN to lower than go version and should be opposite

No, the GOTOOLCHAIN is only used to build golangci-lint, not to run.

ingwarsw · 2024-11-07T13:47:39Z

Yup my bad.. but seems that you have it now.
https://github.com/ldez/cuddly-funicular/actions/runs/11724253936

ldez · 2024-11-07T13:50:31Z

Build with go1.22 + go.mod 1.22 toolchain go1.23.1

~~local Go version 1.22 -> fail~~ Go was downloading go1.23
local Go version 1.23 -> fail

https://github.com/ldez/cuddly-funicular/actions/runs/11724207066
ldez/cuddly-funicular@4a5aced

Build with go1.22 + go.mod 1.22 no toolchain

local Go version 1.22 -> OK
local Go version 1.23 -> fail

https://github.com/ldez/cuddly-funicular/actions/runs/11724253936
ldez/cuddly-funicular@dcbedb3

ingwarsw · 2024-11-07T13:53:36Z

So you have your case?

ldez · 2024-11-07T13:56:02Z

Something that is not reproducible locally is complex to diagnose.
I need to understand what is the exact context inside GitHub Action.

ingwarsw · 2024-11-07T13:56:46Z

why its not reproducible locally?
Just do the same thats on GH and its 100% reproducible..

ldez · 2024-11-07T13:59:00Z

why its not reproducible locally?

This is the question.

ldez · 2024-11-07T14:01:37Z

Build with go1.22 + go.mod 1.22 toolchain go1.23.1 + global GOTOOLCHAIN: local

local Go version 1.22 -> OK
local Go version 1.23 -> fail

So I'm staring to understand.

ldez · 2024-11-07T14:05:33Z

The only case (A) that really fails is: build with go1.22 + go.mod 1.22 toolchain go1.23.1 + local Go version 1.23 (global GOTOOLCHAIN: local).

The other case (B) is: build with go1.22 + go.mod 1.22 toolchain go1.23.1 + local Go version 1.22 (with GOTOOLCHAIN: auto [the default])

The case (B) is the same as the case (A) because actions/setup-go downloads go1.23 (because of toolchain go1.23.1)

ldez · 2024-11-07T14:14:54Z

So now I can reproduce the problem inside a container:

$ docker run --rm -v $(pwd):/app -w /app -it golang:1.23.2-alpine sh  
/app # GOTOOLCHAIN=go1.22.1 go install github.com/golangci/golangci-lint/cmd/[email protected]
...
/app # golangci-lint version
golangci-lint has version v1.61.0 built with go1.22.1 from (unknown, modified: ?, mod sum: "h1:VvbOLaRVWmyxCnUIMTbf1kDsaJbTzH20FAMXTAlQGu8=") on (unknown)

/app # rm -rf /go/pkg/mod/golang.org/toolchain@*

/app # go version
go version go1.23.2 linux/amd64

/app # golangci-lint run
###### High memory and CPU consumption and kill

ldez · 2024-11-07T14:29:33Z

I have a fix.

ldez · 2024-11-07T15:05:53Z

The fix: #5112

jaevans added the bug Something isn't working label Aug 14, 2024

ldez closed this as completed Aug 14, 2024

ldez added question Further information is requested and removed bug Something isn't working labels Aug 14, 2024

ldez mentioned this issue Aug 14, 2024

gosimple + go 1.23 consume all available RAM #4908

Closed

7 tasks

CHC383 mentioned this issue Aug 15, 2024

Plans for versioned tracks? alexmurray/golangci-lint-snap#4

Open

roosterfish mentioned this issue Aug 16, 2024

Build microovn from last compatible version canonical/microcloud#375

Merged

dgl mentioned this issue Aug 19, 2024

Upgrade golangci-lint to v1.60.1 prometheus/prometheus#14691

Merged

weseven mentioned this issue Aug 19, 2024

Update golangci-lint to v1.60.1 (go 1.23 support) giantswarm/architect#1005

Merged

alecthomas mentioned this issue Aug 19, 2024

chore: upgrade to go-1.23.0 TBD54566975/ftl#2430

Merged

github-merge-queue bot pushed a commit to TBD54566975/ftl that referenced this issue Aug 19, 2024

chore: upgrade to go-1.23.0 (#2430)

e9be43b

This also required upgrading golangci-lint due to golangci/golangci-lint#4909

safeer pushed a commit to TBD54566975/ftl that referenced this issue Aug 19, 2024

chore: upgrade to go-1.23.0 (#2430)

92f5e48

This also required upgrading golangci-lint due to golangci/golangci-lint#4909

This was referenced Aug 21, 2024

When running the linter, my PC crashes or the linter dies #4933

Closed

feat: check that Go version use to build is greater or equals to the Go version of the project #4938

Merged

notque mentioned this issue Aug 26, 2024

Int conversion checks sapcc/hermes#248

Closed

bartsmykla mentioned this issue Sep 10, 2024

chore(deps): bump go from 1.22.7 to 1.23.1 kumahq/kuma#11363

Merged

5 tasks

ptnapoleon mentioned this issue Sep 11, 2024

Upgrade to go 1.23 with latest golangci-lint DataDog/chaos-controller#906

Merged

13 tasks

ldez mentioned this issue Sep 19, 2024

Possible memory leak on macOS running in Docker #5030

Closed

7 tasks

ldez mentioned this issue Nov 7, 2024

fix: improve Go detection #5112

Merged

golangci locked as resolved and limited conversation to collaborators Nov 8, 2024

Under go 1.23, golangci-lint uses all available memory #4909

Under go 1.23, golangci-lint uses all available memory #4909

Comments

jaevans commented Aug 14, 2024

Welcome

Description of the problem

Version of golangci-lint

Configuration

Go environment

Verbose output of running

A minimal reproducible example or link to a public repository

Validation

Supporter

boring-cyborg bot commented Aug 14, 2024

ldez commented Aug 14, 2024 • edited Loading

scvna commented Aug 14, 2024

ldez commented Aug 14, 2024

skaji commented Aug 15, 2024

ldez commented Aug 15, 2024

skaji commented Aug 15, 2024

mskonovalov commented Aug 20, 2024

ldez commented Aug 20, 2024

mskonovalov commented Aug 20, 2024 • edited Loading

ldez commented Aug 20, 2024 • edited Loading

mskonovalov commented Aug 20, 2024

pugnascotia commented Sep 9, 2024

ldez commented Sep 9, 2024 • edited Loading

pugnascotia commented Sep 9, 2024

ldez commented Sep 9, 2024

ingwarsw commented Nov 6, 2024 • edited Loading

ldez commented Nov 6, 2024 • edited Loading

ingwarsw commented Nov 7, 2024

ldez commented Nov 7, 2024 • edited Loading

ingwarsw commented Nov 7, 2024 • edited Loading

ldez commented Nov 7, 2024

ingwarsw commented Nov 7, 2024

ingwarsw commented Nov 7, 2024

ldez commented Nov 7, 2024 • edited Loading

ingwarsw commented Nov 7, 2024

ldez commented Nov 7, 2024 • edited Loading

ingwarsw commented Nov 7, 2024

ldez commented Nov 7, 2024

ingwarsw commented Nov 7, 2024

ldez commented Nov 7, 2024

ldez commented Nov 7, 2024

ldez commented Nov 7, 2024

ldez commented Nov 7, 2024 • edited Loading

ldez commented Nov 7, 2024

ldez commented Nov 7, 2024

ldez commented Aug 14, 2024 •

edited

Loading

mskonovalov commented Aug 20, 2024 •

edited

Loading

ldez commented Aug 20, 2024 •

edited

Loading

ldez commented Sep 9, 2024 •

edited

Loading

ingwarsw commented Nov 6, 2024 •

edited

Loading

ldez commented Nov 6, 2024 •

edited

Loading

ldez commented Nov 7, 2024 •

edited

Loading

ingwarsw commented Nov 7, 2024 •

edited

Loading

ldez commented Nov 7, 2024 •

edited

Loading

ldez commented Nov 7, 2024 •

edited

Loading

ldez commented Nov 7, 2024 •

edited

Loading