Excessive memory use with staticcheck and gosimple #5449

ulope · 2025-02-19T17:26:59Z

Welcome

Yes, I'm using a binary release within 2 latest releases. Only such installations are supported.
Yes, I've searched similar issues on GitHub and didn't find any.
Yes, I've read the typecheck section of the FAQ.
Yes, I've tried with the standalone linter if available (e.g., gocritic, go vet, etc.).
I agree to follow this project's Code of Conduct

How did you install golangci-lint?

Official binary

Description of the problem

After upgrading from 1.55.2 to 1.64.5 we got CI failures due to OOM kills.

Bisecting our enabled linters revealed that enabling either staticcheck or gosimple leads to excessive memory use. On my local machine I manually killed the process at around 34GB usage.

Version of golangci-lint

$ golangci-lint --version
golangci-lint has version 1.64.5 built with go1.24.0 from 0a603e49 on 2025-02-13T21:19:55

Configuration

linters-settings:
  unparam:
    check-exported: true
  depguard:
    rules:
      main:
        files:
          - $all
        deny:
          - pkg: github.com/sirupsen/logrus
            description: "Don't use"
  dupl:
    threshold: 100
  exhaustive:
    default-signifies-exhaustive: false
  forbidigo:
    forbid:
      - "pk\\b"
  funlen:
    lines: 100
    statements: 50
  gci:
    custom-order: true
    sections:
      - Standard
      - Default
      - Prefix(github.com/shutter-network/shutter)
      - Prefix(github.com/shutter-network/rolling-shutter)
  goconst:
    min-len: 2
    min-occurrences: 2
  gocritic:
    enabled-tags:
      - diagnostic
      - experimental
      - opinionated
      - performance
      - style
    disabled-checks:
      - dupImport # https://github.com/go-critic/go-critic/issues/845
      - ifElseChain
      - octalLiteral
      - whyNoLint
      - wrapperFunc
      - paramTypeCombine
      - hugeParam
      - unnamedResult
      - rangeValCopy
      - typeDefFirst
  gocyclo:
    min-complexity: 15
  goimports:
    local-prefixes: github.com/shutter-network/shutter
  gomnd:
    settings:
      mnd:
        # don't include the "operation" and "assign"
        checks: argument,case,condition,return
  govet:
    check-shadowing: true
    settings:
      printf:
        funcs:
          - (github.com/golangci/golangci-lint/pkg/logutils.Log).Infof
          - (github.com/golangci/golangci-lint/pkg/logutils.Log).Warnf
          - (github.com/golangci/golangci-lint/pkg/logutils.Log).Errorf
          - (github.com/golangci/golangci-lint/pkg/logutils.Log).Fatalf
  lll:
    line-length: 140
  maligned:
    suggest-new: true
  misspell:
    locale: US
  nolintlint:
    allow-leading-space: true # don't require machine-readable nolint directives (i.e. with no leading space)
    allow-unused: false # report any unused nolint directives
    require-explanation: false # don't require an explanation for nolint directives
    require-specific: false # don't require nolint directives to be specific about which linter is being skipped
  revive:
    ignore-generated-header: true
    severity: warning
    rules:
      - name: blank-imports
      - name: context-as-argument
      - name: context-keys-type
      - name: dot-imports
      - name: error-return
      - name: error-strings
      - name: error-naming
      # - name: exported
      # - name: if-return
      - name: increment-decrement
      #- name: var-declaration
      - name: package-comments
      - name: range
      - name: receiver-naming
      - name: time-naming
      - name: unexported-return
      - name: indent-error-flow
      - name: errorf
      - name: empty-block
      - name: superfluous-else
      - name: unused-parameter
      - name: unreachable-code
      - name: redefines-builtin-id
      - name: unnecessary-stmt
      - name: range-val-in-closure
      - name: atomic
      - name: superfluous-else

linters:
  disable-all: true
  enable:
    - bodyclose
    - copyloopvar
    - dogsled
    - errcheck
    - exhaustive
    - forbidigo
    - funlen
    - gci
    - goconst
    - gocritic
    - gocyclo
    - godot
    - gofumpt
    - goprintffuncname
    - gosec
    - gosimple
    - govet
    - ineffassign
    - lll
    - misspell
    - nakedret
    - nilnesserr
    - noctx
    - nolintlint
    - revive
    - staticcheck
    - stylecheck
    - thelper
    - typecheck
    - unconvert
    - unparam
    - whitespace

  # don't enable:
  # - asciicheck
  # - gochecknoglobals
  # - gocognit
  # - goerr113
  # - maligned
  # - nestif
  # - prealloc
  # - testpackage
  # - wsl
  # - godox

issues:
  exclude:
    - "typeUnparen: could simplify \\(func.* to func\\("
    - "Error return value of `.*Mark.*Flag.*` is not checked"
    - "Error return value of `viper.BindEnv` is not checked"
    - 'shadow: declaration of "err" shadows declaration at line'
    - "Expect WriteFile permissions to be"
    - "GobEncode - result 1 .* is always nil"
  # Excluding configuration per-path, per-linter, per-text and per-source
  exclude-rules:
    - path: _test\.go
      linters:
        - gomnd
        - staticcheck
    - path: evtype/evtype\.go
      linters:
        - gosec
    # https://github.com/go-critic/go-critic/issues/926
    - linters:
        - gocritic
      text: "unnecessaryDefer:"

run:
  timeout: 10m
  skip-dirs:
    - test/testdata_etc
    - internal/cache
    - internal/renameio
    - internal/robustio

Go environment

$ go version && go env
# paste output here

Verbose output of running

On CircleCI with 16GB ram limit:

$ golangci-lint cache clean
$ base=`git merge-base HEAD origin/main`; \
golangci-lint --verbose run --concurrency 3 --print-resources-usage --new-from-rev ${base}
INFO golangci-lint has version 1.64.5 built with go1.24.0 from 0a603e49 on 2025-02-13T21:19:55Z 
INFO [config_reader] Config search paths: [./ /home/circleci/src/rolling-shutter /home/circleci/src /home/circleci /home /] 
INFO [config_reader] Used config file .golangci.yml 
WARN [config_reader] The configuration option `run.skip-dirs` is deprecated, please use `issues.exclude-dirs`. 
WARN [config_reader] The configuration option `linters.govet.check-shadowing` is deprecated. Please enable `shadow` instead, if you are not using `enable-all`. 
INFO [goenv] Read go env for 8.197797ms: map[string]string{"GOCACHE":"/home/circleci/.cache/go-build", "GOROOT":"/home/circleci/go/pkg/mod/golang.org/[email protected]"} 
INFO [lintersdb] Active 31 linters: [bodyclose copyloopvar dogsled errcheck exhaustive forbidigo funlen gci goconst gocritic gocyclo godot gofumpt goprintffuncname gosec gosimple govet ineffassign lll misspell nakedret nilnesserr noctx nolintlint revive staticcheck stylecheck thelper unconvert unparam whitespace] 
INFO [loader] Go packages loading at mode 8767 (files|imports|name|compiled_files|deps|exports_file|types_sizes) took 23.426666103s 
INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 29.407177ms 
Killed
make: *** [Makefile:113: lint-changes] Error 137

A minimal reproducible example or link to a public repository

Public repo (branch): https://github.com/shutter-network/rolling-shutter/tree/chore/upgrade-go-1.23.6
Run make lint-changes

Validation

Yes, I've included all information above (version, config, etc.).

Supporter

I am a sponsor/backer through GitHub or OpenCollective

The text was updated successfully, but these errors were encountered:

boring-cyborg · 2025-02-19T17:27:02Z

Hey, thank you for opening your first Issue ! 🙂 If you would like to contribute we have a guide for contributors.

ldez · 2025-02-19T17:36:29Z

Based on your log I think your problem is not here, you have a compilation issue (typecheck errors).

ulope · 2025-02-19T17:41:49Z

This is with staticcheck and gosimple commented out (sorry should have mentioned this above).

ldez · 2025-02-19T17:41:49Z

I think you are not using the right Go version inside your CI.

https://github.com/shutter-network/rolling-shutter/blob/fd87d6ae09227a7719a22fb13ac0bad0cc6ca418/.circleci/config.yml#L39

https://github.com/shutter-network/rolling-shutter/blob/fd87d6ae09227a7719a22fb13ac0bad0cc6ca418/.circleci/config.yml#L32

ldez · 2025-02-19T17:44:41Z

You are reporting 2 things: a local problem and a CI problem.

For the local problem, you should check the Go version used to build golangci-lint:

$ golangci-lint version
golangci-lint has version 1.64.5 built with go1.24.0 from 0a603e49 on 2025-02-13T21:19:55Z

And your local go version.

$ go version                                                                       
go version go1.24.0 linux/amd64

For the CI, the Go version can be the problem.

ulope · 2025-02-19T17:48:45Z

I think you are not using the right Go version inside your CI.

True, thanks, got overlooked in this branch. But should this matter? We're using the official binary build (via asdf).

Do the go versions that golangci-lint was built with and the local project one have to match?

ldez · 2025-02-19T17:51:57Z

But should this matter?

Yes

We're using the official binary build (via asdf).

I don't know how asdf packages the binaries, but it is not an official way to install binaries.

I need to check how it works.

Do the go versions that golangci-lint was built with and the local project one have to match?

No but it should be consistent: golangci-lint uses some go commands (This is something inside the Go tooling)

ldez · 2025-02-19T17:58:49Z

This is a bit offtopic (and this has no impact no your problem) but I suggest to update your configuration:

.golanci.yml

linters-settings:
  unparam:
    check-exported: true
  depguard:
    rules:
      main:
        files:
          - $all
        deny:
          - pkg: github.com/sirupsen/logrus
            desc: "Don't use"
  dupl:
    threshold: 100
  exhaustive:
    default-signifies-exhaustive: false
  forbidigo:
    forbid:
      - "pk\\b"
  funlen:
    lines: 100
    statements: 50
  gci:
    custom-order: true
    sections:
      - Standard
      - Default
      - Prefix(github.com/shutter-network/shutter)
      - Prefix(github.com/shutter-network/rolling-shutter)
  goconst:
    min-len: 2
    min-occurrences: 2
  gocritic:
    enabled-tags:
      - diagnostic
      - experimental
      - opinionated
      - performance
      - style
    disabled-checks:
      - dupImport # https://github.com/go-critic/go-critic/issues/845
      - ifElseChain
      - octalLiteral
      - whyNoLint
      - wrapperFunc
      - paramTypeCombine
      - hugeParam
      - unnamedResult
      - rangeValCopy
      - typeDefFirst
  gocyclo:
    min-complexity: 15
  goimports:
    local-prefixes: github.com/shutter-network/shutter
  mnd:
      # don't include the "operation" and "assign"
      checks: 
        - argument
        - case
        - condition
        - return
  govet:
    enable:
      - shadow
    settings:
      printf:
        funcs:
          - (github.com/golangci/golangci-lint/pkg/logutils.Log).Infof
          - (github.com/golangci/golangci-lint/pkg/logutils.Log).Warnf
          - (github.com/golangci/golangci-lint/pkg/logutils.Log).Errorf
          - (github.com/golangci/golangci-lint/pkg/logutils.Log).Fatalf
  lll:
    line-length: 140
  misspell:
    locale: US
  nolintlint:
    allow-unused: false # report any unused nolint directives
    require-explanation: false # don't require an explanation for nolint directives
    require-specific: false # don't require nolint directives to be specific about which linter is being skipped
  revive:
    ignore-generated-header: true
    severity: warning
    rules:
      - name: blank-imports
      - name: context-as-argument
      - name: context-keys-type
      - name: dot-imports
      - name: error-return
      - name: error-strings
      - name: error-naming
      # - name: exported
      # - name: if-return
      - name: increment-decrement
      #- name: var-declaration
      - name: package-comments
      - name: range
      - name: receiver-naming
      - name: time-naming
      - name: unexported-return
      - name: indent-error-flow
      - name: errorf
      - name: empty-block
      - name: superfluous-else
      - name: unused-parameter
      - name: unreachable-code
      - name: redefines-builtin-id
      - name: unnecessary-stmt
      - name: range-val-in-closure
      - name: atomic
      - name: superfluous-else

linters:
  disable-all: true
  enable:
    - bodyclose
    - copyloopvar
    - dogsled
    - errcheck
    - exhaustive
    - forbidigo
    - funlen
    - gci
    - goconst
    - gocritic
    - gocyclo
    - godot
    - gofumpt
    - goprintffuncname
    - gosec
    - gosimple
    - govet
    - ineffassign
    - lll
    - misspell
    - nakedret
    - nilnesserr
    - noctx
    - nolintlint
    - revive
    - staticcheck
    - stylecheck
    - thelper
    - unconvert
    - unparam
    - whitespace

issues:
  exclude:
    - "typeUnparen: could simplify \\(func.* to func\\("
    - "Error return value of `.*Mark.*Flag.*` is not checked"
    - "Error return value of `viper.BindEnv` is not checked"
    - 'shadow: declaration of "err" shadows declaration at line'
    - "Expect WriteFile permissions to be"
    - "GobEncode - result 1 .* is always nil"
  # Excluding configuration per-path, per-linter, per-text and per-source
  exclude-rules:
    - path: _test\.go
      linters:
        - gomnd
        - staticcheck
    - path: evtype/evtype\.go
      linters:
        - gosec
    # https://github.com/go-critic/go-critic/issues/926
    - linters:
        - gocritic
      text: "unnecessaryDefer:"

run:
  timeout: 10m

The command golangci-lint config verify can help you to detect configuration problems.

ulope · 2025-02-19T18:00:51Z

I don't know how asdf packages the binaries, but it is not an official way to install binaries.

In the case of golangci-lint it's just downloading the official binary from the github releases.

No but it should be consistent: golangci-lint uses some go commands (This is something inside the Go tooling)

Hm so does that mean it's ok to run a gci-l compiled with go 1.24.0 in a project using go 1.23.6 (which is what we're just now trying to upgrade to) or not?

ldez · 2025-02-19T18:02:37Z

You still have typecheck errors, I recommend reading https://golangci-lint.run/welcome/faq/#why-do-you-have-typecheck-errors

I cloned your project and I have the same typecheck errors.

Hm so does that mean it's ok to run a gci-l compiled with go 1.24.0 in a project using go 1.23.6

Using go 1.23.6 with golangci-lint compiled with go1.24 works, but old versions of Go can be a problem.

ulope · 2025-02-19T18:11:49Z

You still have typecheck errors

Yeah I see them when staticcheck and gosimple are disabled. Interestingly building the project (i.e. go build) doesn't throw any errors.

But in any case even in the presence of typecheck errors I wouldn't expect gci-l to consume huge amounts of memory. Is that an unreasonable assumption?

ldez · 2025-02-19T18:16:08Z

Interestingly building the project (i.e. go build) doesn't throw any errors.

I think that locally you installed some extra dependencies because as you see it doesn't compile in a fresh environment.

$ git clone [email protected]:shutter-network/rolling-shutter.git
...
$ cd rolling-shutter/

$ docker run -it --rm -v $(pwd):/app -w /app golangci/golangci-lint:v1.64.5 sh
...
# go version
go version go1.24.0 linux/amd64
# cd rolling-shutter
# pwd
/app/rolling-shutter
# go build .
...
# github.com/supranational/blst/bindings/go
/go/pkg/mod/github.com/supranational/[email protected]/bindings/go/blst.go:221:11: cannot define new methods on non-local type SecretKey
/go/pkg/mod/github.com/supranational/[email protected]/bindings/go/blst.go:308:15: cannot define new methods on non-local type SecretKey
/go/pkg/mod/github.com/supranational/[email protected]/bindings/go/blst.go:428:11: cannot define new methods on non-local type Fp12
/go/pkg/mod/github.com/supranational/[email protected]/bindings/go/blst.go:432:11: cannot define new methods on non-local type Fp12
/go/pkg/mod/github.com/supranational/[email protected]/bindings/go/blst.go:436:11: cannot define new methods on non-local type Fp12
/go/pkg/mod/github.com/supranational/[email protected]/bindings/go/blst.go:440:11: cannot define new methods on non-local type Fp12
/go/pkg/mod/github.com/supranational/[email protected]/bindings/go/blst.go:446:12: cannot define new methods on non-local type Fp12
/go/pkg/mod/github.com/supranational/[email protected]/bindings/go/blst.go:466:11: cannot define new methods on non-local type P1Affine
/go/pkg/mod/github.com/supranational/[email protected]/bindings/go/blst.go:471:11: cannot define new methods on non-local type P1Affine
/go/pkg/mod/github.com/supranational/[email protected]/bindings/go/blst.go:479:12: cannot define new methods on non-local type P2Affine
/go/pkg/mod/github.com/supranational/[email protected]/bindings/go/blst.go:479:12: too many errors

ldez · 2025-02-20T03:12:53Z

To fix the problem:

update github.com/supranational/blst to v0.3.14 (go get github.com/supranational/[email protected])
update to the min go version to at least 1.23.0 (go mod edit -go=1.23.0)

$ git clone [email protected]:shutter-network/rolling-shutter.git
...
# go version
go version go1.24.0 linux/amd64
$ cd rolling-shutter/
$ docker run -it --rm -v $(pwd):/app -w /app golangci/golangci-lint:v1.64.5 sh
# cd rolling-shutter
# go mod tidy
...
# go build .
...
# github.com/supranational/blst/bindings/go
/go/pkg/mod/github.com/supranational/[email protected]/bindings/go/blst.go:221:11: cannot define new methods on non-local type SecretKey
/go/pkg/mod/github.com/supranational/[email protected]/bindings/go/blst.go:308:15: cannot define new methods on non-local type SecretKey
/go/pkg/mod/github.com/supranational/[email protected]/bindings/go/blst.go:428:11: cannot define new methods on non-local type Fp12
/go/pkg/mod/github.com/supranational/[email protected]/bindings/go/blst.go:432:11: cannot define new methods on non-local type Fp12
/go/pkg/mod/github.com/supranational/[email protected]/bindings/go/blst.go:436:11: cannot define new methods on non-local type Fp12
/go/pkg/mod/github.com/supranational/[email protected]/bindings/go/blst.go:440:11: cannot define new methods on non-local type Fp12
/go/pkg/mod/github.com/supranational/[email protected]/bindings/go/blst.go:446:12: cannot define new methods on non-local type Fp12
/go/pkg/mod/github.com/supranational/[email protected]/bindings/go/blst.go:466:11: cannot define new methods on non-local type P1Affine
/go/pkg/mod/github.com/supranational/[email protected]/bindings/go/blst.go:471:11: cannot define new methods on non-local type P1Affine
/go/pkg/mod/github.com/supranational/[email protected]/bindings/go/blst.go:479:12: cannot define new methods on non-local type P2Affine
/go/pkg/mod/github.com/supranational/[email protected]/bindings/go/blst.go:479:12: too many errors
# go get github.com/supranational/[email protected]
...
# go mod edit -go=1.23.0
...
# go build .
#

In the example, I use your main branch (so with staticcheck and gosimple) and there is no excessive memory consumption.

ulope · 2025-02-21T11:20:49Z

Thanks a lot for taking the time to look into this!

I still don't quite understand where the runaway memory usage is coming from with the outdated blst dependency but I'll gladly take this fix.

Thanks again.

ulope added the bug Something isn't working label Feb 19, 2025

ldez added question Further information is requested and removed bug Something isn't working labels Feb 19, 2025

ldez closed this as completed Feb 21, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Excessive memory use with staticcheck and gosimple #5449

Excessive memory use with staticcheck and gosimple #5449

ulope commented Feb 19, 2025 •

edited by ldez

Loading

boring-cyborg bot commented Feb 19, 2025

ldez commented Feb 19, 2025 •

edited

Loading

ulope commented Feb 19, 2025

ldez commented Feb 19, 2025

ldez commented Feb 19, 2025 •

edited

Loading

ulope commented Feb 19, 2025

ldez commented Feb 19, 2025 •

edited

Loading

ldez commented Feb 19, 2025

ulope commented Feb 19, 2025

ldez commented Feb 19, 2025 •

edited

Loading

ulope commented Feb 19, 2025

ldez commented Feb 19, 2025 •

edited

Loading

ldez commented Feb 20, 2025 •

edited

Loading

ulope commented Feb 21, 2025

Excessive memory use with staticcheck and gosimple #5449

Excessive memory use with staticcheck and gosimple #5449

Comments

ulope commented Feb 19, 2025 • edited by ldez Loading

Welcome

How did you install golangci-lint?

Description of the problem

Version of golangci-lint

Configuration

Go environment

Verbose output of running

A minimal reproducible example or link to a public repository

Validation

Supporter

boring-cyborg bot commented Feb 19, 2025

ldez commented Feb 19, 2025 • edited Loading

ulope commented Feb 19, 2025

ldez commented Feb 19, 2025

ldez commented Feb 19, 2025 • edited Loading

ulope commented Feb 19, 2025

ldez commented Feb 19, 2025 • edited Loading

ldez commented Feb 19, 2025

ulope commented Feb 19, 2025

ldez commented Feb 19, 2025 • edited Loading

ulope commented Feb 19, 2025

ldez commented Feb 19, 2025 • edited Loading

ldez commented Feb 20, 2025 • edited Loading

ulope commented Feb 21, 2025

ulope commented Feb 19, 2025 •

edited by ldez

Loading

ldez commented Feb 19, 2025 •

edited

Loading

ldez commented Feb 19, 2025 •

edited

Loading

ldez commented Feb 19, 2025 •

edited

Loading

ldez commented Feb 19, 2025 •

edited

Loading

ldez commented Feb 19, 2025 •

edited

Loading

ldez commented Feb 20, 2025 •

edited

Loading