[bug] Matching string literals in Rust inner attributes behaves strangely

[PaulDance]

Hi!

⏯ Playground Link

Playground link with relevant code

💻 Code

Code:

#![abc(
    def = "ghi"
)]
#![mno(pqr = "stu")]

fn f() {
    g("123");  // To check that the rule works in the first place.
}

Rule:

id: test
severity: warning
language: Rust

files:
  - "./**/*.rs"

rule:
  kind: string_literal
  not:
    inside:
      kind: attribute
      stopBy: end

🙁 Actual behavior

The first attribute is matched, but not the second, although there is only a formatting difference between the two:

warning[test]: 
  ┌─ ./test.rs:2:11
  │
2 │     def = "ghi"
  │           ^^^^^

warning[test]: 
  ┌─ ./test.rs:7:7
  │
7 │     g("123");
  │       ^^^^^

Weird element: if the inner attributes are switched, i.e. the one written on a single line is put on the first line, then the one on three lines is not matched anymore and only the third string is.

Also weird: if the inner attributes are turned to regular ones (#[...]), then there is no issue anymore.

Still weird: I cannot make the Playground reproduce the issue, but I can only do so locally. See below for detailed version information.

All these behaviors can also be seen if the rule is "inverted" by removing the not. Same if any: - kind: attribute_item - kind: inner_attribute_item is used instead of kind: attribute.

🙂 Expected behavior

Only the third string should be matched, whatever the formatting or relative order of inner attributes:

  ┌─ ./test.rs:7:7
  │
7 │     g("123");
  │       ^^^^^

Meta

$ rustc -vV
rustc 1.77.2 (25ef9e3d8 2024-04-09)
binary: rustc
commit-hash: 25ef9e3d85d934b27d9dada2f9dd52b1dc63bb04
commit-date: 2024-04-09
host: x86_64-unknown-linux-gnu
release: 1.77.2
LLVM version: 17.0.6

$ ast-grep -V
ast-grep 0.20.5

$ uname -a
Linux 6.6.15-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.6.15-2 (2024-02-04) x86_64 GNU/Linux

Ending note: this is the only real issue I encountered while developing a relatively-complex rule for a fairly-sized codebase. It might even be due to a tree-sitter bug. Overall, ast-grep actually works quite well! Thanks ❤️

[HerringtonDarkholme]

This is already reported at #1057.
Let's discuss the issue there.

[PaulDance]

Alright. I saw it, but didn't think it was exactly related. Sorry for the duplicate.

[PaulDance]

Not a duplicate anymore? 😄

[HerringtonDarkholme]

Sorry I misread you case by seeing string literal 🤣

[PaulDance]

OK, no problem 🙂

An idea what the problem might be, then?

[HerringtonDarkholme]

It is probably tree-sitter's issue. I will investigate it later

[HerringtonDarkholme]

I don't know why but the leading #![abc] is parsed as shebang.

I do believe this is tree-sitter-rust's problem

(source_file (line_comment) (shebang) (function_item name: (identifier) parameters: (parameters) body: (block (string_literal))))

[HerringtonDarkholme]

tree-sitter/tree-sitter-rust#204

The issue is also fixed in new tree-sitter release. But unfortunately we are blocked now.

[HerringtonDarkholme]

Closing this in favor of #1104

[PaulDance]

Got it, thanks!

[HerringtonDarkholme]

Fixed in #1119

[PaulDance]

Can confirm it works under main, many thanks!