vim.glob.to_lpeg doesn't always work as expected

[faergeek]

Problem

Group conditions do not properly work when there are multiple groups starting with the same substring and the shorter one appears earlier.

For example, vim.glob.to_lpeg('**/*.{tsx,ts}'):match('test.tsx') matches, but vim.glob.to_lpeg('**/*.{ts,tsx}'):match('test.tsx') doesn't

Steps to reproduce

1. nvim --clean
2. :lua=vim.glob.to_lpeg('**/*.{tsx,ts}'):match('test.tsx') (returns 9)
3. :lua=vim.glob.to_lpeg('**/*.{ts,tsx}'):match('test.tsx') (returns nil)

Expected behavior

Both of the globs match.

Neovim version (nvim -v)

v0.11.0-dev-57+gcd48b72b6

Vim (not Nvim) behaves the same?

n/a

Operating system/version

Arch Linux

Terminal name/version

kitty 0.34.1

$TERM environment variable

xterm-kitty

Installation

build from repo

[faergeek]

Patch with added tests for a particular glob with which I discovered the issue:

diff --git a/test/functional/lua/glob_spec.lua b/test/functional/lua/glob_spec.lua
index 56cd4c9bb..383e66d33 100644
--- a/test/functional/lua/glob_spec.lua
+++ b/test/functional/lua/glob_spec.lua
@@ -223,6 +223,17 @@ describe('glob', function()
       eq(true, match('{[0-9],[a-z]}', '0'))
       eq(true, match('{[0-9],[a-z]}', 'a'))
       eq(false, match('{[0-9],[a-z]}', 'A'))
+
+      -- glob is from willRename filter in typescript-language-server
+      -- https://github.com/typescript-language-server/typescript-language-server/blob/b224b878652438bcdd639137a6b1d1a6630129e4/src/lsp-server.ts#L266
+      eq(true, match('**/*.{ts,js,jsx,tsx,mjs,mts,cjs,cts}', 'test.js'))
+      eq(true, match('**/*.{ts,js,jsx,tsx,mjs,mts,cjs,cts}', 'test.ts'))
+      eq(true, match('**/*.{ts,js,jsx,tsx,mjs,mts,cjs,cts}', 'test.mts'))
+      eq(true, match('**/*.{ts,js,jsx,tsx,mjs,mts,cjs,cts}', 'test.mjs'))
+      eq(true, match('**/*.{ts,js,jsx,tsx,mjs,mts,cjs,cts}', 'test.cjs'))
+      eq(true, match('**/*.{ts,js,jsx,tsx,mjs,mts,cjs,cts}', 'test.cts'))
+      eq(true, match('**/*.{ts,js,jsx,tsx,mjs,mts,cjs,cts}', 'test.jsx'))
+      eq(true, match('**/*.{ts,js,jsx,tsx,mjs,mts,cjs,cts}', 'test.tsx'))
     end)
   end)
 end)

Only last 2 cases do not match.

Not sure if I'll be able to figure out how to fix that myself.

[justinmk]

@stevearc

[stevearc]

I didn't write the lpeg grammar, I merely moved it into the current file. It was added by @nojnhuh in #23788. I spent some time poking around; this seems to be a problem with greedy matching the first option in the list.

-- this fails
eq(true, match('**/*.{ts,tsx}', 'test.tsx'))
-- this succeeds
eq(true, match('**/*.{tsx,ts}', 'test.tsx'))
-- this also succeeds
eq(true, match('**/*.{tsx,ts}', 'test.ts'))

I have no idea how to fix this using purely lpeg. Intuitively I want to take the list of items inside the {} and sort them descending by length, but I couldn't figure out how to make that happen inside the lpeg expression.

[brynne8]

I'm very sure why this happens. It's because vim.glob simply translates regex a | b into a + b, which is incorrect, since it means ordered choice in PEG. Also check the discussion: #29199