Help with busted, local-lua config 🙏

[aaharr]

Very much appreciate you targeting this use case (💯)

When I try to run busted tests, I get syntax errors seemingly from the busted executable. Just the sample busted test from the plenary docs, passes when running busted either via plenary or lua. Would love to be able to debug!

For every test:
E5112: Error while creating lua chunk: /usr/local/bin/busted:3: unexpected symbol near '-'

File is called test_spec.lua and there is an empty minimum_init.spec in the same directory. Plenary seems to be picking it up, but only since I named it test_spec.lua.

            -- Local Lua DAP
            -- https://github.com/mfussenegger/nvim-dap/wiki/Debug-Adapter-installation#local-lua-debugger-vscode
            dap.adapters["local-lua"] = {
                type = "executable",
                command = "node",
                args = {
                    "/home/aaron/local-lua-debugger-vscode/extension/debugAdapter.js",
                },
                enrich_config = function(config, on_config)
                    if not config["extensionPath"] then
                        local c = vim.deepcopy(config)
                        -- 💀 If this is missing or wrong you'll see
                        -- "module 'lldebugger' not found" errors in the dap-repl when trying to launch a debug session
                        c.extensionPath = "/home/aaron/local-lua-debugger-vscode/"
                        on_config(c)
                    else
                        on_config(config)
                    end
                end,
            }

            -- Local Lua Launch Config
            -- https://zignar.net/2023/06/10/debugging-lua-in-neovim/
            dap.configurations.lua = {
                {
                    name = "Launch file (local-lua)",
                    type = "local-lua",
                    request = "launch",
                    cwd = "${workspaceFolder}",
                    console = "integratedTerminal",
                    program = {
                        lua = "lua5.1",
                        file = "${file}",
                    },
                    args = {},
                },
            }
            -- Currently filetypes are not set for lua (why?), so you have to manually run
            -- :lua require('dap').run(require('dap').configurations.lua[1])

                adapters = {
                    -- require("neotest-plenary").setup({
                    --     -- min_init = ".minimal_init.lua",
                    -- }),
                    require("neotest-busted")({
                        -- Leave as nil to let neotest-busted automatically find busted
                        busted_command = "/usr/local/bin/busted",
                        -- Extra arguments to busted
                        -- busted_args = { "--shuffle-files" },
                        -- List of paths to add to package.path in neovim before running busted
                        -- busted_paths = { "my/custom/path/?.lua" },
                        -- List of paths to add to package.cpath in neovim before running busted
                        -- busted_cpaths = { "my/custom/path/?.so" },
                        -- Custom config to load via -u to set up testing.
                        -- If nil, will look for a 'minimal_init.lua' file
                        -- minimal_init = "custom_init.lua",
                    }),
                },

Hoping something jumps out at you!

[MisanthropicBit]

Thanks for the kind words, I'm glad people find it useful 🙂 the adapter is still under development so it really helps when people submit an issue. FYI, the minimal_init.lua file is only necessary if you need to load some test dependencies.

• So running /usr/local/bin/busted test_spec.lua works but you are unable to run the tests via neotest inside the test file? Or is it only debugging that does not work?
• How did you install busted?
• What version of neovim and neotest are you using? The output of :checkhealth neotest-busted check for compatible versions.
• Do you mean the first test file here? I can run the tests and debug them.
• Unfortunately, nothing jumps out at me apart from the console = "integratedTerminal", line which shouldn't matter. Your setup looks pretty much identical to mine.
• You can try to set the log_level to vim.log.levels.debug in the neotest.setup function and check the logs. Perhaps that will give you an idea of what goes wrong. It seems like the command created by neotest-busted might be wrong. Otherwise, insert a vim.print(vim.inspect(test_command)) in the get_strategy_config function.
• Lastly, you could open the busted "executable" which should actually just be a bash script with a shebang. Line 3 would correspond with the line that invokes busted via luarocks exec which might somehow be incorrectly set up.

Hope that helps 🙂

[aaharr]

It certainly helps to see your config and get a sanity check. Enjoying neotest ✅

• Yes I can run that busted path and get expected output on $
• Luarocks.. it shows busted 2.2.0-1, I just use the path from which busted
• will take a look and edit Nvim version is 0.10.0

* neotest-busted ~
- OK has neovim 0.9.0+
- OK `neotest` is installed
- OK `nio` is installed
- OK found no errors in config
- OK found `busted` (type: config) at
  /usr/local/bin/busted

• Yes that's the one
• Cool, worth a try
• I keep hoping someday my lua print statements will go to nvim-dap-ui's console widget or repl, but that is another story
• In the logs I see quite a bit of this, but no explicit errors

<SNR>97_try_cmd returning ['LUAROCKS_SYSCONFDIR=''/usr/local/etc.../busted/2.2.0-1/bin/busted'' "$@"', '']
calling <SNR>97_try_cmd
<SNR>97_try_cmd returning ['LUAROCKS_SYSCONFDIR=''/usr/local/etc.../busted/2.2.0-1/bin/busted'' "$@"', '']
calling <SNR>97_try_cmd
<SNR>97_try_cmd returning ['LUAROCKS_SYSCONFDIR=''/usr/local/etc.../busted/2.2.0-1/bin/busted'' "$@"', '']
calling <SNR>97_try_cmd
<SNR>97_try_cmd returning ['_']
calling <SNR>97_try_cmd
<SNR>97_try_cmd returning ['_']
calling <SNR>97_try_cmd
<SNR>97_try_cmd returning ['_']
calling <SNR>97_try_cmd
<SNR>97_try_cmd returning ['_']

• edit
  Looks like we're just using lua to run busted with some explicit paths in this bash command, and it is working from the cli. I do see a - or two

LUAROCKS_SYSCONFDIR='/usr/local/etc/luarocks' exec '/usr/bin/lua5.1' -e 'package.path="/usr/local/share/lua/5.1/?.lua;/usr/local/share/lua/5.1/?/init.lua;"..package.path;package.cpath="/usr/local/lib/lua/5.1/?.so;"..package.cpath;local k,l,_=pcall(require,"luarocks.loader") _=k and l.add_context("busted","2.2.0-1")' '/usr/local/lib/luarocks/rocks-5.1/busted/2.2.0-1/bin/busted' "$@"

*edit - Command info from vim.inspect

{
  arguments = { "--headless", "-i", "NONE", "-n", "-u", "NONE", "-c", "lua package.path = 'lua/?.lua;lua/?/init.lua;' .. package.path", "-l", "/usr/local/bin/busted", "--verbose", "--output", "/home/aaron/.local/share/nvim/lazy/neotest-busted/lua/neotest-busted/output_handler.lua", "-Xoutput", "/tmp/nvim.aaron/XW7ip6/4.json", "--filter", '"Hello test some other test"', '"/home/aaron/plugins/myplugin/tests/test_spec.lua"', '"--helper"', '"/home/aaron/.local/share/nvim/lazy/neotest-busted/lua/neotest-busted/start_debug.lua"' },
  cpaths = {},
  nvim_command = "/usr/bin/nvim",
  paths = { "lua/?.lua", "lua/?/init.lua" }
}

That's pretty much what I thought as well. I will try these steps and get back. The voice in my head says "it's a path somewhere.. it's always a path!!" Thanks again

[MisanthropicBit]

Thanks for providing the additional info.

In the logs I see quite a bit of this, but no explicit errors

I don't have a single line in my logs (710886 lines) that mention the ...try_cmd thing. Perhaps it's related to neotest which uses vim.fn.jobstart for some stuff under the hood. I can't see the severity of those logs but I guess they are probably just reported as INFO.

Looks like we're just using lua to run busted with some explicit paths in this bash command, and it is working from the cli. I do see a - or two

Everything looks as expected and properly quoted (which might otherwise have meant that a '-' got misinterpreted) and nice that you can run it in the CLI.

*edit - Command info from vim.inspect

I noticed that the additional package.path setup is just lua/?.lua;lua/?/init.lua; (which is included by default) and that package.cpath isn't set up at all. When using a custom busted executable, neotest-busted does not set any additional paths which is why you only see the default ones. If you have a special installation of luarocks/busted that isn't covered by neotest-busted, the paths won't be set up because neotest-busted doesn't know where to look. You can try providing them via busted_paths and busted_cpaths in the setup function.

Do you have an installation in ~/.luarocks (you can also try luarocks list busted)? Neotest-busted should emit debug-level logs about which one it found. You could try to do a directory-local install and see if that setup works since you might have an installation setup that neotest-busted isn't accounting for.

[aaharr]

Thanks @MisanthropicBit for the info and ideas.

Seems like it will take me some time to step through the config combinations and get back. Feel free to close if you need to until then 👍

[MisanthropicBit]

No problem 🙂 Let's keep this issue open for now until it's confirmed whether this is a bug in neotest-busted, a new feature is needed, or something else.

I've added a warning box about the consequences of setting busted_command in another PR for future reference.

[MisanthropicBit]

Hey @aaharr. Did you get a chance to look at the config combinations?

[aaharr]

@MisanthropicBit No.. my Lua flow has been pretty ad hoc. I'm not sure when I'll finally get to TDD my half-finished plugins 🥲

[MisanthropicBit]

No worries, just wanted to follow up on it 🙂

[YaroSpace]

Hi, guys!
I had exactly the same problem. Turns out that the busted executable has the line :3 of LUAROCKS_SYSCONFDIR='/etc/luarocks' exec '/usr/bin/lua5.1' -e 'package.path="/usr/local/share/lua/5.1/?.lua;/usr/local/share/lua/5.1/?/init.lua;"..package.path;package.cpath="/usr/local/lib/lua/5.1/?.so;"..package.cpath;local k,l,_=pcall(require,"luarocks.loader") _=k and l.add_context("busted","2.2.0-1") '/usr/local/lib/luarocks/rocks-5.1/busted/2.2.0-1/bin/busted' "$@", at least in my case. It is supposed to be run from CLI.
However, @MisanthropicBit you are constructing the command as nvim -l .. to run it as Lua script, so it fails at parsing the flag -e.

A workaround is to set busted_command = "/usr/local/lib/luarocks/rocks-5.1/busted/2.2.0-1/bin/busted" directly to busted lua script.

[MisanthropicBit]

Hi @YaroSpace. Thanks for providing some new information.

However, @MisanthropicBit you are constructing the command command as nvim -l .. to run it as Lua script, so it fails at parsing the flag -e.

I missed this in OP's post but I think this actually fails because it is trying to parse a shell script as a lua file as neotest-busted internally runs nvim -l ... as you correctly state.

Btw, what is the reason to run busted like this and not directly from the shell?

Do you mean just running busted ...? That is already possible if you prefer, you just need an appropriate .busted configuration file.

I didn't know busted very well when I started out so this was the avenue I pursued. No reason particularly except that I read about neovim new -l flag and some blog posts that demonstrated using it. If you want to run busted without the config file you need something like nlua and additional setup is needed. See this section for guidance.

A workaround is to set busted_command = "/usr/local/lib/luarocks/rocks-5.1/busted/2.2.0-1/bin/busted" directly to busted lua script.

A better way would be to just let neotest-busted find the executable lua script for you which is the default as is stated in the README. Perhaps I need to make this clearer. Note that setting busted_command will not set busted_paths and busted_cpaths since you are specifying a custom command and neotest-busted doesn't want to assumptions about those paths. By default though, it will only look in your project-local directory so you need to set local_luarocks_only = false if you want to use another user- or global-level install.

I don't recognise that particular path though. How did you install busted and on what system? I may need to add support for multiple paths on different systems (I'm on mac).

Any other feedback on making it easier to set up neotest-busted is most welcome since I'm planning on officially "releasing" it soon 🙂

[YaroSpace]

I have installed busted with luarocks, globally. In init.lua:85 you are checking if there is busted on path. It is found, but then you are passing it to nvim -l .., which does not know about the path, so when the command is run, there is an error that busted is not found. Even if you use vim.fn.exepath() to get the full path, it will be the path to busted shell script, which you cannot pass to nvim -l. I would suggest that for global installation of busted, you use the same technique of locating the executable script as for local.

[YaroSpace]

Btw, thank you for the great work. I spent some time getting familliar with the internals - your code is a pleasure to read.
Two notes:

1. In util.lua:9 you are trimming only double quotes, but should be single quotes too.
2. In logging.lua:19 if the the argument to unpack is nil, it blows up with errors. This happens when there are no test results for example and init.lua:639 log.error with pos_id_key nil.
3. Is there a reason why you left out parsing for pending tests?

[MisanthropicBit]

I have installed busted with luarocks, globally. In init.lua:85 you are checking if there is busted on path…

Good point. I haven’t actually tested the global option since I personally use a per-user luarocks installation so I’ll make a fix for it.

Btw, thank you for the great work. I spent some time getting familliar with the internals - your code is a pleasure to read.

Thanks 🙂 that’s very kind of you to say and very motivating. I feel like I’ve already spent quite a bit of time on this project 😅 so I’m happy to hear it.

1. Can you provide a use case? Internally, the function is used to strip the double-quotes from the components of neotest identifiers such as path::"namespace 1”::"test 1".

2. Nice catch. I’ve seen the error before but didn’t know the exact cause. I’ll make a fix for it.

3. Pending tests are already be supported since parsing an ordinary test should also parse pending tests (docs) e.g.

   describe("busted pending tests", function()
     pending("I should finish this test later")
   end)

   The json output for pending tests is also handled. At least, it works on my machine. Does it not on yours?

[YaroSpace]

Can you provide a use case? Internally, the function is used to strip the double-quotes from the components of neotest identifiers such as path::"namespace 1”::"test 1".

Such identifiers are formed with the same quotes that are actually used in describe/it blocks, in my case I have a mix of it statements with double/sinlge quotes. Since single quotes are not trimmed, there is an error of failed to find matching pos_id_key

[YaroSpace]

3. Pending tests are not being parsed in my case, they do not come up in the neotest summary and there is a pos_id_key mismatch, because they do come up in busted results. I added a treesitter query to match pending to fix that.
4. Regarding json. Actually, on my machine it does break, because pending tests have an attribute of default_fn = func which cannot be encoded into json. I raised an issue with busted here Fails to encode results to json due to non-string error objects being raised lunarmodules/busted#729 (comment). Locally, I fix it, be removing func attributes from handler.pendings in output_handler.lua

[MisanthropicBit]

Such identifiers are formed with the same quotes that are actually used in describe/it blocks, in my case I have a mix of it statements with double/sinlge quotes. Since single quotes are not trimmed, there is an error of failed to find matching pos_id_key

Makes sense. Especially since I have stylua set up to prefer double-quotes so I never tested it with single-quotes. I can reproduce this fine on my machine. I’ll make a fix for it.

Would you mind opening a separate issue(s)? Sounds like you are making PRs for the issues below? Just so we do not duplicate work.

3. I think I understand now. You are talking about pending at the namespace-level right? Like in the example I linked? Those are not supported and I didn’t realise they worked on that level. Calls to pending are supported for individual tests though which is what I meant.
4. Good find. Unfortunately, it seems like busted is largely unmaintained these days.

[YaroSpace]

These are the various options for pending:

describe('Tests the busted pending functions through the commandline', function()

  it('is a test with a pending', function()
    pending('finish this test later')
    error('should never get here')
  end)

  pending('is a pending inside a describe', function()
    it('this test does not run', function()
      error('this should not run')
    end)
  end)

  pending('is a pending inside a describe, without func argument')

end)

[YaroSpace]

Let me open a PR with all the changes I have made to the fork of your repo, so we can comment/cherry pick next to the actual code there and then you can either add the suggestions or I will make a proper PR.

[MisanthropicBit]

Thanks for clarifying pending tests. That should definitely be supported.

Sounds good 🙂 FYI, I’ve already started on a few of the issues and opened PRs for them.