Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: Lazy on linux fails to load plugins when git config --global core.autocrlf is true #1025

Closed
3 tasks done
JL102 opened this issue Sep 11, 2023 · 12 comments
Closed
3 tasks done
Labels
bug Something isn't working

Comments

@JL102
Copy link

JL102 commented Sep 11, 2023

Did you check docs and existing issues?

  • I have read all the lazy.nvim docs
  • I have searched the existing issues of lazy.nvim
  • I have searched the existing issues of plugins related to this issue

Neovim version (nvim -v)

0.9.2

Operating system/version

Tested on Ubuntu 22.04, Pop OS 22.04, and OpenSUSE Tumbleweed

Describe the bug

I was having this issue on AstroNvim and it took me a long time to figure out the true cause of the problem. This was my original post on the AstroNvim discord help forum: https://discord.com/channels/939594913560031363/1128010315217506444

If git config --global core.autocrlf is true, then when Lazy.nvim downloads plugins and attempts to load them, you get errors like this:

Failed to source `/home/drak/.repro/plugins/plenary.nvim/plugin/plenary.vim`

vim/_editor.lua:0: /home/drak/repro.lua..nvim_exec2() called at /home/drak/repro.lua:0../home/drak/.repro/plugins/plenary.nvim/plugin/plenary.vim, line 1: Vim:E492: Not an editor command: ^M

# stacktrace:
  - vim/_editor.lua:0 _in_ **cmd**
  - repro.lua:37

Steps To Reproduce

To reproduce issue:

  1. Set git config --global core.autocrlf true
  2. rm -rf the .repro folder
  3. nvim -u repro.lua repro.lua (Opening the repro.lua file in the editor just so that we have some file open)
    The plugin I included in repro.lua, in this case neo-tree, fails to load because of an unexpected ^M character

To prove that it's core.autocrlf true that is the problem:

  1. Set git config --global core.autocrlf false
  2. rm -rf the .repro folder to force lazy to re-download the plugins
  3. nvim -u repro.lua repro.lua
    Everything works as expected

Expected Behavior

I'd expect either of two things:

  1. Whatever piece of code responsible for loading modules should handle files with CRLF newlines without crashing
    or
  2. When downloading modules from Git, Lazy should force core.autocrlf to be true or false depending on the system.

I am guessing that #2 is the simpler option.

Repro

local root = vim.fn.fnamemodify("./.repro", ":p")

-- set stdpaths to use .repro
for _, name in ipairs({ "config", "data", "state", "cache" }) do
  vim.env[("XDG_%s_HOME"):format(name:upper())] = root .. "/" .. name
end

-- bootstrap lazy
local lazypath = root .. "/plugins/lazy.nvim"
if not vim.loop.fs_stat(lazypath) then
  vim.fn.system({
    "git",
    "clone",
    "--filter=blob:none",
    "--single-branch",
    "https://github.com/folke/lazy.nvim.git",
    lazypath,
  })
end
vim.opt.runtimepath:prepend(lazypath)

-- install plugins
local plugins = {
  -- do not remove the colorscheme!
  "folke/tokyonight.nvim",
  -- add any other pugins here
  {
    "nvim-neo-tree/neo-tree.nvim",
    branch = "v3.x",
    dependencies = {
      "nvim-lua/plenary.nvim",
      "nvim-tree/nvim-web-devicons", -- not strictly required, but recommended
      "MunifTanjim/nui.nvim",
    }
   }
}
require("lazy").setup(plugins, {
  root = root .. "/plugins",
})

-- add anything else here
vim.opt.termguicolors = true
-- do not remove the colorscheme!
vim.cmd([[colorscheme tokyonight]])
@JL102 JL102 added the bug Something isn't working label Sep 11, 2023
@abeldekat
Copy link
Contributor

Hello @JL102,

I think this might be a bug in upstream neovim.

I created a minimal init, using only vim's builtin package manager, testing on arch linux.
The results are the same.

Everything works fine when the global autocrlf is false.
When autocrlf is used, neovim crashes on the first packadd, when that plugin contains a .vim file inside its plugin folder.

Repro

-- test with:
-- git config --global core.autocrlf false
-- git config --global core.autocrlf true
--
-- clean after testing:
-- rm -rf pack .repro

local root = vim.fn.fnamemodify("./.repro", ":p")

-- set stdpaths to use .repro
for _, name in ipairs({ "config", "data", "state", "cache" }) do
  vim.env[("XDG_%s_HOME"):format(name:upper())] = root .. "/" .. name
end

local plugins = {
  {
    "tokyonight.nvim",
    "https://github.com/folke/tokyonight.nvim.git",
  },
  {
    "nui.nvim",
    "https://github.com/MunifTanjim/nui.nvim.git",
  },
  { --has a plugin folder, containing: plenary.vim
    "plenary.nvim",
    "https://github.com/nvim-lua/plenary.nvim.git",
  },
  { -- has a plugin folder, containing: nvim-web-devicons.vim
    "nvim-web-devicons",
    "https://github.com/nvim-tree/nvim-web-devicons.git",
  },
  { -- has a plugin folder, containing: neo-tree.vim
    "neotree.nvim",
    "https://github.com/nvim-neo-tree/neo-tree.nvim.git",
  },
}

local packpath = root .. "/../pack/plugins/opt"
if not vim.loop.fs_stat(packpath) then
  vim.fn.system({ "mkdir", packpath })
end

-- for each plugin: git clone
for _, plugin in pairs(plugins) do
  local pluginpath = packpath .. "/" .. plugin[1]
  if not vim.loop.fs_stat(pluginpath) then
    vim.fn.system({ "git", "clone", "--filter=blob:none", plugin[2], pluginpath })
  end
end

-- for each plugin: packadd
for _, plugin in pairs(plugins) do
  vim.cmd("packadd " .. plugin[1])
end

vim.cmd.colorscheme("tokyonight")

-- When core.autocrlf is true: Crashes on the first packadd command, when a plugin contains a vim file inside its plugin folder.
--[[
Error detected while processing /home/abeldekat/.config/nvim/minimal_packadd.lua:
E5113: Error while calling lua chunk: vim/_editor.lua:0: /home/abeldekat/.config/nvim/minimal_packadd.lua..nvim_exec2() called at /home/abeldekat/.config/nvim/minimal_packadd.lua:0
../home/abeldekat/.config/nvim/pack/plugins/opt/plenary.nvim/plugin/plenary.vim, line 1: Vim:E492: Not an editor command:
stack traceback:
        [C]: in function 'nvim_exec2'
        vim/_editor.lua: in function 'cmd'
        /home/abeldekat/.config/nvim/minimal_packadd.lua:53: in main chunk
Error detected while processing /home/abeldekat/.config/nvim/pack/plugins/opt/plenary.nvim/plugin/plenary.vim:
line    1:
E492: Not an editor command: ^M
line    3:
E182: Invalid command name
line    5:
E492: Not an editor command: ^M
line    6:
E182: Invalid command name
line    8:
E492: Not an editor command: ^M
--]]

Best regards!

@JL102
Copy link
Author

JL102 commented Sep 15, 2023

Thanks for the response!
I can't reproduce with your repro.lua, though. I see that there's a line that mkdirs the required folders, but when I try to run it even with git config --global core.autocrlf false, I get this:

Error detected while processing /home/test/repro.lua:
E5113: Error while calling lua chunk: vim/_editor.lua:0: /home/test/repro.lua..nvim_exec2() called at /home/test/repro.lua:0: Vim(packadd):E919: Directory not found in 'packpath': "pack/*/opt/tokyonight.nvim"
stack traceback:
        [C]: in function 'nvim_exec2'
        vim/_editor.lua: in function 'cmd'
        /home/test/repro.lua:53: in main chunk
Press ENTER or type command to continue

Are you using the same version of nvim as me? (0.9.2)

@abeldekat
Copy link
Contributor

Hello @JL102,

My nvim config is in ~/.config/nvim, using v0.9.2 on arch linux.
In that folder, I save the code as minimal.lua.
When I run nvim -u minimal.lua minimal.lua, the following folders are added: pack and .repro. All plugins are in pack/plugins/opt.

@JL102
Copy link
Author

JL102 commented Sep 21, 2023

@abeldekat hmm. Still doesn't work for me after removing both folders.

In any case, though, I see you wrote vim.fn.system({ "git", "clone", "--filter=blob:none", plugin[2], pluginpath }). I don't know what vim.fn.system is but that looks to me like it's just spawning git clone as a shell process.

After looking, I see that git-clone supports a --config flag: https://www.git-scm.com/docs/git-clone#Documentation/git-clone.txt---configltkeygtltvaluegt
I'm not sure how to detect in code what line endings the current OS prefers, but it seems to me like you could just add --config core.autocrlf=false / --config core.autocrlf=true depending on the currently running OS.

@JL102
Copy link
Author

JL102 commented Sep 21, 2023

Actually, after some cursory testing, it seems like the native Windows build of Neovim doesn't care if the plugins' files are LF or CRLF. In other words, if Windows is the only OS supported by Neovim that uses CRLF, then I think it should be safe to just override core.autocrlf=false in all cases.

@abeldekat
Copy link
Contributor

abeldekat commented Sep 21, 2023

@JL102

Which OS are you using for the test? Your issue only mentions linux:

Tested on Ubuntu 22.04, Pop OS 22.04, and OpenSUSE Tumbleweed

The point of my code snippet is to proof that stock neovim also crashes in the same manner as described in your "steps to reproduce", but without using lazy.nvim. In doing so I was hoping to discover if this issue is indeed a bug in lazy.nvim. I was also hoping to narrow the issue down a bit.

I tested autocrlf for both false and true, on arch linux, setting the autocrlf manually in the terminal.

In any case, though, I see you wrote vim.fn.system({ "git", "clone", "--filter=blob:none", plugin[2], pluginpath }). I don't know what vim.fn.system is but that looks to me like it's just spawning git clone as a shell process.

The provided snippet is intended to mimic the repro you provided, without using lazy.nvim.
In the repro you provided, that same vim.fn.system is used, to initially clone lazy.nvim. See the code containing the following comment: "- bootstrap lazy"
The clone you are referring to is used to retrieve the plugins which would normally have been cloned by lazy.nvim

When testing on linux, I expect the snippet to work in the first case, when autocrlf is off:

  • save as minimal.lua, in the folder that contains your init.lua
  • rm -rf .repro
  • rm -rf pack
  • run nvim -u mininal.lua minimal.lua, in the folder that contains your init.lua

In the second case, when autocrlf is on, I expect the snippet to fail.

If the first case still is not working on your system, could you provide the details of the error? Are there any plugins in the pack folder? Is your neovim folder ~/.config/neovim?

Best regards!

Note: It can take a couple of seconds to git clone the plugins...

@JL102
Copy link
Author

JL102 commented Sep 22, 2023

@abeldekat Sorry, I'm not super used to neovim, so I don't understand exactly what you mean by "in the folder that contains your init.lua. Do you mean I should name the original repro.lua (the one I included in the bug description) to init.lua? Or do you mean my own personal init.lua, or something else? When trying to use your minimal.lua, I was doing it inside a home folder.

I'm primarily using AstroNvim, so normally my init.lua and all that is inside ~/.config/nvim and ~/.config/nvim/lua/user. But I thought that when running nvim with -u minimal.lua / -u repro.lua, it ignores ~/.config/nvim.

I'll respond to everything else you said after I can correctly reproduce your minimal.lua. Don't wanna talk out of my butt until we're on the same page.

@abeldekat
Copy link
Contributor

@JL102 no worries! Thanks for persisting!
You can save the code as repro.lua in ~/.config/nvim. The name of the file is not important.
When using the -u option, the init.lua to AstroNvim will be skipped. Your regular nvim installation will not be changed.

Inside ~/.config/nvim, run nvim -u repro.lua and quit.
The script will create the following folders:
~/.config/nvim/pack
~/.config/nvim/.repro

Before running with autocrlf, make sure to remove the pack and .repro folder.

@JL102
Copy link
Author

JL102 commented Sep 22, 2023

@abeldekat
Got it, thanks! Now I get the same behavior as you. Now to answer the concerns:

Which OS are you using for the test? Your issue only mentions linux:

I'm currently running a Windows desktop and using WSL. When I first noticed the issue, I was on a laptop with Pop OS, but then when testing Ubuntu 22.04 and OpenSUSE Tumbleweed, I was using WSL.

Actually, after some cursory testing, it seems like the native Windows build of Neovim doesn't care if the plugins' files are LF or CRLF. In other words, if Windows is the only OS supported by Neovim that uses CRLF, then I think it should be safe to just override core.autocrlf=false in all cases.

It was only right before I wrote this comment when I downloaded the native Windows build of nvim and tested it. I ran my repro script both with core.autocrlf false and true. In the case where core.autocrlf=false, the files that were downloaded did have LF endings, but everything worked fine.


I'd expect either of two things:

  1. Whatever piece of code responsible for loading modules should handle files with CRLF newlines without crashing
    or
  2. When downloading modules from Git, Lazy should force core.autocrlf to be true or false depending on the system.
    I am guessing that # 2 is the simpler option.

I think you have been focusing on suggestion # 1, while I have been focusing on # 2. After using your example, I understand what you were saying about # 1 being an issue with the runtime. In fact, it looks like it's a "bug" inside both vim AND vi. Neovim doesn't seem to have any issues loading lua files (that's why Lazy ran fine), and the error seems to only pop up when loading a .vim file (and that's why the error only occurred when loading Plenary).

In VS Code, because I can use it to easily control the line ending style, I created a file test.vim with the following contents:

let name = "John"
echo "Hello, " . name

When the file has LF endings, running vim -u test.vim will print "Hello, John" before opening, as expected. When I change the file to have CRLF endings, running vim -u test.vim will give the now-familiar error:

Error detected while processing /home/drak/test.vim:
line    1:
E15: Invalid expression: "John"^M
line    2:
E121: Undefined variable: name
Press ENTER or type command to continue

Maybe the devs of Neovim will be open to fixing it, but I suspect it'll be non trivial, considering we may be dealing with many-decades-old code. I wrote "bug" in quotes, now, after realizing it's present in both vim and vi, because it may actually be intended behavior to expect all files to have LF endings. After this discovery, I feel more strongly now that it would be simpler to adjust Lazy's git clone code to override core.autocrlf.

Since the Windows version of nvim doesn't care if a .vim file has LF or CRLF line endings*, and I think Windows is the only major OS which uses CRLF, and it seems like basically every nvim/vim repo gives their script LF endings, I think it should be safe to force core.autocrlf=false, instead of what I originally suggested, which is to detect what OS you're using and then force core.autocrlf=true on Windows and core.autocrlf=false everywhere else.

*I don't have vim or vi installed on Windows, only nvim. Doing nvim -u test.vim works on Windows no matter the line ending style, but nvim -u test.vim on Ubuntu gives the same error as vi -u test.vim and vim -u test.vim when line endings are CRLF.

@abeldekat
Copy link
Contributor

@JL102,

We are totally on the same page now! I indeed focused on your first suggestion, in order to find out if lazy.nvim is responsible for the issue at hand.
Thanks for the detailed response. Very interesting!
I guess it's up to Folke now.

Best regards!

@folke
Copy link
Owner

folke commented Sep 28, 2023

This is not really a lazy issue as you found. Report it upstream or dont set that global config

@folke folke closed this as not planned Won't fix, can't repro, duplicate, stale Sep 28, 2023
@JL102
Copy link
Author

JL102 commented Sep 28, 2023

This is not really a lazy issue as you found. Report it upstream or dont set that global config

Hi folke,
I respectfully disagree. As discussed later in the thread, even vim and vi don't recognize when a vim file is CRLF, because it goes back to a decades-old assumption that all files on a Unix system have LF endings. I think it would be difficult to convince the developers of Neovim to modify the code responsible for loading vim files, when it could potentially have significant impact on the rest of the program.

Sometimes it's necessary to have core.autocrlf=true as part of your git config, such as if you work on a project with others who develop on Windows. For my part, the core.autocrlf setting was set years ago and it took a long time (2 months) to realize that it was the cause of this issue. There isn't really a logical connection between seeing an error about ^M and git config core.autocrlf, especially if you're not a poweruser & you are using a prepackaged config like AstroNvim or LazyVim.

I'm grateful of your development of Lazy, as it's an excellent and very useful plugin. I respect your time so instead of arguing to get you to fix it for me, I've made a fork and added what I believe is a reasonable implementation of the requested feature and will submit a PR after posting this.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
3 participants