Replies: 25 comments
This comment has been hidden.
This comment has been hidden.
-
Hi all, I'm occasionally overcome with a burning desire for async coloring in vim. I've hacked up something that I hope could be useful, or at least interesting to some of you. I coded an experiment to see if it might be possible/practical to do this with stock vim and some vimscript to communicate with an external process that runs treesitter. Currently it's pretty basic, but it colors! If you have a large buffer, then vim continues to be responsive while the buffer is sent and parsed, but there's a lag when the vimscript applies the colors. In many cases this could be fixed by not applying all the colors immediately. I've found vimscript functions to figure out which lines are on the screen, and to clear coloring from certain lines, but I don't see any functions for removing coloring from character ranges, so I'm not sure the current vim api is sufficient to gracefully handle cases with very long lines. I've been testing in vim8.1 as shipped by debian unstable. I think/hope it'll work just as well in neovim. https://github.com/JasonWoof/vim-treesitter Enjoy! - Jason |
Beta Was this translation helpful? Give feedback.
-
Hi, I would like to know if you could give information on how to build our own parsers for the time being. I tried multiple repositories under the treesitter organization, but they only build statically linked libraries, which cannot be loaded with I started to play around with lua and its quite a great language to use ! I hope we'll see more cool stuff like that in the future ! |
Beta Was this translation helpful? Give feedback.
-
@kyazdani42 depends on your system, but on a POSIX-like system you can clone any language repo and run Then copy |
Beta Was this translation helpful? Give feedback.
-
perfect i wasn't sure which file to compile ! thanks |
Beta Was this translation helpful? Give feedback.
-
There might be some weird interaction between the highlights from inccommand and the lua_attr attributes and/or TSHighlighter being spammed non-stop while ex mode is active. I have a mapping which does something like this:
i.e. prepopulate the command line with the substitution command for the word under the cursor. I had both inccommand and TSHighlighter active for that buffer and this happened about 3 hours into the coding session. When I tried to type anything after executing the mapping, in the command line
I noticed the editor simply froze and memory started to spike up massively. I finally got to exit after mashing Something I noticed is that the screen did not redraw after those freezes, since I did not see the P being replaced as an effect of If I'm not mistaken, I got to use that same mapping a few times without running into this issue, so I can't reproduce it 100% of the time on my end (if ever again). |
Beta Was this translation helpful? Give feedback.
-
Unrelated to the previous issue, but something I also noticed after using the highlighter massively for about a day: making edits (presumably most frequently on insert mode) will mess up the highlighting. When I load up the file, it seems to be rendered perfectly according to the rules. However, say you have
where the
Another issue: two things being highlighted better if they were far apart, but not one after the other. For instance, the following would not render correctly: only the first line and the ones above it would be fine, everything below would be messed up import { foo } from "module"
import { bar } from "module"
(messed up stuff...) but if you added a line between them, it would get a little better import { foo } from "module"
import { bar } from "module" Redoing the same actions would result in the exact same effects, which I think hints at the tree being somehow corrupted by that point. Similar to this, it'd sometimes notice things being highlighted inside of comments, like somehow two attributes were merged for the same row and column. |
Beta Was this translation helpful? Give feedback.
-
yes, byte-resolution updates need to be improved. I plan to do a major refactor soon. |
Beta Was this translation helpful? Give feedback.
-
Today I tried another approach Only rendering it in specific modes local mode = api.nvim_get_mode()
if mode.blocking or (mode.mode ~= "n" and mode.mode ~= "v") then
return
end Not redrawing in case the tick didn't change local tick = api.nvim_buf_get_changedtick(buf)
if tick == _G.buf_ticks[buf] then
return
end Reparsing the entire buffer every time local lines = api.nvim_buf_get_lines(buf, 0, -1, false)
local parser = vim.treesitter.get_parser(0, lang)
local tree = parser:parse()
for cid, node in self.query:iter_captures(tree:root(),buf,0,#lines) do Using the nvim_buf_add_highlight API for coloring the buffer api.nvim_buf_clear_namespace(buf, _G.treesitter_namespace, 0, -1)
-- other stuff...
api.nvim_buf_add_highlight(buf, _G.treesitter_namespace, hl_group, start_row, start_col, next_end_col) While it seemed to fare a little bit better in some regards, I eventually ran into the same problems as the current runtime's approach
|
Beta Was this translation helpful? Give feedback.
-
@bfredl do you think it would make a difference, on the inconsistency issue, if I debounced the callback and ran the highlighting function on the trailing end of the timer? I'm thinking that it gets messed up sometimes because it's too eager, but I don't know... |
Beta Was this translation helpful? Give feedback.
-
Well, if the problems arise at certain types of edits, then the buffer update is likely wrong (tree-sitter already uses more info than the rplugin buffer updates, even if it is not proper byte-level updates yet). Debouncing would only help with performance, not with correctness. It is possible to set a deadline for the parsing, so a large reparse does not freeze the editor. We should do this eventually, but it is unrelated to redraw bugs. |
Beta Was this translation helpful? Give feedback.
-
@bfredl @justinmk Just a follow up to our conversation in #10124 (comment) - there is now a |
Beta Was this translation helpful? Give feedback.
-
@bfredl Also, the syntax for tree queries is changing slightly in tree-sitter/tree-sitter#615. We will be updating the queries in grammar repos like |
Beta Was this translation helpful? Give feedback.
-
Hello everyone, lately I have been working on I just wanted to give some news on this. I (and many others) started working on some items in the list at the top of this issue :
|
Beta Was this translation helpful? Give feedback.
-
@vigoux is tree-sitter going to be in neovim natively in 0.5 or will it always live as a separate extension? |
Beta Was this translation helpful? Give feedback.
-
@tristan957 as I am not part of the core team, I can't tell anything about it and I don't have any clue. My personal goal though is to bring (at least for C) some kind of built-in, and easy-to use highlighting (cc @tjdevries). |
Beta Was this translation helpful? Give feedback.
-
Awesome. I'll install it on my dev machine. Thanks for your work :). |
Beta Was this translation helpful? Give feedback.
-
Personally I use |
Beta Was this translation helpful? Give feedback.
-
@tristan957 I'm not sure I understand your question. Tree sitter support is built-in and compiled into Neovim -- it would not be possible to use otherwise. I'm not sure about long term support for various different languages and how/if we'll ship them, but I just wanted to at least clarify the question there. |
Beta Was this translation helpful? Give feedback.
-
TJ thanks for the clarification. I was naiive when I first entered this issue, but now I understand the relationship between the package and neovim much better. |
Beta Was this translation helpful? Give feedback.
-
Good :) As a personal desire (read: not necessarily as a goal of the core team) I think it would be cool to ship at least C, Vim & Lua treesitter support natively with neovim. For a couple of reasons:
I think we could ship the various associated query files for these languages that As I mentioned, this is just a personal preference at the moment I have been thinking about recently, made possible by @vigoux great work 😁 |
Beta Was this translation helpful? Give feedback.
-
The vim treesitter parser does not exists (yet) fully, but I have been putting up some work here.
Thanks for the kind words ! |
Beta Was this translation helpful? Give feedback.
-
I feel not comfortable about 3.
The configurations for nvim-lsp also have their own repo which can move quicker that polluting the neovim repo. Maybe we can have a minimum everyone can agree on here and update it every half year or so. The vim highlights are also quite minimalistic and almost never change and people use plugins for additional fanciness. |
Beta Was this translation helpful? Give feedback.
-
are there any plans to separate out the lua tree-sitter bindings to their own module? as lua doesn't really have official tree sitter bindings it would be nice to have it as a module on luarocks or something as an officially endorsed binding to tree-sitter. I'm wanting to write some tooling in lua with tree-sitter, but would rather build on top of some work already done rather than rewrite my own bindings from scratch. |
Beta Was this translation helpful? Give feedback.
-
Hi, there is effectively at least the willing to build out of tree Lua bindings. |
Beta Was this translation helpful? Give feedback.
-
As a common question is what the status of tree-sitter is, I figure it would make sense with a tracking issue.
General:
Highlight specific:
@match
predicate (by proper lua regex API) add regex support in treesitter queries #11880Beta Was this translation helpful? Give feedback.
All reactions