thehamsta / nvim-semantic-tokens Goto Github PK
View Code? Open in Web Editor NEWLicense: Apache License 2.0
License: Apache License 2.0
Hi,
Since neovim/neovim#15723 has completely stalled, would you be interested in including the actual LSP functionality into this plugin? I've been using that code for a year now by just copying it to my dotfiles, but your code looks way cleaner and I would rather use it as a plugin.
I think I can create a PR too if you want.
Hi,
I really want to try the semantic highlighting in neovim and searched a lot, finally found this plugin. 😀
So I have
lua im.lsp.buf.semantic_tokens_full()
after entering the buffer, but returns "method textDocument/semanticTokens/full is not supported by any of the servers registered for the current buffer"How could I debug for this? Thank a lot
neovim: 0.7.2
I think this is because of line 71 in table-highlighter.lua.
If i remove the first parameter, i.e. change it to highlight(hls, hl)
, everything works as expected.
can someone suggest how to make this all work for inactive regions (e.g. proprocessor inactive chunks)? I think it would rely on clangd features in https://reviews.llvm.org/D67536. Will these get mapped to a specific hl group by this plugin, or is something else required? Thanks.
I wanted a command like the one treesitter has to debug semantic tokens. So, I came up with this:
local function get_bit(n, k)
if _G.bit then
return _G.bit.band(_G.bit.rshift(n, k), 1)
else
return math.floor((n / math.pow(2, k)) % 2)
end
end
local function modifiers_from_number(x, modifiers_table)
local modifiers = {}
for i = 0, #modifiers_table - 1 do
local bit = get_bit(x, i)
if bit == 1 then
table.insert(modifiers, 1, modifiers_table[i + 1])
end
end
return modifiers
end
local function handler(err, response, ctx, config)
local client = vim.lsp.get_client_by_id(ctx.client_id)
if not client then
return
end
local legend = client.server_capabilities.semanticTokensProvider.legend
local token_types = legend.tokenTypes
local token_modifiers = legend.tokenModifiers
local data = response.data
local line
local start_char = 0
for i = 1, #data, 5 do
local delta_line = data[i]
line = line and line + delta_line or delta_line
local delta_start = data[i + 1]
start_char = delta_line == 0 and start_char + delta_start or delta_start
local token_type = token_types[data[i + 3] + 1]
local modifiers = modifiers_from_number(data[i + 4], token_modifiers)
local token = {
line = line,
start_char = start_char,
length = data[i + 2],
type = token_type,
modifiers = modifiers,
offset_encoding = client.offset_encoding,
}
local line, col = unpack(config.cursor)
if token.line == line - 1 then
if token.start_char <= col and col <= token.start_char + token.length then
vim.pretty_print(token)
end
end
end
end
local function semantic_token_under_cursor()
local cursor = vim.api.nvim_win_get_cursor(0)
local params = { textDocument = require("vim.lsp.util").make_text_document_params() }
local h = vim.lsp.with(handler, { cursor = cursor })
vim.lsp.buf_request(0, "textDocument/semanticTokens/full", params, h)
end
vim.api.nvim_create_user_command("SemanticTokenUnderCursor", semantic_token_under_cursor, {})
It would be better if I could reuse the on_full
handler with a custom on_token
, but it returns at
Some token types are missing for the python server. Everything else works.
{
"tokenTypes": [
"comment",
"keyword",
"string",
"number",
"regexp",
"type",
"class",
"interface",
"enum",
"enumMember",
"typeParameter",
"function",
"method",
"property",
"variable",
"parameter",
"module",
"intrinsic",
"selfParameter",
"clsParameter",
"magicFunction",
"builtinConstant"
],
"tokenModifiers": [
"declaration",
"static",
"abstract",
"async",
"documentation",
"typeHint",
"typeHintComment",
"readonly",
"decorator",
"builtin"
]
}
Maybe this inside the on_attach
would be better than an if statement:
local on_attach = function(client, bufnr)
if client.resolved_capabilities.semantic_tokens_full == true then
vim.cmd [[autocmd BufEnter,CursorHold,InsertLeave <buffer> lua vim.lsp.buf.semantic_tokens_full()]]
end
--...
end
I installed the latest nightly neovim, put this plugin in my plugins/start directory, pasted the setup code from the README.md file, and tried the following in a C file with clangd
running:
:hi link LspEnumMember Keyword
:hi link LspEnum String
etc., using suggestions from the doc file on highlighting.
Nothing changes in my file. No color highlighting changes.
lsp.log shows ASTWorker building file /.../t.c
output of :lua =vim.lsp.semantic_tokens
shows things like
{
end_col = 18,
extmark_added = true,
line = 25,
modifiers = { "declaration", "readonly", "fileScope" },
start_col = 4,
type = "enumMember"
}
and
{
end_col = 14,
extmark_added = true,
line = 33,
modifiers = { "globalScope" },
start_col = 5,
type = "enum"
}
So it looks like the server is sending back the correct info, but the highlight links don't appear to do anything.
The log shows this on a :LspRestart
:
--> workspace/semanticTokens/refresh(0)\n"
<-- $/cancelRequest
I[21:00:16.192] <-- textDocument/semanticTokens/full(3)\n"
<-- reply(0)\n"
--> textDocument/publishDiagnostics\n"
--> reply:textDocument/semanticTokens/full(2) 48 ms, error: Task was cancelled.
I[21:00:16.219] --> reply:textDocument/semanticTokens/full(3) 27 ms\n"
Edit: forgot to mention I'm using:
$ clangd --version
Ubuntu clangd version 14.0.0-1ubuntu1
Features: linux+grpc
Platform: x86_64-pc-linux-gnu
Often (not always) when I trigger the "add missing include" code action by clangd, the following error pops up:
E486: Pattern not found: error:
Error executing vim.schedule lua callback: .../start/nvim-semantic-tokens/lua/nvim-semantic-tokens.lua:35: col value outside range
stack traceback:
[C]: in function 'nvim_buf_set_extmark'
.../start/nvim-semantic-tokens/lua/nvim-semantic-tokens.lua:35: in function 'highlight'
.../start/nvim-semantic-tokens/lua/nvim-semantic-tokens.lua:46: in function 'highlight'
...ic-tokens/lua/nvim-semantic-tokens/table-highlighter.lua:60: in function 'highlight_token'
.../start/nvim-semantic-tokens/lua/nvim-semantic-tokens.lua:50: in function 'on_token'
...ntic-tokens/lua/nvim-semantic-tokens/semantic_tokens.lua:103: in function 'handler'
/home/devel/local/nvim/share/nvim/runtime/lua/vim/lsp.lua:1390: in function ''
vim/_editor.lua: in function <vim/_editor.lua:0>
Is there any way to at least suppress this? It can be quite annoying.
If I understand correctly a variable with both readOnly
and fileScope
will be end up with up to 3 highlight groups applied to it: Variable
, ReadOnlyVariable
and FileScopeVariable
. That seems to assume that all modifiers are independent, but I don't think that always applies. I may want to highlight ReadOnlyFileScopeVariable
specially, or at least differently from both ReadOnlyFunctionScopeVariable
and (mutable) FileScopeVariable
. That doesn't seem to be possible with your current table-based approach.
One option would be to have a user-supplied function like highlightGroups(fileType, tokenKind, modifierList) -> optional<highlightGroupList>
and then fall back to the simple table-based approach if that returns nil. That allows fine tuning for "interesting" cases while still using a simple declarative style for normal cases. It also allows useful but non-trivial logic like looking for variables tagged with fileScope
without readOnly
and highlighting them MutableGlobalVariable
(which is really the case I'd want to highlight).
If there is already some way in (neo)vim to color based on multiple highlight groups rather than just one, that would also work, but as far as I know, they are independent.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.