Switch completion models?

[crenwick]

Is there a setting for the LSP that allows it to switch to 4o as the completion model?

[sircoderin]

It's been over a month now from the release of the new completion model. Can we please have an update?

[AntoineGS]

You can check how we did it for copilot.lua here (Copilot plugin for NeoVim)
You do need to pass the client as if it were vscode, but the maintainer of copilot.vim (official Copilot plugin for NeoVim) says that should change.

[sircoderin]

Tried getting to the bottom of it myself with manual configuration but I couldn't confirm that the correct model was being used in the lsp. How did you check that the configuration actually changes the model?

[AntoineGS]

When you use an incorrect model the LSP outputs an error or warning saying it is falling back to gpt35, so no error means it's a valid one. I also see it pop up every now and then with trace activated, I can post my LSP config when I am back home later it might help.

[sircoderin]

I tried setting the model to something not valid as "gpt-test" and there was no error.

// Initialize method
[2025-03-23T12:35:59+02:00 TRACE lsp] outgoing message:{"jsonrpc":"2.0","id":0,"method":"initialize","params":{"processId":null,"rootUri":"file:///","initializationOptions":{"copilotIntegrationId":"vscode-chat"},"capabilities":{"workspace":{"applyEdit":true,"workspaceEdit":{"documentChanges":true,"resourceOperations":["create","rename","delete"],"snippetEditSupport":true},"didChangeConfiguration":{"dynamicRegistration":true},"didChangeWatchedFiles":{"dynamicRegistration":true,"relativePatternSupport":true},"symbol":{},"executeCommand":{"dynamicRegistration":false},"workspaceFolders":true,"configuration":true,"codeLens":{"refreshSupport":true},"fileOperations":{"dynamicRegistration":false,"didRename":true,"willRename":true},"inlayHint":{"refreshSupport":true},"diagnostic":{}},"textDocument":{"synchronization":{"didSave":true},"completion":{"completionItem":{"snippetSupport":true,"insertReplaceSupport":true,"resolveSupport":{"properties":["additionalTextEdits","command","documentation"]},"labelDetailsSupport":true},"contextSupport":true,"completionList":{"itemDefaults":["commitCharacters","editRange","insertTextMode","insertTextFormat","data"]}},"hover":{"contentFormat":["markdown"]},"signatureHelp":{"signatureInformation":{"documentationFormat":["markdown","plaintext"],"parameterInformation":{"labelOffsetSupport":true},"activeParameterSupport":true}},"formatting":{"dynamicRegistration":true},"rangeFormatting":{"dynamicRegistration":true},"onTypeFormatting":{"dynamicRegistration":true},"definition":{"linkSupport":true},"codeAction":{"codeActionLiteralSupport":{"codeActionKind":{"valueSet":["refactor","quickfix","source"]}},"dataSupport":true,"resolveSupport":{"properties":["kind","diagnostics","isPreferred","disabled","edit","command"]}},"codeLens":{"dynamicRegistration":false},"rename":{"prepareSupport":true,"prepareSupportDefaultBehavior":1},"publishDiagnostics":{"relatedInformation":true},"inlayHint":{"dynamicRegistration":false,"resolveSupport":{"properties":["textEdits","tooltip","label.tooltip","label.location","label.command"]}}},"window":{"workDoneProgress":true,"showMessage":{}},"general":{"positionEncodings":["utf-16"]},"experimental":{"serverStatusNotification":true,"localDocs":true}},"trace":"verbose","workspaceFolders":[],"clientInfo":{"name":"Custom Dev","version":"0.180.0"}}}
[2025-03-23T12:36:00+02:00 TRACE lsp::input_handler] incoming message: {"jsonrpc":"2.0","id":0,"result":{"capabilities":{"textDocumentSync":{"openClose":true,"change":2},"notebookDocumentSync":{"notebookSelector":[{"notebook":"*"}]},"workspace":{"workspaceFolders":{"supported":true,"changeNotifications":true}},"executeCommandProvider":{"commands":["github.copilot.didAcceptCompletionItem","github.copilot.didAcceptPanelCompletionItem"]},"inlineCompletionProvider":{}},"serverInfo":{"name":"GitHub Copilot Language Server","version":"1.236.0","nodeVersion":"22.14.0"}}}

// Set editor info
[2025-03-23T12:34:24+02:00 TRACE lsp] outgoing message:{"jsonrpc":"2.0","id":11,"method":"setEditorInfo","params":{"editorInfo":{"name":"custom","version":"0.1.0"},"editorPluginInfo":{"name":"custom-copilot","version":"0.0.1"},"editorConfiguration":{"github":{"copilot":{"selectedCompletionModel":"gpt-test"}}}}}
[2025-03-23T12:34:24+02:00 TRACE lsp::input_handler] incoming message: {"jsonrpc":"2.0","id":11,"result":"OK"}
[2025-03-23T12:34:24+02:00 TRACE lsp] Took 216.121625ms to receive response to "setEditorInfo" id 11

[AntoineGS]

Here is my configuration for the NeoVim LSP client, so some things might be named differently as it servers as a middle-layer between this configuration and the actual LSP server:

{
 capabilities = {
 copilot = {
 openURL = true
 },
 general = {
 positionEncodings = { "utf-16" }
 },
 textDocument = {
 callHierarchy = {
 dynamicRegistration = false
 },
 codeAction = {
 codeActionLiteralSupport = {
 codeActionKind = {
 valueSet = { "", "quickfix", "refactor", "refactor.extract", "refactor.inline", "refactor.rewrite", "source", "source.organizeImports" }
 }
 },
 dataSupport = true,
 dynamicRegistration = true,
 isPreferredSupport = true,
 resolveSupport = {
 properties = { "edit" }
 }
 },
 completion = {
 completionItem = {
 commitCharactersSupport = false,
 deprecatedSupport = false,
 documentationFormat = { "markdown", "plaintext" },
 preselectSupport = false,
 snippetSupport = false
 },
 completionItemKind = {
 valueSet = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25 }
 },
 completionList = {
 itemDefaults = { "editRange", "insertTextFormat", "insertTextMode", "data" }
 },
 contextSupport = false,
 dynamicRegistration = false
 },
 declaration = {
 linkSupport = true
 },
 definition = {
 dynamicRegistration = true,
 linkSupport = true
 },
 diagnostic = {
 dynamicRegistration = false
 },
 documentHighlight = {
 dynamicRegistration = false
 },
 documentSymbol = {
 dynamicRegistration = false,
 hierarchicalDocumentSymbolSupport = true,
 symbolKind = {
 valueSet = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26 }
 }
 },
 formatting = {
 dynamicRegistration = true
 },
 hover = {
 contentFormat = { "markdown", "plaintext" },
 dynamicRegistration = true
 },
 implementation = {
 linkSupport = true
 },
 inlayHint = {
 dynamicRegistration = true,
 resolveSupport = {
 properties = { "textEdits", "tooltip", "location", "command" }
 }
 },
 publishDiagnostics = {
 dataSupport = true,
 relatedInformation = true,
 tagSupport = {
 valueSet = { 1, 2 }
 }
 },
 rangeFormatting = {
 dynamicRegistration = true
 },
 references = {
 dynamicRegistration = false
 },
 rename = {
 dynamicRegistration = true,
 prepareSupport = true
 },
 semanticTokens = {
 augmentsSyntaxTokens = true,
 dynamicRegistration = false,
 formats = { "relative" },
 multilineTokenSupport = false,
 overlappingTokenSupport = true,
 requests = {
 full = {
 delta = true
 },
 range = false
 },
 serverCancelSupport = false,
 tokenModifiers = { "declaration", "definition", "readonly", "static", "deprecated", "abstract", "async", "modification", "documentation", "defaultLibrary" },
 tokenTypes = { "namespace", "type", "class", "enum", "interface", "struct", "typeParameter", "parameter", "variable", "property", "enumMember", "event", "function", "method", "macro", "keyword", "modifier", "comment", "string", "number", "regexp", "operator", "decorator" }
 },
 signatureHelp = {
 dynamicRegistration = false,
 signatureInformation = {
 activeParameterSupport = true,
 documentationFormat = { "markdown", "plaintext" },
 parameterInformation = {
 labelOffsetSupport = true
 }
 }
 },
 synchronization = {
 didSave = true,
 dynamicRegistration = false,
 willSave = true,
 willSaveWaitUntil = true
 },
 typeDefinition = {
 linkSupport = true
 }
 },
 window = {
 showDocument = {
 support = true
 },
 showMessage = {
 messageActionItem = {
 additionalPropertiesSupport = false
 }
 },
 workDoneProgress = true
 },
 workspace = {
 workspaceFolders = true
 }
 },
 cmd = { "C:/Users/antoi/AppData/Local/nvim-data/lazy/copilot.lua/copilot/win32-x64/copilot-language-server-1.290.0.exe", "--stdio" },
 get_language_id = <function 1>,
 handlers = {
 ["$/logTrace"] = <function 2>,
 ["$/progress"] = <function 3>,
 PanelSolution = <function 4>,
 PanelSolutionsDone = <function 5>,
 ["copilot/openURL"] = <function 6>,
 statusNotification = <function 7>,
 ["window/logMessage"] = <function 8>
 },
 init_options = {
 copilotIntegrationId = "vscode-chat",
 editorInfo = {
 name = "Neovim",
 version = "0.10.4"
 },
 editorPluginInfo = {
 name = "copilot.lua",
 version = "1.290.0"
 }
 },
 name = "copilot",
 on_exit = <function 9>,
 on_init = <function 10>,
 root_dir = "C:\\Users\\antoi\\AppData\\Local\\nvim-data",
 trace = true,
 workspace_folders = { {
 name = "C:\\Users\\antoi\\AppData\\Local\\nvim-data",
 uri = "file:///C:/Users/antoi/AppData/Local/nvim-data"
 } }
}

setEditorInfo call:

{
 editorConfiguration = {
 disabledLanguages = {},
 enableAutoCompletions = true,
 github = {
 copilot = {
 selectedCompletionModel = "gpt-4o-copilot"
 }
 }
 },
 editorInfo = {
 name = "Neovim",
 version = "0.10.4"
 },
 editorPluginInfo = {
 name = "copilot.lua",
 version = "1.290.0"
 }
}

I had to use init_options instead of initialiationOptions but I figured that was because the LSP client was translating it, but it is worth a try.
A better way to see which model you are using would be to call $/setTrace with value='verbose', in which case it will output a log like this:

2025-03-22 23:01:52.256 [INFO]: 51: LSP message: [streamChoices] solution 0 returned. finish reason: [stop]
2025-03-22 23:01:52.256 [WARN]: 53: LSP message: [ghostText] Current completion changed before returning
2025-03-22 23:01:52.256 [INFO]: 52: LSP message: [streamChoices] request done: headerRequestId: [xxxxxxxxxx] model deployment ID: []
2025-03-22 23:01:53.847 [INFO]: 54: LSP message: [fetchCompletions] request.response: [https://proxy.individual.githubcopilot.com/v1/engines/gpt-4o-copilot/completions] took 507.4083999999998 ms

[sircoderin]

So the problem was coming from not using the latest lsp, but the one from copilot.vim which is outdated. On the other hand I removed copilotIntegrationId from init options and the model selection still works on the latest lsp. 🥰

[AntoineGS]

That makes sense haha, thanks for the note about the integrationId I'll be able to remove it too.

[tpope]

Y'all figured it out before I could make it official: The latest language server release supports the github.copilot.selectedCompletionModel workspace configuration setting, without any need for a workaround. I'll be shipping a new release of Copilot.vim with it today too.

Right now there's 2 supported models: the original copilot-codex aka GPT 3.5 Turbo and the brand new gpt-4o-copilot aka GPT 4o. Stay tuned for an API to retrieve the list of supported models dynamically.

One thing I want to call out is that I see a smattering of APIs in the logs shared above that aren't part of the SDK at all (e.g., setEditorInfo). I presume they were cribbed from early versions of Copilot.vim. These have been left in place for now to avoid gratuitously breaking older implementations, but they're unsupported, and will probably go away eventually.

[AntoineGS]

Thank you for the update (and the work you do 🙂)!
Yes you are correct copilot.lua was heavily inspired from copilot.vim (at least for the initial release).
I did notice the warnings for the setEditorInfo and have been trying to stop using them however I had not yet figured out the mapping between the new and old structures, I think I got it now though.

[tpope]

Right now there's 2 supported models: the original copilot-codex aka GPT 3.5 Turbo and the brand new gpt-4o-copilot aka GPT 4o.

Didn't realize when I said this, but the plan is to retire copilot-codex entirely. The only supported value for this option is now gpt-4o-copilot, which means the option no longer actually does anything except complain if you set it to anything else. This is a server-side change, not client-side.

Stay tuned for an API to retrieve the list of supported models dynamically.

This would be a list of 1, so I'm not going to pursue it at this time.

See also this announcement.

[AntoineGS]

Thanks for the info!