When I need to edit files using VS Code on server hosts, I use this: https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-ssh
However, it often times out and requires reconnecting, so for small configuration tasks, I prefer to use vi through an ssh session in WSL. The ssh session times out, but reconnecting is much simpler.
https://code.visualstudio.com/docs/remote/troubleshooting#_configuring-key-based-authentication