I wanted to know how to use GPG keys securly in such a way like I already do with SSH keys: private key always kept on my local host while it is usable on remote hosts via SSH connection.
SSH client side
GPG Agent needs to be run. The agent is usually started on demand by gpg, gpgsm, gpgconf or gpg-connect-agent.
The usual way to run the agent is from the ~/.xsession file:
eval $(gpg-agent --daemon)
For my convenience, I like to add hosts to .ssh/config file:
Host vm HostName 192.168.0.31 RemoteForward /run/user/1000/gnupg/S.gpg-agent
SSH server side
Change SSH daemon's config to allow recreating stale gpg unix sockets.
Edit /etc/ssh/sshd_config file:
GPG agent should be stopped to release related unix socket.
ps aux | grep gpg kill [pid]
Public key must be present
gpg --keyserver keyserver.ubuntu.com --search [EMAIL ADDRESS]
Now reconnect to the SSH sever and validate if all fine:
gpg -K gpg -k