Here is the simple recipie that I have to search for time and again. Check the sources linked below for a much better explanation of how and why.

In my computer

Make your key available to ssh-agent

# for mac
ssh-add -K ~/.ssh/id_rsa

and add the following to ~/.ssh/config

    # ~/.ssh/config
    Host *
        SendEnv LANG LC_*
        ForwardAgent Yes

Configure tmux in the remote server:

    # ~/.tmux.conf
    set-environment -g 'SSH_AUTH_SOCK' ~/.ssh/ssh_auth_sock
    # ~/.ssh/rc
    if [ -S "$SSH_AUTH_SOCK" ]; then
        ln -sf $SSH_AUTH_SOCK ~/.ssh/ssh_auth_sock
    fi

Debbuging: is ssh forwarding working?

    echo "$SSH_AUTH_SOCK"
    # Should print something like this:
    # /tmp/ssh-4hNGMk8AZX/agent.79453

Sources