Update to GnuPG 2.1 or better – “Wot? No secret key?”

The “decryption failed: no secret key” message after upgrading to the latest and greatest GnuPG (i.e. 2.1 or later) can make your heart stop for a moment. But do not despair! Before trying out all the intertwined sequences of commands out there, or even worse generating new keys, you may be safe, and up&running within a few moments with our simple hint.

The issue is that the locations where GnuPG looks for things have changed between version 2.0 and 2.1. In GnuPG 1.x, and 2.0, the secret keyring file ~/.gnupg/secring.gpg not only held your secret keys – as the cunningly chosen name suggests – but also a duplicate copy of your public key data data as well. Due to that, it was entirely self-contained, but at the same time more difficult for the program to maintain since two copies of the same thing had to be kept in sync.

Starting with GnuPG 2.1, secret keys are now stored separately, one per file, and they are maintained by gpg-agent, which keeps them in the ~/.gnupg/private-keys-v1.d/ directory. How do the keys get transferred from ~/.gnupg/secring.gpg to individual files in the new ~/.gnupg/private-keys-v1.d/ when upgrading? In many cases, they don’t; at least not automatically. But do not despair, if GnuPG didn’t automatically migrate your secret keys, you can simply import the old secring directly:

example$ gpg2 --import ~/.gnupg/secring.gpg
Code language: Shell Session (shell)

Note that we are invoking the executable as gpg2 to be sure it will use the new format. Even though gpg and gpg2 may well be the same piece of executable (one symlinked to the other), it is important to make your desire for using the new version behaviour explicit. The GnuPG will detect whether it was invoked as v1 or v2, and will alter its behaviour accordingly. So don’t miss out on the “2”! To verify that the secret keys are available again:

example$ gpg2 --list-secret-keys
Code language: Shell Session (shell)

Why did such a drastic, potentially workflow-breaking change happen with a x.1 release, and not with the x.0 one? Why is there no provision to auto-convert the ~/.gnupg/secring.gpg when no ~/.gnupg/private-keys-v1.d/ exists, or the modification date of ~/.gnupg/secring.gpg is newer than that of ~/.gnupg/private-keys-v1.d/? To find out one would have to get on IRC and ask the developers. I am not sure whether their answers would not be disturbing, however.

In any case, and in general, we highly recommend to make a backup copy of stuff as valuable as your PGP keys. Ultimately they are part of your on-line identity. If you have to change your PGP keys often because you lost your old ones, this will not buy you any confidence with others.

Before complaining that running a full backup to save a few files seems overkill and will take hours (and of course you want to use the latest version right away, don’t you?), here is a small tip how to backup things in seconds. Before upgrading GnuPG X.Y, do the following:

example$ cd example$ tar -cvf .gnupg_pre_X.Y.tar .gnupg/ example$ chmod 0600 .gnupg_pre_X.Y.tar
Code language: Shell Session (shell)

This will create an archive copy of your current ~/.gnupg directory with everything in it. Symlinks, permissions, etc. are all retained. Permissions are tightened to yourself only to be on the safe side.

When in need, you can restore the old version with:

example$ cd example$ mv .gnupg/ .gnupg.DONTUSE/ example$ tar -xvf .gnupg_pre_X.Y.tar
Code language: Shell Session (shell)

To track down complicated problems, you can use the --strip-components option of tar to extract several old versions into directories with different names (e.g. ~/.gnupg_pre_X.Y/), and switch between them using the --homedir option of gpg and gpg2.