Package Signing

Hi Magnus,

It's time to nag you again about package signing. I can give you a script to
batch sign packages, run repo-add, then sign the generated repo with a single
passphrase prompt. Obviously I don't know how well that fits with your current
release method, but it should be possible to set something up that is minimally
invasive and I'll gladly help if I can.

Regards,
Xyne

On Tue, Jul 24, 2012 at 1:20 PM, Xyne  wrote:
> Hi Magnus,
>
> It's time to nag you again about package signing. I can give you a script to
> batch sign packages, run repo-add, then sign the generated repo with a single
> passphrase prompt. Obviously I don't know how well that fits with your current
> release method, but it should be possible to set something up that is minimally
> invasive and I'll gladly help if I can.Good that you nag!

I'd love getting that script, and possibly hints on key
generation/storage/management/etc as well.

/M-- 
Magnus Therning                      OpenPGP: 0xAB4DFBA4
email:    jabber: 
twitter: magthe               http://therning.org/magnus

Magnus Therning wrote:

> On Tue, Jul 24, 2012 at 1:20 PM, Xyne  wrote:
> > Hi Magnus,
> >
> > It's time to nag you again about package signing. I can give you a script to
> > batch sign packages, run repo-add, then sign the generated repo with a single
> > passphrase prompt. Obviously I don't know how well that fits with your current
> > release method, but it should be possible to set something up that is minimally
> > invasive and I'll gladly help if I can.
> 
> Good that you nag!
> 
> I'd love getting that script, and possibly hints on key
> generation/storage/management/etc as well.I've put together a clean script using various code snippets that I have in my
release scripts:

http://xyne.archlinux.ca/scripts/pacman/#repo...

Just ask if anything is unclear or if you think you've found a bug. If you need
something customized to your build system, give me some details and I'll work
on it.

I'm going to announce it on the forum too. If there's any interest, I'll
probably package it. If I do, the link will also change, so check the projects
page if the one above dies.


For key generation/etc, I would suggest generating a new key pair dedicated to
package signing, but that's just a personal preference. You could just as well
use the same key pair that you already use to sign your email. Management is not
really any different either: keep the private key secure, have a revocation key
ready, etc.

If you create a new key pair, upload the public one to e.g. pgp.mit.edu and
post the fingerprint in a few different places so users can verify it before
trusting it.

That's all I can think of for now.

Xyne wrote:

> I've put together a clean script using various code snippets that I have in my
> release scripts:
> 
> http://xyne.archlinux.ca/scripts/pacman/#repo...
> 
> Just ask if anything is unclear or if you think you've found a bug. If you need
> something customized to your build system, give me some details and I'll work
> on it.I have just uploaded a new version that fixes a bug in the creation of the
$repo.db.sig symlink.

On Tue, Jul 24, 2012 at 11:19:54PM +0000, Xyne wrote:
> Magnus Therning wrote:
> 
>> On Tue, Jul 24, 2012 at 1:20 PM, Xyne  wrote:
>>> Hi Magnus,
>>>
>>> It's time to nag you again about package signing. I can give you a script to
>>> batch sign packages, run repo-add, then sign the generated repo with a single
>>> passphrase prompt. Obviously I don't know how well that fits with your current
>>> release method, but it should be possible to set something up that is minimally
>>> invasive and I'll gladly help if I can.
>> 
>> Good that you nag!
>> 
>> I'd love getting that script, and possibly hints on key
>> generation/storage/management/etc as well.
> 
> 
> I've put together a clean script using various code snippets that I have in my
> release scripts:
> 
> http://xyne.archlinux.ca/scripts/pacman/#repo...
> 
> Just ask if anything is unclear or if you think you've found a bug.
> If you need something customized to your build system, give me some
> details and I'll work on it.[...]> For key generation/etc, I would suggest generating a new key pair
> dedicated to package signing, but that's just a personal preference.
> You could just as well use the same key pair that you already use to
> sign your email. Management is not really any different either: keep
> the private key secure, have a revocation key ready, etc.Correct me if I'm wrong in this assumption, but I need to have the
following three items available when running the script:

1. The newly-built package.
2. The repo database (x.db.tar.gz) I'm adding the package to.
3. The secret key.

This is a slight problem for me.  I build on kiwilight (where I'm not
alone in having root access), the database is on xsounds.org (where I
don't have root access at all), and to be fully comfortable I'd like
to keep the secret key and perform the signing on my own machine :-)

Is there some way to simply extract the actual data that is to be
signed (the hashes), and perform the actual signing manually?

(I've found a need for this sort of thing with other package managers
as well, especially RPM, but never found a way to do that.  I would
find it unfortunate if the pacman developers have painted themselves
into the same corner as the RPM developers.)

/M

On 2012-07-29 18:48 +0200Magnus Therning wrote:

>Correct me if I'm wrong in this assumption, but I need to have the
>following three items available when running the script:
>
>1. The newly-built package.
>2. The repo database (x.db.tar.gz) I'm adding the package to.
>3. The secret key.1 & 3, yes.

If you have all of the packages then the full database will be recreated so you
don't actually need 2, but if it's present then it will be updated with the
selected packages.>This is a slight problem for me.  I build on kiwilight (where I'm not
>alone in having root access), the database is on xsounds.org (where I
>don't have root access at all), and to be fully comfortable I'd like
>to keep the secret key and perform the signing on my own machine :-)
>
>Is there some way to simply extract the actual data that is to be
>signed (the hashes), and perform the actual signing manually?I'm not sure, but I think gpg needs the full file to generate the signature.
There might be some way to dig the hashing algorithm out of the source code and
then create your own remote signing function with it, but that would require
knowledge of gpg internals.

One solution might be to build the packages on kiwilight, then mount the
directory of built packages with sshfs.  You could then run the signing script
locally. I don't know much bandwith that will use, but I think it's worth
trying. In the worst case scenario, it will be equivalent to downloading the
packages. Whether or not that's a problem depends on your connection.

If I understand the problem correctly, you do not generate the database
yourself. That should not be a problem for package signatures, as repo-add will
include them in the database as long as the signature files are present when it
adds the packages. If you can't remote-mount xsounds with sshfs and sign the
database there, just download it and sign it locally then upload the database
signature file.

If that is not possible for whatever reason, just having package signatures is
better than nothing. However, given what you've said about not being the only
one to have access to these repos, I think package signing in this case is very
important.

I would also like to know who does have access to these files. On kiwilight
I believe that it is only Kaiting, who is a TU. Who has access on xsounds?





Could you simply make kiwilight the main host and have xsounds mirror it? The
process would then be the following:
1) ssh into kiwilight, build, and move to haskell/$arch if necessary
2) mount haskell/$arch via ssh and run the signing script locally

You would then have a fully signed repo in haskell/$arch that can be mirrored
by xsounds.

On Mon, Jul 30, 2012 at 8:35 AM, Xyne  wrote:
> On 2012-07-29 18:48 +0200
> Magnus Therning wrote:
>
>>Correct me if I'm wrong in this assumption, but I need to have the
>>following three items available when running the script:
>>
>>1. The newly-built package.
>>2. The repo database (x.db.tar.gz) I'm adding the package to.
>>3. The secret key.
>
> 1 & 3, yes.
>
> If you have all of the packages then the full database will be recreated so you
> don't actually need 2, but if it's present then it will be updated with the
> selected packages.
>
>
>>This is a slight problem for me.  I build on kiwilight (where I'm not
>>alone in having root access), the database is on xsounds.org (where I
>>don't have root access at all), and to be fully comfortable I'd like
>>to keep the secret key and perform the signing on my own machine :-)
>>
>>Is there some way to simply extract the actual data that is to be
>>signed (the hashes), and perform the actual signing manually?
>
> I'm not sure, but I think gpg needs the full file to generate the signature.
> There might be some way to dig the hashing algorithm out of the source code and
> then create your own remote signing function with it, but that would require
> knowledge of gpg internals.Yes, there seems to be no way to split the two steps of
creating-digest and encrypting-digest-with-secret-key (these two
together make up the signing action).  It's very unfortunate that the
design of signing in pacman is designed to use gpg directly on the
package.> One solution might be to build the packages on kiwilight, then mount the
> directory of built packages with sshfs.  You could then run the signing script
> locally. I don't know much bandwith that will use, but I think it's worth
> trying. In the worst case scenario, it will be equivalent to downloading the
> packages. Whether or not that's a problem depends on your connection.I don't see how that could be anything else but the worst case, GnuPG
on the machine needs to consume the entire package in order to create
the digest, hence all built packages must be downloaded :-(> If I understand the problem correctly, you do not generate the database
> yourself. That should not be a problem for package signatures, as repo-add will
> include them in the database as long as the signature files are present when it
> adds the packages. If you can't remote-mount xsounds with sshfs and sign the
> database there, just download it and sign it locally then upload the database
> signature file.I am generating the DB myself, I just do it on another machine than
where I perform the builds.> If that is not possible for whatever reason, just having package signatures is
> better than nothing. However, given what you've said about not being the only
> one to have access to these repos, I think package signing in this case is very
> important.I agree, but it's important that the signing actually adds security,
which is why I'd like to keep the secret key on my own machine.
Performing the signing (and keeping the secret key) on either
kiwilight or xsounds might only create a false sense of security,
which arguably is worse than *no* sense of security ;-)> I would also like to know who does have access to these files. On kiwilight
> I believe that it is only Kaiting, who is a TU. Who has access on xsounds?Not sure.> Could you simply make kiwilight the main host and have xsounds mirror it? The
> process would then be the following:
> 1) ssh into kiwilight, build, and move to haskell/$arch if necessary
> 2) mount haskell/$arch via ssh and run the signing script locally
>
> You would then have a fully signed repo in haskell/$arch that can be mirrored
> by xsounds.Of course I could.  It could be argued that since the building happens
on kiwilight we all are forced to trust all (root) users of that
system anyway.  Adding a signature (which is created on kiwilight)
won't decrease the trustworthiness of the packages, but a signature
would mean that the trustworthiness is kept as the packages are copied
over to xsounds.

I find security to be very tricky, so any comments and corrections to
my thinking is more than welcome.

/M-- 
Magnus Therning                      OpenPGP: 0xAB4DFBA4
email:    jabber: 
twitter: magthe               http://therning.org/magnus

>> One solution might be to build the packages on kiwilight, then mount the
>> directory of built packages with sshfs.  You could then run the signing script
>> locally. I don't know much bandwith that will use, but I think it's worth
>> trying. In the worst case scenario, it will be equivalent to downloading the
>> packages. Whether or not that's a problem depends on your connection.
>
>I don't see how that could be anything else but the worst case, GnuPG
>on the machine needs to consume the entire package in order to create
>the digest, hence all built packages must be downloaded :-(I understand that such a solution is not ideal. but is it not possible in the
absence of other solutions? Aside from ghc itself, are there any really big
packages? For ghc, if you are still using the one from community then it is
already signed.>> Could you simply make kiwilight the main host and have xsounds mirror it? The
>> process would then be the following:
>> 1) ssh into kiwilight, build, and move to haskell/$arch if necessary
>> 2) mount haskell/$arch via ssh and run the signing script locally
>>
>> You would then have a fully signed repo in haskell/$arch that can be mirrored
>> by xsounds.
>
>Of course I could.  It could be argued that since the building happens
>on kiwilight we all are forced to trust all (root) users of that
>system anyway.  Adding a signature (which is created on kiwilight)
>won't decrease the trustworthiness of the packages, but a signature
>would mean that the trustworthiness is kept as the packages are copied
>over to xsounds.
>
>I find security to be very tricky, so any comments and corrections to
>my thinking is more than welcome.This may well be the best solution. Kiwilight is already run by a trusted user,
so I think it can be trusted as much as [community] can, provided that no one
else has root access. You should ask Kaiting about that.

I think this is a scenario for using subkeys. As I understand it, you should
generate a new local master key for package signing. From that key you can then
generate a signing subkey that you can upload to sign packages on kiwilight.

I have never used subkeys myself, so I have no practical advice to give. This
page from the Debian Wiki may be a good starting point:
http://wiki.debian.org/subkeys

By keeping the master key yourself, you can always revoke the signing subkey,
regardless of what happens on the server.

I would use a relatively strong password on the uploaded key for added
security.


/X