Posted by anarcat on July 13, 2009 at 11:15pm
There has been some talks in the issue tracker recently about implementing SSL support in Aegir. It's an exciting feature I've been thinking about for a while, and I'm at the point of trying to figure out how it would actually be implemented. I wrote up a first spec draft of what it would look like and I would like to seek wider discussion on the subject. How do you think this should be handled? How does your provider deal with SSL right now? Any specific issues we need to think about?
Flames, simple comments and trolls here, patches in the issue queue. :)
Comments
@theanarcat
My idea is simple: don't touch anything Apache related in Aegir for SSL and create simple module for uploading .pem file and modify Pound instance on the front-end for SSL. You need just simple Pound config file (one file for all SSL enabled vhosts) parsing like this is for Apache and only 2 lines in Pound to change per vhost: IP and path to one-in-all .pem file (can be uploaded and not stored in a database).
ListenHTTPSAddress 127.0.0.1
Port 443
Cert "/path/to/domain.pem"
As a bonus - if you really can't live without Apache, you can keep the Apache smaller, since you don't need to load SSL module in this setup.
OK, I know we need something for Apache based SSL in Aegir, but think for a moment you can make your life easier :)
In any case: enabling SSL can't be fully automated - you have to create private key and CSR request and figure out how to bundle it all together depending on SSL cert provider (intermediate certs etc.)
It would be cool to have a new SSL toy in the Aegir, but let's remember about all manual work you can't avoid here.
~Grace
[edit]
To clarify something: when using Pound as a front-end SSL, you don't need to change/add separate IP/port for back-end in Aegir/Apache. It can run without any change. You just need unique IP per SSL enabled domain where Pound can listen on port 443. (DNS change involved - but this is not Aegir job, anyway). In this scenario Pound works as a transparent SSL layer without any change in the backend config.
[/edit]
Wow, I wish my wife could
Wow, I wish my wife could speak like that :D
Irrelevant?
I fail to see how this has anything to do with this (technical) discussion...
So I know about pound and
So I know about pound and SSL load balancers. It doesn't change anything in my proposal: you still need to manage the SSL certificates, per domain (so per site). This needs to be managed in the frontend and the backend. In my spec, the only thing that actually relates to Apache is step 3: configuring vhosts, which will also need to be done for pound (unless you use wildcards).
About manual intervention: my spec is designed so there is no required manual intervention (assuming password-less certificates) from the administrator. There 's also no intervention required from the user if we can implement a communication layer with an existing SSL provider API (like gandi, but they don't have it yet), or it will require minimal intervention from the user ("copy-paste the certificate from your provider here") in the worst case.
As for DNS: it will (as of 0.4) be Aegir's job to take care of DNS, so that's something we'll have to take care of if we're going to play with load balancers.
In short, I do not want to discard the idea of working with pound, ng, relayd or any load balancing solution for SSL support: I'm just saying it should not be the first objective and we have some work to do regardless of what will use the actual SSL cert.
understand that, but..
...you invited "trolls here" so... :-)
OK, I agree, in any case we need something to manage SSL setup, no matter what kind of server will be used. My true intention here is making Aegir less "Apache-only" so it can work with popular new web servers, proxies/caches etc. without hacking the code manually.
~Grace
Hehe... i'm the troll. :)
Hehe... i'm the troll. :) I'm all for being implementation-agnostic, but as long as we're pre-1.0, I don't think it's realistic to start implementing other backends than typical ones, read: LAMP (linux, apache, mysql, php).
It's totally part of the project objectives to have clear APIs here, however, that API will stabilise as time goes on, and as people write modules for it. :) So if you want to start hacking at a Pound module, be my guest: it will be a fine test of the API. However, be aware that we will severely refactor it for 0.4, see this issue in particular: http://drupal.org/node/344967.
Comments on SSL
I like the functionality that's described in the spec. A few comments:
a) One thing that wasn't explicit in the spec was the grain of the SSL policy. When an Aegir admin chooses a way to provide certificates (e.g. auto-generate a self-signed cert; use an API from an SSL vendor; ask clients to upload a cert), she could make that decision for an individual site, for a platform, or for all sites hosted in Aegir. My impression is that this should be done on a per-site basis.
b) How should SSL interact with site aliases? Is it appropriate to provision certificates for aliases?
c) It sounds like there will be several ways to produce a cert (e.g. uploading a cert to the hosting DB; auto-generating a cert with a CA's API), and there will be several ways to consume a cert (e.g. configure Apache, configure Pound). The spec doesn't touch on how a consumer would get a copy of the certificate from the provider.
d) The spec equivocates a bit on whether to store the key+cert in the hosting DB. I understand and agree with the complaint that it seems less secure. What's the upside of storing it in the hosting DB? ... It's probably necessary for some SSL policies/workflows (e.g. if clients upload their own keys and certs through the web UI), but it isn't necessary with others (e.g. auto-generate self-signed cert; use a shared wildcard cert; use an API from an external CA; auto-generate the CSR but client uploads cert).
An API like the following one would provide loose enough coupling to work with different consumers/providers; it's also loose enough that we can apply least-privilege (i.e. one cert provider might store data in the hosting DB as a user-editable object; another cert provider stores it as a hidden, automagic file on the backend). I apologize in advance because I haven't worked with Aegir 0.2+ yet, so I'm not familiar with the latest code conventions/structure. However, this should show the basic dynamic.
## provision.module (SSL Certificate API) function provision_cert(&$site) { if (isset($site['cert'])) { return $site['cert']; } else { $cert = array( 'enabled' => false, 'key-file' => null, 'cert-file' => null, 'ca-cert-file' => null ); drupal_alter('provision_cert', $cert, $site); $site['cert'] = $cert; return $cert; } } ## provision_apache.module (SSL Certificate Consumer) /** Add SSL directives to the Apache config file */ function provision_apache_create_vhost_config(&$site) { ... $cert = provision_cert($site); if ($cert['enabled']) { $vhost_config .= "SSLEngine on\n"; $vhost_config .= "SSLCertificateFile " . $cert['cert-file']; $vhost_config .= "SSLKeyFile ".$cert['key-file']; ... } ... } ## provision_gandi.module (SSL Certificate Provider) /** Use Gandi API to auto-generate a certificate */ function provision_gandi_provision_cert_alter(&$cert, &$site) { if ($site['cert-provider'] != 'gandi') return; $gkey = ... generate key ... $gcsr = ... generate CSR ... $gcert = ... submit CSR to Gandi for signing... # Put the cert/key files in the site's dir so we can backup/restore/migrate ... mkdir /sites//gandi with mode 700... $cert['key-file'] = $keydir . '/key.pem'; ... write $gkey to $cert['key-file'] with mode 400 ... $cert['cert-file'] = $keydir . '/cert.perm'; ... write $gcert to $cert['cert-file'] with mode 400 ... $cert['enabled'] = true; } ## provision_usercert.module (SSL Certificate Provider) /** Fetch the key and cert from the hosting DB */ function provision_usercert_provision_cert_alter(&$cert, &$site) { if ($site['cert-provider'] != 'usercert') return; db_connect(HOSTING_DB); $key_query = db_query('select node->body from node where ...'); $key_data = db_fetch_array($key_query); $cert['key-file'] = sprintf('/var/aegir/config/ssl.d/%s', $site[url]); file_put_contents($cert['key-file'], $key_data['body']); ... }some comments
a) agreed: certificates are per-site. However, my spec specifies that certs can have a life of their own and can be reused in multiple sites (think wildcards)
b) i'm not sure what to do about aliases. it's the trickiest part: with wildcards, it's likely to work properly if all aliases are under the same domain, otherwise it will just not work. if we can parse the certificate, we can identify which aliaes will have SSL enabled however, so we could enable ssl only for the certs that are available...
c) sure it does: the spec says the certs are stored in the database, regardless of the provider (which would then basically interfaces with node_load/node_save or maybe a higher-level API). the spec also says the certs are written in ~aegir/config/ssl.d to be consumed by whoever needs them.
d) i agree that storing in the database is a problem, but given the distributed nature of Aegir, it's hard to do otherwise: in 0.4, we will support multiple servers, and we can't rely on ~aegir/config/ssl.d being the same everywhere. that's why I chose to put the certs in the central database. besides, if aegir has to manipulate the certificates, it will need to read/write it, which is pretty close to storing them in the database (as long as we got inter-site site security right). but yeah, maybe an alternative here could be to allow a path (including the server name) to the certificate to be put instead of the full certificate... not sure where i stand on this.
I like the API you are proposing, but I'm not clear on what all functions do... what's provision_cert()?
It needs to be modified to take into account multiple servers too.
Finally, it seems to associate the certificate with a site, whereas a certificate could be associated with multiple sites. basically, a cert is associated with N domains (FQDNs, with a flag for wildcards), although we don't have the concept of domain in aegir (yet?).
Ok
a) Well, I think it makes sense that certificates have a lifecycle independent of the site lifecycle. My impulse would be to manage its lifecycle in the backend (much like MySQL credentials are managed in the backend), but I appreciate that -- if the front-end user provides the certificate -- then some of that has to go in the frontend.
d) A few comments in no particular order:
d.1) A CSR and certificate can be safely stored anywhere -- they only contain contact information and a public key. The problem is the RSA private key.
d.2) The RSA private key only needs to pass through the hosting database if the end-user uploads the key. This use-case seems a bit smelly to me -- e.g. if the user uploads the key, then the key sits on his personal computer, so the key can be compromised by run-of-the-mill malware.
d.3) I can't really speak to adding multi-server support because I don't know the basics of the 0.4 design. (e.g. For a given site, how many servers will simultaneously run the site? What's the mechanism for server-to-server signalling? How many agents will be watching Aegir's task queue?) However, I'm optimistic that a solution would be possible because a multi-server solution will have to solve some related problems -- e.g. Aegir will manage MySQL credentials (which are auto-generated, random, confidential, and currently outside the hosting DB). Aegir will have to migrate or synchronize a site's data files.
d.4) If the database needs to store a reference to a cert/key without storing the cert/key, then you need an ID. The path could work, but it does seem hard to maintain across servers. The public-key would also be a unique, natural ID.
d.5) The drafted provision_cert() produces a reference to a certificate. In an object-oriented system, one might call it a factory method -- it will create a certificate, but the caller doesn't care how it gets done, and there are multiple modules which can handle it. The hook_provision_cert_alter() provides a way to plug in different logic for creating/assigning certificates.
d.6) The suggested API call seems to associate the key+cert with a site, but that's deceptive -- it's actually very hands-off. The SSL cert consumer (e.g. Apache's vhost config) only cares about the certificate of one site, so the API call only provides the certificate for one site. There may be other sites or services that use the same key+cert, but it's irrelevant when you're writing a vhost config file.
Now, the cert <-> site relationship could be 1-to-1 or 1-to-N. The relationship could be tracked explicitly with a SQL table, or it could be matched automatically by the domain name in the cert. The cert could be auto-generated on-demand or uploaded. It could be created by front-end logic or back-end logic. The canonical datastore could be a local directory, network share, or MySQL. The logic goes in hook_provision_cert_alter() so that alternative modules can take responsibility for it.
The two hook implementations provide different trade-offs:
d.6.i.) provision_gandi_provision_cert_alter() creates certificates on-demand without any user interaction. (I'm overselling a little here -- obviously, gandi.net doesn't have an API for this. I used this as the example because that vendor was mentioned before. Realistically, opensrs or geotrust look more promising.) The certificate is stored with the other site data (sites/$url/gandi). Installation can run any server, and sites be migrated among servers, and the private key is well sequestered. Of course, this doesn't do 1-to-N or wildcard certs, but that's OK -- if this module is activated, it's because the admin wants to create a new cert for each site.
d.6.ii) provision_usercert_provision_cert_alter() reads certificate data from a database and writes it to a local file. Multiple sites could be reading from the same DB records, so it works with 1-to-N relationship. Of course, in this case, the backend doesn't have any responsibility for creating the certificate -- that's handled in the front-end, where Aegir can prompt the user to paste the certificate data.
My point here is to keep the contract between the consumers and providers small and simple. That should make it easier to support different consumers (e.g. apache, nginx, pound) even while enhancing / redesigning / replacing the certificate provisioning logic.
Let's code this now ;)
a) I guess we agree. My only concern is to make sure the frontend has enough flexibility to add certificates and manage the backend associations.
d.1 & 2) we also agree here, mostly: the issue is the RSA private key. However, I don't mind having the user be responsible for that part of the security. tools, not policy. :)
d.3) quick q&a on the server front: a) in 0.4, I would expect a given site to be on a single server, but I have given thoughts of multi-server sites support. b) server to server signaling is implemented using SSH commands, with SSH keys as the authentication token, deployed out of aegir by the site administrators. c) only one aegir "agent" (the hosting module, really) should be watching the queue, but the question in itself is interesting, maybe the queue could be designed to support multiple frontends. It's in the mental 0.4 spec for now as far as I know.
now, the sql credentials issue is different, as there is a one to one mapping there, while there isn't such a mapping for SSL certificates: you will want to apply the same certificate to multiple sites, maybe on multiple servers. so just storing them on a random server is inconvenient. maybe it should be stored on the master aegir server, but I'm trying not to rely too much on that premise, and besides, I'm not sure it gives us a lot more security than storing it in the aegir database (although that database is directly accessible by the web server, through the hosting frontend, while the files can be protected from that influence).
i would welcome such a change in direction, but we must acknowledge it will yield more complexity.
d.4) the key hash could be the ID, and if the key is stored in a common location across servers, then you just need to know the hash and the server name, which would uniquely identify a key and its location.
d.5) i understand, thank you for the clarification.
d.6) thank you again for those clarifications. it makes sense, although I have a few objections:
A) do not store the certificate in a path accessible by the webserver (sites/foo/cert pops up like a real bad idea in my head, worse than being in the database. :)
B) I am totally open with starting with 1 to 1 as long as we allow for further extensibility, which your API supports.
C) you should probably write a provision_ssl (or provision_cert?) module that would implement that API (with your above code as a starting point). it will need some frontend glue too...
I would very much like to see a patch land in the issue queue now. ;) You can also open a ssl branch from git if you want...
http://groups.drupal.org/node/24343
Submitted a summary of this
Submitted a summary of this discussion here: http://drupal.org/node/394452#comment-1876056
Help Needed
I just need some clarification with the Pound2.4.
With Regards,
Ahamed Mukthaar
Jr.Design Engineer,
Adeptte Microsystems,
Chennai,
Tamilnadu,India
Mobile:91-9894168796.
http://fearlessdragon.yolasite.com
@awniyya1
When you are using Pound as a front-end proxy, you don't need SSL enabled in your Apache configuration. It's a Pound job to listen on default 443 port for SSL connections and on default 80 port for non-SSL connections.
Apache can listen on 82 port only.
Your configuration for Pound can look like below:
# Global Directives
User "apache"
Group "apache"
Alive 2
LogLevel 0
# Main HTTP
ListenHTTP
Address 192.168.72.101
Port 80
Client 10
MaxRequest 19453000
RewriteLocation 2
Err500 "/etc/pound/errors/500.html"
Err501 "/etc/pound/errors/501.html"
Err503 "/etc/pound/errors/503.html"
Err414 "/etc/pound/errors/414.html"
# Catch-all static/dynamic content server
Service
URL ".*"
BackEnd
Address 192.168.72.100
Port 82
TimeOut 120
End
Session
Type COOKIE
ID "OMEGA"
TTL 3600
End
End
End
# SSL
ListenHTTPS
Address 192.168.72.101
Port 443
Cert "/etc/pound/YourKeyCert.pem"
Client 180
NoHTTPS11 2
MaxRequest 19453000
Ciphers "ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL"
# Catch-all static/dynamic content server
Service
URL ".*"
BackEnd
Address 192.168.72.100
Port 82
TimeOut 120
End
Session
Type COOKIE
ID "OMEGA"
TTL 3600
End
End
End
HTH
~Grace
-- Turnkey Drupal Hosting on Steroids -- http://twitter.com/omega8cc
@omega
Hi,
Thankyou very much for your reply..........
With Regards,
Ahamed Mukthaar
Jr.Design Engineer,
Adeptte Microsystems,
Chennai,
Tamilnadu,India
Mobile:91-9894168796
http://fearlessdragon.yolasite.com
With Regards,
Ahamed Mukthaar
Jr.Design Engineer,
Adeptte Microsystems,
Chennai,
Tamilnadu,India
Mobile:91-9894168796.
http://fearlessdragon.yolasite.com
@ omega
Hi Omega,
I understood your clarification. But wat I want to do is that I need both frontendssl as well as the backend ssl.
So that only am trying tto connect the ssl port of apache. Wen I contacted the pound mailing list group I found their reply to be that the pound does not support the backend ssl.
Do you have any idea regarding how to make the pound to support the backend ssl. If any please let me know...........
Am eagerly waiting for your reply................
With Regards,
Ahamed Mukthaar
Jr.Design Engineer,
Adeptte Microsystems,
Chennai,
Tamilnadu,India
Mobile:91-9894168796.
http://fearlessdragon.yolasite.com
With Regards,
Ahamed Mukthaar
Jr.Design Engineer,
Adeptte Microsystems,
Chennai,
Tamilnadu,India
Mobile:91-9894168796.
http://fearlessdragon.yolasite.com
@awniyya1
As far as I know, Pound doesn't support SSL/encrypted BackEnd. Pound is a simple proxy and not web-server. If you need encrypted connection between Pound and BackEnd (Apache), you will need something like http://openvpn.net/index.php/open-source.html, but this is Off Topic here.
HTH
~Grace
-- Turnkey Drupal Hosting on Steroids -- http://twitter.com/omega8cc
https://letsencrypt.org/ just
https://letsencrypt.org/ just announced free and automated certs. Would be nice if Aegir supports that or at least provides an API so that modules can add certs by using the letsencrypt api.