2018-01-11, Warning from letsencrypt about outdated SSL certificate#
Summary#
A team member received a warning from letsencrypt saying that the SSL certificate
for beta.mybinder.org was going to expire soon. This was unexpected because
we use kube-lego to automatically register new SSL certificates for various
sub-domains of mybinder.org. After a few days, we re-checked the SSL
certificate on beta.mybinder.org and it seemed to have renewed properly,
so this was a noop from our perspective. However it revealed a few things we
should do differently to make sure this doesn’t happen again.
Timeline#
All times in PST
2018-01-11#
A team member received an email that our SSL for beta.mybinder.org was going to expire.
He opened https://github.com/jupyterhub/mybinder.org-deploy/issues/283.
2018-01-18#
Another team member used the SSL certificate checking website below:
https://www.ssllabs.com/ssltest/analyze.html?d=beta.mybinder.org&latest
to inspect the current SSL certificate of beta.mybinder.org. This seemed
to be correctly renewed, and the immediate problem considered resolved.
2018-01-19 11:00#
We received another email saying that both beta and docs were out
of date.
Upon looking at the Google Analytics history, we realized that the date
these certificates were scheduled to run out was exactly 3 months from the
day we switched mybinder.org to point to the beta deployment.
The letsencrypt expiration emails doc
says that if the name / details of the certificate you request change at all,
you may receive these emails even though you’ve successfully renewed your certificate.
We double checked that the cert for beta, docs, and * look correct, which they did.
So, we concluded that we’re getting these notices because the SSL details changed and letsencrypt has (expectedly) failed to link the two.
2018-01-19 11:00#
A team member noticed that this is because our kubernetes deployment has an
account that’s unique to the domain we were using. So when we changed domains
(from beta to *), we also switched accounts on letsencrypt. Our old account
is what is triggering the emails, but our new account is working fine.
Action items#
Process#
Do not use a single team member’s email address for letsencrypt
Instead, use a shared google groups email account so we all get pinged
This has been done: binder-team@googlegroups.com
keep an eye on the SSL once the first expiration date comes around and make sure this is a correct assumption.