Comments (11)
jessuppi
commented on May 27, 2024
Example error message on new installs:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Account registered.
Requesting a certificate for example.com and www.example.com
Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
Domain: example.com
Type: unauthorized
Detail: 2606:4700:3034::6815:238a: Invalid response from http://example.com/.well-known/acme-challenge/5vnlI6sdSN5ixd0467ij9wZgoaWr2NiS3dsmdmj54k4: 404
Domain: www.example.com
Type: unauthorized
Detail: 2606:4700:3034::6815:238a: Invalid response from http://example.com/.well-known/acme-challenge/1SgtlSd0B60jZWGy2LEUlHZ4jgBIhjouVeqH65OS44Q: 404
Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
This is strange, because unless the Certbot team has carelessly written their error messages (unlikely) then it means the verification tests for both example.com and www.example.com are trying to load from http://example.com which doesn't make very much sense to me... I would expect separate domain verifications.
And because SlickStack runs over HTTPS by default and has HSTS enabled by default, the HTTP verification is going to fail which means we need a way to tell Certbot to run the tests over https://... instead I think.
from slickstack.
jessuppi
commented on May 27, 2024
Also see:
from slickstack.
jessuppi
commented on May 27, 2024
Also see:
As per this discussion, the Certbot Let's Encrypt gurus think that something is not redirecting to HTTPS version properly on the first attempt at generating certs... however the Certbot errors are HTTP, which makes even less sense. How could SlickStack be properly redirecting www to non-www scheme, but failing to redirect http to https?
Our production server block force redirects all requests to HTTPS and HSTS is also enabled too...
from slickstack.
rom0x
commented on May 27, 2024
Hey. I can confirm the issue still exists (even after running the installation multiple times + a reboot). Just got a brand new server instance and tried installing SlickStack (Now using KVM at Hetzner.de). Kind regards.
Maybe it would be possible to provide SlickStack with "Full SSL Strict" enabled, using the SSL certs you can create directly under CloudFlare instead of having self-signed certificates through letsencrypt?
from slickstack.
jessuppi
commented on May 27, 2024
https://slickstack.io/forum/topic/why-does-slickstack-fail-to-activate-lets-encrypt-on-first-attempt
from slickstack.
jessuppi
commented on May 27, 2024
So part of my confusion was discussing this issue with the Lets Encrypt community, who maybe were unaware of some Certbot-specific issues, specifically that port 80 is still required. I also failed to properly specify how SlickStack was integrating Certbot with our Nginx configuration and server blocks, i.e. only port 443 for canonical.
But after confirming HSTS was not an issue, and thoroughly discussing this and reviewing dozens of forum threads, Stack Exchange threads, blog posts, and beyond... I think this is the cause:
Ref: https://letsencrypt.org/docs/allow-port-80/
We occasionally get reports from people who have trouble using the HTTP-01 challenge type because they’ve firewalled off port 80 to their web server. Our recommendation is that all servers meant for general web use should offer both HTTP on port 80 and HTTPS on port 443. They should also send redirects for all port 80 requests, and possibly an HSTS header (on port 443 requests).
Allowing port 80 doesn’t introduce a larger attack surface on your server, because requests on port 80 are generally served by the same software that runs on port 443.
Since SlickStack only allows port 80 on the catch-all Nginx server block (which is not domain-matched), then Certbot is very likely "hanging" because SlickStack doesn't 301 redirect those requests to HTTPS via port 80. In other words, simply redirecting HTTP to HTTPS via Cloudflare or Nginx is not enough...
If we don't want to change our Nginx configuration, we would have to use TLS-ALPN-01 and switch to a different ACME client instead of Certbot.
Ref: https://community.letsencrypt.org/t/which-client-support-tls-alpn-challenge/75859
Ref: https://community.letsencrypt.org/t/confused-about-tls-alpn-01-authorization-type-for-certbot/170881
And for the record, this is not impossible, but it's more involved than I would hope for really:
from slickstack.
jessuppi
commented on May 27, 2024
TLDR I'm not entirely opposed to ditching Certbot for another ACME client, however, because Certbot is sponsored by EFF among other established organizations, there's perhaps stronger trust and longevity with their project... and Certbot is already supported in the Ubuntu packages and such.
What if Certbot or Nginx begin supporting TLS-ALPN-01 in a few years, are we going to switch back again? Although strong security is a top priority for SlickStack, and I hate having to support port 80, I think the logical solution here is the one with the least dependencies and complications... adding a port 80 specific server block in Nginx.
from slickstack.
jessuppi
commented on May 27, 2024
Update: adding the below snippet to production, staging, and development server blocks seems to have improved things, and the "404 Not Found" error is no longer returned by Certbot on brand new servers:
#### for Certbot only ##
server {
listen 80;
listen [::]:80;
server_name @SITE_DOMAIN_INCLUDING_WWW @SITE_DOMAIN_EXCLUDING_WWW;
return 301 https://@SITE_DOMAIN$request_uri;
}
However, there was still a 52x error returned by Certbot on my last attempt on a virgin SlickStack server, this might be related to the Linux kernel issue we've been discussing separately.
from slickstack.
jessuppi
commented on May 27, 2024
I checked the Nginx access log, there was no attempt by Certbot shown until the 2nd install, which was:
2400:cb00:397:1024::ac46:7fa0 - - [12/Feb/2023:15:43:25 +0000] "GET /.well-known/acme-challenge/73sWayf0TQJd_m8v956DrOM7cKZsGJnSJQjcrYQOmwY HTTP/1.1" 301 162 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
2400:cb00:98:1024::ac44:2312 - - [12/Feb/2023:15:43:25 +0000] "GET /.well-known/acme-challenge/73sWayf0TQJd_m8v956DrOM7cKZsGJnSJQjcrYQOmwY HTTP/1.1" 301 162 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
172.71.147.30 - - [12/Feb/2023:15:43:25 +0000] "GET /.well-known/acme-challenge/q873S9W4yfmxiwLWs4Pu0ldxxSHK1m1L7zL5jkr33wM HTTP/1.1" 301 162 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
2400:cb00:28:1024::6ca2:f504 - - [12/Feb/2023:15:43:25 +0000] "GET /.well-known/acme-challenge/73sWayf0TQJd_m8v956DrOM7cKZsGJnSJQjcrYQOmwY HTTP/1.1" 301 162 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
172.68.34.69 - - [12/Feb/2023:15:43:25 +0000] "GET /.well-known/acme-challenge/q873S9W4yfmxiwLWs4Pu0ldxxSHK1m1L7zL5jkr33wM HTTP/1.1" 301 162 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
172.70.126.238 - - [12/Feb/2023:15:43:25 +0000] "GET /.well-known/acme-challenge/q873S9W4yfmxiwLWs4Pu0ldxxSHK1m1L7zL5jkr33wM HTTP/1.1" 301 162 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
2400:cb00:612:1024::ac47:fe30 - - [12/Feb/2023:15:43:25 +0000] "GET /.well-known/acme-challenge/73sWayf0TQJd_m8v956DrOM7cKZsGJnSJQjcrYQOmwY HTTP/2.0" 200 87 "http://example.com/.well-known/acme-challenge/73sWayf0TQJd_m8v956DrOM7cKZsGJnSJQjcrYQOmwY" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
2400:cb00:98:1024::ac44:2311 - - [12/Feb/2023:15:43:25 +0000] "GET /.well-known/acme-challenge/q873S9W4yfmxiwLWs4Pu0ldxxSHK1m1L7zL5jkr33wM HTTP/2.0" 200 87 "http://www.example.com/.well-known/acme-challenge/q873S9W4yfmxiwLWs4Pu0ldxxSHK1m1L7zL5jkr33wM" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
2400:cb00:98:1024::ac44:22f4 - - [12/Feb/2023:15:43:25 +0000] "GET /.well-known/acme-challenge/73sWayf0TQJd_m8v956DrOM7cKZsGJnSJQjcrYQOmwY HTTP/2.0" 200 87 "http://example.com/.well-known/acme-challenge/73sWayf0TQJd_m8v956DrOM7cKZsGJnSJQjcrYQOmwY" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
2400:cb00:398:1024::ac46:8285 - - [12/Feb/2023:15:43:25 +0000] "GET /.well-known/acme-challenge/q873S9W4yfmxiwLWs4Pu0ldxxSHK1m1L7zL5jkr33wM HTTP/2.0" 200 87 "http://www.example.com/.well-known/acme-challenge/q873S9W4yfmxiwLWs4Pu0ldxxSHK1m1L7zL5jkr33wM" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
2400:cb00:28:1024::6ca2:f56c - - [12/Feb/2023:15:43:25 +0000] "GET /.well-known/acme-challenge/q873S9W4yfmxiwLWs4Pu0ldxxSHK1m1L7zL5jkr33wM HTTP/2.0" 200 87 "http://www.example.com/.well-known/acme-challenge/q873S9W4yfmxiwLWs4Pu0ldxxSHK1m1L7zL5jkr33wM" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
2400:cb00:543:1024::ac47:9670 - - [12/Feb/2023:15:43:25 +0000] "GET /.well-known/acme-challenge/73sWayf0TQJd_m8v956DrOM7cKZsGJnSJQjcrYQOmwY HTTP/2.0" 200 87 "http://example.com/.well-known/acme-challenge/73sWayf0TQJd_m8v956DrOM7cKZsGJnSJQjcrYQOmwY" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)"
So there's no longer 404 errors, but timeout errors like 522 instead.
I assume this means Nginx is not even properly active at the time of the first attempts...
from slickstack.
jessuppi
commented on May 27, 2024
Related: https://community.letsencrypt.org/t/522-error-trying-to-use-certbot-for-a-nextcloud-server/188534
from slickstack.
jessuppi
commented on May 27, 2024
Changed server blocks a bit:
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name @SITE_DOMAIN_INCLUDING_WWW @SITE_DOMAIN_EXCLUDING_WWW;
location /.well-known/acme-challenge/ {
allow all;
auth_basic off;
default_type "text/plain";
try_files $uri =404;
root /var/www/html;
}
.....
Now getting "Timeout during connect (likely firewall problem)" but port 80 scan shows fine. Not sure if IPv6 related, however in the Certbot logs it shows the connection attempt was to the IPv4 address so should be unrelated...
Really blows my mind how finicky Certbot is, not sure we should keep using it.
from slickstack.
Related Issues (20)
- SS_ADMINER_PUBLIC="false" does not work as intended. HOT 9
- Allow tuning of PHP8 JIT settings (opcache.jit options in php.ini) HOT 4
- Cloudflare real visitor IP support in Nginx config HOT 18
- Option to allow only Cloudflare IPs to connect to origin server HOT 2
- OpenVZ PHP-FPM "Unable to set priority for the master process: Permission denied" HOT 8
- Support for custom Permissions Policy HTTP header in Nginx HOT 6
- Improve WP-Cron robustness for Multisite environments HOT 3
- Scanning WordPress core files to check if intact or broken HOT 4
- Redirect .php extension to WordPress if not exists HOT 4
- Why deny /wp-admin/load-styles.php and /wp-admin/load-scripts.php? HOT 3
- Nginx access log enabled by default but nginx.conf says not HOT 5
- Spam Content HOT 1
- Expose additional options for ss remote backup HOT 2
- support for local development ? HOT 1
- 403 Error on Static Assets with Query Strings on SlickStack.io HOT 1
- Implement Server-Wide Connection Limit to Mitigate Request Overload HOT 1
- ss-config had a build version update, but ss-update-config didn't HOT 4
- Can't get it to work out of the box? HOT 1
- 403 google bot error
- Consider optional cache warming script that can fire on cron schedules
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from slickstack.