Using LetsEncrypt with a webserver in a docker

I have a docker which contains a webserver (caddy) that runs on ports 8080 and 8443 (instead of 80 and 443).

Code:CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
820e77fbb3b9 api-platform_caddy "caddy run --config ..." 29 minutes ago Up 24 minutes 2019/tcp, 0.0.0.0:8080->80/tcp, :::8080->80/tcp, 0.0.0.0:8443->443/tcp, 0.0.0.0:8443->443/udp, :::8443->443/tcp, :::8443->443/udp api-platform_caddy_1
067da3c186d5 api-platform_php "docker-entrypoint p..." 30 minutes ago Up 24 minutes (healthy) 9000/tcp api-platform_php_1
a8522cb58871 api-platform_pwa "docker-entrypoint.s..." 30 minutes ago Up 24 minutes 3000/tcp api-platform_pwa_1
0e34c1583cfe postgres:13-alpine "docker-entrypoint.s..." 30 minutes ago Up 24 minutes 0.0.0.0:5433->5432/tcp, :::5433->5432/tcp api-platform_database_1
I then have an Apache virtual host which acts as a proxy between 80 to 8080 and 443 to 8443 (80 just redirects to 443 and is not shown), and use letsencrypt for certificates.

Code:<IfModule mod_ssl.c>
<VirtualHost *:443>
ServerName my-site.net
ProxyPreserveHost On
ProxyRequests Off
# <Proxy *>
# Order deny,allow
# Allow from all
# </Proxy>
SSLProxyEngine On
SSLProxyVerify none
SSLProxyCheckPeerCN Off
SSLProxyCheckPeerExpire Off
SSLProxyCheckPeerName Off
#SSLEngine On #Not sure what this is
ProxyPass / https://127.0.0.1:8443/
ProxyPassReverse / https://127.0.0.1:8443/

Include /etc/letsencrypt/options-ssl-apache.conf
SSLCertificateFile /etc/letsencrypt/live/my-site.net/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/my-site.net/privkey.pem
</VirtualHost>
</IfModule>I've gotten it working in the past but then I start over and I get TLS errors. I thought I did the exact steps but can't get things working. I am not sure whether it has something to do with executing "sudo certbot --apache" when the docker is up or not or whether I need to first make a traditional virtual host pointing to some directory and then execute cerbot. I also thought that maybe it was selinux related and have it disabled. Tried uninstalling and reinstalling cerbot and not sure whether doing so would have any effect.

I am going round and round and hoping for some advise how to troubleshoot and fix this. Thanks!!!

Code:[michael@my-site api-platform]$ docker-compose up
Creating network "api-platform_default" with the default driver
Creating api-platform_database_1 ... done
Creating api-platform_pwa_1 ... done
Creating api-platform_php_1 ... done
Creating api-platform_caddy_1 ... done
Attaching to api-platform_database_1, api-platform_php_1, api-platform_pwa_1, api-platform_caddy_1
database_1 |
database_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
database_1 |
php_1 | Installing dependencies from lock file (including require-dev)
database_1 | 2021-09-25 14:22:25.743 UTC [1] LOG: starting PostgreSQL 13.4 on x86_64-pc-linux-musl, compiled by gcc (Alpine 10.3.1_git20210424) 10.3.1 20210424, 64-bit
database_1 | 2021-09-25 14:22:25.743 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
database_1 | 2021-09-25 14:22:25.743 UTC [1] LOG: listening on IPv6 address "::", port 5432
php_1 | Verifying lock file contents can be installed on current platform.
pwa_1 | yarn run v1.22.5
php_1 | Nothing to install, update or remove
pwa_1 | $ next dev
pwa_1 | ready - started server on 0.0.0.0:3000, url: http://localhost:3000
database_1 | 2021-09-25 14:22:25.754 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
php_1 | Generating optimized autoload files
php_1 | composer/package-versions-deprecated: Generating version class...
caddy_1 | {"level":"info","ts":1632579750.9612656,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
php_1 | composer/package-versions-deprecated: ...done generating version class
php_1 | 86 packages you are using are looking for funding.
php_1 | Use the `composer fund` command to find out more!
database_1 | 2021-09-25 14:22:25.771 UTC [22] LOG: database system was shut down at 2021-09-25 14:22:16 UTC
php_1 |
php_1 | Run composer recipes at any time to see the status of your Symfony recipes.
php_1 |
database_1 | 2021-09-25 14:22:25.776 UTC [1] LOG: database system is ready to accept connections
php_1 | Executing script cache:clear [OK]
caddy_1 | {"level":"warn","ts":1632579750.966388,"msg":"input is not formatted with 'caddy fmt'","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
caddy_1 | {"level":"info","ts":1632579750.967962,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["[::1]:2019","127.0.0.1:2019","localhost:2019"]}
caddy_1 | {"level":"info","ts":1632579750.9681666,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
caddy_1 | {"level":"info","ts":1632579750.9681878,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
caddy_1 | {"level":"info","ts":1632579750.9682405,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc00023c620"}
caddy_1 | {"level":"info","ts":1632579750.9960306,"logger":"http","msg":"enabling experimental HTTP/3 listener","addr":":443"}
caddy_1 | {"level":"info","ts":1632579750.9960551,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/data/caddy"}
caddy_1 | {"level":"info","ts":1632579750.9961016,"logger":"tls","msg":"finished cleaning storage units"}
caddy_1 | {"level":"info","ts":1632579750.9961822,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["my-site.net"]}
caddy_1 | 2021/09/25 14:22:30 failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size for details.
caddy_1 | {"level":"info","ts":1632579750.9966419,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
caddy_1 | {"level":"info","ts":1632579750.996675,"msg":"serving initial configuration"}
caddy_1 | {"level":"info","ts":1632579750.996811,"logger":"tls.obtain","msg":"acquiring lock","identifier":"my-site.net"}
caddy_1 | {"level":"info","ts":1632579751.0015178,"logger":"tls.obtain","msg":"lock acquired","identifier":"my-site.net"}
caddy_1 | {"level":"info","ts":1632579751.0027425,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["my-site.net"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":""}
caddy_1 | {"level":"info","ts":1632579751.0027647,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["my-site.net"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":""}
pwa_1 | info - Using webpack 5. Reason: no custom webpack configuration in next.config.js https://nextjs.org/docs/messages/webpack5
php_1 | Executing script assets:install public [OK]
php_1 |
php_1 | Waiting for database to be ready...
php_1 | The database is now ready and reachable
caddy_1 | {"level":"info","ts":1632579751.5622032,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"my-site.net","challenge_type":"tls-alpn-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
php_1 |
php_1 | [OK] Already at the latest version ("DoctrineMigrations\Version20200828181711")
php_1 |
php_1 | [25-Sep-2021 14:22:31] NOTICE: fpm is running, pid 1
php_1 | [25-Sep-2021 14:22:31] NOTICE: ready to handle connections
pwa_1 | event - compiled successfully
caddy_1 | {"level":"error","ts":1632579753.0027852,"logger":"tls.issuance.acme.acme_client","msg":"challenge failed","identifier":"my-site.net","challenge_type":"tls-alpn-01","status_code":400,"problem_type":"urn:ietf:params:acme:error:connection","error":"Error getting validation data"}
caddy_1 | {"level":"error","ts":1632579753.0028174,"logger":"tls.issuance.acme.acme_client","msg":"validating authorization","identifier":"my-site.net","error":"authorization failed: HTTP 400 urn:ietf:params:acme:error:connection - Error getting validation data","order":"https://acme-v02.api.letsencrypt.org/acme/order/214512190/27121325760","attempt":1,"max_attempts":3}
caddy_1 | {"level":"info","ts":1632579754.2444031,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"my-site.net","challenge_type":"http-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
caddy_1 | {"level":"error","ts":1632579754.9741173,"logger":"tls.issuance.acme.acme_client","msg":"challenge failed","identifier":"my-site.net","challenge_type":"http-01","status_code":403,"problem_type":"urn:ietf:params:acme:error:unauthorized","error":"Invalid response from https://my-site.net/.well-known/acme-challenge/Bu2hYZHx9JZjzEVOW8F447Hcc_wwySx2Q0g1WOoE4g8 [12.34.56.78]: \"<!DOCTYPE HTML PUBLIC \\\"-//IETF//DTD HTML 2.0//EN\\\">\\n<html><head>\\n<title>502 Proxy Error</title>\\n</head><body>\\n<h1>Proxy Error</h1\""}
caddy_1 | {"level":"error","ts":1632579754.974146,"logger":"tls.issuance.acme.acme_client","msg":"validating authorization","identifier":"my-site.net","error":"authorization failed: HTTP 403 urn:ietf:params:acme:error:unauthorized - Invalid response from https://my-site.net/.well-known/acme-challenge/Bu2hYZHx9JZjzEVOW8F447Hcc_wwySx2Q0g1WOoE4g8 [12.34.56.78]: \"<!DOCTYPE HTML PUBLIC \\\"-//IETF//DTD HTML 2.0//EN\\\">\\n<html><head>\\n<title>502 Proxy Error</title>\\n</head><body>\\n<h1>Proxy Error</h1\"","order":"https://acme-v02.api.letsencrypt.org/acme/order/214512190/27121341370","attempt":2,"max_attempts":3}
caddy_1 | {"level":"error","ts":1632579756.206941,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"my-site.net","issuer":"acme-v02.api.letsencrypt.org-directory","error":"[my-site.net] solving challenges: my-site.net: no solvers available for remaining challenges (configured=[http-01 tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[dns-01]) (order=https://acme-v02.api.letsencrypt.org/acme/order/214512190/27121347190) (ca=https://acme-v02.api.letsencrypt.org/directory)"}
caddy_1 | {"level":"warn","ts":1632579756.207511,"logger":"tls.issuance.zerossl","msg":"missing email address for ZeroSSL; it is strongly recommended to set one for next time"}
php_1 | - - 25/Sep/2021:14:22:36 +0000 "GET /ping" 200
caddy_1 | {"level":"info","ts":1632579757.8707886,"logger":"tls.issuance.zerossl","msg":"generated EAB credentials","key_id":"GvpqzyR0WQ1UQkPFUyguLQ"}
caddy_1 | {"level":"info","ts":1632579760.632394,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["my-site.net"],"ca":"https://acme.zerossl.com/v2/DV90","account":""}
caddy_1 | {"level":"info","ts":1632579760.6324198,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["my-site.net"],"ca":"https://acme.zerossl.com/v2/DV90","account":""}
caddy_1 | {"level":"info","ts":1632579762.0576348,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"my-site.net","challenge_type":"http-01","ca":"https://acme.zerossl.com/v2/DV90"}
php_1 | - - 25/Sep/2021:14:22:46 +0000 "GET /ping" 200
php_1 | - - 25/Sep/2021:14:22:57 +0000 "GET /ping" 200
^CGracefully stopping... (press Ctrl+C again to force)
Stopping api-platform_caddy_1 ... done
Stopping api-platform_php_1 ... done
Stopping api-platform_pwa_1 ... done
Stopping api-platform_database_1 ... done
[michael@my-site api-platform]$