entered username and password is correct but 'user authentication failed' error is received about ikev2-setup HOT 6 CLOSED

I'm not sure why you're having these problems. Have you tried cat /etc/ipsec.secrets on the server to ensure your username and password are correct?

Also on the server, what do you see in sudo tail -f /var/log/syslog as you try to connect?

I think the NAT message is just a result of the use of 'forceencaps' in the server-side strongSwan config (see https://wiki.strongswan.org/projects/strongswan/wiki/connsection).

from ikev2-setup.

Hi, Thanks for your help.
Im sure that the entered password is correct and I double checked it with cat /etc/ipsec.secrets
Here is the syslog

Sep 7 05:25:59 srv194863 charon: 15[NET] received packet: from 2.190.173.254[500] to 185.110.188.169[500] (376 bytes)
Sep 7 05:25:59 srv194863 charon: 15[ENC] parsed IKE_SA_INIT request 0 [ SA KE No N(FRAG_SUP) N(NATD_S_IP) N(NATD_D_IP) V V V V ]
Sep 7 05:25:59 srv194863 charon: 15[IKE] received MS NT5 ISAKMPOAKLEY v9 vendor ID
Sep 7 05:25:59 srv194863 charon: 15[IKE] received MS-Negotiation Discovery Capable vendor ID
Sep 7 05:25:59 srv194863 charon: 15[IKE] received Vid-Initial-Contact vendor ID
Sep 7 05:25:59 srv194863 charon: 15[ENC] received unknown vendor ID: 01:52:8b:bb:c0:06:96:12:18:49:ab:9a:1c:5b:2a:51:00:00:00:02
Sep 7 05:25:59 srv194863 charon: 15[IKE] 2.190.173.254 is initiating an IKE_SA
Sep 7 05:25:59 srv194863 charon: 15[CFG] selected proposal: IKE:AES_GCM_16_256/PRF_HMAC_SHA2_384/ECP_384
Sep 7 05:25:59 srv194863 charon: 15[IKE] remote host is behind NAT
Sep 7 05:25:59 srv194863 charon: 15[ENC] generating IKE_SA_INIT response 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(CHDLESS_SUP) N(MULT_AUTH) ]
Sep 7 05:25:59 srv194863 charon: 15[NET] sending packet: from 185.110.188.169[500] to 2.190.173.254[500] (288 bytes)
Sep 7 05:25:59 srv194863 charon: 16[NET] received packet: from 2.190.173.254[4500] to 185.110.188.169[4500] (572 bytes)
Sep 7 05:25:59 srv194863 charon: 16[ENC] parsed IKE_AUTH request 1 [ EF(1/3) ]
Sep 7 05:25:59 srv194863 charon: 16[ENC] received fragment #1 of 3, waiting for complete IKE message
Sep 7 05:25:59 srv194863 charon: 06[NET] received packet: from 2.190.173.254[4500] to 185.110.188.169[4500] (572 bytes)
Sep 7 05:25:59 srv194863 charon: 06[ENC] parsed IKE_AUTH request 1 [ EF(2/3) ]
Sep 7 05:25:59 srv194863 charon: 06[ENC] received fragment #2 of 3, waiting for complete IKE message
Sep 7 05:25:59 srv194863 charon: 05[NET] received packet: from 2.190.173.254[4500] to 185.110.188.169[4500] (556 bytes)
Sep 7 05:25:59 srv194863 charon: 05[ENC] parsed IKE_AUTH request 1 [ EF(3/3) ]
Sep 7 05:25:59 srv194863 charon: 05[ENC] received fragment #3 of 3, reassembled fragmented IKE message (1542 bytes)
Sep 7 05:25:59 srv194863 charon: 05[ENC] parsed IKE_AUTH request 1 [ IDi CERTREQ N(MOBIKE_SUP) CPRQ(ADDR DNS NBNS SRV ADDR6 DNS6 SRV6) SA TSi TSr ]
Sep 7 05:25:59 srv194863 charon: 05[IKE] received 63 cert requests for an unknown ca
Sep 7 05:25:59 srv194863 charon: 05[CFG] looking for peer configs matching 185.110.188.169[%any]...2.190.173.254[192.168.1.156]
Sep 7 05:25:59 srv194863 charon: 05[CFG] selected peer config 'roadwarrior'
Sep 7 05:25:59 srv194863 charon: 05[IKE] initiating EAP_IDENTITY method (id 0x00)
Sep 7 05:25:59 srv194863 charon: 05[IKE] peer supports MOBIKE
Sep 7 05:25:59 srv194863 charon: 05[IKE] no private key found for 'dev1.noatrader.ir'
Sep 7 05:25:59 srv194863 charon: 05[ENC] generating IKE_AUTH response 1 [ N(AUTH_FAILED) ]
Sep 7 05:25:59 srv194863 charon: 05[NET] sending packet: from 185.110.188.169[4500] to 2.190.173.254[4500] (65 bytes)

from this:

Sep 7 05:25:59 srv194863 charon: 05[IKE] no private key found for 'dev1.noatrader.ir'

I found this one:
https://lists.strongswan.org/pipermail/users/2010-June/000378.html
and the output for my ipsec listcerts is this:

List of X.509 End Entity Certificates

subject: "CN=dev1.noatrader.ir"
issuer: "C=US, O=Let's Encrypt, CN=R3"
validity: not before Sep 06 07:26:39 2022, ok
not after Dec 05 06:26:38 2022, ok (expires in 89 days)
serial: 04:17:20:e7:bb:7a:e8:82:b6:36:8f:fc:62:a8:46:6c:cf:01
altNames: dev1.noatrader.ir
flags: serverAuth clientAuth
OCSP URIs: http://r3.o.lencr.org
certificatePolicies:
2.23.140.1.2.1
1.3.6.1.4.1.44947.1.1.1
CPS: http://cps.letsencrypt.org
authkeyId: 14:2e:b3:17:b7:58:56:cb:ae:50:09:40:e6:1f:af:9d:8b:14:c2:c6
subjkeyId: e7:55:b5:30:14:fa:3b:76:63:6d:bc:67:97:3c:c4:9d:f0:38:08:da
pubkey: RSA 4096 bits
keyid: f1:2e:25:28:d4:00:eb:a3:e5:e7:66:d4:9b:6e:9a:70:5e:f6:25:0c
subjkey: e7:55:b5:30:14:fa:3b:76:63:6d:bc:67:97:3c:c4:9d:f0:38:08:da

which does not have "has private key" in front of pubkey as the link says...
What should I do?

from ikev2-setup.

Right. When I run ipsec listcerts I get pubkey: RSA 4096 bits, has private key, so this seems to be your problem.

I guess this probably narrows the issue down to certbot and AppArmor.

First, are your certificates there?

certbot certificates
ll /etc/ipsec.d/private
ll /etc/letsencrypt/live/dev1.noatrader.ir/
ll /etc/letsencrypt/archive/dev1.noatrader.ir/

Next, it's probably worth checking whether strongSwan is giving any errors at startup time, when it tries to read your certificates. Run sudo tail -f /var/log/syslog, while issuing a sudo ipsec restart in another session.

from ikev2-setup.

All the certificates seem to exist.
certbot certificates

Saving debug log to /var/log/letsencrypt/letsencrypt.log

---

Found the following certs:
Certificate Name: dev1.noatrader.ir
Domains: dev1.noatrader.ir
Expiry Date: 2022-12-05 06:26:38+00:00 (VALID: 88 days)
Certificate Path: /etc/letsencrypt/live/dev1.noatrader.ir/fullchain.pem
Private Key Path: /etc/letsencrypt/live/dev1.noatrader.ir/privkey.pem

---

ll /etc/ipsec.d/private

lrwxrwxrwx 1 root root 51 Sep 7 05:35 privkey.pem -> /etc/letsencrypt/live/dev1.noatrader.ir/privkey.pem

ll /etc/letsencrypt/live/dev1.noatrader.ir/

lrwxrwxrwx 1 root root 41 Sep 6 08:26 cert.pem -> ../../archive/dev1.noatrader.ir/cert1.pem
lrwxrwxrwx 1 root root 42 Sep 6 08:26 chain.pem -> ../../archive/dev1.noatrader.ir/chain1.pem
lrwxrwxrwx 1 root root 46 Sep 6 08:26 fullchain.pem -> ../../archive/dev1.noatrader.ir/fullchain1.pem
lrwxrwxrwx 1 root root 44 Sep 6 08:26 privkey.pem -> ../../archive/dev1.noatrader.ir/privkey1.pem
-rw-r--r-- 1 root root 692 Sep 6 08:26 README

ll /etc/letsencrypt/archive/dev1.noatrader.ir/

-rw-r--r-- 1 root root 2195 Sep 6 08:26 cert1.pem
-rw-r--r-- 1 root root 3750 Sep 6 08:26 chain1.pem
-rw-r--r-- 1 root root 5945 Sep 6 08:26 fullchain1.pem
-rw------- 1 root root 3272 Sep 6 08:26 privkey1.pem

and here is the tail -f /var/log/syslog after ipsec restart

Sep 8 06:19:57 dev1 charon: 00[DMN] Starting IKE charon daemon (strongSwan 5.8.2, Linux 5.4.0-125-generic, x86_64)
Sep 8 06:19:57 dev1 charon: 00[CFG] loading ca certificates from '/etc/ipsec.d/cacerts'
Sep 8 06:19:57 dev1 charon: 00[CFG] loaded ca certificate "C=US, O=Let's Encrypt, CN=R3" from '/etc/ipsec.d/cacerts/chain.pem'
Sep 8 06:19:57 dev1 charon: 00[CFG] loading aa certificates from '/etc/ipsec.d/aacerts'
Sep 8 06:19:57 dev1 charon: 00[CFG] loading ocsp signer certificates from '/etc/ipsec.d/ocspcerts'
Sep 8 06:19:57 dev1 charon: 00[CFG] loading attribute certificates from '/etc/ipsec.d/acerts'
Sep 8 06:19:57 dev1 charon: 00[CFG] loading crls from '/etc/ipsec.d/crls'
Sep 8 06:19:57 dev1 charon: 00[CFG] loading secrets from '/etc/ipsec.secrets'
Sep 8 06:19:57 dev1 charon: 00[LIB] opening '/etc/ipsec.d/private/privkey.pem' failed: Permission denied
Sep 8 06:19:57 dev1 charon: 00[LIB] building CRED_PRIVATE_KEY - RSA failed, tried 9 builders
Sep 8 06:19:57 dev1 charon: 00[CFG] loading private key from '/etc/ipsec.d/private/privkey.pem' failed
Sep 8 06:19:57 dev1 charon: 00[CFG] loaded EAP secret for xxxx
Sep 8 06:19:57 dev1 charon: 00[CFG] loaded EAP secret for xxxx
Sep 8 06:19:57 dev1 charon: 00[CFG] loaded 0 RADIUS server configurations
Sep 8 06:19:57 dev1 charon: 00[CFG] HA config misses local/remote address
Sep 8 06:19:57 dev1 kernel: [98392.072945] kauditd_printk_skb: 7 callbacks suppressed
Sep 8 06:19:57 dev1 kernel: [98392.072946] audit: type=1400 audit(1662614397.683:68): apparmor="DENIED" operation="open" profile="/usr/lib/ipsec/charon" name="/etc/letsencrypt/archive/dev1.noatrader.ir/privkey1.pem" pid=8123 comm="charon" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Sep 8 06:19:57 dev1 charon: 00[LIB] loaded plugins: charon aesni aes rc2 sha2 sha1 md5 mgf1 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl fips-prf gmp agent xcbc hmac gcm drbg attr kernel-netlink resolve socket-default connmark farp stroke updown eap-identity eap-aka eap-md5 eap-gtc eap-mschapv2 eap-dynamic eap-radius eap-tls eap-ttls eap-peap eap-tnc xauth-generic xauth-eap xauth-pam tnc-tnccs dhcp lookip error-notify certexpire led addrblock unity counters
Sep 8 06:19:57 dev1 charon: 00[LIB] dropped capabilities, running as uid 0, gid 0
Sep 8 06:19:57 dev1 charon: 00[JOB] spawning 16 worker threads
Sep 8 06:19:57 dev1 charon: 05[CFG] received stroke: add connection 'roadwarrior'
Sep 8 06:19:57 dev1 charon: 05[CFG] adding virtual IP address pool 10.101.0.0/16
Sep 8 06:19:57 dev1 charon: 05[CFG] loaded certificate "CN=dev1.noatrader.ir" from 'cert.pem'
Sep 8 06:19:57 dev1 charon: 05[CFG] added configuration 'roadwarrior'
Sep 8 06:20:20 dev1 python3.8[651]: Stats for 08.09.2022 06:20:20

so the problem is opening '/etc/ipsec.d/private/privkey.pem' failed: Permission denied
I searched a bit and found similar problems and the only way it worked was copying the original file from /etc/letsencrypt/archive/dev1.noatrader.ir/privkey1.pem to the /etc/ipsec.d/private/privkey.pem
Now it Works Fine!
It was a fun problem and Thanks alot for your help.
My only remaining question is will it work after the certificats' 90 days renew ?

from ikev2-setup.

Good that this is fixed for now, but I think you'll find it will go wrong again in around 90 days when Let's Encrypt renews the certificate in /etc/letsencrypt/archive but your copied certificate expires.

Given the slightly later log line, ... apparmor="DENIED" ..., it looks like you have an AppArmor problem. This is meant to be fixed by the following part of the script, but something must be going wrong here.

grep -Fq 'jawj/IKEv2-setup' /etc/apparmor.d/local/usr.lib.ipsec.charon || echo "
# https://github.com/jawj/IKEv2-setup
/etc/letsencrypt/archive/${VPNHOST}/* r,
" >> /etc/apparmor.d/local/usr.lib.ipsec.charon

aa-status --enabled && invoke-rc.d apparmor reload

Can you check what you have in /etc/apparmor.d/local/usr.lib.ipsec.charon?

from ikev2-setup.

@jawj I had the same problem today after re-running the ./setup.sh script with a different domain name. After doing this, the file /etc/apparmor.d/local/usr.lib.ipsec.charon still only contained a reference to the old letsencrypt certificate folder:

$ cat /etc/apparmor.d/local/usr.lib.ipsec.charon
# https://github.com/jawj/IKEv2-setup
/etc/letsencrypt/archive/OLD.DOMAIN.NAME/* r,

Solution: I appended this file with a path to the new letsencrypt certificate folder and the problem was fixed.

from ikev2-setup.