These steps are intended to help troubleshoot IPSec VPN connectivity issues. They are divided into two parts, one for each Phase of an IPSec VPN.
Phase 1 -
To rule out ISP-related issues, try pinging the peer IP from the PA external interface. Ensurethat pings are enabled on the peer’s external interface.
If pings have been blocked per security requirements, see if the other peer is responding to the main/aggressive mode messages, or the DPDs. Check for the responses of the “Are you there?” messages from the peer in the system logs under the Monitor tab or under ikemgrlogs.
Check that the IKE identity is configured correctly. Check that the policy is in place to permit IKE and IP Sec applications. Usually this policy is not required if there is no clean-up rule configured on the box. If a clean-up rule is configured, the policy is configured usually from the external zone to the external zone. Check that proposals are correct. If incorrect, logs about the mismatch can be found under the system logs, or by using the following CLI command
less mp-log ikemgr.log
Check that pre shared key is correct. If incorrect, logs about the mismatch can be found under the system logs, or by using the following CLI command:
less mp-log ikemgr.log
Take packet captures to analyse the traffic. Use filters to narrow the scope of the captured traffic.
Useful CLI commands:
show vpn ike-sa gateway
test vpn ike-sa gateway
debug ike stat
Advanced CLI commands:
For detailed logging, turn on the logging level to debug:
debug ike global on debug
less mp-log ikemgr.log
To view the main/aggressive and quick mode negotiations, it is possible to turn on pcaps forcapturing these negotiations.
Messages 5 and 6 onwards in the main mode and all the packets in the quick mode have their data payload encrypted:
debug ike pcap on
view-pcap no-dns-lookup yes no-port-lookup yes debug-pcap ikemgr.pcap
Turn off debugs
debug ike pcap off
Configuring packet filter and captures restricts pcaps only to the one worked on,
debug IKE pcap on shows pcaps for all VPN traffic.To check if NAT-T is enabled, packets will be on port 4500 instead of 500 from the 5th and 6thmessages of main mode. Check if vendor id of the peer is supported on the Palo Alto Networks deviceand vice-versa.Phase 2 Check if the firewalls are negotiating the tunnels, and ensure that 2 unidirectional SPIs exist:
show vpn ipsec-sashow vpn ipsec-sa tunnel
Check if proposals are correct. If incorrect, logs about the mismatch can be found under thesystem logs under the monitor tab, or by using the following command:
less mp-log ikemgr.log
Check if pfs is enabled on both ends. If incorrect, logs about the mismatch can be found underthe system logs under the monitor tab, or by using the command
less mp-log ikemgr.log -
Check the proxy-id configuration. This is usually not required when the tunnel is between twoPalo Alto Networks firewalls, but when the peer is from another vendor, IDs usually need to beconfigured.A mismatch would be indicated under the system logs, or by using the command:
less mp-log ikemgr.log
Useful CLI commands -
show vpn flow nameshow vpn flow name | match bytes
Check if encapsulation and decapsulation bytes are increasing. If the firewall is passing traffic,then both values should be increasing.
show vpn flow name | match bytes
If encapsulation bytes are increasing and decapsulation is constant, then the firewall is sending butnot receiving packets. Check to see if a policy is dropping the traffic, or if a port translating device in front of PANthat might be dropping the ESP packets.
test routing fib-lookup virtual-router default ip
runtime route lookup
virtual-router: defaultdestination: 10.5.1.1result: interface tunnel.1
Show routing routetest vpn ipsec-sa tunnel
Advanced CLI commands:debug ike global on debugless mp-log ikemgr.logdebug ike pcap onview-pcap no-dns-lookup yes no-port-lookup yes debug-pcap ikemgr.pcapdebug ike pcap off
If tunnels are up but traffic is not passing through the tunnel
Check security policy and routing. Check for any devices upstream that perform port-and-address-translations. Because ESP is alayer 3 protocol, ESP packets do not have port numbers. When such devices receive ESPpackets, there is a high possibility they may silently drop them, because they do not see theport numbers to translate. Apply debug packet filters, captures or logs, if necessary, to isolate the issue where the trafficis getting dropped.
—> VPN IP Sec Tunnel Status is Red = When it comes to working with IP Sec VPNs, it can be tricky to understand the status properly, which is why I chose this topic to talk about. Let’s start with the IP Sec tunnel status window, which can be accessed from the Web GUI > Network >IP Sec Tunnels. Inside that window, you see the status of all of the IP Sec VPN tunnels that you have configured on this firewall
The confusing part about the IP Sec Tunnel status window is that there are actually 3 areas that show the current status. I have detailed the “status” below
Phase 1 – IKE status – Green indicates a valid IKE phase-1 SA or IKEv2 IKE SA. Red indicatesthat IKE phase-1 SA is not available or has expired.
Phase 2 – IPSec status – Green indicates an IPSec phase-2 security association (SA) tunnel.Red indicates that IPSec phase-2 SA is not available or has expired.
– IPSec Tunnel Interface status – Green indicates that the tunnel interface is up (becausetunnel monitor is disabled or because tunnel monitor status is UP and the monitoring IPaddress is reachable). Red indicates that the tunnel interface is down because the tunnelmonitor is enabled and the remote tunnel monitoring IP address is unreachable.
I have personally seen the Phase 2 IPSec showing Green, with Phase 1 IKE showing a Red status, eventhough the tunnel is showing green, because it is still active. The next time that IKE is to berenegotiated, it may or may not have an issue, but it’s good to be aware of.To get more detailed information on what is going on when the IPSec Tunnel Interface is showingRed, you will need to go into your logs and look for any errors that may help indicate what theproblem is. You also can click the “Tunnel Info” and “IKE Info” text to the right of the “bubble” statusto get more info. (A window will appear showing the IKE or IPSec info).