ISA Server 2006 - IPsec Tunnel Mode Site-to-Site
VPN Connections: A Couple of Things That Are Not Supported
- 1. Lack of support for
Diffie-Hellman MODP Group 5 (1536-bit)
- 2. Lack of support for AES
- 3. The local subnet issue
- 4. IKE authentication with
- 5. No IPComp
- 6. The overlapping subnets
issue(And no advanced NAT policies)
Using L2TP/IPsec for a VPN site-to-site connection is recommended when both VPN
gateways are ISA 2004/2006 Firewalls. This is specified in clear in many docs on
Microsoft's web site.
IPsec Tunnel Mode site-to-site VPN connections are used when interoperability
with a VPN gateway from another vendor is required. ISA Server 2006 has passed
Basic Interoperability Test. If you scroll this document, you will find
against which VPN gateways ISA Server 2006 was tested.
If you need to create an IPsec Tunnel Mode site-to-site VPN connection with a VPN
gateway from another vendor, you may ran into a couple of unsupported settings
Let's point out some of them.
1. Lack of support for
Diffie-Hellman MODP Group 5 (1536-bit)
A easy to spot one will be the lack of support for Diffie-Hellman MODP Group 5
(1536-bit). Popular VPN gateways often use this group. ISA Server 2006 can use
instead the stronger Diffie-Hellman MODP Group 14, see Figure1.
Actually, Diffie-Hellman MODP Group 14 matches the strength of 3DES.
Unfortunetely, some vendors are "stuck" at the Diffie-Hellman MODP Group 5
"level", and they do not support Diffie-Hellman MODP Group 14. You may like to
search the VPNC
Members Supported Features - IPsec to see which vendor listed there has
support for Diffie-Hellman MODP Group 14.
Figure1: Diffie-Hellman Groups
2. Lack of support for
Another easy to spot one will be the lack of support for AES. AES is now
widely available on popular VPN gateways.
ISA Server 2006 only supports DES
and 3DES, see Figure2.
Although AES is present on popular
VPN gateways, there is a hint: this does not automatically mean that you can
benefit from its strength. The derivation of AES keys requires the use of
algorithms of certain strength. For example, since above we've talked about
Diffie-Hellman Groups, AES with 128 bits keys requires for comparable strength,
Diffie-Hellman MODP Group 15 (3072-bit). This may not be very practical (too
slow lowering system's performance).
Still, even when for
example, Diffie-Hellman MODP Group 5 is used for keys derivation, AES would be
the preferred choiced over 3DES due to efficiency reasons.
Figure2: Symmetric Encryption Algorithms
3. The local subnet
The one that follows is not that easy to discover, but if you are in the "right"
place you will definetelly have headaches.
In certain situations, you may
need to specify that the local site IP addresses range to include only a subnet
or just a single IP address from the internal network.
the local office uses the 192.168.40.0/24 network and the remote office uses the
You just need to specify as the local network a
single host, 192.168.40.2 and for the remote network, again just a single host,
192.168.30.2 for the site-to-site VPN connection.
However it is not
possible to specify from ISA's GUI that just a single host from the internal
network will be used for the VPN site-to-site connection.
Let's visualize this a little bit. One endpoint of the VPN tunnel uses an ISA
Server 2006 Server installed on Windows 2003 Standard R2 SP2 and the other
endpoint of the VPN tunnel uses a Cisco router.
I've run the VPN
site-to-site wizard, and skipped to add the network rule and the access rule.
For the remote network I've specified only the 192.168.30.2 IP address. No
problem with that.
But the VPN site-to-site wizard does not ask me anything
about the local range of IP addresses that are to be used.
If I take a look
at the site-to-site summary, I can notice that, although I did not entered
anything, a local network appears to be configured (includes all the IP
addresses used on ISA's Internal Network) - this ISA Server has only two NICs,
internal and external -.
Figure3: IPsec Tunnel Mode VPN site-to-site Summary - No Network Rule
Even more, if I take a look at the Quick Mode Filters, I can spot that the proxy
identities that are to be used during IKE Quick Mode negotiations have been
defined, see Figure4 and Figure5. You can view
this filters either using the "IP Security Monitor" snap-in or "netsh"
Figure4: IP Security Monitor Quick Mode Generic Filters
Figure5: netsh Quick Mode Generic Filters
Obviously the proxy identities are wrong, instead of 192.168.40.0/24, I want to
And, as you might know, if the proxy identities do not
match at both endpoints of the VPN tunnels, IKE Phase II (QM) negotiations will
fail, thus the "tunnel" will not be established.
You may think: "what about the required network rule, I will define a network
rule between the remote site and this local host(192.168.40.2), and maybe things
Well, it does not work like so. I've created a Computer
Object for this local machine, seeFigure6.
Figure6: Computer Object
And a network rule with a route relationship between the remote site and the
Figure7: Network Rule
If we look now at the site-to-site
summary (see Figure8), the local network still includes the
entire network 192.168.40.0/24. Only 192.168.40.2/32 is shown as a routable
local IP address. And, I'm suggested that on the other end of the tunnel, to use
192.168.40.2/32 as the remote network.
Figure8: IPsec Tunnel Mode VPN site-to-site New Summary
Analyze again the Quick Mode filters and nothing has changed, still the wrong
proxy identities are used, see Figure9.
Figure9: IP Security Monitor Quick Mode Same Generic Filters
Moving at the other endpoint of the VPN tunnel, and looking at the proxy
identities defined on the Cisco router, we can see they look fine.
Figure10: Cisco Router IPsec Tunnel Mode VPN Expected Proxy Identities
Although we know that IKE QM negotiations will fail, let's play the game and try
to bring the tunnel up. I will initiate the tunnel from ISA's side, by pinging
from host 192.168.40.2 to host 192.168.30.2.
I've enabled "crypto ipsec" debugging on the Cisco router. As expected,
IKE QM will fail. InFigure 11 we can see the error shown on a
Cisco 3620 router(IOS version 12.3-16), "proxy identities not supported", and
in Figure 12, the error shown on a Cisco 3640 router (IOS
version 12.4-7), "no IPSEC cryptomap exists" - note that the error
from Figure 12 was shown too on a Cisco 3725 router (IOS
We can clearly see that the proxy identities proposed by
ISA were not the ones expected by the Cisco router.
Figure11: Cisco 3620 Router (IOS version 12.3-16): "proxy identities not
Figure12: Cisco 3640 Router (IOS version 12.4-7): "no IPSEC cryptomap exists"
If we analize the Oakley.log from ISA, in case of the VPN site-to-site connection
with the Cisco 3620 router, we can notice that IKE Main Mode negotiations were
completed, and then ISA attempted to negotiate IKE QM with the wrong proxy
identities, see Figure13. Thus, the Cisco router informs ISA
that it has not accepted its proposal (NO-PROPOSAL-CHOSEN, see RFC2408
Internet Security Association and Key Management Protocol (ISAKMP), an "Internet
Official Protocol Standards" (STD 1), for more details about Notify
Messages Error - Types), see Figure14.
Figure13: ISA -> Cisco 3620 Router Oakley.log: Wrong Proxy IDs
Figure14: ISA -> Cisco 3620 Router Oakley.log: NO-PROPOSAL-CHOSEN
Same story with the Oakley.log from ISA in case of the VPN site-to-site
connection with the Cisco 3640 router. See Figure15 and Figure16.
Figure15: ISA -> Cisco 3640 Router Oakley.log: Wrong Proxy IDs
Figure16: ISA -> Cisco 3640 Router Oakley.log: NO-PROPOSAL-CHOSEN
Interesting, if instead of trying to initiate the tunnel from ISA's side, I will
initiate the tunnel from the Cisco router's side (by pinging from host
192.168.30.2 to host 192.168.40.2), the connection will be established.
the Cisco router starts the IKE QM negotiations and proposes the correct proxy
IDs, ISA "senses" that its policy is "too general", adds on the fly new QM
filters and accepts Cisco router's proposal. See Figure17.
By the way, if you post your Oakley.log on a forum searching for help when you
have a problem with your VPN connection, and you need to remove your public IP
addresses from the Oakley.log for privacy issues, make sure you do not miss
their hex values too, see Figure17, InTunnelEndpt(192.168.22.240
here) and OutTunnelEndpt (192.168.22.222 here).
Figure17: CiscoRouter-> ISA Oakley.log: Policy Too General
If we look now at the Quick Mode filters, we can notice the newly added ones
(correct ones), see Figure18.
Figure18: IP Security Monitor Quick Mode New Generic Filters
Obviously, the "result" cannot be considered a solution, there is no gambling
here, the tunnel must be intiated from any direction and must stay up. Amazing
how, although ISA Server 2006 features a powerful GUI which greatly simplifies
the creation of VPN site-to-site connections, for something plain simple, we
have a Las Vegas story.
4. IKE authentication
with certificates issue
ISA Server 2006 is capable of using certificates for authentication in case of
IPsec Tunnel Mode site-to-site VPN connections. However there is a problem: you
can specify only from which CA will accept ISA certificates, see Figure19.
This means that ISA will not perform additional checks over the certificate
received from its IPsec peer (say check the name from the certificate).
Therefore, the posibility that somebody in possesion of a valid certificate
issued by the trusted CA, to establish the VPN connection, exists, because ISA
cannot verify if indeed that certificate was issued to the expected VPN peer (if
ISA could check the name from the certificate, this situation would be avoided,
assuming that no one can obtain a certificate containing the correct name under
a false identity from the CA).
Notice the big exclamation mark from Figure19 ,
which informs you to use a privately issued certificate (from a private
certification authority, maybe your local private CA). If you click on the help
link you will find more details about certificates and VPN connections. So it's
always a good idea to read properly the documentation first. If you do so, you
would be advised about the above described situation.
Figure19: IPsec Tunnel Mode VPN Site-to-Site Connection: Authentication with
5. No IPComp
If you want to use compression with IPsec Tunnel Mode VPN Site-to-Site
Connections you cannot do so. With other VPN gateways, compression occurs
through IPComp (before encryption). ISA Server 2006 does not support IPComp.
If you want compression with VPN Site-to-Site Connections, you must use
L2TP/IPsec site-to-site VPN connections (MPPC is used).
6. The overlapping
subnets issue (And no advanced NAT policies)
Another situation that may appear will be with overlapping subnets. Say, the
local office uses 192.168.50.0/24 on its internal network, and so does the
remote office. This can occur when you create a VPN site-to-site to a partner's
office or so. Since both offices use IP addresses from private IP space, and
they do not belong to the same company, there is a chance for subnets
to overlap. A solution to this will be to apply policy based NAT(or whatever
will be the marketing term) over VPN traffic, so hosts on one side can
communicate with hosts on the other side using mapped IP addresses. Say, on the
local VPN gateway, local network 192.168.50.0/24 will be NATed as
192.168.100.0/24 and the remote network will be 192.168.200.0/24. On the remote
VPN gateway, local network 192.168.50.0/24 will be NATed as 192.168.200.0/24 and
the remote network will be 192.168.100.0/24. Stating mappings(static NAT) can be
defined for servers, say for a local server 192.168.50.10 -> 192.168.100.10. So
when a remote host (192.168.50.100) needs to access our local server
192.168.50.10, it will initiate communications with the 192.168.100.10 host.
Unfortunetely, ISA does not have such flexibility regarding the NAT process, so
we cannot do these. ISA's design was focused on advanced firewall capabilities
rather than on advanced NAT policies, which are more common to routers.