As we mentioned in a previous blog entry the KB980346 update published by Microsoft upgrades the underlying Windows SChannel used by Forefront TMG 2010 to support the secure TLS renegotiation extension.
What must be noted that after applying KB980346 on the TMG machine, both the secure and insecure forms of renegotiation may be supported, TMG being in (server) Compatible mode, thus clients not supporting the secure TLS renegotiation extension may use the old insecure form of renegotiation for example in the case of the scenario of a secure web server publishing.
You can adjust this by setting the AllowInsecureRenegoClients registry entry to 0, path:
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\SecurityProviders\SCHANNEL
This will put TMG as a “TLS server” in server Strict mode, thus TMG will allow only clients supporting the TLS renegotiation extension to connect for example for the scenario of a secure web server publishing(when TMG acts as a “TLS server”), if the clients’ TLS Client Hello does not “flag” support for the TLS renegotiation extension, TMG will terminate such requests from the clients.
Other possible interesting “issue”: there was a report in respect with RFC 5746 on the McAfee forums that the McAfee Web Gateway “is reintroducing this vulnerability into connections where both the client and server have been updated”(when doing outbound HTTPS Inspection).
More exactly some version of Google Chrome 6 has some sort of a static list of web sites supporting the TLS renegotiation extension, and since the McAfee Web Gateway was acting as a MITM for the SSL/TLS traffic between the client and server and was not supporting RFC 5746, as a result this prompted Google Chrome to throw an error.
More about this error here(not sure if again MWG was on the path or other web gateways doing outbound HTTPS Inspection):
http://www.google.ro/support/forum/p/Chrome/thread?tid=4af55758316034e2&hl=en
On Chrome 6.0.472.51 beta(currently the latest from the beta channel for Chrome, http://googlechromereleases.blogspot.com/) if TMG was not updated with KB980346, seems that Chrome is “silently” notice(similar with what Opera does) the user about this, if the user clicks the SSL padlock(as suggested by a (disappointed I would add) user on the McAfee forums).
On Chrome 6.0.472.51 beta, if TMG was updated with KB980346, again Chrome similar with what Opera does, will not display the message about the TLS reneg extension to the user, if the user clicks the SSL padlock.
I’ve tried myself the “culprit” Chrome 6.0.472.0 behind TMG doing outbound HTTPS inspection and I did not have any troubles when KB980346 was applied on TMG(if it was not applied Chrome did throw that error).
Opera also has an option to display a warning about a secure web site not supporting the secure TLS reneg extension but this is not currently enabled by default.
Do note that while TMG(with KB980346 applied on TMG) is on the path doing outbound HTTPS Inspection(thus for the client appearing as the SSL/TLS server) the client will not know if the targeted end server really supports the secure TLS reneg extension.
You can put TMG in client Strict mode, but this will make TMG to not be able to set up TLS sessions at all with servers which do not support the secure TLS reneg extension, which will likely currently break a lot of connections(for the outbound HTTPS inspection scenario when TMG acts as a client and not only).