Sometime ago we’ve played a little bit with TMG Beta 2’s IPS. There we've alter a little bit some code found on the web.
There might be some unsaid stuff: the IPS on TMG Beta 3 (for example inspecting HTTP traffic from internal clients to external web servers) might have some dependencies on the HTTP compression setting.
By default, TMG Beta 3 is configured to request compression when internal clients(located on the Internal Network) access external web servers-maybe not the most fortunate expression-:
Note that, with the Request Compressed Data tab of the Configure HTTP compression option, TMG Beta 3 will request HTTP compression say from an external web server, and the HTTP compression occurs between the TMG Beta 3 and that web server, the server’s response is not passed compressed between the TMG Beta 3 and the internal client(for that you use another setting). For example, see this Microsoft article for more details(although is for TMG Medium Business Edition).
Now if you go and remove External from the above setting, you may face a situation.
According to:
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
”If no Accept-Encoding field is present in a request, the server MAY assume that the client will accept any content coding.”
Which means that, even if you(TMG Beta 3) do not request compression, the server may compress its response(varying on its configuration, say compress text/html).
The trick: Today, most of the browsers always send the Accept-Encoding: gzip, deflate option, so the browser is "prepared" for compression. One such browser may be IE 6, which is quite “infamous” for its number of vulnerabilities found over the years. And the sad truth is that, big companies, may have yet to migrate to newer and more secure versions of IE(corporate policies).
So you may have one nice reason to fit in the TMG’s Beta 3 IPS and do some tests(beta stuff).
As said, you go and remove External from the Request Compressed Data tab of the Configure HTTP compression option. And configure TMG Beta 3 to request HTTP compression just from some selected destinations.
TMG will not request compression from a public web server, so one may expect the web server to "obey". However one such “public” web server may use compression “no matter what”.
The request will reach the web server without the Accept-Encoding: gzip, deflate or so option.
But the server replies with Vary: Accept-Encoding and Content-Encoding: gzip. Say, with a simple html web page containing some “funky” JavaScript.
The response passes through TMG Beta 3 compressed, TMG Beta 3 does not react(Malware Inspection may interfere, so I’ve disabled it to view what happens), does not drop it saying compression was not expected or so, IPS Scan Result says Inspected, nothing is detected(although you might expect the needed NIS signature to detect the attempt as it detected when TMG Beta 3 was configured to request compression).
TMG Beta 3’s logs might show compression client=no, server=no.
And finally this response will reach the client(which browser’s has requested compressions) and … .
Once upon a time, ISA Server 2004 used to drop such a response from the web server, if it did not request compression.
But it looks that TMG Beta 3 “behaves” different, and it has a certain default setting for the Request Compressed Data tab of the Configure HTTP compression option.
If you still want to not have TMG to request HTTP compression, you can configure the HTTP filter on your access rules on TMG Beta 3 with signatures to block the web server’s response if it is compressed and you did not request compression.
Also, if you deploy TMG Beta 3 to test a complex scenario, with multiple NICs and corresponding TMG Beta 3 Networks for these NICs, you may double check to see if TMG Beta 3 request HTTP compression for example when a client from a TMG Network access a web server located on another TMG Network(thus through TMB Beta 3), to make sure IPS inspection is not affected by unwanted HTTP compression from that web server.
The trick described above may be quite tempting to use in bypassing an IPS, along with other HTTP evasion tricks, as there are on the market various IPS that do not have the intelligence to inspect HTTP traffic at a certain level and it is pretty simple to use.
Actually I’ve noticed this when playing with TMG Beta 2 and its IPS. At that time, the Request Compressed Data tab of the Configure HTTP compression option seemed to affect the Malware Inspection too. But now, after some quick tests with TMG Beta 3 and its Malware Inspection, I was not able to repro the old findings for the Malware inspection if I remove External from the Request Compressed Data tab of the Configure HTTP compression option. Even if I’ve “split with TCP” the compressed server’s response, the Malware Inspection seem to reassemble and decompress the server’s response. I have not tried many tricks right now, but it looks fine.
In fact, the Malware Inspection on TMG Beta 3 appears to be pretty good, during my quick tests “resisted”(with the help of the Block suspicious files setting –it was not sure what was in my packets-) when for example the Kaspersky Anti-Virus for Microsoft ISA Server(on ISA Server 2006) failed to detected me. Not that these quick tests would have such a great relevance…
Anyway, remember that all these levels of protection on TMG Beta 3, Malware Inspection+IPS(applied to HTTPS traffic too due to the new HTTPS inspection feature), should not replace the client side protection, and only to complement your defense-in-depth solution.
And more important, you should always patch your machines. An IPS is not a silver bullet.
Also you should use the new URL filtering option to block access to “bad destinations”, thus decreasing at some extent the likeliness for internal users to access(willingly or unwillingly) potentially malicious web sites.