Network security devices perform security detection on the traffic passing through the devices based on signature rules. When the network communication data matches the signature rules, the network security devices will block the data. However, sometimes, the security devices block normal network business communication as well. This is a typical case for our reference.

Problem Description

An extractor host needs to extract files from the FTP server on a payment platform. When extracting files, it often fails because the extracting scripts fail and stop running. By analyzing the files failed to be extracted, it’s found that the name of failed files ends with “227”.

The communication process is as the following image. The extractor host communicates with the Payment Platform FTP Server by passing through a switch, a router, and firewalls. Since the change of file name caused the stop of extracting scripts, so the fault is located to the application layer. Therefore, layer 3 devices such as firewalls should be focused. As the diagram below, a Colasoft nChronos Server is deployed to monitor the traffic at Capture Point 1 before FW-1, Capture Point 2 before FW-2, and Capture Point 3 after FW-3.

communicaiton-process

Analysis

To reproduce the fault, create a test file “TEST227.DAT” on the Payment Platform FTP Server, and then run the extracting scripts to extract the file “TEST227.DAT”.

Since Colasoft nChronos Server is already deployed to collect the traffic from the three capture points, the whole test process is captured. Use nChronos to analyze the traffic captured at Capture Point 1 and Capture Point 2, and the packet decoding shows that the name of the test file is changed to "TEST22_", as the screenshot below. Therefore, it can be judged that the fault point is on the right side, which is close to the Payment Platform.

packet-decoding-1

Then use nChronos to analyze the traffic captured at Capture Point 3, and the packet decoding shows that the name of the test file is “TEST227.DAT”, which doesn’t change, as the screenshot below.

packet-decoding-2

After several tests, it was found that if the name of extracted files is "XXXXXX227. DAT", the file name will be changed to "XXXXXX22_. DAT" during the extraction via FTP protocol, the extracting scripts will have something wrong, resulting in the stop, and the files will not be transmitted.

Analysis Conclusion

Therefore, it can be judged that the fault happens between the Capture Point 2 and Capture Point 3, i.e. firewalls FW-2 and FW-3. Because there is no way to mirror the traffic between the two firewalls, so it is roughly judged that one of the two firewalls has modified FTP data.

After communicating with other departments, it’s told that FW-3 is a Check Point firewall, which enables FTP detecting. The FTP detecting function diagnoses the occurrence of 227 (file name/file size) as FTP Bounce attack, which results in file name change.

Because the Check Point firewall has more meticulous security detection options for protocols than other firewalls, this kind of problem has been solved after modifying the security detection settings for 227.

Relevant solutions can also be found on Check Point official forum.

nChronos Value

Due to the continuous development and upgrading of network architectures, various application failures are always emerging. Application access has to pass through a large number of middle devices. It is difficult to locate the fault point by traditional analysis methods.

Network analysis technology can decode packets from layer 2-7 and restore the raw communication data. And, it can discover the abnormal transmission among middle devices, and find out the root cause for application abnormalities.