Gladiator Research and Security

Part 1 – Recognizing an Infection
Part 2 – Incident Response Plans and Procedures

In the first part of our Gladiator guide on Malware Basics, we discussed different methods of determining whether a device or network has a malware infection.  Having discussed how to find these malicious infections, we will now review what to do when your phone begins ringing off the hook with users claiming that “the Internet is slow,” or that their workstations are exhibiting signs of a malware infection.  Administrators’ questions at that point usually include, “Where do we go from here?  Whom do we notify and what actions need to be performed to ‘clean up’ the device?”  The answer to all of these questions should be documented in a formal Malware Incident Response Plan.

As many administrators know from experience, no network connected to the Internet is completely impervious to every threat that exists, and with Zero-day vulnerabilities ever present, it is quite possible that at some point a malware infection could occur.  In case such an event were to transpire, it would be of great benefit to have a thorough Plan in place to effectively minimize the damage done and quickly bring the infected network or device back into production.  Such a Plan can greatly expedite the process and also help ensure that valuable lessons are learned at the same time. Lessons learned can then be iteratively applied to further tailor and enhance the Plan to better prepare for any future incidents.

This Malware Incident Response Plan is not intended to replace the institution’s current internal Incident Response Plan (IRP).  Instead, this Plan is intended to supplement the existing IRP and is to be used in the specific case of malware infection.  Your full IRP should include information such as when customer notification is necessary, whether any law enforcement agencies need to be contacted and other information regarding escalation procedures.  All of these steps should be followed properly, and the Malware IRP should come into play when it is time to address finding and removing malicious applications or files.

Preparation

• Establish procedures for removal and restoration of the affected systems.
• Select an individual or group of individuals as the Incident Response Team.  This Team should be familiar with the Incident Response Plan.  Knowledge of where to find and how to use detection and removal tools is also necessary.
• Create and maintain a list of any contacts that will need to be notified or contacted for assistance.  This includes internal users and third party support groups.
• Obtain and install any necessary security applications on networks PCs.  This includes software such as anti-virus (AV), anti-spyware, etc.
• Create an incident response tool on some form of non-writable, removable media such as a CD or DVD.  USB thumb drives and external hard drives are not recommended, as some infections can spread to these devices.  This incident response tool would include any software used for removal of malicious applications (like Malware Bytes) and backup copies of the current anti-virus software (as some malicious files can remove anti-virus software).

Detection

• Pay careful attention to alerts generated by monitoring software.  Notify users to alert the IT staff or Response Team if alerts are generated on workstations.  If you have a centralized solution, utilize the alerting functionality to notify the right users.  Many popular AV solutions can alert users via email.
• If you have a third party monitoring your network, ensure that these alerts are received by the necessary internal users.  These third parties may have access to different monitoring tools and utilities and can also provide valuable information.
• Do not ignore some common symptoms that infected machines may exhibit.  Look out for multiple or reoccurring server or workstation crashes.  Workstations or servers can also become sluggish to respond to mouse or keyboard commands.  Some worms or trojans can generate large amounts of traffic, so look out for Internet access that has slowed dramatically.
• Monitor your email queues for a large backup in your outbound mail queue.  Botted workstations or servers can flood your mail server with spam.  Also lookout for a large surge in bounced-back messages, as this is another indicator of a device infected with a bot.

Removal / Containment

• Remove the infected or suspect device from the network by unplugging the device’s Ethernet cable.  This step is imperative as worms can quickly spread through vulnerable networks.
• Notify the necessary parties that an incident has occurred and start the necessary incident response procedures.
• If a team will be monitoring the infection to measure damage or research, ensure that the device is connected to a network that has no access to the production network.  If possible, the device should be connected to completely different LAN and WAN networks than production.
• Save all available security and event logs on the device.  This includes the Windows or Operating event logs, and any logs from security software.  These logs will be essential in aiding with research into the severity of the infection, and eventually, in recovering the device.
• Record and save all details of any findings and all actions taken.  If a team dedicated to malware response or breaches comes in to research the incident, any notes available will be extremely helpful to this group.
• Run any AV, anti-spyware, or anti-malware removal tools that are available.  After scanning is finished, monitor the device’s health and connection information to ensure that all remnants of the infection have been removed.  Monitoring should be done over a period of time, as any malicious connections may be on a scheduled basis.  The device will still need to be isolated from the rest of the network.
• If the device is still not functioning at previous levels, or if malicious connections are still detected after scanning, a complete rebuild will be necessary.  Some malware can install root kits which can make removal nearly impossible.  For any data stealing or unknown malware, a rebuild is highly recommended in all cases, as this is the only way to ensure all remnants of the infection are removed.

Restoration

• Try to pinpoint the source of the infection.  Determine how the infection was able to enter the network.  If the infection was able to propagate, research how it spread throughout the network.
• Change all passwords on any affected accounts and all the local passwords on the infected machine.  Ensure that all new passwords conform or are more secure than internal policy password standards.
• If a user’s workstation was infected, instruct the user to change any passwords on pages that were viewed while the infection occurred or went undiscovered.  This includes personal banking pages and even social networking sites, as these may store sensitive information.  These passwords will need to be changed on a device that was not affected by the infection.
• Notify all monitoring parties before placing the device back onto the network.  These parties can increase alerting levels or monitor the device’s outbound connections.  If possible, increase internal alerting levels on security software.

Lessons Learned

• Determine if your incident response procedures were effective.  Determine if any changes are needed in the processes.  Was the incident handled thoroughly?  Were all parties given all of the information necessary to diagnose and restore operations?  Were there any delays or stopping points in the process?  Why did these problems arise?
• If user workstations were involved, develop a plan to inform users of the attack and how to avoid similar situations.  For many of the web-based attacks “in the wild,” some form of user interaction is required.  User education is just as important as security devices and software in minimizing incidents.
• Determine if steps need to be added to the response plan.  Some steps may prove unnecessary and may need to be removed.  Your response plan will need to be as flexible and fluid as the incidents that may arise.
• Prepare detailed reports for management.  These reports should cover all aspects of the incident response including effectiveness, any failures, estimated damages, etc.  These reports could also detail any new security products that may need to be included for future protection.