Gladiator Research and Security

Part 1 – Recognizing an Infection
Part 2 – Incident Response Plans and Procedures

Introduction

We’ve all been faced with the following situation at one time or another.  Imagine you’ve just walked in the door of your office, and one of your coworkers comes up to you complaining that his computer is running slowly.  You tell him that is normal in the morning, but then he says his web browser keeps popping up new windows and then crashing.  Of course, he forgets to mention that he was on a flash game website when it all started, until you get to his desk and discover this for yourself.  So, now what do you do?  You suspect this machine has a virus or some other type of malware, but you aren’t sure.  The computer has fully-updated virus definitions and a full system scan didn’t find any malware.  But is the machine safe?

In Part 1 of Malware Basics, we’re going to review some signs that a machine may have malware.  Then we will go over some useful tools for identifying suspicious files even if your antivirus suite does not detect anything.  Finally, we’ll talk about how to identify what type of malware you found.  Part 2 of this series will deal with Malware Incident Response.

Signs of an Infection

The scenario described in the introduction is just one sign indicating that a machine may be infected with malware.  Let’s review some other signs so you will know what to look for.

• Programs running slowly, crashing – many types of malware like to piggy-back on other applications, like web browsers, to monitor what they are doing.  This can be resource intensive.  On the other hand, some malware is just badly written and can slow down a machine or even crash other applications.  Crashing an application may also be on purpose, or a result of a buffer overflow.
• New services running on the machine – Malware likes to stay in memory between reboots.  It also likes to run regardless of if a user is logged in or not.  A great way to do both is with a Windows Service.  New, unknown services in the service management console could mean the PC is infected with malware.
• Suspicious network traffic – This could indicate an infection on a broader scale, with multiple machines affected by malware.  If you notice a very slow network and your switches seem to have much more traffic going across them than usual, it could be an indication of malware or possibly a misconfigured application.  Traffic destined for suspicious Internet IP addresses or websites could also indicate an infection on internal workstations.
• New listening ports – Some malware applications, especially backdoor Trojans and rootkits, will open new listening ports on a machine and wait for instructions.  Become familiar with what ports are normally open on your workstations so suspicious ports will stand out.
• Antivirus warnings – Antivirus software cannot be expected to find all malware, but it does detect about 75%.  Some malware will attempt to download other malware to augment its capabilities (like a rootkit trying to download a Trojan, etc.).  Antivirus software may detect one of these applications but not both.  An AV warning, combined with other signs, is a good indication of an infection, especially if you’re not currently browsing the web, copying files.
• Warning from service providers – If a monitoring provider notices suspicious behavior coming from your network, there may be malware involved.  Hopefully the internal IP address of the workstation can be provided; otherwise, you may have to go on a bit of a goose chase to determine which machine is the source.

Tools

Before we get in to actually analyzing a machine, we should talk about what free tools are available for live system analysis (for Windows).  The tools below are designed to help users determine what applications are running and which ones may be malicious.

(Note: The following sections are designed to aid in a cursory analysis, not to gather evidence for a law enforcement investigation.  If you feel there is a chance of charges being filed, your best bet is to involve a professional computer forensics investigator.)

System Analysis

• Process Explorer – Process Explorer is similar to Windows Task Manager, but with more information and features.  Not only does it do everything Task Manager does, but it also shows all running processes organized by how they were started, what DLLs are attached to them, what services are dependent upon them, etc.  Furthermore, it identifies packed DLL files and executables, which could indicate malware.
  Vendor: Microsoft / SysInternals
  Link: http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx
• Autoruns – Autoruns displays all programs set to run during boot or logon.  It has multiple tabs to differentiate where the applications are set to start from.  Some examples include the registry, explorer.exe, scheduled tasks, services, etc.  Autoruns can also hide signed Microsoft entries from its output, allowing you to pinpoint less trustworthy applications.
  Vendor: Microsoft / SysInternals
  Link: http://technet.microsoft.com/en-us/sysinternals/bb963902.aspx
• Service Console – The service console comes with Windows and is used to start and stop services.  It also lists all currently configured services, along with their status and startup type.  You can look at the service properties to determine what executable is actually running, as well.
  Vendor: Microsoft
  Link: Comes with Windows.  (Start > Run > “services.msc”)

Traffic Analysis

• TCP View – TCP View is a Windows program that shows all open TCP and UDP connections for all running processes.  It lists the local address, remote address, and the state of the connection.
  Vendor: Microsoft / SysInternals
  Link: http://technet.microsoft.com/en-us/sysinternals/bb897437.aspx
• Wireshark – Wireshark is a network traffic analyzer and sniffer.  It can parse multiple types of network traffic.
  Vendor: Open Source
  Link: http://www.wireshark.org/download.html
• Netwitness Investigator – Netwitness Investigator is a network traffic capture analysis tool.  This application can read packet dumps from Wireshark or perform its own captures.  It then analyzes the traffic for different content types.
  Vendor: Netwitness
  Link: http://download.netwitness.com/download.php?src=DIRECT
• Netstat – Netstat is a command-line program that is shipped with Windows.  It reports protocol statistics and current network connections.  More information and available parameters can be found at http://technet.microsoft.com/en-us/library/bb490947.aspx.
  Vendor: Microsoft
  Link: Included with Windows.   (The command line can be started by going to Start > Run > “cmd”)

File Analysis

• Virus Total – Virus Total is a free, online file antivirus scanning service.  It takes a submitted file and scans it with over 30 different antivirus engines.  A report is generated to determine which engines detected the file, as well as what malware they detected.
  Link: http://www.virustotal.com
• CW Sandbox – CW Sandbox is a free, online automated malware analysis service.  CW Sandbox will analyze the behavior of the submitted sample and send a summary report back via email.
  Link: http://www.cwsandbox.org
• ThreatExpert – ThreatExpert is another free online malware analysis service.  The analysis engine is different than CW Sandbox and may pick up on different aspects of the program’s behavior.
  Link: http://www.threatexpert.com
• Google –The Internet can be a very valuable source of information when researching malware; Therefore, Google searches for executable or DLL names can provide useful information.  Many times the searches will identify the executable as malware, or at least describe some observed behavior.

(Note: Many of the tools above involve uploading local files to a website.  Do not upload any files containing confidential information.)

Analysis

Preperation

The first step for any good incident analysis is preparation.  There are a lot of tools and techniques that may be new to you, so it’s a good idea to practice first.  Also, it’s a good idea to look at the output of these tools on a computer that doesn’t have malware so you can become familiar with what is “normal.”  Of course, if you’ve already got a machine infected with malware, this process may take time you don’t have to spare.  Here are some actions I recommend before trying a live analysis:

1)    Download the tools listed in this article and keep them handy. It is a good idea to put them on some read-only media, such as a CD or a write-protected USB drive.  Don’t put them on a network share, as you don’t want an infected machine connecting to one of your file servers.  Plus, if you were forced to disconnect a machine from the network, it would leave that file share (and the Internet) unavailable.  Be sure to update these tools on a regular basis.

2)    Educate your users on signs of an infection. Users are your first line of defense, and if they know what to look for, they can catch an infection before it spreads.  They should also be aware of whom to contact when they think there is a problem, as well as what NOT to do (like continuing to access confidential information using their PC) before talking to someone else.

3)    Create a malware incident response plan. The plan should designate an incident response group specifically for malware.  This group can and should include vendors used for network and security services.  Furthermore, it should list step-by-step procedures for reacting to a reported malware infection.

Investigation

Okay, so you’re sitting at the PC that you think may have malware, and you’ve put the CD containing your tools in the drive.  So, now what?  Depending on what you are looking for, you may start your investigation using different tools than what I’m going to describe, but here’s a general investigative approach that should work until you are more familiar with what to look for.

(Note: Try to organize your windows so you can see all of these tools at the same time.  It might help you correlate a starting or dying process with certain web traffic, etc.)

1)    Start up Process Explorer.  This will give you an idea of what applications are running right now.  If you are using Vista, be sure to run this application as an Administrator (you need to right-click and specify “Run as Administrator”).  You will notice that many of the processes are highlighted in some way.  The default highlight settings as of this writing are shown below.

Pay particular attention to the packed processes (shown as “Packed Images”).  A packed process has been compressed or encrypted, usually to make reverse engineering more difficult.  Many malware processes are packed, whereas few standard processes are.  Another indication of malware is randomly named executables.  If there is a process running called “Pu0NxX67.exe,” it is probably not supposed to be there.  If you have any questions about a running process, search the Internet using the process name and there will likely be information on it.  Actually, there are many Internet listings which describe executables that may be running on a computer.

2)    Start up TCP View.  Look at all incoming and outgoing connections for suspicious traffic. Examples of suspicious traffic could include strange source or destination addresses, as well as non-standard ports.  Close out applications you are not using, like Internet Explorer or Firefox, as they could clutter this view.

3)    Start up Autoruns.  Look through the list of applications that are set to run upon startup.  Also, be sure to check the scheduled tasks tab, as many new malware programs set tasks to run on a regular basis so that they remain in memory.  Another common location is the Registry’s “Run” key, which specifies what applications are loaded upon startup.

4)    If you find an executable or file you believe could be malicious, upload it to a malware analysis service.  There are many available, but Gladiator recommends the following: Virus Total, CWSandbox and ThreatExpert.

5)    If you haven’t found anything but you are still suspicious, try running a network capture.  Install Wireshark on the system and capture traffic for at least 20 or 30 minutes (if you don’t have the space, you can run this for a shorter amount of time).  Then save the capture to a file.  Load this file in NetWitness Investigator for analysis.  Hopefully, any suspicious traffic will be identified in this way.

Cleanup

The cleanup method used depends upon what was found during your investigation.  There are three different scenarios to deal with:  1) Malware was found and identified; 2) Malware was found but not identified; and, 3) No malware was found.  If malware was found, remember that it might have spread to other machines on the network.  Notify users of the specific signs of infection to ensure they know what to look for.  More machines may have to be examined to ensure the outbreak is contained.

Malware Found and Identified

This is the best case scenario, because not only did you find the answer as to why the system is acting up, but you also identified it as a known piece of malware.  Once you know what you are dealing with it, is much easier to figure out how to clean the system.  Presumably, the malware was identified by one or more antivirus engines on Virus Total.  If so, it should provide you with the name of the malware.  Now you can search for removal instructions and more information using that name, either on the antivirus vendor’s website or the Internet using a search engine.  Remember that different antivirus vendors name malware differently, so the same malware can be called multiple things.

Malware Found but Not Identified

Handling this scenario is a bit trickier.  In this case, you have found something, but you’re not sure what it is.  It could be that Virus Total identified it as malicious, but didn’t give you a specific name.  Virus Total may have called it just a “Suspicious File” or “Generic Trojan.”  You may not be able to find removal instructions for something you can’t specifically identify.  And there is no way to know if just deleting the file will remove it, or if it will copy itself back from memory.  And how many copies of it are on the system?  Certainly, there are a lot of questions that probably cannot be answered.  Still, there are a few options here, as described below, in no particular order:

• Rebuild the system
• Contact a network support provider
• Make a copy of the hard drive

Rebuilding the system is probably the first option and the one most likely to restore the system to a usable state.  The problem with rebuilding is somewhat surprising – you do get rid of the malware, but if the malware is gone, you may never know what it did.  And you may never know if it somehow compromised your network in some major way.  For example, what if it was a key logger, and it sent customer information to some external entity?  Unfortunately, the purpose of the program may never be known.  But the number one priority is usually to get back online and productive, and rebuilding the system usually does the trick.

Another option is to contact a network support provider who may be more familiar with malware analysis.  They may be able to run the malware in an isolated environment to determine exactly what it is doing.  This option could give you more information before you decide how to proceed.

A safer way to deal with the malware would be to make a copy of the hard drive for later analysis.  This way, the PC can be rebuilt, but you still have a copy of the hard drive in case there are questions about what the malware actually did.

No Malware Found

This result might seem like the best case scenario, but it can sometimes be the most frustrating.  Here, you still have a system that has symptoms of malware, but you didn’t find any suspicious files.  The options for this scenario depend on how carefully you want to proceed.  You can treat this scenario in the same way as if you found malware but couldn’t identify it, with a few additions.  First, you can leave the system alone and do nothing, since no malicious files were found.  Second, calling a network support provider might be a good idea, as you may just have a hardware problem.  Lastly, you can revisit the system in a week or so, when antivirus vendors may have caught up with what is infecting the system.