Incident Response

[Here is another in our series of  posts from Joe Tinucci, Assistant Treasurer of the University of Colorado. The best practices Joe shares today center on incident response, with which we all need to be concerned. Be prepared and don't think about a breach in terms of "if", but in terms of "when". Thanks for this excellent advice, Joe! --gaw]

Given the continuing publicity surrounding merchant credit card breaches, it is vital that every merchant has a tested Incident Response (IR) plan in place.  Not only is this a requirement of the PCI DSS, but it also makes good business and practical sense.  One of the major benefits of a tested IR plan is the roadmap that it provides when everyone is tempted to panic and run around like chickens with their heads cut off -- believe me, your situation will be difficult enough without some sort of guide or plan in place to deal with the incident and the fallout.

And, the security consensus is starting to gel around the idea that everyone should simply plan on getting breached and so your best defense is to be able to rapidly detect breaches and to move quickly to contain the damage and/or exfiltration of data.  In other words, it is in your future and you might as well be ready for it...

Why Should You Care?

Just about anyone can list a bunch of reasons why you should care about being breached.  But, it would be a good exercise to list as many as possible just in case you need to lay them out for management at some point:

• Your contract with your acquiring bank requires you to be in compliance with the PCI DSS; being breached is considered prima-facie evidence that you were not in compliance with both your bank contract and the PCI DSS.
• There are fines assessed by the card associations and by your acquiring bank.  These fines can run to thousands, or hundreds of thousands of dollars.  Someone is going to have to figure out how to pay these unanticipated costs out of their departmental or campus budget.
• There are investigation and response costs that could easily run into five or more figures.  Just bringing in an outside QSA to perform a Report on Compliance (ROC, also known as the IT Audit From Hell) could be prohibitively costly.  It is likely that the IT budget wasn't set up in expectation of having one (or more) of these audits.  And, like fines, it will have to come unexpectedly out of someone's budget.
• If there is fraud committed with the stolen cardholder data it is likely that the merchant will have to absorb some or all of the losses.  Given the trends today of issuing banks attempting to transfer liability for fraud directly to merchants, these costs could outweigh all others combined.
• There will be remediation costs as well as business process changes; the impact of these could be large and could include the cost of new equipment and software and lots of hours of consultant time.
• There will be the cost of business interruption for the merchant.  Depending on the timing of the incident this cost could have enormous impact, including significant nonfinancial impacts -- imaging not being able to take online admission applications for two weeks before the application deadline.
• Your ability to accept cards for payments could be restricted or denied.  While this might not be a big deal for some departments, any department doing a significant amount of online business would have to re-implement manual payment processing.  And, completely losing the ability to process cardholder payments could be beyond disastrous for the organization.
• Finally, your reputation with your customers -- students, staff, parents, the local community, the general public -- will be impacted.  If you screw up in handling the incident the reputational hit could have long lasting effects.

How Do You Know That You Have an Incident?

Studies have shown that most merchants do not detect their own breaches but instead are informed that they have a problem by outside entities such as the card associations, law enforcement, a bank, or through an extortion attempt.  So, in many cases the first sign that you have had a breach comes via a phone call or email.

Another warning sign of an incident is if staff fall prey to any of the many phishing attempts that hit their desk -- if they are trained to report that they fell for it.  Of course, if they don't realize that there is a problem responding to phishing emails or they are too embarrassed to report it, you are back to hoping that you eventually receive the phone call above.

Other signs include "funny" computer or system behavior -- sending out unauthorized emails or files, slow boot times, unexpected or unusual program behavior, changes to file fingerprints, and so forth.

Your staff (and really, any staff members connected anywhere to your network) need to be trained to ask for help when they see these signs so their systems can be investigated and cleaned as quickly as possible.

Before the Incident

There are several key steps that you need to take to prepare for implementing your IR plan:

• Understand what data is held where in your networks.  This is a data inventory and classification process; everyone says that it is a good idea but not everyone has done it.  At the very least you need to identify where cardholder data is stored and processed so you can designate those PCs/systems as high-risk systems holding critical data.
• Update / patch all equipment / systems that touch cardholder data.  Most organizations have processes in place for updating their PC operating systems and anti-virus / anti-malware programs, but are there other types of equipment that come in contact with cardholder data that need updating (such as copiers and firewalls)?
• Identify your primary IR team.  At a minimum, the team should be composed of representatives from Treasury, IT security, Legal, Public Relations / Communications, senior management, the merchant department, and your external IT security vendor (if applicable).
• Identify your supplemental IR team members, if their expertise becomes necessary.  This will most certainly include someone from your acquiring bank, an IT forensics vendor, local police, the FBI or Secret Service, an outside crisis communications expert, your trusted mailing house, a wholesale credit report provider, your telephone hotline provider, and possibly counselors and advisors to your customers.  Other representatives should be included as the situation requires.
• Finally, you should draw up the plan and make sure that it gets distributed to all parties holding critical data, their system owners, and their designated staff backups. I have found that a checklist format works well, or you can create it in a "Do this first, then that, then..." format.

IR Plan Components

1. Immediately contain the data exposure and minimize data loss
2. Preserve the evidence
•   Do not access the system
•   Remove network and web access
•   DO NOT TURN OFF SYSTEM
3. Alert all necessary parties immediately
4. Call for forensics help
5. Gather the relevant merchant data (merchant ID, merchant contacts, merchant transaction volume, last PCI DSS status, etc.)
6. Continue with your IR plan

This document can be found at:  http://usa.visa.com/download/merchants/cisp-what-to-do-if-compromised.pdf

There are many step-by-step frameworks for creating an IR plan; find one and customize it for your organization if you have to start from scratch (see resource suggestions below).

 

During the Incident

 

First – Clear your calendar for the remainder of the week.  This incident will take an enormous amount of time between phone calls, in-person meetings, working with the incident response team, documenting, and so forth.  Kiss your calendar goodbye!

 

Second – Notify and assemble the team.  Decide when to bring the acquiring bank into the loop, and then bring them in.  If you consider the acquirer to be your partner, you would bring them in sooner than if you consider them to be an adversary; in any case you will need to let them know eventually.

 

Third – Contain the damage.  Make sure that everyone involved at the merchant understands that they must:

• STOP -- do not touch the machine, do not unplug the power
• ISOLATE -- remove the machine / system from the network by unplugging the network (or Internet) connection
• CALL FOR HELP

Fourth – Figure out what happened to which system and how to work without it.  Remember that you have a clock ticking -- the getting-back-into-business clock.  While you do not want to do anything in haste, you cannot dawdle...

Fifth – Communicate as appropriate to your constituencies, as directed by the IR team.

Sixth – Remediate.  Fix whatever business processes went wrong, and make sure it doesn't happen again.

After the Incident

Once you are through the mad rush of responding to the incident, you should debrief the IR process.  What did you learn from the incident, as well as the response to the incident?  What weak technical and/or business practices were identified in this process and how can they be fixed and/or strengthened?  While it is easy to skip this step, it will help strengthen your response the next time the IR plan is activated.

Resources

There are many incident response-related resources available.  As noted above, the Visa What To Do If Compromised document is a great resource upon which to base a merchant IR plan.  One resource that we have found very helpful is our bank; they shared access to their risk management portal from which we drew several useful IR templates.  Likewise, your organization's risk management department (if you have one) is a good resource if you need to start from scratch to draw up an IR plan, as are any insurance relationships that your organization might have.  Finally, the usual security websites have lots of IR plan templates -- SANS, GIAC, ISACA, REN-ISAC, and so forth.

Good luck and may all your incidents be only tests of the plan…

Joe Tinucci is the Assistant Treasurer at the University of Colorado, where he manages the University's banking relationships.  As part of that job, he also drives the PCI DSS compliance process for approximately 160 card-accepting merchants across diverse card-acceptance environments in four campuses. Joe can be reached at (303) 837-2185 or joe.tinucci@cu.edu).