Risk Assessments Case Study

Q. Who was your customer and what was their role?

Our customer was the Consumer line of business (LoB) within Bank of America (BoA), which is headquartered in Charlotte, North Carolina. The Consumer LoB provides services to individual consumers, such as savings and transactional accounts, mortgages, personal loans, debit cards, and credit cards.

Q. What was Definitive’s role with Bank of America?

They hired us to support the Supplier Management Compliance Review (SMCR) Program. The goal of the program was to help the Consumer LOB:
1) Protect personally identifiable information (PII), such as customer names, account numbers, social security numbers, and other non-public personal information;
2) Ensure that operationally-critical suppliers had viable business continuity plans in-place to quickly recover from a business interruption

The goal was to be achieved by conducting an operational risk assessment of key suppliers to the Consumer LoB. A supplier assessment measured the operational risk associated with the supplier by assessing their compliance with hundreds of controls, spanning a broad range of domains. Upon the conclusion of each assessment, an assessment report was presented to the senior management of the supplier and Consumer LoB that included the findings and recommended remediation actions. In the weeks and months following an assessment, the Consumer LoB and supplier personnel worked together to address the remediation actions and provide a monthly status update on the number of open findings.

Q. What was Bank of America trying to achieve and what challenges were they facing?

Upon joining the team, the customer shared that they were experiencing two acute challenges:

First, the assessment team had recently completed approximately fifty (50) on-site supplier risk assessments and was having difficulty synthesizing the voluminous amount of data to accurately convey the risk posed to the customer. The existing reports focused on the number of findings opened/closed, and the number of controls with a red/yellow/green status, providing little value to executive management. The inability to draw business insights from the data was concerning to the customer. They knew that the analytics and reporting issue was going to become more pronounced in the coming year, as the assessment team was just days away from launching a 6-month assessment period that would add approximately one hundred (100) additional assessments to the database.

Secondly, during the remediation phase after each supplier assessment, the customer was unable to determine when the completed remediation actions for a given supplier were sufficient to declare the remediation phase completed. The existing practice was to require every finding to be fully resolved. The inability to close-out the remediation phase by analyzing the residual risk would result in over allocating the available staff, as the assessment team would need to continue managing and monitoring the remediation associated with the assessments from the prior year while simultaneously initiating new ones.

Q. What did the Analytic Hierarchy Process involve and why was it important?

To the bank’s credit, with ~10,000 suppliers, they recognized that some suppliers have more inherent risk than others due to the nature of the products and services that they provide to the bank. As such, they were able to first prioritize the suppliers for an assessment based upon their inherent risk score.

A quick review of the information security and business continuity controls being used in the assessments yielded that they were connected to the goal (of an assessment) through a series of intermediary entities – where the goal (at the top) was decomposed into domains, and domains were decomposed to functions, and functions were decomposed to requirements, and finally, requirements were decomposed to controls.

With this tree-like structure (assessment goal --> domains --> functions --> requirements --> controls) already in-place, we recommended the use the Analytic Hierarchy Process (AHP) to build a collection of weighted, hierarchical assessment models that reflected the relative importance of each domain, function, and requirement for each category of supplier (e.g., call centers, card printers, credit agencies, etc.)

The weighted assessment models were used to aggregate the data at the lowest level (i.e. control level) of the hierarchy to provide insight into how the supplier fared at the requirement, function, and domain levels – resulting in an overall rating at the top of the hierarchy (i.e. goal). The weighted assessment models doubled as highly visual dashboards that more accurately depicted the risk exposure associated with a given supplier. The dashboards enabled the customer to answer questions such as:
• Should we enter a remediation phase with the supplier?
• Should we replace the supplier?
• Should we sunset the supplier relationship and rely on the other suppliers in that same category?

To determine when to declare the radiation phase complete, the weighted assessment models enabled the assessment team to determine when the successful remediation of findings at the control level were sufficient to move the aggregated assessment at the requirement, function, domain, and goal levels to an acceptable state. By remediating the most consequential (higher weighted) controls, the upper levels of the weighted scoring model moved “to green”, shifting the emphasis (and resources) away from trying to remediate every finding – and particularly, away from those that were inconsequential (lower weighted) for that category of supplier.

The assessment dashboard could be used to quickly determine which findings necessitated immediate and complete remediation and which did not. By addressing those controls that would contribute most to moving the top level of the model (the goal) to “green” optimized the allocation of resources. Simply put, neither the bank nor the suppliers had infinite resources to mitigate every finding to get a favorable rating on every control. We knew that if resources and time were being applied to remediating the findings associated with less consequential (lower weighted) controls for a given supplier, that we would be diluting our ability to identify and mitigate more consequential risks with that same supplier or other suppliers.

Q. How did Definitive add value to the supplier assessment program?

By adopting the solution described, the customer significantly improved its visibility and insight into their operational risk. The executives were no longer limited to the descriptive and diagnostic analytic reporting that they been receiving. Although not yet fully matured, the customer was able to begin analyzing where they had significant risk of a data breech across their supply chain. Although it’s never easy to anticipate where and how the next data breach might occur, the executives could, for the first time, begin to proactively make strategic sourcing decisions to minimize their operational risk exposure.