What is the current state of the model risk management purview on cybersecurity solutions? It wasn’t that long ago (March 2021) that a regulatory agency in Europe fell victim to an attack on its servers that affected its email system and potentially led to theft of personal data.
In a similar vein, eight months later in North America, a large American stock-trading platform suffered a data breach after adversaries were able to hack into its systems. Hackers gained access to the personal information of more than seven million customers of the trading platform. Moreover, threats to institutions continue to rise because of a more connected world. As a result of continued and ever-evolving cyber threats, the US Department of Homeland Security and the European Central Bank have issued notifications to financial institutions to strengthen practices, remain vigilant, and demonstrate resilience.
With cyber attacks becoming more frequent and increasingly sophisticated, financial institutions are using more analytical solutions and tools to analyze and mitigate cyber threats. As the use of cyber models grows, so too do the risks related to the design and use of those models. This is why the focus on model risk management (MRM) for cybersecurity solutions is on the rise, in an effort to identify key risks in organizational cyber solutions and to help mitigate them. MRM monitors risks from potential adverse consequences of decisions based on incorrect or misused models.
The first step of MRM is to identify the use of these analytical tools and include them in the model inventory. Financial institutions across the globe are adding cyber solutions to their model inventory, with banks in North America leading the way compared with banks in Europe, the Middle East, and Africa (EMEA). According to a 2021 McKinsey survey, 70 percent of the respondent banks in North America (NA) are aiming to include cyber risk model types into the scope of MRM governance. On the other hand, 38 percent of the respondent banks in the EMEA region have plans to include cyber risk model types in the MRM scope. Compared with EMEA, the scope of MRM in NA is more comprehensive. Any approach that meets the definition of a model, beyond regulatory models, is also considered part of the MRM scope.
Consequently, since most cyber solutions leverage advanced analytics including machine learning (ML) techniques, they are considered to be meeting the definition of a model by institutions in NA and, thereby, are being included in the inventory.
What does the typical cyber risk solutions/tool landscape look like?
Cybersecurity remains at the top of the executive agenda, and banks are allocating more resources and investment to strengthen their cybersecurity defenses. Banks use a combination of analytical models and deterministic solutions as part of a robust cyber framework. Cyber solutions are used in banking for the following three priorities: safeguard web and mobile applications, identify risk exposure, and review existing cyber defenses.
- Safeguard web and mobile applications. With the increased transition to a digital economy, there is a need to implement the right solutions to protect a bank’s applications. Cybersecurity solutions are required to fulfill a set of objectives including detection and prevention of intrusions, data and messaging security, and access management. A range of solutions from advanced analytics (for example, ML) to rule-based approaches (for instance, expert-driven nonmodels) can be leveraged to fulfill these objectives. As an example, ML solutions play a key role in detecting and preventing intrusions and denial of service attacks. Similarly, analytical solutions are leveraged by banks to ensure data and messaging security and provide overall response to end point threats. On the other end of the spectrum, rule-based approaches are leveraged by banks to implement controls for managing the Internet of Things (IoT) and preventing fraud from transactions. Additionally, qualitative, expert-based approaches are used for identity and access management, which are important since the introduction of shared data repositories and a cloud-connected world.
- Identify risk exposure. With the increasing uncertainty associated with cyber threats, banks are eager to understand the likelihood of suffering business or data disruptions and to identify cybersecurity risk exposure. Any kind of disruption can lead to economic impact, and it is important for banks to quantify the dollar value exposure they potentially have because of cyber risk. To measure the risk, organizations start by qualitatively creating a catalog of cyber risk areas the organization is exposed to. Next, the dollar value risk is simulated across a range of scenarios and compared against the risk appetite of the organization. Then, controls are designed and identified to mitigate or reduce risk. Once that is accomplished, it is essential to qualitatively monitor and periodically review the risk environment to identify emerging threats and ensure compliance with cybersecurity guidelines.
- Review existing cyber defenses. Finally, with the dynamic nature of cybersecurity, banks need to periodically review and challenge existing cybersecurity defenses. To achieve this, certain tools and solutions are used to simulate an attack to identify system vulnerabilities. Qualitative approaches are used to set up a review and challenge framework around the cybersecurity landscape. In addition, a security incident process is established to investigate potential cyber attacks and act as a feedback loop (see sidebar, “Cybersecurity solutions are typically used for three key purposes”).
What are the key model risks of cybersecurity solutions?
Adding cyber solutions under a bank’s MRM purview can shed new light on risks the organization never knew it faced. MRM can help identify the areas of model risks and then point toward solutions, mitigating the risk. In the subsequent section, we present the risk areas that can be introduced by the cyber solutions and the implication for the model risk management of cybersecurity models.
- Methodology obsolescence. Organizations face adversarial risk that is constantly evolving. The adversary, whether a hacker or insider, is always trying to outthink the defenders by devising innovative techniques or simply to social engineer a victim to hand over their credentials. In both cases, the model methodology may become obsolete over time, which means it needs to be adaptable to emerging threats.
- High-speed and sudden attack. Oftentimes, an attack appears to come out of nowhere, and it hits with little to no warning. When this occurs, the victim is performing triage, and triage provides limited opportunity to ward off the attack. In this scenario, the model methodology needs to be aware and be able to think ahead and capture such threats before they become attacks.
- High stakes and extended downtime. In terms of attacks, one of the keys behind cybersecurity is to eliminate any unplanned downtime. To that end, an attack may lead to significant financial impact, data leakage, a long period of downtime, or an impact on the reputation of the organization, especially in the case of a ransomware attack. That is why the cyber model should have effective governance and robust controls.
- Vendor risk. An extensive level of collaboration is required when implementing a vendor’s cyber solution. In most cases, solutions are developed by vendors and implemented within an integrated infrastructure. The technology behind the solution, though, is kept as a trade secret by the vendors. With that in mind, the organization must conduct an effective review and challenge of the vendor-based cyber solutions.
- Infrastructure challenges. Cyber solutions require a large amount of data in the form of internet traffic, browser fingerprint, and constantly changing traffic behavior, which require a highly developed IT infrastructure. A simple misprocessed signal could cause large losses and expose a bank’s applications to external threats. That is why there needs to be effective penetration testing to ensure the solutions are well integrated and properly functioning within the suite of applications.
What are the key considerations for model risk management of cybersecurity solutions?
The list of potential risks highlighted in the prior section shows the importance of MRM and its application to cybersecurity. McKinsey survey results show NA and EMEA banks mentioned that cyber solutions are being added to their model inventory. In order to effectively manage the risk of cyber solutions, risk-management procedures need to be advanced to capture the dynamics introduced by the cybersecurity models. Following are the advanced practices and key considerations to note for effective risk management of cyber solutions:
- Model inventory management. Most advanced banks carry out a review of the entire cybersecurity environment to provide a transparent view of the landscape and classify the subcomponents into models and nonmodels. Any solution that either has the complexity of the quantification methodology or has a degree of uncertainty of the output are categorized as a model. An exhaustive and accurate inventory could serve as a tool for the banks to monitor model risk and manage the overall portfolio.
- Materiality assessment. A materiality assessment is typically run by the banks on the identified models based on the intrinsic-modeling technique and its criticality. The guiding factors behind the severity assessment of cyber solutions are the motivation and use case of the model. The materiality assessment could consider the following approaches: include questions around the cyber risk losses (that is, quantum of loss in the past that led to the development of a specific solution); identify financial impact by running hypothetical simulations; and rank the solutions by importance for the same use case (that is, core versus add-on solution).
- Vendor engagement. Cyber solutions are generally onboarded from third-party vendors that typically will not provide full disclosure around key elements of their technology. Banks, therefore, enhance their model documentation standards for vendor models to provide sufficient information for understanding of the conceptual framework. Further, the model documents are elaborated on with the output testing and governance actions implemented by the vendor to ensure that the model performs as intended. Continuous dialogue between the vendor and the bank security team could act as a feedback loop on any surprises or challenges.
- Model monitoring. Threat actors are getting more innovative and sophisticated. That could also mean the cybersecurity solution may lose its impact over time and, therefore, need to be monitored periodically. Advanced banks set up a live monitoring to watch for degradation, periodic output performance testing, and audit trail generation. In addition, the models are reviewed on a periodic basis for retraining or any adjustments, as needed. Periodic penetration testing is also performed to identify any unique scenario that may not be in the production version of the model.
- Independent validation. Banks are also strengthening their second line of defense to perform an independent review of the cyber solutions for identification of model risk. Validation standards used by the MRM function of the bank are expanded to focus on the conceptual soundness of the model, exhaustiveness of the assessment testing performed by the first line, and the robustness of the governance and controls around the model. In the exhibit, we outline a customized validation approach following a validation framework across five dimensions: input data, methodology, model performance, implementation, and governance.
How should banks start their journey?
The importance of model risk management of cyber solutions is now clear and will only continue to grow in the future. Banks have begun to understand the cyber analytics landscape and customize their MRM standards to incorporate the specifics of cyber solutions. The sooner banks start their journey and establish an effective approach, the quicker they will be able to manage risk and establish controls. The following is a path an organization could take toward establishing effective governance and standards:
- Conduct awareness workshops. Banks should begin by organizing workshops with the business unit, risk function, and management to raise awareness of model risks and model risk management in network security, technology, and cyber risk teams. Further, the workshops should also aim to increase depth of understanding of the bank’s cybersecurity landscape and how any gap affects the organization.
- Build capabilities. Banks should develop and train teams in the first line and second line of defense with the relevant skills, including advanced technical and programming skills, knowledge of governance and controls, and awareness of regulatory guidance on model risk management.
- Review model landscape and identify inventory. Banks should conduct a model identification exercise to identify and classify cyber solutions. While going through this process, the banks would also get an opportunity to identify missing solutions for any specific use case.
- Customize MRM standards. The existing bank’s MRM standards—including validation practices—should be customized to incorporate cybersecurity-specific dynamics. The refined standards should end up applied to the cybersecurity models to identify a potential source of model risks.
- Conduct independent validation. The second-line function should conduct initial validations of the identified models in line with enhanced and customized practices. The focus should be on the key sources of risk identified in the individual cybersecurity model.
- Measure model risk and start to report it internally. Banks should score the model risk across individual models and report it internally to key stakeholders. Such periodic reviews and reports would be critical for identifying gaps, making decisions, and building resilience.
Now is the time to employ cybersecurity MRM to ensure everyone is on the same page to prevent threat actors from causing a costly attack.
