The macroeconomic environment over the past year has been characterized by rising uncertainty, bouts of volatility, and a sharp increase in event risk. These factors, and an uneven economic recovery, have motivated many financial institutions to leverage new analytics capabilities for a range of business processes. In parallel, the commercial landscape has continued to evolve, amid accelerating digitization and a wave of acquisition activity that has led to the expansion of model inventories in both the United States and Europe.
Over the past year, McKinsey has invited groups of risk managers to come together to discuss the state of the art in risk modeling and model risk management (MRM). At roundtables and through our global MRM survey, we have gathered insights from institutions in the United States and Europe on a range of modeling challenges and opportunities.1
The outputs from our discussions shed light on the state-of-the-art in bank MRM and reveal a range of themes that are likely to shape institutional approaches over the coming year. In particular, they reveal three key transformations: an increased focus on efficiency, digitization, and automation of the model life cycle; an expansion of the scope of MRM into new areas, including climate, cyber, sales and marketing, and even human resources; and a focus on derisking and maximizing the potential of artificial intelligence and big data. All of these have informed a range of strategic and tactical adjustments that will define the parameters of MRM in the year ahead.
Transforming efficiency, digitization, and automation of the model life cycle
In response to an increasingly complex economic and business environment, and the powerful economic impact of the COVID-19 pandemic, many banks have expanded their model inventories over the recent period. US banks have seen as much as a 25 percent jump in number of models since 2019, while European institutions report a 13 percent rise. Still, the process remains a challenge. In Europe, initial validation for Tier 1 models takes 20 weeks on average, while Tiers 2 and 3 models take 13 and nine weeks, respectively. For periodic validation, the timelines are 11 weeks, six weeks, and four weeks, respectively. In the United States, the validation timelines are typically lower across banks, with initial validation for Tier 1 models taking 12 weeks, while Tiers 2 and 3 models take six and four weeks, respectively. For periodic validation, the timelines are on average seven weeks, five weeks, and four weeks, respectively.
As activity has ramped up, banks report that costs in areas including inventory management, reporting, and risk-limit setting have risen. In response, a large number have taken steps to improve the efficiency of the MRM function. Team leaders have tried to ensure that overlaps and redundancies are minimized, processes are optimized, and risk-based approaches are operationalized across the organization.
With capacity pressure rising, automation has become an increasingly urgent priority, supported by ever-more standardized workflows. Commonly cited benefits of automation include increased effectiveness (more consistency and rigor across activities) and greater efficiency (for example, freeing up of model validation capacity).
In Europe, the most automated process among survey respondents is ongoing monitoring and testing, particularly for models subject to frequent testing, followed by periodic validation testing. There is no particular variance across model types. Looking forward, the highest priority is automation of MRM workflows, followed by automation of validation, testing, and documentation. US banks are also focused on automation of MRM workflows, as well as managing validation frequency for some models. Many report they have acted to align validation depth with model tiers.
Still, many banks report that automation remains at an early stage, with automated testing and standardized codes used sporadically rather than over the whole model life cycle. Indeed, despite advancements over the past year, 50 percent of European banks have yet to commence automation across all model life cycle activities. No more than 30 percent of the group report automation being fully implemented in any single function. Among US banks, 60 percent are prioritizing automation of development and validation activities for models subject to frequent testing or documentation activities. Second in line are models sharing a similar methodology. Across banks, manual inputs are still most dominant in model documentation and initial validation documentation.
Effective automation is contingent on clear standards across model types. However, many banks are constrained by challenges in implementing tiering effectively, which is a precondition of setting the right standards for different levels of materiality. Looking ahead, the priority for many is to focus on the next level of automation, which is to put in place dedicated teams to drive efficiency. Time-consuming tasks such as documentation and reporting are high on executive agendas.
Enhanced standards
Many banks report that they have started the process of introducing more granulated standards for MRM, including drafting model-specific or tier-specific documents, prioritized by risk exposures, regulatory needs, and the potential for reputational damage. Among other efficiency initiatives, roundtable participants highlighted the migration to model life cycle digitization and the positive effects of cloud transformation programs, with efficiency benefits again seen as significant—or at least potentially so. However, these kinds of transformations also bring challenges. Life cycle digitization is tough in the absence of strong data processes (collection, quality, and management), the lack of which can undermine repeatability and reproducibility. In addition, the broad scope and continuous evolution of model families requires frequent adaptions, a task complicated by the common involvement of multiple stakeholders across functions and lines of defense as well as the need for senior stakeholder buy-in.
Roundtable participants agreed that the appropriate response to these challenges is to ramp up MRM team capabilities, with many now seeing this as a priority. Cloud migration was also commonly described as a potentially important enabler but was seen as contingent on high levels of standardization and process simplification. In addition, many banks said that they required a risk-based approach to tiering. With these building blocks in place, it may be possible to achieve an “automation leapfrog,” putting the program at the top of the strategic agenda and working to automate across the board.
Finally, in building out their digital capabilities, increasing numbers of banks reported moving toward an agile approach, characterized by short sprints, test-and-learn environments, and program flexibility. Key enablers for agile methodologies include a unified technology platform for data and systems and the use of advanced IT tools to capture model information on the fly. This may comprise proactively screening data warehouses and analytics platforms, telemetry, and alternative approaches to information capture. Once these are in place, perceived advantages include better consistency and reproduction in reporting, efficiency gains through a refocus on high-value activities, and faster turnarounds. Several banks said they are increasingly reliant on external data, which requires more management attention but can produce excellent analytical outcomes.
Empowering MRM in new areas
One driver of inventory expansion over the past two years has been the emergence of a range of new use cases, including relating to emerging risks relating to cyber, climate, and COVID-19, as well as the redevelopment of some categories to cater to structural market changes. The emergence of the new secured overnight financing rate is one example. Moreover, there is a consensus that the validation burden will rise over the next two years, amid higher levels of demand in areas such as climate and AI.
Against this backdrop, a large number of institutions have worked to recalibrate their organizational setups and have expanded mandates to widen the scope of MRM. There has been a wave of investment in new tool kits and validation approaches to support risk management activities. A common trend in the EU has been for banks to divide their MRM resources into two primary teams, with one focusing on regulatory models and the other tasked with the remainder. Another dominant trend has been to elevate oversight at senior levels. To that end, many banks have started to incorporate model risk in their broader assessments of risk appetite. Indeed, 81 percent of European banks have formulated a statement of risk appetite for model risk. This shows the growing importance that institutions attach to the subject, and a high level of C-suite engagement in setting tolerances.
Statements of risk appetite are often based on standard types of metrics. The most common are the quality of models, compliance with MRM policy, and risk capital add-ons. Banks commonly use a score card to put a number on risk and provide a benchmark for reporting. When it comes to model risk capital, many European banks report subsuming the cost under operational risk capital, with a sizeable minority assigning the budget to margin of conservation frameworks. One in four hold no specific capital against model risk.
To ensure effective oversight at all levels, the majority of banks in both the United States and Europe have centralized their MRM and validation functions (the United States for some time now)—albeit split into regulatory and nonregulatory capabilities. European banks report adding support through colocated teams or, less commonly, localized teams. A few have adopted a hybrid federated and localized approach to MRM. For model development, the vast majority of banks operate teams that are fragmented across business and model types. However, around 28 percent of European banks have set up single or multiple centers of excellence, for example in credit risk, market risk, and AI/machine learning.
Heads of MRM and validation often have different reporting lines, particularly in Europe, with validation heads seeing more variance than others. In the United States, MRM reports tend to be directly to the chief risk officer or another senior executive on risk committees. The challenge amid this mix of approaches, beyond meeting regulatory expectations, is to raise skill levels to match the diversity and range of models, and to ensure that validators become de facto “risk managers” in the way they approach their work.
Across most banks, MRM policies and standards are typically shared between the first and second line of defense (LoD) in about equal measure, with operations and technology teams taking responsibility for end user computing, alongside operational risk management. However, given the events of the past year, across geographies, a majority of MRM teams are planning to work closely with the first LoD to assess the impact of the COVID-19 pandemic on models and standards, with a focus on model performance-monitoring activities.
As banks develop their internal standards, they are aware that the regulatory burden is set to intensify. Many participants in the US roundtable highlighted recent discussions on the Office of the Comptroller of the Currency’s MRM Handbook and the interagency statement on MRM for Bank Secrecy Act and Anti-Money Laundering (BSA/AML) compliance. They noted that, in practice, the impact is likely to be largest on small and medium-size banks. Discussions highlighted continuing uncertainty about how to best meet supervisory expectations for governance of non-models tools.
Derisking and maximizing the potential of AI/ML and big data
One of the most exciting areas of innovation in modeling is in artificial intelligence, machine learning (ML), and deep learning, the development of which has enabled banks to ask more nuanced questions of much larger data sets. As a result, risk areas such as financial crime compliance and cyber have become much more amenable to interrogation. Sales and marketing has also been a key beneficiary, with banks able to analyze customer data to offer a more streamlined and tailored proposition.
As AI and ML have become core elements of the tool kit, many banks have worked to manage risks (data ethics, black box, biases) through enhanced model governance, validation frameworks, and more powerful knowledge capabilities, supported by training where appropriate. Many have built or acquired digital tools and infrastructure to ensure they maximize the value of advanced modeling techniques. Banks also reported a heightened regulatory focus, which reflects the extent of potential ethical and reputational risks associated with complex models.
Many roundtable participants said they had ramped up efforts to enhance AI/ML model definitions. However, a large number still need to tackle AI validation standards and tools, as well as the deficit in AI talent. In addition, there are practical challenges in the use of AI. For example, when it comes to financial crime compliance models, banks need to pay close attention to their obligations under regulations such as the European Union’s General Data Protection Regulation. Regulators are often not fully prepared for new models and lack the frameworks or responsible persons to make full assessments, bankers said. The same kinds of challenges are presented by climate models, where colleagues may have an intimate understanding of risk, but are not acquainted with the data and methodologies that support modeling. This implies the need for a more flexible approach to validation, so that development in these fast-evolving areas can proceed in parallel.
Often in AI use cases, a lack of historical data can inhibit comparability. Roundtable participants emphasized that validation approaches might initially require flexibility and pragmatism, reflecting the fact that the data in AI models and frequency of calibration are different to traditional models. Given the novelty of AI use cases, the flexibility enabled by agile approaches was seen as a good match.
The number of AI models varies across banks globally, with some using 60 or 70 (or as many as 290 at the margins) while others are comfortable with ten or 15. However, in Europe, 60 percent of banks plan to develop at least ten AI/ML models in the next two years, and 30 percent plan to validate at least ten models next year, our survey shows.
The emergence of AI and its use cases has brought a distinct set of challenges, described by some market participants as “cultural.” At a minimum, new use cases require an agile/interactive engagement model for the first and second LoD, and the early involvement of support functions such as legal and IT. Roundtable participants highlighted the advantages of a dedicated support structure and of securing strong sponsorship, which should be accompanied by a tailored risk-based approach to validation and review. In addition, bankers acknowledged the need to foster validation awareness among modeling teams, many of which are populated by data scientists who are unfamiliar with validation protocols. With that in mind, one task is to clearly define standards and expectations, supporting deeper collaboration between modeling teams in the nonregulatory space and validation.
One of the primary roles for MRM teams is to define explainability requirements for the first LoD, alongside working to enhance monitoring standards. At larger banks, MRM teams are often required to benchmark AI/ML models against simpler approaches. Among common initiatives, banks have updated their model definitions, provided new guidelines to the first LoD, and renewed their governance frameworks. Still, institutions report being at varying stages of planning and implementation. In addition, there are still distinctive gaps in coverage. For example, most banks do not have defined roles for assessment of bias.
Looking to the year ahead, bankers cited two key priorities: to enhance their MRM framework capabilities (including risk culture, standards, and procedures) and to upgrade their validation resources. “MRM 2.0” is firmly on the agenda, which for many banks will mean getting to the next level of reporting and KPIs, strengthening risk appetite frameworks, and embedding good governance and the right culture. Culture is seen to be particularly important at senior levels, so that decision makers fully understand the potential risks and impacts that models, and analytics in general, may bring. These efforts should be built on the three pillars of increased efficiency, supported by digitization, a more empowered MRM function, and advancements in the use of AI. In a world that continues to be defined by uncertainty, much work, but also much opportunity, lies ahead.
