SUPERVISORY GUIDANCE ON MODEL RISK MANAGEMENT CONTENTS I . PDF

15d ago
0 Views
0 Downloads
243.79 KB
18 Pages
Transcription

Federal Deposit Insurance CorporationSUPERVISORY GUIDANCE ONMODEL RISK MANAGEMENTCONTENTSI. Introduction.1II. Purpose and Scope.2III. Overview of Model Risk Management.2IV. Model Development, Implementation, and Use.4V. Model Validation.7VI. Governance, Policies, and Controls.14VII. Conclusion.18I. INTRODUCTIONBanks rely heavily on quantitative analysis and models in most aspects of financial decision making.1They routinely use models for a broad range of activities, including underwriting credits; valuingexposures, instruments, and positions; measuring risk; managing and safeguarding client assets;determining capital and reserve adequacy; and many other activities. In recent years, banks haveapplied models to more complex products and with more ambitious scope, such as enterprise-widerisk measurement, while the markets in which they are used have also broadened and changed.Changes in regulation have spurred some of the recent developments, particularly the U.S. regulatorycapital rules for market, credit, and operational risk based on the framework developed by the BaselCommittee on Banking Supervision. Even apart from these regulatory considerations, however,banks have been increasing the use of data-driven, quantitative decision-making tools for a numberof years.The expanding use of models in all aspects of banking reflects the extent to which models canimprove business decisions, but models also come with costs. There is the direct cost of devotingresources to develop and implement models properly. There are also the potential indirect costs ofrelying on models, such as the possible adverse consequences (including financial loss) of decisionsbased on models that are incorrect or misused. Those consequences should be addressed by activemanagement of model risk.This guidance describes the key aspects of effective model risk management. Section II explains thepurpose and scope of the guidance, and Section III gives an overview of model risk management.1Unless otherwise indicated, banks refers to state non-member banks, state savings associations, and all otherinstitutions for which the Federal Deposit Insurance Corporation is the primary supervisor. It is not expected thatthis guidance will pertain to FDIC-supervised institutions with under 1 billion in total assets unless the institution’smodel use is significant, complex, or poses elevated risk to the institution.1

Section IV discusses robust model development, implementation, and use. Section V describes thecomponents of an effective validation framework. Section VI explains the salient features of soundgovernance, policies, and controls over model development, implementation, use, and validation.Section VII concludes.II. PURPOSE AND SCOPEThe purpose of this document is to provide comprehensive guidance for banks on effective modelrisk management. Rigorous model validation plays a critical role in model risk management;however, sound development, implementation, and use of models are also vital elements.Furthermore, model risk management encompasses governance and control mechanisms such asboard and senior management oversight, policies and procedures, controls and compliance, and anappropriate incentive and organizational structure.Previous guidance and other publications issued by the FDIC on the use of models address aspects ofmodel risk management for specific types of models or pay particular attention to model validation.2Based on supervisory and industry experience over the past several years, this document expands onexisting guidance—most importantly by broadening the scope to include all aspects of model riskmanagement. Many banks may already have in place a large portion of these practices, but banksshould ensure that internal policies and procedures are consistent with the risk managementprinciples and supervisory expectations contained in this guidance. Details may vary from bank tobank, as practical application of this guidance should be customized to be commensurate with abank’s risk exposures, its business activities, and the complexity and extent of its model use. Forexample, steps taken to apply this guidance at banks using relatively few models of only moderatecomplexity might be significantly less involved than those at a bank where use of models is moreextensive or complex.III. OVERVIEW OF MODEL RISK MANAGEMENTFor the purposes of this document, the term model refers to a quantitative method, system, orapproach that applies statistical, economic, financial, or mathematical theories, techniques, andassumptions to process input data into quantitative estimates. A model consists of three components:an information input component, which delivers assumptions and data to the model; a processingcomponent, which transforms inputs into estimates; and a reporting component, which translates theestimates into useful business information. Models meeting this definition might be used foranalyzing business strategies, informing business decisions, identifying and measuring risks, valuingexposures, instruments or positions, conducting stress testing, assessing adequacy of capital,managing client assets, measuring compliance with internal limits, maintaining the formal controlapparatus of the bank, or meeting financial or regulatory reporting requirements and issuing public2For instance, the FDIC has addressed aspects of model risk management in guidance related to different activities;see Joint Agency Policy Statement on Interest Rate Risk (FIL-52-96), FFIEC Advisory on Interest Rate RiskManagement (FIL-2-2010), Interagency Advisory on Interest Rate Risk Management Frequently Asked Questions(FIL-2-2012), FDIC’s Credit Card Activities Manual dit card/),and Supervisory Guidance on Implementing Dodd-Frank Act Company-Run Stress Tests for Banking OrganizationsWith Total Consolidated Assets of More Than 10 Billion but Less Than 50 Billion (79 FR 14153). In addition,the advanced-approaches risk-based capital rules (12 CFR 325, Appendix D) contain explicit validationrequirements for subject banking organizations.2

disclosures. The definition of model also covers quantitative approaches whose inputs are partially orwholly qualitative or based on expert judgment, provided that the output is quantitative in nature.3Models are simplified representations of real-world relationships among observed characteristics,values, and events. Simplification is inevitable, due to the inherent complexity of those relationships,but also intentional, to focus attention on particular aspects considered to be most important for agiven model application. Model quality can be measured in many ways: precision, accuracy,discriminatory power, robustness, stability, and reliability, to name a few. Models are never perfect,and the appropriate metrics of quality, and the effort that should be put into improving quality,depend on the situation. For example, precision and accuracy are relevant for models that forecastfuture values, while discriminatory power applies to models that rank order risks. In all situations, itis important to understand a model's capabilities and limitations given its simplifications andassumptions.The use of models invariably presents model risk, which is the potential for adverse consequencesfrom decisions based on incorrect or misused model outputs and reports. Model risk can lead tofinancial loss, poor business and strategic decision making, or damage to a bank’s reputation. Modelrisk occurs primarily for two reasons: The model may have fundamental errors and may produce inaccurate outputs when viewedagainst the design objective and intended business uses. The mathematical calculation andquantification exercise underlying any model generally involves application of theory, choiceof sample design and numerical routines, selection of inputs and estimation, andimplementation in information systems. Errors can occur at any point from design throughimplementation. In addition, shortcuts, simplifications, or approximations used to managecomplicated problems could compromise the integrity and reliability of outputs from thosecalculations. Finally, the quality of model outputs depends on the quality of input data andassumptions, and errors in inputs or incorrect assumptions will lead to inaccurate outputs.The model may be used incorrectly or inappropriately. Even a fundamentally sound modelproducing accurate outputs consistent with the design objective of the model may exhibithigh model risk if it is misapplied or misused. Models by their nature are simplifications ofreality, and real-world events may prove those simplifications inappropriate. This is evenmore of a concern if a model is used outside the environment for which it was designed.Banks may do this intentionally as they apply existing models to new products or markets, orinadvertently as market conditions or customer behavior changes. Decision makers need tounderstand the limitations of a model to avoid using it in ways that are not consistent with theoriginal intent. Limitations come in part from weaknesses in the model due to its variousshortcomings, approximations, and uncertainties. Limitations are also a consequence ofassumptions underlying a model that may restrict the scope to a limited set of specificcircumstances and situations.Model risk should be managed like other types of risk. Banks should identify the sources of risk andassess the magnitude. Model risk increases with greater model complexity, higher uncertainty aboutinputs and assumptions, broader use, and larger potential impact. Banks should consider risk fromindividual models and in the aggregate. Aggregate model risk is affected by interaction anddependencies among models; reliance on common assumptions, data, or methodologies; and any3While outside the scope of this guidance, more qualitative approaches used by banking organizations—i.e., thosenot defined as models according to this guidance—should also be subject to a rigorous control process.3

other factors that could adversely affect several models and their outputs at the same time. With anunderstanding of the source and magnitude of model risk in place, the next step is to manage itproperly.A guiding principle for managing model risk is "effective challenge" of models, that is, criticalanalysis by objective, informed parties who can identify model limitations and assumptions andproduce appropriate changes. Effective challenge depends on a combination of incentives,competence, and influence. Incentives to provide effective challenge to models are stronger whenthere is greater separation of that challenge from the model development process and when challengeis supported by well-designed compensation practices and corporate culture. Competence is a key toeffectiveness since technical knowledge and modeling skills are necessary to conduct appropriateanalysis and critique. Finally, challenge may fail to be effective without the influence to ensure thatactions are taken to address model issues. Such influence comes from a combination of explicitauthority, stature within the organization, and commitment and support from higher levels ofmanagement.Even with skilled modeling and robust validation, model risk cannot be eliminated, so other toolsshould be used to manage model risk effectively. Among these are establishing limits on model use,monitoring model performance, adjusting or revising models over time, and supplementing modelresults with other analysis and information. Informed conservatism, in either the inputs or the designof a model or through explicit adjustments to outputs, can be an effective tool, though not an excuseto avoid improving models.As is generally the case with other risks, materiality is an important consideration in model riskmanagement. If at some banks the use of models is less pervasive and has less impact on theirfinancial condition, then those banks may not need as complex an approach to model riskmanagement in order to meet supervisory expectations. However, where models and model outputhave a material impact on business decisions, including decisions related to risk management andcapital and liquidity planning, and where model failure would have a particularly harmful impact ona bank’s financial condition, a bank’s model risk management framework should be more extensiveand rigorous.Model risk management begins with robust model development, implementation, and use. Anotheressential element is a sound model validation process. A third element is governance, which sets aneffective framework with defined roles and responsibilities for clear communication of modellimitations and assumptions, as well as the authority to restrict model usage. The following sectionsof this document cover each of these elements.IV. MODEL DEVELOPMENT, IMPLEMENTATION, AND USEModel risk management should include disciplined and knowledgeable development andimplementation processes that are consistent with the situation and goals of the model user and withbank policy. Model development is not a straightforward or routine technical process. The experienceand judgment of developers, as much as their technical knowledge, greatly influence the appropriateselection of inputs and processing components. The training and experience of developers exercisingsuch judgment affects the extent of model risk. Moreover, the modeling exercise is often amultidisciplinary activity drawing on economics, finance, statistics, mathematics, and other fields.Models are employed in real-world markets and events and therefore should be tailored for specific4

applications and informed by business uses. In addition, a considerable amount of subjectivejudgment is exercised at various stages of model development, implementation, use, and validation.It is important for decision makers to recognize that this subjectivity elevates the importance ofsound and comprehensive model risk management processes.4Model Development and ImplementationAn effective development process begins with a clear statement of purpose to ensure that modeldevelopment is aligned with the intended use. The design, theory, and logic underlying the modelshould be well documented and generally supported by published research and sound industrypractice. The model methodologies and processing components that implement the theory, includingthe mathematical specification

Banks rely heavily on quantitative analysis and models in most aspects of financial decision making.1 ... quantitative decision-making tools for a number ... model risk management for specific types of models or pay particular attention to model validation.2