The Center for Audit Quality (CAQ) has published Audit Committee Oversight in the Age of Generative AI, a resource for audit committee members that provides an overview of the use of generative AI (genAI) in financial reporting processes and internal control over financial reporting (ICFR). The paper includes questions audit committees can ask to understand management’s approach to using genAI in financial reporting and the related risks. The CAQ notes that “As the use of genAI in financial reporting processes and ICFR presents new risks and considerations for companies, audit committees will have an important oversight role to play.”
The CAQ’s publication discusses three topics – an overview of genAI, the impact of genAI on financial reporting and ICFR, and the AI regulatory environment.
Overview of GenAI
GenAI is one category of the broader field of artificial intelligence. The CAQ provides this taxonomy:
Artificial intelligence (AI) broadly refers to machines that mimic human cognitive abilities. AI includes capabilities such as natural language processing, problem-solving, pattern recognition, anomaly identification, and decision-making.
Machine learning is a subset of AI that uses algorithms to learn from, and make predictions or decisions based on, data. Machine learning algorithms are designed to learn and improve from experience.
Deep learning is a subset of machine learning that uses algorithms that roughly approximate the structure and capabilities of the human brain and enable the technology to handle complex tasks.
GenAI refers to a subset of deep learning based on probabilistic technology that can create content, including text, images, audio, or video, when prompted by a user. GenAI creates responses using algorithms that are often trained on open-source information, such as text and images from the internet.
Other points in the CAQ’s explanation of genAI Include:
Predictive technology: Gen AI technologies are trained on large datasets and learn patterns, structures, and representations from the training data. GenAI technologies make “predictions of the next character, word, phrase, pixel, etc. to formulate a probable response to the user prompt.” Asking genAI the same question several times may produce different answers.
Foundation models and customization. Foundation models are large language models that can be adapted to a range of uses. Companies can build their own customizations on top of foundation models, such as by training the model with the company’s data.
The ”black box” concept. The process by which genAI derives output is not readily “explainable or interpretable” because of the inherent complexity of AI algorithms and the nonlinearity of the relationships between the underlying data and the outputs. The importance of being able to explain or interpret output depends on the use.
Impact of GenAI on Financial Reporting and ICFR
GenAI can streamline some financial reporting activities, such as those that involve drafting, summarizing data, and working with unstructured data. Further, genAI can identify trends, patterns, and anomalies in data that would be difficult for humans to uncover. While the use of genAI in financial reporting can make employees more efficient, “humans continue to be involved to oversee, understand, and evaluate the relevance and reliability of the outputs from genAI technology”.
The CAQ lists five sources of risk that the audit committee should consider in understanding and overseeing management’s use of genAI in financial reporting. For each risk topic, the CAQ suggests questions the committee may want to discuss with management and the auditor. Below is a summary of each risk and examples of suggested questions.
Governance. Establishing strong oversight and governance of the use of genAI is foundational to successfully deploying genAI technologies. Oversight/governance questions the audit committee might ask include:
For management: Does the company have the requisite expertise to select, develop, deploy, and monitor genAI technologies? Will management need to engage third parties to select, develop, deploy, and monitor genAI technologies? What are the company’s objectives and related success criteria for deploying genAI technologies? Are genAI technologies intended to augment or automate existing processes?
For the auditor: What risks has the auditor identified based on how the company has deployed genAI technologies? How will the auditor address such risks in the audit? Has the auditor identified any deficiencies or lack of internal controls to mitigate against risks related to the company’s use of genAI technologies that fall within the scope of the audit?
Data Privacy and Security. Privacy and security risks depend on how genAI is used. Publicly available genAI may track and save user prompts and user data inputs. These inputs may be used to train the model and could affect the responses provided to other users. GenAI technologies may also be susceptible to cyber-attacks that could impact the reliability of outputs or put confidential company data at risk. Data privacy/security questions the audit committee might ask include:
For management: Does the company use a public instance of genAI technologies or a private instance? How does management consider cybersecurity risks when selecting or developing genAI technologies? Has the company performed a cybersecurity risk assessment for genAI technologies to evaluate threats and safeguards?
For the auditor: Has the auditor identified any risks related to data privacy or security of genAI technologies that are relevant to the audit?
Selection and Design of GenAI Technologies. The audit committee should understand where genAI is deployed in the financial reporting process and why management selected those aspects of the process for genAI support. The committee should also understand how management determined whether to build or buy the genAI technologies it is employing and how it determined the technology’s capabilities. Questions the audit committee might ask include:
For management: How does management design genAI technologies, including determining which genAI technologies to use (such as selecting an existing genAI technology, using a foundation model with added customizations, or developing the company’s own model) and the data needed for those technologies?
For the auditor: How does the company’s use of a foundation model or development of its own model impact the auditor’s risk assessment?
Deploying and Monitoring GenAI Technologies. The audit committee should understand how management tests genAI technology prior to deployment, how it determines the appropriate level of human involvement, and how management monitors the effectiveness of genAI technologies. Questions the audit committee might ask include:
For management: How does the company test genAI technologies prior to deployment to determine that they operate as designed? Does the company measure, track, and communicate performance metrics related to the functioning of the genAI technologies, including the precision of the technology? How has the company trained employees about genAI technologies?
Fraud. The use of genAI can increase fraud risk. For example, employees could use genAI to create documentation for fraudulent transactions or third parties could use genAI to create deepfake videos or audio files to convince company employees to provide them with money or confidential information. Fraud risk questions the audit committee might ask include:
For management: What fraud risks associated with the use of genAI technologies has management identified and how have they been addressed?
For the auditor: Has the auditor identified any fraud risks related to the company’s use of genAI technologies? How has the auditor addressed such risks in the audit? Has the auditor identified any deficiencies in or lack of internal controls to mitigate against fraud risks arising from the company’s use of genAI technologies?
Regulatory Environment
Both existing regulations governing technology and data protection and new regulations to mitigate security and safety risks may apply to the company’s use of AI. The audit committee should understand how management monitors, evaluates, and complies with applicable laws and regulations. Questions the audit committee might ask include:
For management: Has management considered any new compliance or regulatory risks that are introduced by the use of genAI technologies? How has management addressed such risks?
For the auditor: Has the auditor identified any risks of material misstatement related to noncompliance with laws and regulations related to the use of genAI?
Audit Committee Takeaways
Oversight of the use of genAI in financial reporting processes and ICFR will be a new challenge for many audit committees. The CAQ’s paper is an excellent primer on genAI and the risk areas that audit committees should consider. The questions it suggests committees discuss with the management and the auditor would be a good starting point for committees seeking to educate themselves on how genAI will impact financial reporting and on the new risks that will arise as it becomes ubiquitous.
Comments