The Dark Side of AI

by Makaela Rae Clapp, CPA, Senior Associate

Posted on September 2, 2025

Artificial Intelligence, more commonly referred to as AI, has taken multiple industries by storm, including the world of finance and accounting. Its capabilities continue to grow and impress us all – after all, who wouldn’t appreciate a little help analyzing large sets of data, summarizing an article, or writing a difficult email? Tasks that used to take hours can now be done within minutes, allowing you to spend more time on value-added work. Of course, these benefits are not without risks. Below are some key risks to keep in mind when utilizing AI in your work.

First, AI programs are not secure when it comes to personally identifiable information, or PII. What is PII? According to the US Department of Labor, personally identifiable information is defined as “information that can be used to distinguish or trace an individual’s identity, either alone or when combined with other information that is linked or linkable to a specific individual.” Your department or organization most likely already has policies regarding PII – who has access to it, how it can be securely transmitted or disposed of, whom it can be shared with, etc. You may be tempted to run your files through an AI program to gain some efficiencies. However, it is important to note that the way AI works is by learning from large datasets, and as such there is not guarantee of data privacy or security once you give files to an AI program.

Artificial intelligence is also not always correct! The goal is that AI can improve accuracy when it comes to accounting by eliminating human errors and biases. While these programs are getting smarter every day, you may still run across instances where the AI program produces incorrect or misleading information. Therefore, we should not fully rely on AI deliverables but rather utilize them as a resource to gain efficiencies and reduce tedious, time-consuming tasks. The responsibility still falls on you as the professional to review the output for errors, accuracy, and quality.

Lastly and most importantly, while AI technology can help detect fraud through the data analysis of outliers and the identification of unusual trends (as just a few examples), the evolution of this technology has also allowed fraud schemes to become more complex and easier to perpetrate. Fraudsters have exploited the capabilities of AI to:

  • Automate phishing scams
  • Hack user accounts on a substantial scale,
  • Impersonate CFOs or other management to trick them into sending money to scammers (referred to as “deepfakes”)
  • Steal personal information (another reason to consider PII!)
  • Identify behavioral patterns from individuals or software systems to exploit vulnerabilities in internal controls/existing technology controls. (Important to note how trend & pattern detection can be used both to detect and perpetrate fraud!)

This list is not comprehensive; there are numerous other schemes being created every day as machine learning becomes smarter.

The world of technology has always been rapidly changing, and it is often hard to keep up. This should not serve as a deterrent to using AI to assist you with your job duties, but rather, as always, exercise caution with new technology. To summarize, below are some key takeaways to consider when using AI:

  • Ensure that unnecessary personal information is removed from any files before utilizing AI software.
  • Review all outputs and deliverables for accuracy.
  • There is an increased risk of fraud, especially as AI continues to learn and bad actors exploit its capabilities. Crack down on internal controls and technology controls.
  • Implement an AI policy at your organization to address these risks. Only allow trusted personnel to utilize the software and limit access based on employees’ job functions.
  • Start slow when implementing AI into your day-to-day job duties.