Americans reported losing $665 million to phone call fraud in the first half of 2026, according to Federal Trade Commission data as of June, with nearly $128 million of that total attributed to scammers impersonating a business. As synthetic voice technology accelerates, financial institutions are urgently funding new authentication technologies to verify caller identity without creating friction for legitimate customers.
The Rising Threat of Artificial Intelligence in Phone Fraud
Voice scams have been around for decades, but deepfakes add believability and scalability by removing the staffing constraints that once limited scam operations. According to background data, Microsoft researchers demonstrated in 2023 that their VALL-E system could reproduce someone’s voice from a three-second recording, preserving characteristics such as emotion and the acoustic environment. Anyone who has posted a video or audio clip online may have left material that scammers can use to imitate their voice, and caller ID numbers can be easily spoofed.
The financial toll of these synthetic impersonations extends well beyond individual retail consumers. In 2024, a finance employee at the engineering firm Arup joined a video call with people he believed were his colleagues, including the chief financial officer, and authorized $25.6 million in transfers before Hong Kong police confirmed the other participants were deepfakes. Furthermore, research from Rutgers University in 2025 showed that artificial intelligence models could complete scam dialogues successfully between 43% and 74% of the time without human intervention by fielding unscripted questions and absorbing pushback.
The overlap between legitimate commerce and criminal capability is also widening. According to a 2026 Sinch survey of 2,527 decision-makers across 10 countries, nearly half of North American companies have started using artificial intelligence on their phone lines, further blurring the line between legitimate and fraudulent calls. Yet, familiarity with the technology offers no built-in immunity for users. Research from Harvard and Meta published in 2026 found that nearly a third of artificial intelligence voices went unrecognized in an experiment asking participants to judge recorded scam calls, and frequent artificial intelligence users did no better at spotting a synthetic caller than people who never use it.
How Speaker Verification Operates as a Gatekeeper
To understand the defense mechanisms currently being deployed, it helps to examine the underlying mechanics of voice technology. According to historical data on the field, speaker recognition has a history dating back some four decades as of 2019, using the acoustic features of speech that differ between individuals due to anatomy and learned behavioral patterns. Speaker recognition answers the question of who is speaking, differing from speech recognition or speaker diarisation.
Within security applications, a vital distinction exists between identification and verification. Speaker verification, also called speaker authentication, is a one-to-one match where one speaker’s voice is matched to a particular template, acting as a gatekeeper to provide access to secure systems with the user’s knowledge and cooperation. This contrasts with speaker identification, which is a one-to-many match where an unknown voice is compared against multiple templates. While identification can be run covertly, verification requires the speaker to claim an identity and use their voice to confirm it.
Adaptive Authentication and New Industry Standards
Financial institutions are seeking to strengthen security without adding friction to the customer experience through adaptive multi-factor authentication. Voice security fintech Illuma, founded in 2016, specializes in voice authentication solutions for credit unions and community banks. According to Crunchbase, Illuma has raised a total of $11.5 million in disclosed funding, bolstered by a recent investment from existing clients Financial Plus Credit Union, Dort Financial Credit Union, and CUSO One Washington Financial.
The company’s flagship offering, IllumaSHIELD, uses passive voice authentication to authenticate trusted account holders in seconds while helping credit unions and banks detect artificial intelligence voice fraud, deepfakes, and account takeover attempts in real time. According to statements from Illuma Founder and CEO Dr. Milind Borkar, the funding will advance deepfake detection, fraud prevention, and adaptive authentication capabilities. Clients have already reported operational improvements; Financial Plus Credit Union Senior Vice President and Chief Innovation Officer Jessica McNier stated that the technology has provided faster authentication, fewer escalations, and an improved ability to identify fraudulent callers since onboarding.
Forensic Applications and Speaker Identification
Beyond the gatekeeping role of verification, speaker recognition technology serves critical functions in forensic investigations. While verification is a one-to-one match used for secure system access, speaker identification is a one-to-many process used to determine an unknown speaker’s identity by comparing a voice against multiple templates. Unlike verification, which requires the user’s knowledge and cooperation, speaker identification systems can be implemented covertly to identify participants in a discussion or alert automated systems to speaker changes.
In forensic settings, professionals often utilize a multi-stage approach to analyze potential fraud. Investigators typically perform an initial speaker identification process to generate a list of “best matches” from a database. Following this, they conduct a series of verification processes to determine a conclusive match. By analyzing the similarities and differences between samples from the speaker and the identified best matches, both the prosecution and defense can use these findings as evidence to determine if a suspect is the actual offender.
Technical Gap Between Legitimate AI Adoption and Scam Capabilities
Despite the widespread adoption of AI in legitimate business operations, a significant technical gap exists between the capabilities of AI used for fraud and the AI deployed for security purposes. According to the 2026 Sinch survey, nearly half of North American companies have started using AI on their phone lines, yet this same technology is being weaponized by scammers with increasing sophistication. For example, Microsoft’s VALL-E system, demonstrated in 2023, can reproduce a voice from a three-second recording, preserving emotional and acoustic characteristics. However, the same level of precision and adaptability is not commonly found in the AI systems used for speaker verification or fraud detection, which often rely on older or less advanced models.
Research from Rutgers University in 2025 highlights that AI models can now handle complex scam dialogues, including unscripted questions and pushback, with success rates between 43% and 74%. In contrast, financial institutions are still relying on speaker verification systems that, according to historical data on the field, date back to the 1980s and use acoustic features that differ between individuals due to anatomy and learned behavior. This lag in technological advancement leaves a critical window of opportunity for scammers to exploit. As noted in the 2026 research from Harvard and Meta, even frequent users of AI do not perform significantly better at identifying synthetic voices, indicating that current security measures are not keeping pace with the evolving threat landscape.
Sources
- Businesses Are Embracing AI in Calls, and So Are Scammers. How Can You Trust a Voice on the Line?
- Speaker recognition
- Illuma Raises Funds to Bring Voice Security to Credit Unions – Finovate
Photo: Timur Seyfelmlyukov / Unsplash (Unsplash License)
