Why gen AI is a double-edged sword in cybersecurity

My daughter calls me out of the blue and tells me she needs money while she’s on a trip overseas. She adds that she’ll fill me in on the details later. Reacting as a parent, I just give in.

In this hypothetical scenario, my money has now been stolen by a fraudster posing as my child using a digital voice clone created with generative AI. You may be thinking, “That’s not going to fool me. I know what my own child sounds like.” But I assure you that if you are talking over the phone with one of these digital twins, you will not be able to distinguish the difference. That’s how sophisticated this spoofing technology has become.

This is just one example of the challenges consumers, businesses and governments are now facing as gen AI has quickly become mainstream. It’s creating new threats in the cybersecurity space, but also opportunities. On the one hand, gen AI has made creating and executing complex forms of fraud much cheaper and more accessible. On the other, companies, including Mastercard, are using gen AI to add more contextual understanding to our cybersecurity tools, making them smarter, improving the experience for consumers and fighting fraud faster than before.

How to fend off a digital twin

A digital twin can be a copy of your voice or a combination of your voice and a video of you. In both cases, fraudsters can train this twin to say whatever they want.

Creating these digital clones used to be very expensive. Today, someone can sign up for any number of gen AI-powered services online, upload a series of voice recordings or videos and create a digital twin virtually for free.

In other cases, a fraudster can use the text generation capabilities of gen AI to create highly targeted and personalized messages at scale to potentially reach many victims via a messaging app.

In these examples, fraudsters are now armed with powerful tools at almost no cost. These scams are also already happening. In a major incident in February, a finance worker in Hong Kong was duped into sending scammers $25 million after talking on a video conference call with digital twins of several co-workers, including the company’s chief financial officer.

The serious implication here is that this is an unfair fight for us humans. We can’t learn or adapt as quickly as an AI, no matter how hard we try.

Going back to that scenario with my daughter, there’s an easy way to address this deception. We have a family password that we can use for these types of situations. A digital twin wouldn’t know that information. Even without a shared password, you could ask a potential digital twin to confirm information that it also wouldn’t know — like a recent restaurant you went to with the person you’re talking to.

Suddenly, the entire facade of a digital twin starts to crumble, no matter how clever it is or how quickly it can evolve its practices. This is how we can avoid trying to run the same race as an AI. Instead, let’s lean in to being more human.

Sometimes, a non-high-tech solution might work just fine to fight such fraud.

How we use gen AI to strengthen cybersecurity

Gen AI, of course, isn’t only being used by fraudsters. And low-tech solutions aren’t always the answer. Mastercard’s teams today are using gen AI and AI tools to improve the security of the digital world.

In another example, we can build more sophisticated fraud models with the help of AI, too. Today, many fraud models are built using your buying habits and actions. That means that if you typically spend $100 a day around Philadelphia, a fraud model may flag if you suddenly tried to purchase a $2,000 item in the Philippines.

We can use AI models to understand spending habits more broadly — not just at the individual level.  Let’s say you never gamble, but a gambling expense is flagged by your bank’s fraud system. Looking at a broader range of data, an AI system can help us identify that many people who stay at a given casino resort do gamble — and since your transactions show you’re at that resort, this gambling expense is no longer viewed as suspicious, avoiding the need to send an alert or suspending your card. That creates a better experience for you.

In just a few years, gen AI has changed how we interact with technology, write code and it’s even caused us to question whether what we’re hearing or seeing is actually real — even a call from my own daughter.

The story about gen AI is complex — and the risks are clear. But I know we can direct it in the right ways so people and businesses can benefit from the new wave of innovation it’s creating.

Banner image via Adobe Stock and created using generative AI.