The Art of the Con, Now Available at Scale

I spend a lot of time talking with colleagues about how we counter social engineering. I'm continuously fascinated (and slightly horrified) by how it evolves every 5–10 years, and by the lengths people will go to manipulate the behaviour of others for financial or intelligence gain.

Inspired by the recent attack against Binance, I thought it'd be fun to go down a quick rabbit hole into the history of social engineering in the digital era.

Of course, long before computers existed, humans were manipulating each other for gain. Pretending to be someone they're not, exploiting trust and authority, and convincing people to let them into places they really shouldn't be.

One of the earlier chapters of this in the digital world was phone phreaking in the 1960s and 70s. People discovered they could recreate signalling tones to manipulate telephone networks and make free long-distance calls. Not exactly an exploitation of human trust, but phone phreakers also became pretty adept at manipulating the system to gain information and access.

The 90s then brought us the Spice Girls, blue eyeshadow and computer-based social engineering.

Kevin Mitnick became famous for combining social engineering with technical intrusion, tricking employees into handing over passwords, access codes and sensitive information. He helped demonstrate something the security industry still grapples with today: sometimes it's much easier (and quieter) to convince someone to open the door than to find a way to break through it.

Then we hit the 2000s, which brought an explosion of e-commerce, email and enterprise connectivity. And with that came an enormous opportunity to industrialise social engineering. Phishing became a scalable business. Attackers could impersonate banks, businesses and trusted organisations, convincing thousands of people to disclose credentials, open malicious attachments or visit fraudulent websites. What once required individual effort could now be executed at scale.

Following the phish, things started to get a little more sophisticated with the spear phish.

The 2010s saw spear phishing and business email compromise (BEC) become increasingly prominent. Attackers researched their victims, impersonated executives and trusted colleagues, and exploited our inherent trust in hierarchy, familiarity and authority. More effort, but the rewards were much bigger. This certainly hasn't died off either, with BEC seemingly experiencing a resurgence at scale.

Anyway, that brings us to today.

Generative AI is enabling attackers to produce incredibly convincing, highly personalised messages, impersonate voices and faces, and automate interactions that previously required significant time and human effort. And these aren't just one-off attempts. They're being packaged into campaigns, run at scale and hyper-targeted at the people in companies who have direct access to the crown jewels.

The recent Binance attack is the most recent example of how far this can go, and likely a sign of the new normal. According to Binance's initial findings, attackers used fake recruitment interviews to build trust with an engineer, eventually convincing them to download a malicious coding assignment. That compromise became part of a much larger attack against the company's wallet infrastructure.

What I found particularly interesting during this quick dive is that the fundamental techniques haven't really changed all that much. But the scale, sophistication and resources required to execute them have.

AI is creating a step change in the economics of social engineering. Highly personalised, carefully researched and convincing campaigns that once required significant resources can increasingly be executed by almost anyone with an internet connection and enough tokens. And the breadth of potential victims is expanding with it.

Whenever I read about these incidents, I always wonder how the damage could've been avoided. With the might of technology being thrown at developers lately, what are we doing to protect both them and the organisations they work for from the damage that can follow? (In this case, unfortunately, to the tune of US$6.3 billion in affected assets.)

Do we give them more cyber awareness training? Restrict their access? Introduce more verification steps?

Those things matter, but perhaps we also need to spend more time designing security around the assumption that, eventually, someone will be successfully manipulated.

Rather than putting all of our eggs in the basket of prevention, we put some in the basket of "if it happens, it won't be so bad". Otherwise known as the basket of minimised consequence.

Looking ahead, though, I'm not sure I'm asking the same question anymore: What does social engineering look like in a world where humans are no longer the only interface for access?

What happens when the target of manipulation is an AI agent with access to systems, information and the authority to act on our behalf?

What if, just like humans in the pre-digital era, agents pretend to be someone or something they're not, exploit trust and authority, and convince others to let them into places they really shouldn't be?

We're doing some research on this at the moment. We'll let you know how it goes. 😉