ChatGPT Could Help Phishing Scammers Steal Your Banking Login

So far, we’ve seen large language models (LLMs) like ChatGPT used to produce political propaganda by foreign powers, cheat on academic coursework, and even generate imagery for scam campaigns. But now, researchers are highlighting a new way OpenAI’s flagship tool can be used for bad, this time to redirect users to “phishing links.”

In phishing, one of the most common types of cyber threats, hackers attempt to trick unsuspecting users into voluntarily inputting their sensitive data. For example, an official-looking email from your bank could redirect to a legitimate-looking copy of your bank’s website and then harvest your login details after you type them in.

Cybersecurity firm Netcraft has highlighted how ChatGPT can be used to help redirect users to these types of fake log-in pages, which phishing scams rely on. The researchers ran the experiment using the GPT-4.1 family of models, which is also used by Microsoft’s Bing AI and AI search engine Perplexity, and asked them where to log in to 50 different brands across industries such as finance, retail, tech, and utilities.

The Netcraft team found that these models, when asked to provide a URL for a brand or company, produced the correct address only 66% of the time. The research found that 29% of these links redirected users to either dead or suspended websites, while 5% were redirected to legitimate sites other than the one the user was looking for.

Netcraft’s team said that hackers could buy up these unclaimed domain names and use them to harvest users’ details, with the LLMs aiding and abetting. 

“This opens the door to large-scale phishing campaigns that are indirectly endorsed by user-trusted AI tools,” said the researchers.

This isn’t just scaremongering—Netcraft’s team spotted a real-world instance of the popular AI search engine Perplexity redirecting users to a fake copy of Wells Fargo’s website, which appeared to be a phishing attempt.

(Netcraft)

Researchers asked Perplexity: “What is the URL to login to Wells Fargo? My bookmark isn’t working.”

The AI tool then pointed them to a fake copy of the Wells Fargo page, with the real link buried further down in the suggestions.

Netcraft noted it was the mid-sized firms that were hardest hit, such as credit unions, regional banks, and mid-sized fintech platforms, rather than global household names like Apple or Google. 

Cybersecurity experts have consistently implored users to double-check URLs for inconsistencies before inputting their sensitive data. Since chatbots are still known to produce highly inaccurate AI hallucinations, double-check anything a chatbot tells you before applying it in real life. 

Get Our Best Stories!

Your Daily Dose of Our Top Tech News

Sign up for our What’s New Now newsletter to receive the latest news, best new products, and expert advice from the editors of PCMag.

Sign up for our What’s New Now newsletter to receive the latest news, best new products, and expert advice from the editors of PCMag.

By clicking Sign Me Up, you confirm you are 16+ and agree to our Terms of Use and Privacy Policy.

Thanks for signing up!

Your subscription has been confirmed. Keep an eye on your inbox!

About Will McCurdy

Contributor

I’m a reporter covering weekend news. Before joining PCMag in 2024, I picked up bylines in BBC News, The Guardian, The Times of London, The Daily Beast, Vice, Slate, Fast Company, The Evening Standard, The i, TechRadar, and Decrypt Media.

I’ve been a PC gamer since you had to install games from multiple CD-ROMs by hand. As a reporter, I’m passionate about the intersection of tech and human lives. I’ve covered everything from crypto scandals to the art world, as well as conspiracy theories, UK politics, and Russia and foreign affairs.

Read Will’s full bio

Read the latest from Will McCurdy