Home / AI Abuse / Researchers tricked AI into creating malware

Researchers tricked AI into creating malware

March 21, 2025, 7:01 pm EDT

ZDNET: “Generative AI has stirred up as many conflicts as it has innovations — especially when it comes to security infrastructure.”

“Enterprise security provider Cato Networks says it has discovered a new way to manipulate AI chatbots. On Tuesday, the company published its 2025 Cato CTRL Threat Report, which showed how a researcher — who Cato clarifies had ‘no prior malware coding experience’ — was able to trick models, including DeepSeek R1 and V3, Microsoft Copilot, and OpenAI’s GPT-4o, into creating “fully functional” Chrome infostealers, or malware that steals saved login information from Chrome. This can include passwords, financial information, and other sensitive details.”

Topic: / /

Source:

Sign up to get the latest AI news roundup direct to your inbox.

    Recent News

    Google builds AI animal recognition tool

    April 1, 2025, 10:43 am EDT

    TCD: “A new artificial intelligence model could make it significantly easier for researchers to track and monitor wildlife across the globe.”

    “Google recently released SpeciesNet, an open-source AI that can be used to identify animal species. The AI analyzes photos from camera traps — digital cameras that are triggered when infrared sensors capture motion — and can quickly identify species from the photos.”

    Minecraft developer rejects AI

    March 31, 2025, 11:54 am EDT

    “It seems like you don’t need to worry about generative AI (artificial intelligence) worming its way into the hit open-world survival game Minecraft.” reports TechRadar.

    “According to IGN, Minecraft Vanilla game director Agnes Larsson discussed the issue of AI in game development at a recent event and wasn’t too enthusiastic about the prospect. ‘Here for us, just like Minecraft is about creativity and creating,’ they said.”

    It follows a similar trend recently noted at Nintendo.

    Zuckerberg approved illegally downloading material to train Llama

    March 31, 2025, 11:43 am EDT

    Futurism: “In January, Meta lost a huge fight with a group of authors who sued the company for using their books to train its AI. The case uncovered the fact that Meta had illegally downloaded an infamous pirate library, LibGen, to procure millions of legally protected texts. Those books were then fed to Meta’s LLM, Llama, after software engineers got approval from the Zuck himself. In other words, one of the largest companies in the world didn’t even bother to pay for a single copy of each book it used to build its AI.”

    AI is better than humans at making meme captions

    March 30, 2025, 9:30 pm EDT

    “A new study examining meme creation found that AI-generated meme captions on existing famous meme images scored higher on average for humor, creativity, and ‘shareability’ than those made by people. Even so, people still created the most exceptional individual examples.” reports Ars Technica.

    OpenAI researcher thinks AI could have arrived decades ago

    March 30, 2025, 7:20 pm EDT

    “Noam Brown, who leads AI reasoning research at OpenAI, says certain forms of ‘reasoning’ AI models could’ve arrived 20 years earlier had researchers ‘known [the right] approach’ and algorithms.” reports TechCrunch.

    “‘There were various reasons why this research direction was neglected,’ Brown said during a panel at Nvidia’s GTC conference in San Jose on Wednesday. ‘I noticed over the course of my research that, OK, there’s something missing. Humans spend a lot of time thinking before they act in a tough situation. Maybe this would be very useful [in AI].'”

    Fully AI-generated newspaper turns out to be horrific

    March 30, 2025, 7:02 am EDT

    “The image reeks of generative artificial intelligence. The warped, watercolor-esque style. Reuters spelled ‘Redutrs’ and El País with an accent over the wrong letter.” writes Alex Mahadevan, at Poynter.

    “It’s currently the main image of the Italian newspaper Il Foglio’s article about AI use in newsrooms — which was written with AI. The image was created with ChatGPT, according to the cutline, as were several others in a new insert in its daily editions called Foglio AI.”

    AI is ‘Alien Intelligence’ – Harvard Professor

    March 29, 2025, 9:06 am EDT

    Popular Mechanics: “The human brain is a marvel of neurons firing electrical impulses through complex networks of neural pathways. It can also be notoriously slow when it comes to figuring out new solutions for complicated tasks like designing new computer chips. That’s why a research team from Princeton University and the Indian Institute of Technology decided to hand the job over to artificial intelligence.”

    “What the AI came up with was almost alien.”

    Cloudflare’s AI Labyrinth diverts bots to nonsense honeypot

    March 29, 2025, 7:31 am EDT

    “Cloudflare, one of the biggest network internet infrastructure companies in the world, has announced AI Labyrinth, a new tool to fight web-crawling bots that scrape sites for AI training data without permission. The company says in a blog post that when it detects “inappropriate bot behavior,” the free, opt-in tool lures crawlers down a path of links to AI-generated decoy pages that ‘slow down, confuse, and waste the resources’ of those acting in bad faith.” reports The Verge.

    AI is pirating millions of books for training

    March 29, 2025, 6:00 am EDT

    The Atlantic: “When employees at Meta started developing their flagship AI model, Llama 3, they faced a simple ethical question. The program would need to be trained on a huge amount of high-quality writing to be competitive with products such as ChatGPT, and acquiring all of that text legally could take time. Should they just pirate it instead?”

    People are using Grok for fact-checking

    March 28, 2025, 6:06 pm EDT

    TechCrunch: “Some users on Elon Musk’s X are turning to Musk’s AI bot Grok for fact-checking, raising concerns among human fact-checkers that this could fuel misinformation.”

    UK tech secretary uses ChatGPT as an advisor

    March 28, 2025, 4:00 pm EDT

    “The UK’s technology secretary, Peter Kyle, has asked ChatGPT for advice on why the adoption of artificial intelligence is so slow in the UK business community – and which podcasts he should appear on.” New Scientist reports.

    Sweden’s government wants police to use AI face-recognition

    March 28, 2025, 3:46 pm EDT

    Reuters: “Swedish police should be allowed to use real-time, AI-powered face-recognition to combat crime, Sweden’s government proposed on Thursday, as it seeks new tools to stop sometimes violent offences rocking the Nordic country in recent years.”

    Studio Ghibli and fans disgusted by OpenAI’s new image generator

    March 28, 2025, 3:42 pm EDT

    Forbes: “The launch of OpenAI’s new image generator, powered by GPT-4o, has flooded the internet with Studio Ghibli-inspired images in a trend that, ironically, goes directly against the ethos of Studio Ghibli.”

    “Animator Hayao Miyazaki, co-founder of Japanese animation house Studio Ghibli, famously delivered a passionate condemnation of AI that is often quoted by critics of the technology.”

    NatWest teams up with OpenAI to improve customer service

    March 28, 2025, 3:18 pm EDT

    “NatWest (NWG.L), opens new tab and OpenAI have joined forces to enhance the lender’s digital assistants and customer support processes using artificial intelligence, in the first collaboration of its kind with a UK-headquartered bank.” reports Reuters.

    Nintendo wants to stay original by rejecting AI

    March 27, 2025, 3:09 pm EDT

    Creative Bloq: Amid the growing AI art trend, Nintendo stands apart. While other household names embrace the use of AI to enhance game development and improve user experience, Nintendo holds fast in its pursuit of player connection, creativity and originality. Spearheaded by the vision of Shigeru Miyamoto, this decision exemplifies Nintendo’s commitment to offering unique experiences in a rapidly evolving market. It’s a reason why Nintendo Switch 2 is one of the year’s most-anticipated releases.

    New game dev platform launches with AI testing

    March 27, 2025, 12:38 pm EDT

    “Razer is getting into AI with a new developer platform called Wyvrn. It encompasses dev-focused automation tools like the Razer AI QA Copilot to assist with quality assurance / game testing and the AI Gamer Copilot (formerly called Project Ava) for real-time tips and guides via voice assistant while playing games.” reports The Verge.

    Film to be dubbed entirely with AI voice and mouth movements

    March 27, 2025, 12:20 pm EDT

    “A foreign language sci-fi movie is headed to U.S. movie theaters this spring, but audiences won’t have to groan about subtitles. For the first time, an international feature film will look and sound as if it was made in English thanks to artificial intelligence.” reports Variety.

    Anthropic introduces web searching via Claude

    March 26, 2025, 12:17 pm EDT

    “Anthropic’s AI-powered chatbot, Claude, can now search the web — a capability that had long eluded it.” reports TechCrunch.

    “Web search is available now in preview for paid Claude users in the U.S., Anthropic said in its blog, with support for free users and additional countries coming soon.”

    New AI prediction model can replace weather supercomputers

    March 26, 2025, 9:08 am EDT

    Independent: Cambridge scientists have made a major breakthrough in weather forecasting after developing a new AI prediction model that is tens of times better than current systems.

    New Scientist: “Weather forecasting has, since the 1950s, relied on physics-based models that extrapolate from observations made using satellites, balloons and weather stations. But these calculations, known as numerical weather prediction (NWP), are extremely intensive and rely on vast, expensive and energy-hungry supercomputers.”

    Google’s AI Overviews will soon give health advice

    March 25, 2025, 8:49 pm EDT

    “Google’s crappy ‘AI Overviews’ regularly spits out dangerous and incorrect answers — and now it’s being entrusted with medical advice.” writes Futurism.

    “In a self-congratulatory blog post, Google’s chief health officer Karen DeSalvo claimed that ‘recent health-focused advancements on Gemini models’ and ‘our best-in-class quality and ranking systems’ will allow the janky feature to ‘cover thousands more health topics.'”

    Sign up to get the latest AI news roundup direct to your inbox.