The Dark Side of Generative AI: When Security Meets Surveillance A recent report from Cisco's Talos intelligence group has highlighted the disturbing trend of hackers exploiting top of the line generative AI models to develop malware, automate cyberattacks, and hunt for software vulnerabilities.
The findings, based on an analysis of prompt histories and chat logs accidentally exposed online by hackers, reveal how threat actors are circumventing safety guardrails designed to protect these tools.
Hackers have been using simple social engineering tactics to bypass model restrictions. They claim to be participating in authorized hacking competitions or assert administrative permission to perform tasks.