Death by a Thousand Prompts: Open Model Vulnerability Analysis

[ad_1] AI fashions have turn out to be more and more democratized, and the proliferation and adoption of open weight fashions has contributed considerably to this actuality. Open-weight fashions present researchers, builders, and AI fans with a strong basis for limitless use circumstances and purposes. As of August 2025, main U.S., Chinese language, and European fashions have round 400M whole downloads on HuggingFace. With an abundance of selection within the open weight mannequin ecosystem and the flexibility to fine-tune open fashions for particular purposes, it's extra vital than ever to grasp what precisely you’re getting with an open-weight mannequin—together with its safety posture.

Cisco AI Protection safety researchers performed a comparative AI safety evaluation of eight open-weight giant language fashions (LLMs), revealing profound susceptibility to adversarial manipulation, significantly in multi-turn situations the place success charges had been noticed to be 2x to 10x larger than single-turn assaults. Utilizing Cisco’s AI Validation platform, which performs automated algorithmic vulnerability testing, we evaluated fashions from Alibaba (Qwen3-32B), DeepSeek (v3.1), Google (Gemma 3-1B-IT), Meta (Llama 3.3-70B-Instruct), Microsoft (Phi-4), Mistral (Massive-2 also called Massive-Instruct-2047), OpenAI (GPT-OSS-20b), and Zhipu AI (GLM 4.5-Air).

Beneath, we’ll present an summary of our mannequin safety evaluation, evaluate findings, and share the complete report which gives a whole breakdown of our evaluation.

Evaluating Open-Supply Mannequin Safety For this report, we used AI Validation, which is a part of our full AI Protection resolution that performs automated, algorithmic assessments of a mannequin’s security and safety vulnerabilities. This report highlights particular failures reminiscent of susceptibility to jailbreaks. tracked by MITRE ATLAS and OWASP as AML.T0054 and LLM01:2025 respectively. The chance evaluation was carried out as a black field engagement the place the main points of the appliance structure, design, and current guardrails, if any, weren't disclosed previous to testing.

Throughout all fashions, multi-turn jailbreak assaults, the place we leveraged quite a few strategies to steer a mannequin to output disallowed content material, proved extremely efficient, with assault success charges reaching 92.78 p.c. The sharp rise between single-turn and multi-turn vulnerability underscores the dearth of mechanisms inside fashions to take care of and implement security and safety guardrails throughout longer dialogues.

These findings affirm that multi-turn assaults stay a dominant and unsolved sample in AI safety. This might translate into real-world threats, together with dangers of delicate information exfiltration, content material manipulation resulting in compromise of integrity of knowledge and data, moral breaches by way of biased outputs, and even operational disruptions in built-in techniques like chatbots or decision-support instruments. As an illustration, in enterprise settings, such vulnerabilities may allow unauthorized entry to proprietary data, whereas in public-facing purposes, they may facilitate the unfold of dangerous content material at scale.

We infer, from our assessments and evaluation of AI labs technical reviews, that alignment methods and mannequin provenance could issue into fashions’ resilience in opposition to jailbreaks. For instance, fashions that concentrate on capabilities (e.g., Llama) did exhibit the best multi-turn gaps, with Meta explaining that builders are “within the driver seat to tailor security for his or her use case” in post-training. Fashions that targeted closely on alignment (e.g., Google Gemma-3-1B-IT) did exhibit a extra balanced profile between single- and multi-turn methods deployed in opposition to it, indicating a deal with “rigorous security protocols” and “low threat stage” for misuse.

Open-weight fashions, reminiscent of those we examined, present a strong basis that, when mixed with malicious fine-tuning methods, could doubtlessly introduce harmful AI purposes that bypass commonplace security and safety measures. We don't discourage the continued funding and improvement into open-source and open-weight fashions. Moderately, we concurrently encourage AI labs that launch open-weight fashions to take measures to forestall customers from fine-tuning the safety away, whereas additionally encouraging organizations to grasp what AI labs prioritize of their mannequin improvement (reminiscent of robust security baselines versus capability-first baselines) earlier than they select a mannequin for fine-tuning and deployment.

To counter the chance of adopting or deploying unsafe or insecure fashions, organizations should think about adopting superior AI safety options. This consists of adversarial coaching to bolster mannequin robustness, specialised defenses in opposition to multi-turn exploits (e.g., context-aware guardrails), real-time monitoring for anomalous interactions, and common red-teaming workout routines. By prioritizing these measures, stakeholders can remodel open-weight fashions from liability-prone property into safe, dependable elements for manufacturing environments, fostering innovation with out compromising safety or security.

Comparative vulnerability evaluation displaying assault success charges throughout examined fashions for each single-turn and multi-turn situations.

Findings As we analyzed the info that emerged from our analysis of those open-source fashions, we appeared for key menace patterns, mannequin behaviors, and implications for real-world deployments. Key findings included:

Multi-turn Assaults Stay the Major Failure Mode: All fashions demonstrated excessive susceptibility to multi-turn assaults, with success charges starting from 25.86% (Google Gemma-3-1B-IT) to 92.78% (Mistral Massive-2), representing as much as a 10x enhance over single-turn baselines. See Desk 1 beneath:

Alignment Method Drives Safety Gaps: Safety gaps had been predominantly constructive, indicating heightened multi-turn dangers (e.g., +73.48% for Alibaba Qwen3-32B and +70% for Mistral Massive-2 and Meta Llama 3.3-70B-Instruct). Fashions that exhibited smaller gaps could exhibit each weaker single-turn protection however stronger multi-turn protection. We infer that the safety gaps stem from alignment method to open-weight fashions: labs reminiscent of Meta and Alibaba targeted on capabilities and purposes deferred to builders so as to add further security and safety insurance policies, whereas lab with a stronger safety and security posture reminiscent of Google and OpenAI exhibited extra conservative gaps between single- and multi-turn methods. Regardless, given the variation of single- and multi-turn assault method success charges throughout fashions, end-users ought to think about dangers holistically throughout assault methods. Menace Class Patterns and Sub-threat Focus: Excessive-risk menace courses reminiscent of manipulation, misinformation, and malicious code era, exhibited persistently elevated success charges, with model-specific weaknesses; multi-turn assaults reveal class variations and clear vulnerability profiles. See Desk 2 beneath for a way completely different fashions carried out in opposition to varied multi-turn methods. The highest 15 sub-threats demonstrated extraordinarily excessive success charges and are price prioritization for defensive mitigation. <img loading="lazy" decoding="async" class="aligncenter wp-image-481263 " src="https://blogs.cisco.com/gcs/ciscoblogs/1/2025/11/Screenshot-2025-11-05-at-6.02.24 PM-1.png" alt="" width="1219" height="411" srcset="https://blogs.cisco.com/gcs/ciscoblogs/1/2025/11/Screenshot-2025-11-05-at-6.02.24 PM-1-300x101.png 300w, https://blogs.cisco.com/gcs/ciscoblogs/1/2025/11/Screenshot-2025-11-05-at-6.02.24 PM-1-1024x345.png 1024w, https://blogs.cisco.com/gcs/ciscoblogs/1/2025/11/Screenshot-2025-11-05-at-6.02.24 PM-1-768x259.png 768w, https://blogs.cisco.com/gcs/ciscoblogs/1/2025/11/Screenshot-2025-11-05-at-6.02.24 PM-1-1536x518.png 1536w, https://blogs.cisco.com/gcs/ciscoblogs/1/2025/11/Screenshot-2025-11-05-at-6.02.24 PM-1.png 1852w" sizes="auto, (max-width: 1219px) 100vw, 1219px"/> Assault Methods and Methods: Sure methods and multi-turn methods achieved excessive success and every mannequin’s resistance different; the collection of completely different assault methods and methods have the potential to critically affect outcomes. Total Implications: The two-10x superiority of multi-turn assaults in opposition to the mannequin’s guardrails calls for fast safety enhancements to mitigate manufacturing dangers.

The outcomes in opposition to GPT-OSS-20b, for instance, aligned carefully with OpenAI’s personal evaluations: the general assault success charges for the mannequin had been comparatively low, however the charges had been roughly in keeping with the “Jailbreak analysis” part of the GPT-OSS mannequin card paper the place refusals ranged from 0.960 and 0.982 for GPT-OSS-20b. This end result underscores the continued susceptibility of frontier fashions to adversarial assaults.

An AI lab’s purpose in growing a particular mannequin might also affect evaluation outcomes. For instance, Qwen’s instruction tuning tends to prioritize helpfulness and breadth, which attackers can exploit by reframing their prompts as “for analysis,” “fictional situations”, therefore, the next multi-turn assault success charge. Meta, however, tends to ship open weights with the expectation the builders add their very own moderation and security layers. Whereas baseline alignment is nice (indicated by a modest single-turn charge), with none further security and safety guardrails (e.g., retaining security insurance policies throughout conversations or classes or tool-based moderation reminiscent of filtering, refusal fashions), multi-turn jailbreaks also can escalate shortly. Open-weight centric labs reminiscent of Mistral and Meta usually ship capability-first bases with lighter built-in security options. These are interesting for analysis and customization, however they push defenses onto the deployer. Finish-users who're in search of open-weights fashions to deploy ought to think about what elements of a mannequin they prioritize (security and safety alignment versus high-capability open weights with fewer safeguards).

Builders also can fine-tune open-weight fashions to be extra strong to jailbreaks and different adversarial assaults, although we're additionally conscious that nefarious actors can conversely fine-tune the open-weight fashions for malicious functions. Some mannequin builders, reminiscent of Google, OpenAI, Meta, Microsoft, have famous of their technical reviews and mannequin playing cards that they took steps to scale back the chance of malicious fine-tuning, whereas others, reminiscent of Alibaba, DeepSeek, and Mistral, didn't acknowledge security or safety of their technical reviews. Zhipu evaluated GLM-4.5 in opposition to security benchmarks and famous robust efficiency throughout some classes, whereas recognizing “room for enchancment” in others. On account of inconsistent security and safety requirements throughout the open-weight mannequin panorama, there are attendant safety, operational, technical, and moral dangers that stakeholders (from end-users to builders to organizations and enterprises that undertake these use) should think about when both adopting or utilizing these open-weight fashions. An emphasis on security and safety, from improvement to analysis to launch, ought to stay a high precedence amongst AI builders and AI practitioners.