CWE-1434 Base Draft

Insecure Setting of Generative AI/ML Model Inference Parameters

This vulnerability occurs when a generative AI or ML model is deployed with inference parameters that are too permissive, causing it to frequently generate incorrect, nonsensical, or unpredictable…

Definition

What is CWE-1434?

This vulnerability occurs when a generative AI or ML model is deployed with inference parameters that are too permissive, causing it to frequently generate incorrect, nonsensical, or unpredictable outputs.

Generative AI models, like those for text or image creation, use settings such as temperature, Top P, and Top K to control their creativity and decision-making. Setting these values incorrectly—often too high—forces the model to make overly random guesses, leading to 'hallucinations,' incoherent content, or wildly unrealistic results. When these flawed outputs are used in decision-making processes, data pipelines, or user-facing features, they can corrupt data integrity and create serious reliability issues. For developers, securing AI components means rigorously validating and constraining these inference parameters in production, similar to sanitizing user input. This requires testing across diverse inputs to find safe thresholds. Managing these configurations at scale across multiple models is challenging; an ASPM platform like Plexicus can help by automatically detecting insecure AI parameter settings and tracking these flaws alongside traditional vulnerabilities in your application stack.

Auswirkungen in der Praxis

Real-world CVEs caused by CWE-1434

Bisher sind in MITREs Katalog keine öffentlichen CVE-Referenzen mit dieser CWE verknüpft.

Wie Angreifer es ausnutzen

Angreiferpfad Schritt für Schritt

1
Assume the product offers an LLM-based AI coding assistant to help users to write code as part of an Integrated Development Environment (IDE). Assume the model has been trained on real-world code, and the model behaves normally under its default settings. Suppose there is a default temperature of 1, with a range of temperature values from 0 (most deterministic) to 2. Consider the following configuration.
2
The problem is that the configuration contains a temperature hyperparameter that is higher than the default. This significantly increases the likelihood that the LLM will suggest a package that did not exist at training time, a behavior sometimes referred to as "package hallucination." Note that other possible behaviors could arise from higher temperature, not just package hallucination. An adversary could anticipate which package names could be generated and create a malicious package. For example, it has been observed that the same LLM might hallucinate the same package regularly. Any code that is generated by the LLM, when run by the user, would download and execute the malicious package. This is similar to typosquatting. The risk could be reduced by lowering the temperature so that it reduces the unpredictable outputs and has a better chance of staying more in line with the training data. If the temperature is set too low, then some of the power of the model will be lost, and it may be less capable of producing solutions for rarely-encountered problems that are not reflected in the training data. However, if the temperature is not set low enough, the risk of hallucinating package names may still be too high. Unfortunately, the "best" temperature cannot be determined a priori, and sufficient empirical testing is needed.
3
In addition to more restrictive temperature settings, consider adding guardrails that test that independently verify any referenced package to ensure that it exists, is not obsolete, and comes from a trusted party. Note that reducing temperature does not entirely eliminate the risk of package hallucination. Even with very low temperatures or other settings, there is still a small chance that a non-existent package name will be generated.

Verwundbares Codebeispiel

Vulnerable JSON

Assume the product offers an LLM-based AI coding assistant to help users to write code as part of an Integrated Development Environment (IDE). Assume the model has been trained on real-world code, and the model behaves normally under its default settings. Suppose there is a default temperature of 1, with a range of temperature values from 0 (most deterministic) to 2. Consider the following configuration.

Verwundbar JSON

{

```
   "model": "my-coding-model",
   "context_window": 8192,
   "max_output_tokens": 4096,
   "temperature", 1.5,
   ...
 }

Sicheres Codebeispiel

Secure JSON

The problem is that the configuration contains a temperature hyperparameter that is higher than the default. This significantly increases the likelihood that the LLM will suggest a package that did not exist at training time, a behavior sometimes referred to as "package hallucination." Note that other possible behaviors could arise from higher temperature, not just package hallucination. An adversary could anticipate which package names could be generated and create a malicious package. For example, it has been observed that the same LLM might hallucinate the same package regularly. Any code that is generated by the LLM, when run by the user, would download and execute the malicious package. This is similar to typosquatting. The risk could be reduced by lowering the temperature so that it reduces the unpredictable outputs and has a better chance of staying more in line with the training data. If the temperature is set too low, then some of the power of the model will be lost, and it may be less capable of producing solutions for rarely-encountered problems that are not reflected in the training data. However, if the temperature is not set low enough, the risk of hallucinating package names may still be too high. Unfortunately, the "best" temperature cannot be determined a priori, and sufficient empirical testing is needed.

Sicher JSON

{

```
   ...
   "temperature", 0.2,
   ...
 }

What changed: the unsafe sink is replaced (or the input is validated/escaped) so the same payload no longer triggers the weakness.

Präventions-Checkliste

How to prevent CWE-1434

Implementation / System Configuration / Operation Develop and adhere to robust parameter tuning processes that include extensive testing and validation.
Implementation / System Configuration / Operation Implement feedback mechanisms to continuously assess and adjust model performance.
Documentation Provide comprehensive documentation and guidelines for parameter settings to ensure consistent and accurate model behavior.

Erkennungssignale

How to detect CWE-1434

Automated Dynamic Analysis Moderate

Manipulate inference parameters and perform comparative evaluation to assess the impact of selected values. Build a suite of systems using targeted tools that detect problems such as prompt injection (CWE-1427) and other problems. Consider statistically measuring token distribution to see if it is consistent with expected results.

Manual Dynamic Analysis Moderate

Plexicus Auto-Fix

Plexicus erkennt CWE-1434 automatisch und öffnet in unter 60 Sekunden einen Fix-PR.

Codex Remedium scannt jeden Commit, identifiziert genau diese Schwachstelle und liefert einen reviewer-ready Pull Request mit dem Patch. Keine Tickets. Keine Hand-offs.

Demo anfordern Plexicus kostenlos testen

Häufig gestellte Fragen

Frequently asked questions

Was ist CWE-1434?

Wie gravierend ist CWE-1434?

MITRE hat für diese Schwachstelle keine Exploit-Wahrscheinlichkeit veröffentlicht. Behandle sie als mittlere Auswirkung, bis dein Threat Model anderes belegt.

Welche Sprachen oder Plattformen sind von CWE-1434 betroffen?

MITRE lists the following affected platforms: Not Architecture-Specific, AI/ML, Not Technology-Specific.

Wie kann ich CWE-1434 verhindern?

Develop and adhere to robust parameter tuning processes that include extensive testing and validation. Implement feedback mechanisms to continuously assess and adjust model performance.

Wie erkennt und behebt Plexicus CWE-1434?

Die SAST-Engine von Plexicus erkennt die Datenfluss-Signatur von CWE-1434 bei jedem Commit. Bei einem Treffer öffnet unser Codex-Remedium-Agent einen Fix-PR mit korrigiertem Code, Tests und einer einzeiligen Zusammenfassung für den Reviewer.

Wo erfahre ich mehr über CWE-1434?

MITRE veröffentlicht die kanonische Definition unter https://cwe.mitre.org/data/definitions/1434.html. Für ergänzende Hinweise kannst du auch die OWASP- und NIST-Dokumentation heranziehen.

Verwandte Schwachstellen

Weaknesses related to CWE-1434

CWE-440 Parent

Expected Behavior Violation

This weakness occurs when a software component, such as a function, API, or feature, fails to act as documented or intended. The system's…

CWE-684 Kann vorausgehen

Incorrect Provision of Specified Functionality

This weakness occurs when software behaves differently than its documented specifications, which can mislead users and create security…

CWE-691 Peer

Insufficient Control Flow Management

This vulnerability occurs when a program's execution flow isn't properly managed, allowing attackers to bypass critical checks, trigger…

Ressourcen

Schluss mit dem Bezahlen pro Entwickler.
Schließ den Kreislauf.

Plexicus ist die KI-native ASPM, die scannt, filtert, fixt, pentestet und erklärt — autonom. Unbegrenzte Entwickler, unbegrenzte Repos, Fair-Use-KI-Aktionen. Echter kostenloser Tarif, €269/mo jährlich, wenn du bereit bist.

Kostenlos starten Demo buchen