Improper Handling of Mixed Encoding

Description

This vulnerability occurs when software fails to correctly process input that contains multiple character encodings within the same data stream.

Extended Description

Mixed encoding vulnerabilities happen when an application receives data that switches between different character sets—like UTF-8, UTF-16, ISO-8859-1, or others—without proper normalization. Attackers exploit this by injecting malicious payloads in one encoding that, when misinterpreted by the parser, bypass validation routines designed for a different encoding. This mismatch between what the security checks see and what the backend interpreter processes creates a dangerous gap in your defenses. To prevent this, developers should implement a strict input validation pipeline that first decodes all incoming data into a single, consistent internal character set (like UTF-8) before performing any security checks or processing. This normalization step ensures that validation logic and downstream components are all evaluating the same canonical representation of the data, eliminating the ambiguity that mixed encodings introduce.

Common Consequences 1

Scope: Integrity

Impact: Unexpected State

Potential Mitigations 4

Phase: Architecture and Design

Strategy: Input Validation

Avoid making decisions based on names of resources (e.g. files) if those resources can have alternate names.

Phase: Implementation

Strategy: Input Validation

Assume all input is malicious. Use an "accept known good" input validation strategy, i.e., use a list of acceptable inputs that strictly conform to specifications. Reject any input that does not strictly conform to specifications, or transform it into something that does. When performing input validation, consider all potentially relevant properties, including length, type of input, the full range of acceptable values, missing or extra inputs, syntax, consistency across related fields, and conformance to business rules. As an example of business rule logic, "boat" may be syntactically valid because it only contains alphanumeric characters, but it is not valid if the input is only expected to contain colors such as "red" or "blue." Do not rely exclusively on looking for malicious or malformed inputs. This is likely to miss at least one undesirable input, especially if the code's environment changes. This can give attackers enough room to bypass the intended validation. However, denylists can be useful for detecting potential attacks or determining which inputs are so malformed that they should be rejected outright.

Phase: Implementation

Strategy: Output Encoding

Use and specify an output encoding that can be handled by the downstream component that is reading the output. Common encodings include ISO-8859-1, UTF-7, and UTF-8. When an encoding is not specified, a downstream component may choose a different encoding, either by assuming a default encoding or automatically inferring which encoding is being used, which can be erroneous. When the encodings are inconsistent, the downstream component might treat some character or byte sequences as special, even if they are not special in the original encoding. Attackers might then be able to exploit this discrepancy and conduct injection attacks; they even might be able to bypass protection mechanisms that assume the original encoding is also being used by the downstream component.

Phase: Implementation

Strategy: Input Validation

Inputs should be decoded and canonicalized to the application's current internal representation before being validated (Incorrect Behavior Order: Validate Before Canonicalize). Make sure that the application does not decode the same input twice (Double Decoding of the Same Data). Such errors could be used to bypass allowlist validation schemes by introducing dangerous inputs after they have been checked.