Improper Handling of Alternate Encoding

Description

This vulnerability occurs when software fails to correctly process input that arrives in a different character encoding than expected, allowing that input to bypass security checks or cause misinterpretation.

Extended Description

Attackers can exploit encoding mismatches by crafting inputs in alternate formats like UTF-7, UTF-16, or other multi-byte encodings. When security filters or validation logic only check for common encodings like UTF-8 or ASCII, these specially encoded payloads can slip through undetected. This often leads to successful injection attacks, as the malicious content is decoded later in a different component that interprets it as active code or commands. To prevent this, developers must normalize all incoming data to a single, known encoding immediately upon entry into the application's trust boundary. Security checks must be performed after this normalization step, not before. Additionally, consistently specifying and enforcing character encoding standards across all system interfaces—including databases, web requests, and file processing—closes the gaps where interpretation differences create security risks.

Common Consequences 1

Scope: Access Control

Impact: Bypass Protection Mechanism

Potential Mitigations 4

Phase: Architecture and Design

Strategy: Input Validation

Avoid making decisions based on names of resources (e.g. files) if those resources can have alternate names.

Phase: Implementation

Strategy: Input Validation

Assume all input is malicious. Use an "accept known good" input validation strategy, i.e., use a list of acceptable inputs that strictly conform to specifications. Reject any input that does not strictly conform to specifications, or transform it into something that does. When performing input validation, consider all potentially relevant properties, including length, type of input, the full range of acceptable values, missing or extra inputs, syntax, consistency across related fields, and conformance to business rules. As an example of business rule logic, "boat" may be syntactically valid because it only contains alphanumeric characters, but it is not valid if the input is only expected to contain colors such as "red" or "blue." Do not rely exclusively on looking for malicious or malformed inputs. This is likely to miss at least one undesirable input, especially if the code's environment changes. This can give attackers enough room to bypass the intended validation. However, denylists can be useful for detecting potential attacks or determining which inputs are so malformed that they should be rejected outright.

Phase: Implementation

Strategy: Output Encoding

Use and specify an output encoding that can be handled by the downstream component that is reading the output. Common encodings include ISO-8859-1, UTF-7, and UTF-8. When an encoding is not specified, a downstream component may choose a different encoding, either by assuming a default encoding or automatically inferring which encoding is being used, which can be erroneous. When the encodings are inconsistent, the downstream component might treat some character or byte sequences as special, even if they are not special in the original encoding. Attackers might then be able to exploit this discrepancy and conduct injection attacks; they even might be able to bypass protection mechanisms that assume the original encoding is also being used by the downstream component.

Phase: Implementation

Strategy: Input Validation

Inputs should be decoded and canonicalized to the application's current internal representation before being validated (Incorrect Behavior Order: Validate Before Canonicalize). Make sure that the application does not decode the same input twice (Double Decoding of the Same Data). Such errors could be used to bypass allowlist validation schemes by introducing dangerous inputs after they have been checked.