Encoding Error

Description

This vulnerability occurs when software incorrectly transforms data between different formats, leading to corrupted or misinterpreted information that can break functionality or create security gaps.

Extended Description

Encoding errors happen when a system fails to properly convert data from one character set or format to another, such as between UTF-8, ASCII, or URL encoding. This often surfaces when handling user input, transferring data across systems, or preparing information for display. The result is 'mojibake' (garbled text), lost data, or unexpected characters that can crash applications, corrupt data storage, or bypass validation checks. For developers, the core issue is assuming data will always be in a single, expected format. To prevent this, explicitly define and validate character encodings at every system boundary—when reading input, sending output, or storing data. Use standardized, well-tested libraries for all encoding/decoding operations instead of custom logic, and consistently apply these transformations across your entire application stack to maintain data integrity.

Common Consequences 1

Scope: Integrity

Impact: Unexpected State

Potential Mitigations 3

Phase: Implementation

Strategy: Input Validation

Assume all input is malicious. Use an "accept known good" input validation strategy, i.e., use a list of acceptable inputs that strictly conform to specifications. Reject any input that does not strictly conform to specifications, or transform it into something that does. When performing input validation, consider all potentially relevant properties, including length, type of input, the full range of acceptable values, missing or extra inputs, syntax, consistency across related fields, and conformance to business rules. As an example of business rule logic, "boat" may be syntactically valid because it only contains alphanumeric characters, but it is not valid if the input is only expected to contain colors such as "red" or "blue." Do not rely exclusively on looking for malicious or malformed inputs. This is likely to miss at least one undesirable input, especially if the code's environment changes. This can give attackers enough room to bypass the intended validation. However, denylists can be useful for detecting potential attacks or determining which inputs are so malformed that they should be rejected outright.

Phase: Implementation

Strategy: Output Encoding

While it is risky to use dynamically-generated query strings, code, or commands that mix control and data together, sometimes it may be unavoidable. Properly quote arguments and escape any special characters within those arguments. The most conservative approach is to escape or filter all characters that do not pass an extremely strict allowlist (such as everything that is not alphanumeric or white space). If some special characters are still needed, such as white space, wrap each argument in quotes after the escaping/filtering step. Be careful of argument injection (Improper Neutralization of Argument Delimiters in a Command ('Argument Injection')).

Phase: Implementation

Strategy: Input Validation

Inputs should be decoded and canonicalized to the application's current internal representation before being validated (Incorrect Behavior Order: Validate Before Canonicalize). Make sure that the application does not decode the same input twice (Double Decoding of the Same Data). Such errors could be used to bypass allowlist validation schemes by introducing dangerous inputs after they have been checked.

Observed Examples 4

CVE-2004-1315Forum software improperly URL decodes the highlight parameter when extracting text to highlight, which allows remote attackers to execute arbitrary PHP code by double-encoding the highlight value so that special characters are inserted into the result.

CVE-2004-1939XSS protection mechanism attempts to remove "/" that could be used to close tags, but it can be bypassed using double encoded slashes (%252F)

CVE-2001-0709Server allows a remote attacker to obtain source code of ASP files via a URL encoded with Unicode.

CVE-2005-2256Hex-encoded path traversal variants - "%2e%2e", "%2e%2e%2f", "%5c%2e%2e"

Applicable Platforms

Languages:

Not Language-Specific : Undetermined

Modes of Introduction

Implementation

Related Attack Patterns

Related Weaknesses

ChildOf:

Improper Neutralization (CWE-707)

CanPrecede:

Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal') (CWE-22)

CanPrecede:

Improper Resolution of Path Equivalence (CWE-41)

Taxonomy Mapping

PLOVER

Notes

RelationshipPartially overlaps path traversal and equivalence weaknesses.

MaintenanceThis is more like a category than a weakness.

MaintenanceMany other types of encodings should be listed in this category.