JavaScript String Internals: UTF-16 Code Units, V8 Representation Topologies, and Memory Immutability

1. Executive Summary & Semantic Core

In JavaScript, a string is an immutable primitive sequence of 16-bit code units formatted under the UTF-16 encoding standard. Every character index and length calculation operates on 16-bit code units ($2^{16}$ range) rather than Unicode code points, causing surrogate pairs (e.g., emojis, complex ideographs) to register a .length of 2. Under the Google V8 engine, strings are represented not as raw flat arrays, but through advanced heap topologies—such as ConsStrings (concatenation trees), SlicedStrings (zero-copy substring views), and Internalized Strings (hash-consed global string tables). Understanding these underlying storage models is critical for avoiding memory bloat in high-throughput data processing and parser implementations.

2. Low-Level Memory Model: UTF-16, Code Units vs. Code Points

The ECMAScript specification mandates UTF-16 representation for all primitive strings. While characters within the Basic Multilingual Plane (BMP, code points U+0000 to U+FFFF) map 1:1 to a single 16-bit code unit, supplementary characters (such as emojis or rare historical glyphs, U+10000 to U+10FFFF) are encoded as surrogate pairs (two consecutive 16-bit units).

Unicode Code Point: U+1F680 (🚀 Rocket)
UTF-16 Representation: High Surrogate (0xD83D) + Low Surrogate (0xDE80)
Memory Allocated: 2 x 16-bit = 32 bits (4 bytes)
.length Evaluation: 2 (Reflects code units, NOT rendered characters)

const rocket = "🚀";

// The Code Unit Pitfall
console.log(rocket.length);           // 2 (Surrogate pair: 2 code units)
console.log(rocket.charAt(0));        // "\uD83D" (Malformed solitary surrogate)
console.log(rocket.charCodeAt(0));    // 55357 (0xD83D)

// Accurate Code Point Introspection
console.log(rocket.codePointAt(0));   // 128640 (0x1F680)

// Safe Unicode-Aware Iteration (Uses @@iterator to resolve full code points)
const characterArray = [...rocket];
console.log(characterArray.length);    // 1 (Accurately counted glyph)

The String Length & Indexing Matrix

3. V8 Engine Representation Topologies (ConsString vs. FlatString)

To prevent severe heap fragmentation and eliminate redundant $O(N)$ allocations during string operations, V8 uses specialized internal C++ structures on the managed heap:

V8 String Internal Topologies:
1. SeqOneByteString / SeqTwoByteString: Flat sequential character arrays in memory.
2. ConsString: Binary tree combining two strings (LHS + RHS) without copying data.
3. SlicedString: Pointer to a parent string with an offset and length (zero-copy view).
4. ThinString: Indirection pointer replacing a duplicate string with its internalized canonical symbol.

1. The ConsString Allocation Tree

When chaining strings via + or +=, V8 avoids copying the characters into a new flat buffer. Instead, it creates a ConsString - a lightweight tree node holding pointers to both substrings:

let packetLog = "HEADER;";
// Does NOT allocate a new 15-byte flat string; allocates a ConsString node pointing to both
packetLog += "PAYLOAD_CHUNK_1;"; 
packetLog += "CHECKSUM_VALID;";

Performance Guard: Reading characters via indexing (packetLog[i]) or passing the string into native C++ bindings forces V8 to flatten the ConsString tree into a contiguous SeqString. Excessive tree depth without flattening can introduce traversal stalls.

2. The SlicedString Memory Retention Trap

Calling str.slice() or str.substring() creates a SlicedString, which references the original string's buffer rather than duplicating characters:

let largeBuffer = "X".repeat(10_000_000); // 10MB string allocated on heap

function getShortHeader() {
  // SlicedString: Contains only { parent: largeBuffer, offset: 0, length: 10 }
  return largeBuffer.slice(0, 10); 
}

const header = getShortHeader();
// Retaining 'header' keeps the ENTIRE 10MB largeBuffer alive in heap memory,
// preventing V8 Garbage Collection!

// Production Fix: Force flattening / allocation detachment
const detachedHeader = (" " + header).slice(1);

4. Modern String Interpolation: Tagged Templates AST Interception

Template literals (enclosed in backticks) compute dynamic strings, while Tagged Templates allow intercepting the raw Abstract Syntax Tree (AST) tokens before string resolution. This enables SQL injection defense, safe HTML sanitization, and parameterized shell commands:

function safeSql(strings, ...values) {
  return strings.reduce((accumulator, currentString, index) => {
    const parameter = values[index - 1];
    // Sanitize parameter to eliminate SQL Injection vectors
    const sanitized = typeof parameter === "string" 
      ? `'${parameter.replace(/'/g, "''")}'` 
      : String(parameter);
    
    return accumulator + sanitized + currentString;
  });
}

const userInput = "admin' OR '1'='1";
const query = safeSql`SELECT * FROM users WHERE username = ${userInput};`;
// Safely compiled to: SELECT * FROM users WHERE username = 'admin'' OR ''1''=''1';

5. Defensive String Processing & Regex Invariants

• Escape Sequences & Control Characters: Escape sequences (\n, \t, \r) consume 1 code unit despite consisting of 2 source code characters.
• String Wrapper Object Equality: Never use new String("val"). Primitive strings use value identity; wrapper objects use heap reference addresses:

const primitiveStr = "status";
const wrapperA = new String("status");
const wrapperB = new String("status");

console.log(primitiveStr === wrapperA); // false (Type mismatch: string !== object)
console.log(wrapperA === wrapperB);     // false (Distinct heap pointers)

6. Enterprise Best Practices & ESLint Governance

• Prohibit Wrapper Constructors: Enforce no-new-wrappers in ESLint to block new String(), new Number(), and new Boolean().
• Normalize Multilingual Strings: Always execute .normalize("NFC") on user-submitted textual content before persisting to databases to resolve Unicode canonical equivalence mismatches (e.g., combining diacritics vs. single composite characters).
• Use Intl.Segmenter for Text Clipping: When truncating UI copy or enforcing length constraints across internationalized text, use Intl.Segmenter instead of naive .slice(0, N) to avoid splitting surrogate pairs or emojis in half.