📄 Paper Walkthrough

Fletch: File-System Metadata Caching in Programmable Switches Qingxiu Liu, Jiazhen Cai, Siyuan Sheng, Yuhui Chen, Lu Tang, Zhirong Shen, Patrick P. C. Lee arXiv:2510.08351v3 · May 2026 · 17 pages Code: https://github.com/adslabcuhk/fletch

TL;DR

Fletch moves HDFS metadata caching from clients into the programmable switch’s data plane, compiles it to a Tofino chip via P4, and serves metadata requests directly on the network’s critical path. It drops cache-coherence complexity from O(N) to O(1) and uses in-switch centralization to absorb hot requests and fix load imbalance on the backend.

Why Fletch

• 67–96% of filesystem requests in Baidu AI Cloud are metadata operations. In Yahoo’s HDFS clusters, 3% of files serve 34–39% of requests — extreme access skew.
• Three pain points of client-side caching:
  1. Consistency maintenance is O(N): every write must notify all clients.
  2. Client caches are siloed and can’t cooperate to fix load imbalance.
  3. Existing schemes (IndexFS, InfiniFS) only cache directory permissions; attribute reads still go to servers.
• Programmable switches flip the picture: they sit on every client↔server path, so centralization fixes both problems at once — O(1) consistency and natural hot-spot absorption.

Three Core Techniques

Multi-level Locking: P4 Pseudocode

The following is P4-style pseudocode derived from §V. P4_16 has no loops or recursion; multi-level path resolution is implemented via packet recirculation (one level per pass).

Data structures

// === Lock counter arrays: 8 banks, 64K slots × 16-bit each ===
register lock_array_1 { Width: 16; Size: 65536; }  // level 1: /a
register lock_array_2 { Width: 16; Size: 65536; }  // level 2: /a/b
register lock_array_3 { Width: 16; Size: 65536; }  // level 3: /a/b/c
// ... 7 banks; level 8+ share lock_array_8

// === Validation flags: 1 bit per cached path ===
register validation { Width: 1; Size: <hash_space>; }

// === Cache values ===
register cache_value_lo { Width: 32; Size: <num_slots>; }
register cache_value_hi { Width: 32; Size: <num_slots>; }

Read path (single-path reads: open / stat)

Each level: ① take lock → ② validate → ③ permission check → ④ release previous level’s lock → ⑤ either advance via recirculation or finish.

action process_level() {
    lvl = cur_level;
    idx = hash_low16_at_level(lvl);  // from PHV, precomputed by client

    valid = validation.read(idx);
    if (valid == 1) {
        lock_val = lock_array_X.read(idx);
        lock_array_X.write(idx, lock_val + 1);   // read lock: counter++

        meta = read_cache(idx);
        if (!permission_check(meta)) {
            failed = 1; fail_level = lvl;
        }
    } else {
        failed = 1; fail_level = lvl;
    }

    if (lvl > 1) release_lock(lvl - 1);

    if (cur_level < depth && !failed) {
        cur_level = lvl + 1;
        recirculate();
    } else {
        release_lock(lvl);
        if (!failed) send_response_to_client();
        else        forward_to_server();
    }
}

Write path (single-path writes: chmod / chown)

action process_write() {
    idx = hash_low16_at_level(depth_of_target);
    cur = lock_array_X.read(idx);
    if (cur > 0) { recirculate(); return; }  // spin-wait
    validation.write(idx, 0);                  // exclusive
    forward_to_server();                       // write-through
}

action on_write_response() {
    if (success) update_cache(idx, new_metadata);
    validation.write(idx, 1);
    // sequence number protocol prevents ACK loss causing double decrement
    send_ack_to_server();
}

Writer starvation (paper-acknowledged limitation): the design is reader-preferring. While a write recirculates waiting for the counter to drop to zero, new reads keep incrementing it, so the writer can starve under continuous read load.

Multi-path writes (chmod -R): top-down ordering

The parent’s validation flag stays at 0 until every cached descendant has been updated. Any in-flight reads blocked by validation=0 are forwarded to the server, so they never see a mix of “parent updated, children not yet”.

How “loops” map to P4

Key Numbers (128 simulated servers)

• Latency (read-heavy @ 0.26 MOPS): Fletch cuts NoCache’s avg / p95 / p99 by 64.6% / 89.8% / 87.7%.
• Switch resources: SRAM 58.4% / Stage 100% / ALU 98% / PHV 93% — slightly more than NetCache / FarReach, but within budget.
• Recovery: server 0.5–2.1 s / controller < 40 ms / switch 14.6–75.5 s (slowest, replays admission per path).

Paper-acknowledged limitations

• Writer starvation under reader-preferring locking
• Write-heavy workloads degrade beyond ~50% chmod
• PHV at 93% leaves little headroom for new features
• Strong assumption: root is permanently cached, root permissions immutable
• Multi-switch topology is not addressed — only a single Tofino is evaluated

My take

The idea is right, the implementation is complete, and the evaluation is solid. Two engineering red flags worth noting:

1. Writer starvation is a real production risk — any workload with non-trivial chmod / rename traffic could stall.
2. PHV at 93% is uncomfortably close to the hardware ceiling; any future feature hits the wall.

References

• Paper: https://arxiv.org/abs/2510.08351
• Code: https://github.com/adslabcuhk/fletch