The Ghost in the Machine

Our code is telling stories. One of these is about how it will behave at runtime.

As your codebase grows, it can be hard to keep track of the trail where changes can have subtle side-effects and a pattern emerges:

We make a change, we push, tests pass, reviews clear up and yet it fails when it finally runs in production.

At Unfault we don’t see code as just a collection of files, we treat it as a web of execution paths.

As developers, we walk our code continuously. Sometimes this is more akin to a hike. Even more often, it feels like a hike without a map.

So if we are struggling to orient ourselves in the code, how can we ensure the execution path we are working on is going to be fine?

Copyright DC Studio

Introducing the fault add-on for Unfault

Unfault sees your codebase as a graph and uses this as a foundation to give developers tooling they can use to explore these execution paths for impact when they misbehave.

• Trace downstream impacts: See exactly which functions or services will be affected by a change in a data contract.
• Identify critical paths: Pinpoint the specific network calls that, if compromised, would degrade the user experience.

$ unfault graph
Workspace: waiter
Nodes: 3 functions, 0 classes, 3 routes, 1 remote servers
Edges: 0 calls, 2 http calls, 0 imports
Listens on: 8080

Entry points:
  GET    /health
           └─ health
  POST   /orders
           └─ create_order
               └─ → POST   localhost:8081/orders (kitchen)
  GET    /kitchen/status
           └─ kitchen_status
               └─ → GET    localhost:8081/status (kitchen)

Graph ready. Run 'unfault ask' to explore.

This is where the fault add-on comes in. Because unfault already understands the network routes and service boundaries within your graph, we can use that metadata to inject network faults directly into those paths.

$ $(unfault addon fault plan -q "latency 200ms for 30s" --target $KITCHEN_API_URL)

    Welcome to fault. - Your Reliability Toolbox!

    Enabled Proxies:
     - 0.0.0.0:9090 => 127.0.0.1:8000 [tcp]

    Plugins:
     No plugins provided.

    Configured Faults:
     - Latency: per read/write: false, side: Server, direction: Ingress, distribution: Uniform, min: 200ms, max: 200ms

    Faults Schedule:
        Latency: ███████████████████████████████████████████████████████

    Status:
     ⠙ Progress: ------------------🐢-------------------------------- 🏁
     Elapsed 00:00:10  | Remaining 66.66% | Total 30 seconds
     ⠏ Waiting for incoming traffic...

In the next article, we will demonstrate how to use fault to move from “seeing the links” to “testing the resilience” of a specific communication path.