Vix.cpp v2.7.0 is here Read the blog
Vix.cpp/Docs
Vix.cpp

Appearance

Cache and Run History ​

The agent module can store local cache entries and local run history inside the workspace. These two features serve different purposes. Cache helps repeated requests avoid unnecessary model calls when a response is safe to reuse. Run history makes agent execution easier to inspect after the program or command has finished.

Both features are local. They belong to the workspace selected for the run, and they are controlled by the agent configuration. This keeps the behavior predictable for developer tools, CLI commands, and local applications.

Use the public Vix AI aggregator in normal application code:

cpp
#include <vix/ai.hpp>

For code that works directly with the lower-level runtime, custom providers, or run storage:

cpp
#include <vix/ai/agent/AgentRuntime.hpp>

For examples that print output:

cpp
#include <vix/print.hpp>

Local agent data ​

By default, the agent stores local data under the workspace:

text
.vix/agent/memory
.vix/agent/cache
.vix/agent/runs
.vix/agent/logs

This layout keeps agent data close to the project that produced it. A developer can inspect the run history of a project without searching through global state, and repeated local work can benefit from cache without mixing unrelated projects together.

The workspace remains the boundary. Cache, run history, file reads, and tool working directories are all tied to the workspace chosen for the request.

Enable cache ​

Cache is controlled by AgentConfig and by AgentRequest.

cpp
config.use_cache = true;
config.cache_ttl_ms = 5 * 60 * 1000;

For one request:

cpp
request.use_cache = true;

Both values matter. The configuration defines whether the agent instance may use cache. The request decides whether this specific run should use it. If cache is disabled in the configuration, setting request.use_cache = true should not make cache available for that run.

Cache TTL ​

cache_ttl_ms controls how long a cached response may be reused.

cpp
config.cache_ttl_ms = 5 * 60 * 1000;

A short TTL is useful while developing prompts because it avoids stale answers for too long. A longer TTL can be useful for repeated local explanations that do not depend on changing files or command output.

The cache is meant to be conservative. It should make repeated local requests faster without hiding behavior that should be rechecked.

Cache hits ​

AgentResponse tells the caller whether the final response came from cache.

cpp
const auto &response = result.value();

vix::print("From cache:", response.from_cache);

This is useful for CLI output and diagnostics. A cached response may still be correct, but the user should be able to tell that the provider was not called again for that answer.

What gets cached ​

The agent cache is designed for safe model responses. A cache key is based on the provider, model, prompt, and prepared context fingerprint. This means two different models or two different providers should not share the same cached response as if they were the same backend.

Responses involving tools are not treated like plain cached model responses. Tool execution represents local work performed during a run, so it should not be silently skipped later as if it were only a text completion.

Disable cache ​

Disable cache when every request should call the provider.

cpp
config.use_cache = false;
request.use_cache = false;

This is useful when testing providers, debugging prompts, or working with commands and tool output. It makes the behavior direct: every run asks the provider again.

Cache example ​

The following example uses a custom provider that counts how many times it was called. The first run calls the provider. The second run uses the same request with cache enabled, so it can return from cache instead of calling the provider again.

cpp
#include <memory>
#include <string_view>

#include <vix/ai/agent/AgentRuntime.hpp>
#include <vix/print.hpp>

namespace
{
  class CountingProvider final : public vix::ai::agent::ModelProvider
  {
  public:
    [[nodiscard]] std::string_view name() const noexcept override
    {
      return "counting";
    }

    [[nodiscard]] bool local() const noexcept override
    {
      return true;
    }

    [[nodiscard]] vix::ai::agent::AgentResult<bool> available() const override
    {
      return true;
    }

    [[nodiscard]] vix::ai::agent::AgentResult<vix::ai::agent::ModelResponse>
    generate(const vix::ai::agent::ModelRequest &request) override
    {
      ++calls_;

      vix::ai::agent::ModelResponse response;

      response.text = "This response was generated by the provider.";
      response.model = request.model;
      response.provider = "counting";
      response.status = vix::ai::agent::ModelResponseStatus::Completed;

      return response;
    }

    [[nodiscard]] int calls() const noexcept
    {
      return calls_;
    }

  private:
    int calls_{0};
  };
}

int main()
{
  vix::ai::agent::AgentConfig config;

  config.provider = "counting";
  config.model = "counting-model";

  config.allow_file_read = false;
  config.allow_process = false;
  config.allow_file_write = false;

  config.use_cache = true;
  config.cache_ttl_ms = 5 * 60 * 1000;
  config.persist_memory = true;

  auto provider = std::make_shared<CountingProvider>();

  vix::ai::agent::Agent agent(config, provider);

  vix::ai::agent::AgentRequest request;

  request.workspace = ".";
  request.input = "Explain local-first software in simple words.";
  request.mode = vix::ai::agent::AgentRequestMode::Explain;

  request.allow_tools = false;
  request.allow_file_read = false;
  request.allow_process = false;
  request.allow_file_write = false;
  request.use_cache = true;

  auto first = agent.run(request);

  if (!first)
  {
    vix::print("First run error:", first.error().message());
    return 1;
  }

  vix::print("First run:");
  vix::print(first.value().text);
  vix::print("From cache:", first.value().from_cache);
  vix::print("Provider calls:", provider->calls());
  vix::print();

  auto second = agent.run(request);

  if (!second)
  {
    vix::print("Second run error:", second.error().message());
    return 1;
  }

  vix::print("Second run:");
  vix::print(second.value().text);
  vix::print("From cache:", second.value().from_cache);
  vix::print("Provider calls:", provider->calls());

  return 0;
}

This kind of example is useful because it makes cache behavior visible. The provider has no hidden state except the call counter, so the output shows whether the second run reused the cached response.

Enable run history ​

Run history is controlled by persist_memory.

cpp
config.persist_memory = true;

When persistence is enabled, the agent writes local run data under:

text
.vix/agent/runs/<run_id>/

The run id is returned in AgentResponse.

cpp
const auto &response = result.value();

vix::print("Run id:", response.run_id);

The run id connects the program output to the files stored on disk. This is useful when debugging a failed request, comparing two model responses, or understanding which tools were used during a run.

Run files ​

A run directory may contain files such as:

text
run.json
prompt.txt
model_response.json
tools.json
response.json
response.txt
error.json

Not every run needs every file. A successful run may store the prompt, model response, final response, and tool summaries. A failed run may store error information instead. The purpose is to make the run inspectable without forcing the application to print every internal detail to the terminal.

Inspecting a response ​

An application can show the most useful run metadata directly.

cpp
const auto &response = result.value();

vix::print(response.text);
vix::print();
vix::print("Run id:", response.run_id);
vix::print("Provider:", response.provider);
vix::print("Model:", response.model);
vix::print("From cache:", response.from_cache);
vix::print("Duration:", response.duration_ms, "ms");

For tool-based runs, the response also contains tool summaries.

cpp
vix::print("Tools used:", response.tools.size());

for (const auto &tool : response.tools)
{
  vix::print("-", tool.name, tool.ok ? "ok" : "failed");
}

This gives a CLI or application enough information to explain what happened without exposing the full run directory in normal output.

Disable run history ​

Disable persistence when the run should not write local run data.

cpp
config.persist_memory = false;

This is useful for temporary experiments, tests that should not leave files behind, or applications that manage their own storage. Cache and run history are separate choices, so a program can disable persistence while also disabling cache for a fully transient run.

cpp
config.use_cache = false;
config.persist_memory = false;

Cache and tools ​

Tool-based requests need extra care. A tool result depends on the local state at the time of the run. A file may change, a command may produce different output, and a workspace may evolve between runs.

For that reason, tool-based workflows are often clearer with cache disabled during development.

cpp
config.use_cache = false;
request.use_cache = false;

After the workflow is understood, cache can be enabled for requests that are safe to reuse. The important point is that cache should not make the user believe that a file or command was checked again when it was not.

CLI behavior ​

The vix agent command uses the same runtime concepts. Cache and memory are enabled by default in the CLI options, and they can be disabled for a run.

Disable cache:

bash
vix agent ask "Explain this project" --no-cache

Disable local memory and run persistence:

bash
vix agent ask "Explain this project" --no-memory

Analyze a workspace with more time for the local model:

bash
vix agent analyze . --timeout 120000

These flags are useful when testing prompts or checking whether a response is fresh.

Complete example ​

The following example runs a project analysis request with cache and run history enabled, then prints the run metadata.

cpp
#include <vix/ai.hpp>
#include <vix/print.hpp>

int main()
{
  vix::ai::agent::AgentConfig config;

  config.provider = "ollama";
  config.model = "llama3";
  config.model_url = "http://127.0.0.1:11434";

  config.timeout_ms = 120'000;

  config.allow_file_read = true;
  config.allow_process = false;
  config.allow_file_write = false;

  config.max_files = 200;
  config.max_file_size = 256 * 1024;
  config.max_context_chars = 32'000;

  config.use_cache = true;
  config.cache_ttl_ms = 5 * 60 * 1000;
  config.persist_memory = true;

  vix::ai::agent::Agent agent(config);

  vix::ai::agent::AgentRequest request;

  request.workspace = ".";
  request.input =
      "Look at this project structure and explain the most important files.";
  request.mode = vix::ai::agent::AgentRequestMode::Analyze;

  request.allow_tools = true;
  request.allow_file_read = true;
  request.allow_process = false;
  request.allow_file_write = false;
  request.use_cache = true;
  request.timeout_ms = config.timeout_ms;

  auto result = agent.run(request);

  if (!result)
  {
    vix::print("Agent error:", result.error().message());
    return 1;
  }

  const auto &response = result.value();

  vix::print(response.text);
  vix::print();
  vix::print("Run id:", response.run_id);
  vix::print("Provider:", response.provider);
  vix::print("Model:", response.model);
  vix::print("From cache:", response.from_cache);
  vix::print("Duration:", response.duration_ms, "ms");

  return 0;
}

This configuration keeps the run observable. The response tells the caller whether cache was used, and the run id points to the local run history when persistence is enabled.

Common mistakes ​

Expecting cache to mean the model checked the project again ​

A cache hit means the response was reused. It does not mean the provider was called again or that the workspace was rechecked in the same way as a fresh run.

cpp
if (response.from_cache)
{
  vix::print("Cache:", "hit");
}

Show cache status when it matters to the user.

Caching during tool debugging ​

When debugging file reads, command tools, or tool call behavior, disable cache first.

cpp
config.use_cache = false;
request.use_cache = false;

This avoids confusing a cached model response with the behavior of the current tool run.

Disabling persistence and expecting run files ​

Run history is written only when persistence is enabled.

cpp
config.persist_memory = true;

If persist_memory is false, the application should not expect a useful run directory for inspection.

Using the wrong workspace ​

Cache and run history belong to the workspace. Running the same program from a different directory can create a different local agent data location.

cpp
request.workspace = ".";

For tools, prefer setting a clear workspace path instead of relying on the process working directory when that directory may change.

Next step ​

Continue with the custom providers page to see how to implement a ModelProvider for tests, demos, or alternate model backends.

Released under the MIT License.

Copyright © 2026 Vix.cpp