Architecture

Vein uses a database + filesystem architecture: a metadata store tracks what’s cached and where, while the actual package files live on disk.

Request flow

flowchart TD
    A[Client request] --> B{Cached?}
    B -->|Hit| C[Serve from filesystem]
    B -->|Miss| D[Fetch from upstream\nRubyGems / crates.io / npm]
    D --> E[Cache locally]
    E --> F[Serve to client]

• Permanent caching - artifact files (gems, crates, npm tarballs) are cached on first fetch and served locally on every subsequent request.
• TTL + revalidation - index and metadata endpoints (RubyGems index, crates.io sparse index, npm package metadata) are cached for a limited time and revalidated against upstream.
• Corruption recovery - a cache entry that fails checksum verification is refetched from upstream rather than served.

Metadata store

SQLite (default, vein.db) or PostgreSQL. This is the authoritative source of truth for every cached package:

• Name, version, platform
• Filesystem path
• SHA256 checksum
• File size
• Last-accessed timestamp

It’s consulted on every cache miss (to decide whether a fetch is needed) and updated on every cache write.

SQLite and PostgreSQL adapter features are mutually exclusive at compile time (sqlite vs postgres Cargo features) - a build picks one.

Filesystem storage

Cached artifacts live under [storage] path (default ./cache), organized into per-ecosystem subfolders. This is separate from the metadata store so large binary payloads never touch the database.

Hot cache

A background job periodically refreshes an in-memory “hot cache” index from the SQLite/PostgreSQL store, controlled by [hotcache] refresh_schedule (a six-field cron expression). Set it to an empty string to disable automatic refresh.

Runtime

Vein is built on Rama, a modular Rust HTTP framework, running on Tokio. It’s a single binary with no external runtime dependencies beyond the chosen database.

Reliability

Database and upstream HTTP calls both support configurable retry policies ([database.reliability.retry], [upstream.reliability.retry]) with exponential, fibonacci, or constant backoff. See Configuration for the full option list.