Skip to content
tatami

Guides

Task-oriented walkthroughs for the things people actually do with tatami: writing and reading files from Go, converting crawl shards, managing a collection, and building and serving a search index.

Each guide is built around a job rather than an API surface: writing typed columns and reading them back, bringing existing Parquet output into the format, stitching many files into one collection, turning a corpus into a keyword search index that answers in microseconds, serving a fleet of shards behind one query, and running that broker over HTTP. They assume you have worked through the quick start.

Writing and reading files Use the Go library to write typed columns to a tatami file, read them back with column projection, and prune rows with predicate pushdown and key lookups. Converting crawl shards Bring existing Parquet crawl output into tatami with the convert command, control which columns are separated, filtered, and dictionary-hinted, and see why the file comes out smaller. Managing a collection Stitch many tatami files into one logical dataset with a manifest, prune whole files before opening them, look a key up across the set, and compact the log. Searching a corpus Build a search segment from documents, run keyword queries with BM25 ranking and block-max WAND retrieval, and fetch the matching documents from the same file in microseconds. Merging and serving at scale Delete documents from a sealed segment, fold many small segments into large ones with a tiered merge, and serve a whole fleet of segments behind one query that still answers in under a millisecond. Distributed serving at scale Serve a fan of cold shards behind one query with a routed pruned broker, drop the document body to ship search-only segments, and fan a query across leaf brokers with an aggregator for an exact fleet-wide top-k. Serving search over HTTP Run tatami serve over a directory of search segments: a concurrent lock-free broker with admission control, a per-request deadline, and a smart segment cache sized to hold the working set.