Data Estate

A personal data warehouse with a self-training AI brain, running every day on a Raspberry Pi. The reference build behind Estate Box. In development. Python…

What it is

Data Estate is Bit Primitive's own reference implementation: a complete personal data platform that has run every day on a Raspberry Pi, and the working case study behind Estate Box.

It brings personal finance, crypto and public market data into one warehouse, trains an AI on it each night, and serves the results to a phone app. It is private and not distributed; this page describes what it does, not anyone's data.

The warehouse

• One dbt project on DuckDB, modelled Kimball style: staging, intermediate, facts and dimensions, then app views that are the only thing consumers read.
• More than a thousand dbt nodes and 800+ data tests, including tests that fail the build if a sensitive string could reach the app.
• Blue/green builds. Each run builds a copy, checks it and swaps it in atomically, so readers are never blocked and a failed build never touches the live data.
• Strict parity checks against the previous system, record for record, before anything was switched over.
• Crypto capital gains worked out lot by lot (first in, first out) under South African rules, with staking and other rewards treated as income at their value on receipt.
• A daily schedule with catch-up: if the box was off at the scheduled time, it runs the missed feeds and build when it comes back.

The estate brain

A family of small models trained from scratch on the estate's own data, retrained after every nightly build:

• Transaction categoriser that suggests categories for new lines and learns from each review.
• Spending anomalies against each merchant's and category's own history.
• Cash-flow and net-worth forecasts with 80% ranges, picked on rolling backtests and graded once the dates arrive.
• News intelligence that tags topics and entities in the day's headlines.
• Ask the estate: plain-language questions answered by retrieval from facts the warehouse holds, with their source.
• Money optimiser that looks for recurring charges, price rises, fees and spending above its own baseline.
• Pipeline health: a model pointed at the estate itself, using the dbt lineage and row-count history to spot data problems.

Every model follows the same rule: a challenger is promoted only when it beats the current champion on held-out data it has never seen, and beats a naive baseline. Every version, score and decision is kept, so the brain's progress is a record, not a claim.

The app

• A static, installable phone app built from the warehouse's app views, served behind access control.
• Net worth, cash flow, crypto, forecasts, insights and the brain's history in one place.
• A 3D data galaxy that draws the warehouse itself: every table, how they connect, and how they grow.
• Statements can be uploaded from the phone; the box picks them up and loads them in the next build.

Privacy

Account numbers are masked before they leave staging, wallet details never reach the app, logs or this site, and the brain masks every string before it becomes a feature. Automated tests plant fake identifiers and check that they appear nowhere downstream.