How it works and install

How it works and install for chainstar. Raw blockchain data to clean dimensional tables in your own warehouse, from one command.

chainstar pulls blocks, transactions and token transfers from an RPC endpoint (yours, or a free public one), saves them as Parquet, loads them into your warehouse (DuckDB, Postgres, MotherDuck, ClickHouse, BigQuery or Snowflake) or a Parquet data lake, and builds a small star schema with dbt: facts for transfers, fees, swaps and USD prices, dimensions for wallets, tokens, contracts, pools and dates. All chains share the same tables, so one warehouse can hold Ethereum, Base, Bitcoin and Solana side by side.

extract  →  decode  →  load  →  model
 cryo        cryo      DuckDB    dbt        (Ethereum, Base)
 JSON-RPC    -         DuckDB    dbt        (Bitcoin, Solana)

extract: on Ethereum and Base, cryo fetches blocks, transactions, ERC-20 Transfer events and DEX Swap events for your block range into Parquet files, decoding each Transfer log into token, sender, recipient and value; chainstar decodes the swaps. cryo does not support Bitcoin or Solana, so for those chainstar reads blocks over plain JSON-RPC itself and writes Parquet files in the same layout (see Chains), deriving Solana swaps from the transactions.
decode (Ethereum and Base): for every pool seen, chainstar reads its tokens and factory and asks the factory to confirm it; for every token contract seen, cryo calls name(), symbol() and decimals().
load: the Parquet files are copied into raw tables (schema chainstar_raw).
model: a bundled dbt project builds the star schema (schema chainstar) and tests it.

This is an early release (0.1, unreleased). It supports Ethereum, Base, Bitcoin and Solana, and as targets DuckDB, Postgres, MotherDuck, ClickHouse, BigQuery, Snowflake and a Parquet data lake on local disk or S3-compatible storage; see Targets for how each was tested.

Install

There are three ways; pick one.

Docker image (no Python or Rust needed, cryo included). Published to the GitHub Container Registry by this repository's Docker image workflow, for amd64 and arm64 (Apple silicon and other 64-bit ARM machines):

docker run --rm -v "$PWD:/work" ghcr.io/bit-primitive/chainstar --help

ghcr.io/bit-primitive/chainstar:latest follows main; version tags such as :0.1.0 follow releases. To build the image yourself: docker build -t chainstar . in a checkout (the cryo compile takes about 5 minutes).

How the container sees your files: the folder you mount at /work is chainstar's working folder, so warehouse.duckdb, data/ and exports appear in it, and paths you pass (--output mine.xlsx, --target duckdb:my.duckdb) are relative to it. Pass your own RPC URL with -e CHAINSTAR_RPC_URL (it is read from your environment, not written into the command). Add -it for quickstart's questions; without a terminal it uses the defaults, as with --yes. The image runs as an unprivileged user (uid 1000); on Linux, --user "$(id -u):$(id -g)" makes the files yours. DuckDB's postgres, excel, httpfs, aws and motherduck extensions and the ClickHouse driver are included, so DuckDB, Postgres, MotherDuck, ClickHouse and the Parquet lake all work without downloads, in an image of about 1 GB unpacked. The BigQuery and Snowflake drivers would nearly double that, so they are left out of the published image; to write to those, build your own: docker build --build-arg EXTRAS="all" -t chainstar . (or EXTRAS="bigquery", EXTRAS="snowflake", or EXTRAS="" for no drivers).

uv or pipx (Python 3.11 or newer):

uv tool install git+https://github.com/Bit-Primitive/chainstar
# or: pipx install git+https://github.com/Bit-Primitive/chainstar

chainstar is not on PyPI yet; once it is, uv tool install chainstar or pipx install chainstar.

DuckDB, Postgres, MotherDuck and Parquet lakes need nothing more. ClickHouse, BigQuery and Snowflake need their drivers, as an extra: clickhouse, bigquery, snowflake, or all:

uv tool install "chainstar[clickhouse] @ git+https://github.com/Bit-Primitive/chainstar"

The plug-ins for dlt and Airbyte have their own extras, dlt and airbyte, which all does not include; see Using chainstar from dlt and Airbyte.

cryo, for Ethereum and Base outside Docker. It is written in Rust; get Rust from https://rustup.rs, then:

cargo install cryo_cli

This installs cryo 0.3.2 from crates.io, the version chainstar is tested with. Bitcoin and Solana need nothing else.