Local Cluster Demo

This starts the three-node Compose cluster, then reads the example table from the leader and from a follower.

Prerequisites

• Docker
• The mysql client
• openssl

Create the test CA and server certificate before the first up. They are not committed. examples/gencerts.sh writes them to examples/certs. Compose mounts that directory at /certs, and Raft reads ca.crt, server.crt, and server.key from there. Without ca.crt each node logs open /certs/ca.crt: no such file or directory and exits, and Compose restarts it.

The server certificate is signed by that CA, carries serverAuth and clientAuth, and names the addresses clients dial: 127.0.0.1 from the host, the node names, and the private addresses Compose assigns.

examples/gencerts.sh
docker compose up --build

A stack that is already restarting picks the files up after docker compose restart. A volume created before these addresses still has the old Raft peer list. Remove it first: docker compose down -v.

Ports

n1 bootstraps the Raft group. n2 and n3 join it. Raft on port 7001 and the write-forward port 7002 stay on the vpc network (10.116.0.0/24), are not published to the host, and require mutual TLS. The server certificate is the client certificate as well. Each node keeps /data in its own volume. The same addresses, with host networking and a Cloud Firewall, are what a DigitalOcean deployment uses; see droplets/README.md (opens in a new tab). docker compose up requires HARDHATDB_BOOTSTRAP_PASSWORD. That value is used only when a node has no accounts yet. HARDHATDB_SEED_EXAMPLE=1 creates mydb.accounts and mydb.notes. Leave it unset for an empty data directory. accounts.id is the shard column. A volume created for the previous example still has mydb.mytable. Remove it before the first start on this schema: docker compose down -v.

Check the leader, then a follower

mysql --host=127.0.0.1 --port=3306 --user=root --password="$HARDHATDB_BOOTSTRAP_PASSWORD" \
  --ssl-mode=REQUIRED --ssl-ca=examples/certs/ca.crt \
  mydb --execute="SELECT name, email FROM accounts;"
mysql --host=127.0.0.1 --port=3307 --user=root --password="$HARDHATDB_BOOTSTRAP_PASSWORD" \
  --ssl-mode=REQUIRED --ssl-ca=examples/certs/ca.crt \
  mydb --execute="SELECT name, email FROM accounts;"

Writes succeed on the leader. A write to port 3307 or 3308 is forwarded. Ask any node who the leader is:

mysql --host=127.0.0.1 --port=3308 --user=root --password="$HARDHATDB_BOOTSTRAP_PASSWORD" \
  --ssl-mode=REQUIRED --ssl-ca=examples/certs/ca.crt \
  --execute="SHOW RAFT STATUS;"

Clients

Each example app lives in its own directory and has a compose file that starts the app with this cluster. Run examples/gencerts.sh first, and stop the root stack. These stacks use the same MySQL ports. One file starts the cluster and all three apps:

docker compose -f examples/compose.yaml up --build

Or start one language:

docker compose -f examples/webapp/compose.yaml up --build
docker compose -f examples/rails/compose.yaml up --build
docker compose -f examples/fastapi/compose.yaml up --build

The page on port 8080 shows each node's SHOW RAFT STATUS, which address receives the next write, and whether another node can see that row yet.

Inside those stacks the apps dial n1, n2, and n3 on port 3306 and trust /certs/ca.crt. From the host, the same clients take every published address. A broken connection tries the next one:

MYSQL_ADDRS=127.0.0.1:3306,127.0.0.1:3307,127.0.0.1:3308

The example clients trust examples/certs/ca.crt unless MYSQL_TLS_CA is set. MYSQL_TLS_CA=off connects without TLS. Leave MYSQL_ADDRS unset to use only localhost:3306.

One address

A client that stores one address uses hardhatdb-proxy (opens in a new tab), from a checkout next to this repo. Run this from the proxy repo:

HARDHATDB_BOOTSTRAP_PASSWORD=secret docker compose \
  -f ../hardhatdb/compose.yaml -f compose.yaml up --build

The proxy listens on host port 3390 and copies each connection to a node whose /readyz returns 200. The example apps keep using MYSQL_ADDRS.

Two groups

This stack is alpha. examples/compose.sharded.yaml places accounts.id on two Raft groups. Ids below 3 are on g1–g3. Ids from 3 up are on g4–g6. g1–g3 also vote in the meta group. Host MySQL ports are 3416–3421, and the apps are on 8081, 8001, and 3001, so this stack can run beside the single group. A new account includes id.

docker compose -f examples/compose.sharded.yaml up --build
examples/placeshards.sh

Stress test

The workload lives in hardhatdb-stress (opens in a new tab). ./run.sh there pulls ghcr.io/bongani-m/hardhatdb and runs the same client against one node, the cluster, MySQL, ElyraSQL, TiDB, and the ranged cluster.

MediaWiki on one HardhatDB node lives in the sibling directory hardhatdb-mediawiki.

Stop

docker compose down

docker compose down -v also deletes the data volumes.