Overview

Buz's schema registry functionality consists of several parts:

1. An onboard cache which stores schemas so incoming payloads can be validated and annotated in microseconds.
2. A configurable backend, from which schemas are sourced before being stored in the onboard cache.
3. HTTP routes used to interact with schemas directly or purge the onboard cache.

Onboard Schema Registry Cache

The onboard schema registry cache stores schemas. If a schema is not available from the cache when needed, said schema is sourced from the configured backend.

The onboard schema registry cache ttl and the max size are both configurable.

If a schema is not used within the configured $registry.ttlSeconds it is purged and will be sourced from the backend when needed next. If the onboard schema cache exceeds $registry.maxSizeBytes it will purge schemas in LRU fashion.

For example configuration, please refer to the registry section of the bootstrap configuration.

Schema Registry Backends

Buz supports storing schemas in many different backends which are outlined along with their associated docs below:

Backend	Type	Supported?	Maintainer	More Details
File	Buz	Yes ✅	🐝	Docs
HTTP/S	Buz	Yes ✅	🐝	Docs
Google Cloud Storage	Object Storage	Yes ✅	🐝	Docs
AWS S3	Object Storage	Yes ✅	🐝	Docs
Minio	Object Storage	Yes ✅	🐝	Docs
Postgres	Database	Yes ✅	🐝	Docs
Mysql	Database	Yes ✅	🐝	Docs
Mongodb	Database	Yes ✅	🐝	Docs
Clickhouse	Database	Yes ✅	🐝	Docs
Materialize	Saas	Yes ✅	🐝	Docs
---	---	---	---	---
Kafka Schema Registry	Registry	Not yet ⚠️	This could be you.	-
Snowplow IGLU	Registry	Not yet ⚠️	This could be you.	-

Schema Registry HTTP Routes

Schemas

Any schema in the configured backend can be accessed under the /s/ route using /s/$PATH_TO_SCHEMA or /s/$SCHEMA_NAME.

For example

When running Buz using either the quickstart or make boostrap, you can access the Buz config schema at http://localhost:8080/s/io.silverton/buz/internal/config/app/v1.0.json.

Cache Purge

The onboard schema registry cache can be explicitly purged by sending either a GET or POST to /c/purge.

This is simply an operational lever to get the most-recent schema versions onto collectors while they are running.

For example

After running the quickstart or make boostrap you can easily purge all locally-cached schemas by navigating to http://localhost:8080/c/purge