Buz's schema registry functionality consists of several parts:
- An onboard cache which stores schemas so incoming payloads can be validated and annotated in microseconds.
- A configurable backend, from which schemas are sourced before being stored in the onboard cache.
- HTTP routes used to interact with schemas directly or purge the onboard cache.
Onboard Schema Registry Cache
The onboard schema registry cache stores schemas. If a schema is not available from the cache when needed, said schema is sourced from the configured backend.
The onboard schema registry cache ttl and the max size are both configurable.
If a schema is not used within the configured
$registry.ttlSeconds it is purged and will be sourced from the backend when needed next. If the onboard schema cache exceeds
$registry.maxSizeBytes it will purge schemas in LRU fashion.
For example configuration, please refer to the
registry section of the bootstrap configuration.
Schema Registry Backends
Buz supports storing schemas in many different backends which are outlined along with their associated docs below:
|Google Cloud Storage||Object Storage||Yes ✅||🐝||Docs|
|AWS S3||Object Storage||Yes ✅||🐝||Docs|
|Minio||Object Storage||Yes ✅||🐝||Docs|
|Kafka Schema Registry||Registry||Not yet ⚠️||This could be you.||-|
|Snowplow IGLU||Registry||Not yet ⚠️||This could be you.||-|
Schema Registry HTTP Routes
Any schema in the configured backend can be accessed under the
/s/ route using
When running Buz using either the quickstart or
make boostrap, you can access the Buz config schema at http://localhost:8080/s/io.silverton/buz/internal/config/app/v1.0.json.
The onboard schema registry cache can be explicitly purged by sending either a
This is simply an operational lever to get the most-recent schema versions onto collectors while they are running.
After running the quickstart or
make boostrap you can easily purge all locally-cached schemas by navigating to http://localhost:8080/c/purge