Remove required dependency on Confluent Schema Registry
Is your feature request related to a problem? Please describe.
Confluent Schema Registry is a required component for DataHub. This makes adoption of DataHub harder for companies that don’t prefer the licensing.
Describe the solution you'd like
DataHub uses the Schema Registry more as a convenience than a necessity. The Kafka Topic schemas are built along-side the code, and with the MCP/MCL pathway are no longer going to change as often.
We could remove the dependency on Schema Registry as a required component, and default to in-memory (compiled) schemas on the classpath, if a schema registry is not found to be configured on startup.
This will simplify deployment for certain environments and companies.
Describe alternatives you've considered
We could also use another schema registry like Karapace for this, but since our requirements are so simple, we might be able to just remove this requirement altogether.
Additional context
https://datahubspace.slack.com/archives/CV2KB471C/p1643033131052800
Subscribe to post
Get notified by email when there are changes.