Extend Data Containers to Curated Data Containers / Data Products
Is your feature request related to a problem? Please describe.
The UI of datahub recently started to support "Data Containers and Domains" π
This is awesome, but it only allows physical assets to be grouped together.
I would like to create data products that offer different access ways for the data consumer depending on their needs.
Describe the solution you'd like
I would like to curate datasets and other datahubs entities in one common location, like the data container, but beyond the physical level.
This will allow a common list of documentation (incl. tags, domain assignment) and workflows to start from.
In this common location, I would add a data container of MySQL tables that belongs to this topic. Also, I would add Apache Kafka streams that you could use alternatively.
Describe alternatives you've considered
Currently, I bend the paths of datahub, but then you have no common documentation.
You could only have one source, but that's not very modern.
Another alternative would be the new data domains, but there are no sub-domains, and it also feels hacky.
Additional context
This has been discussed a few times:
https://datahubspace.slack.com/archives/CV2UXSE9L/p1644945899276919
https://datahubspace.slack.com/archives/C01HPV8EKTK/p1628634684022100
https://datahubspace.slack.com/archives/C02FWNS2F08/p1632430177289200
https://datahubspace.slack.com/archives/C01HPV8EKTK/p1628634628021200
This extended data container would fit into the data mesh vision as a data product.
Subscribe to post
Get email updates when post changes