Roadmap

Extend Data Containers to Curated Data Containers / Data Products

Is your feature request related to a problem? Please describe.

The UI of datahub recently started to support "Data Containers and Domains" πŸ‘

This is awesome, but it only allows physical assets to be grouped together.

I would like to create data products that offer different access ways for the data consumer depending on their needs.

Describe the solution you'd like

I would like to curate datasets and other datahubs entities in one common location, like the data container, but beyond the physical level.

This will allow a common list of documentation (incl. tags, domain assignment) and workflows to start from.

In this common location, I would add a data container of MySQL tables that belongs to this topic. Also, I would add Apache Kafka streams that you could use alternatively.

Describe alternatives you've considered

  • Currently, I bend the paths of datahub, but then you have no common documentation.

  • You could only have one source, but that's not very modern.

  • Another alternative would be the new data domains, but there are no sub-domains, and it also feels hacky.

Additional context

This has been discussed a few times:

This extended data container would fit into the data mesh vision as a data product.