Advanced Dataset Schema Properties/Partition Support
Eg. Partition key, cluster by, distkey/sortkey
“It will be nice to support ingesting table optional clauses similar to primary/foreign key for datasets, or bake it into the recipe itself. Will be nice to ingest metadata about partition by, cluster by for bigquery and distkey and sortkey for redshift. Throwing these as custom properties now but will be nice to see it in the schema tab.” - Original Slack post
There’s also interest in displaying data profiling statistics for each partition.
Post Information
Subscribe to post
Get notified by email when there are changes.
Upvoters
+9
Downvoters
Post Details