Preview your hierarchical data before running your recipe. Allocate your concurrent data prep jobs to meet your orchestration needs. Get information on when each data prep job started and was canceled.
- Visualize Hierarchies and Flatten Transformations with Preview in the Recipe Editor
Preview the results of hierarchical data in aggregate nodes and flatten transformations to ensure that you have the correct output before running your recipe. For example, if you’re flattening a role hierarchy to implement row-level security on a dataset, you can preview the results to verify that they’re as expected before proceeding. - Control Your Data Prep Concurrency Allocation
Allocate your dataflow concurrency towards recipe jobs to meet your orchestration needs. For example, instead of being limited to running two dataflows and one recipe at a time, you can share dataflow concurrency towards recipes and run three recipes at the same time. - Audit Details of Running and Canceled Data Prep Jobs
Enhanced data prep job information provides details for data prep jobs that are running off schedule or why they were canceled. Now you can see when each job started, who started it, and if a job was canceled, who canceled it and when. You can also view the last person who scheduled or unscheduled a recipe or connection in the job scheduler. - Get More Usage Information in Data Manager
Manage your data limits with more statistics on the Data Manager Usage page. New usage information includes the number of external uploads and the total number of dataflow and recipe runs in a rolling 24-hour period. You can also see the monthly count of rows output by recipes. - Trigger Recipes to Run When an External Connection Syncs or a CSV Uploads
Set your event-based recipe schedules to run when an external connection syncs or a CSV upload finishes. Instead of being limited to one event for your trigger, you can select multiple events. Create a recipe schedule to run after a local connection syncs and after an external connection syncs, keeping your recipe up to date regardless of the input source. - Experience Better Recipe Error Messaging and Reliability
Your recipes are now more reliable and easier to troubleshoot. If an issue occurs, the error message helps you identify the problem and resolve issues faster. In addition, if a recipe requires more resources to run successfully, the resources are automatically increased after the first failure and the job is restarted. The need for increased resources is remembered for subsequent runs. Previously, you needed to open a support case to increase a recipe’s resources. - Append Data Faster with Incremental Uploads (Generally Available)
Improve your data upload time when using the external data API by loading CSV files incrementally. By registering data faster, you can bring in more data to your existing datasets without causing the overall run time to grow exponentially. - Connect to Amazon Athena (Generally Available)
Sync Amazon Athena data to Data Manager by creating a remote connection. - Connect to Databricks (Generally Available)
Create a remote connection using the Databricks connector to sync data from Databricks to Data Manager.