There are many ways to ingest the data in standard file formats from cloud storage to Delta Lake, but is there a way to ingest data from 100s of files within few seconds as soon as it lands in a storage account folder? I have recently come across a Customer with requirements to ingest ~1500 […]
If you are already working on building an Azure Data Engineering solution using Azure Data Factory as an orchestration tool and Azure Cosmos DB in a scenario where you may have to delete documents from a particular SQL container programmatically, then you might have already figured out that there is no easy way to do […]
I have recently come across a Customer who is migrating On-prem DW workloads to Azure cloud (using Azure Synapse Analytics, Data Factory, Azure Storage, Power BI) and has a scenario where they have a bunch of 7z compressed data files that are being unzipped using a utility installed in a VM for further processing in […]
Follow My Blog
Get new content delivered directly to your inbox.