Using Azure Synapse Serverless Pools for Data Engineering | Data Exposed

Data engineering is one of the most involved, dull, and repetitive tasks in the data industry today. Bringing up the subject is a sure way to instantly clear the room at a party or make your friends groan in despair. The work includes a lot of complicated tools, many, many lines of code, and a not-insignificant amount of gnashing of teeth. But what if I told you there is a way to quickly, easily, and cheaply do both data engineering and data exploration on most data without even having to import it into a database? What if I told you that Azure Synapse has a just about magical tool that can instantly connect to flat files and make them available as nice, well-behaved tables? I'm talking about Azure Synapse serverless pools, a great tool for both initial exploration and surprisingly complex data engineering. I'll walk you through what it is, how to set it up, and give you a few examples of basic data exploration, as well as a tad more complex data engineering. Oh, and did I mention you can actually store the end result in several different formats as well...?[00:56] Azure Synapse Serverless Pools Overview[01:54] Demo[11:05] Getting started Resources:Andy Cutler's blog on serverless poolsAndy Mallon's blog on bucketing with T-SQLAbout Alexander Arvidsson:Alexander is a principal solutions architect at Attollo in Sweden. Nobody really knows what it means to be a principal solutions architect, but he spends his days speaking, teaching courses or helping clients of all shapes and sizes to take better care of their data.He has spent the last 25 years toying with data, databases and related infrastructure services such as storage, networking and virtualization, occasionally emerging from the technical darkness to attend a Star Wars convention somewhere in the world.He is a frequent international speaker, Data Platform MVP, podcaster, Pluralsight author, blogger and a Microsoft Certified Trainer, focusing on the Microsoft data platform stack.About MVPs:Microsoft Most Valuab