Real-time ingestion with Event Hub and Synapse

Real-time ingestion with Event Hub and Synapse

Published Jun 20 2022 08:49 AM 611 Views

Skip to footer content

Some weeks ago, my manager encouraged us to create some relevant scenarios that help us to learn new things and before that I was already learning Azure Synapse Analytics. There are some “Get Started Labs” that I see are like the most, regularly about getting the information from a source and sending it to a target, and that’s totally fine but I want to explore more options about Real-time ingestion. I worked several years in a big solution for a customer who needed information with a short time of delay and in that time the options were limited.

Now, we have a lot of options, services, platforms, and it could be complex to try to figure out which is our best option. One of the main problems we had some years ago is the capacity of an artifact or a service to receive massive information and have all those services available 24/7.

With Azure you have these options to implement real-time ingestion in your solutions.

I won’t go in so much detail about each one because the documentation is amazing, if you want to know more about it, go to:  Choose a real-time message ingestion technology in Azure

There’s not a best option or a right or wrong choice, everything depends on the scenario and sometimes is about time, about contracts, about money, just because the architect says, or the circumstances makes to have one or another. In this case I picked Event Hubs because, as I’ll show in the example, we’ll receive different types of information, and we reduce the amount of code in our solution with the Capture feature which automatically stores your data in a data lake.

To have a complete scenario we have ingestion, transformation, analysis, and visualization as you can see in the image below. It’s possible to have different sources in the ingestion, combining streaming and batching and everything goes into our Data Lake (in this case we are using Azure Synapse).

To implement this first part of the ingestion you can follow and check the following GitHub repository: vasegovi/synapse-demo (github.com)

But let me tell you here some of lessons I learned when I was implementing this, first of all I search for some similar examples on internet and found this one Trying out Event Hub Capture to Synapse. It covers almost all the things that I was interested in so you can also go there and follow it, and then come here and review some topics that you can add in your learning.

Event Hub

With the capture feature you can easily configure the way and the place where you need your data. Some important aspects that you should consider are:

To extend all this information visit: Capture events through Azure Event Hubs in Azure Blob Storage or Azure Data Lake Storage

Blob Permissions

This is one of the most interesting points that you want to know about in detail because it was the most complex task that I had to deal with.

The table shows us all the users and access that you must add into your blob storage, the one that is working as Data Lake. the is the user that is going to run scripts manually, or for unit test, then the managed identity that we created. Then the name of your synapse workspace creates a Service Principal which you also have to add in the permissions because is the one that executes the script once is published, and last one the Azure service principal for Azure Data Factory to run triggers.

Why Azure Data Factory? Because inside Azure Synapse engine the ADF engine is involved and is the one that will execute the scripts when they are launched with the trigger event.

role/user	description	purpose
your user	this is for the user logged	When you are running scripts manually
managed identity	the managed identity we created	Because it will be assigned as administrator
wssynapse (synapse workspace name)	the workspace synapse resource	For pipelines
	ADF Service Principal	For triggers

For deeper explanations please visit: Create a trigger that runs a pipeline in response to a storage event

Git Integration

There are many branch strategies that you can follow in your application and infrastructure as well depending on several factors but there are important points that you should consider.

Python Client

This Python client is useful to send information to your Event Hubs. The unique goal of this piece is to simulate a device which sends all data to Event hubs, and ideal architecture looks like the image below, where you have several sources to capture the info needed for your solution.

Challenge

When you finish the steps in the repo and complete the challenge, please come back and let me know in the comments how was it

Next Steps:

I will continue with my end-to-end scenario so keep in touch to see part 2 when we’ll add Machine Learning to analyze this information, meanwhile you can learn more about streaming and synapse let you here some useful links

Capture Event Hubs data in Azure Storage and read it by using Python

High throughput stream ingestion to Azure Synapse

Analytics end-to-end with Azure Synapse

You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.

%3CLINGO-SUB%20id%3D%22lingo-sub-3518504%22%20slang%3D%22en-US%22%3EReal-time%20ingestion%20with%20Event%20Hub%20and%20Synapse%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-3518504%22%20slang%3D%22en-US%22%3E%3CP%3ESome%20weeks%20ago%2C%20my%20manager%20encouraged%20us%20to%20create%20some%20relevant%20scenarios%20that%20help%20us%20to%20learn%20new%20things%20and%20before%20that%20I%20was%20already%20learning%20Azure%20Synapse%20Analytics.%20There%20are%20some%20%E2%80%9CGet%20Started%20Labs%E2%80%9D%20that%20I%20see%20are%20like%20the%20most%2C%20regularly%20about%20getting%20the%20information%20from%20a%20source%20and%20sending%20it%20to%20a%20target%2C%20and%20that%E2%80%99s%20totally%20fine%20but%20I%20want%20to%20explore%20more%20options%20about%20Real-time%20ingestion.%20I%20worked%20several%20years%20in%20a%20big%20solution%20for%20a%20customer%20who%20needed%20information%20with%20a%20short%20time%20of%20delay%20and%20in%20that%20time%20the%20options%20were%20limited.%3C%2FP%3E%0A%3CP%3ENow%2C%20we%20have%20a%20lot%20of%20options%2C%20services%2C%20platforms%2C%20and%20it%20could%20be%20complex%20to%20try%20to%20figure%20out%20which%20is%20our%20best%20option.%20One%20of%20the%20main%20problems%20we%20had%20some%20years%20ago%20is%20the%20capacity%20of%20an%20artifact%20or%20a%20service%20to%20receive%20massive%20information%20and%20have%20all%20those%20services%20available%2024%2F7.%3C%2FP%3E%0A%3CP%3EWith%20Azure%20you%20have%20these%20options%20to%20implement%20real-time%20ingestion%20in%20your%20solutions.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fevent-hubs%2F%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3EAzure%20Event%20Hubs%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fiot-hub%2F%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3EAzure%20IoT%20Hub%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fhdinsight%2Fkafka%2Fapache-kafka-get-started%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3EKafka%20on%20HDInsight%3C%2FA%3E%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EI%20won%E2%80%99t%20go%20in%20so%20much%20detail%20about%20each%20one%20because%20the%20documentation%20is%20amazing%2C%20if%20you%20want%20to%20know%20more%20about%20it%2C%20go%20to%3A%26nbsp%3B%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Farchitecture%2Fdata-guide%2Ftechnology-choices%2Freal-time-ingestion%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3EChoose%20a%20real-time%20message%20ingestion%20technology%20in%20Azure%3C%2FA%3E%3C%2FP%3E%0A%3CP%3EThere%E2%80%99s%20not%20a%20best%20option%20or%20a%20right%20or%20wrong%20choice%2C%20everything%20depends%20on%20the%20scenario%20and%20sometimes%20is%20about%20time%2C%20about%20contracts%2C%20about%20money%2C%20just%20because%20the%20architect%20says%2C%20or%20the%20circumstances%20makes%20to%20have%20one%20or%20another.%20In%20this%20case%20I%20picked%20Event%20Hubs%20because%2C%20as%20I%E2%80%99ll%20show%20in%20the%20example%2C%20we%E2%80%99ll%20receive%20different%20types%20of%20information%2C%20and%20we%20reduce%20the%20amount%20of%20code%20in%20our%20solution%20with%20the%20Capture%20feature%20which%20automatically%20stores%20your%20data%20in%20a%20data%20lake.%3C%2FP%3E%0A%3CP%3ETo%20have%20a%20complete%20scenario%20we%20have%20ingestion%2C%20transformation%2C%20analysis%2C%20and%20visualization%20as%20you%20can%20see%20in%20the%20image%20below.%20It%E2%80%99s%20possible%20to%20have%20different%20sources%20in%20the%20ingestion%2C%20combining%20streaming%20and%20batching%20and%20everything%20goes%20into%20our%20Data%20Lake%20(in%20this%20case%20we%20are%20using%20Azure%20Synapse).%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22architecture.jpg%22%20style%3D%22width%3A%20999px%3B%22%3E%3Cspan%20class%3D%22lia-inline-image-display-wrapper%22%20image-alt%3D%22architecture.jpg%22%20style%3D%22width%3A%20999px%3B%22%3E%3Cimg%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F381081iA0EADFC6C8C04E1D%2Fimage-size%2Flarge%3Fv%3Dv2%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22architecture.jpg%22%20alt%3D%22architecture.jpg%22%20%2F%3E%3C%2Fspan%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3ETo%20implement%20this%20first%20part%20of%20the%20ingestion%20you%20can%20follow%20and%20check%20the%20following%20GitHub%20repository%3A%20%3CA%20href%3D%22https%3A%2F%2Fgithub.com%2Fvasegovi%2Fsynapse-demo%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3Evasegovi%2Fsynapse-demo%20(github.com)%3C%2FA%3E%3C%2FP%3E%0A%3CP%3EBut%20let%20me%20tell%20you%20here%20some%20of%20lessons%20I%20learned%20when%20I%20was%20implementing%20this%2C%20first%20of%20all%20I%20search%20for%20some%20similar%20examples%20on%20internet%20and%20found%20this%20one%20%3CA%20href%3D%22https%3A%2F%2Fwww.mikestephenson.me%2F2021%2F08%2F08%2Ftrying-out-event-hub-capture-to-synapse-part-1%2F%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3ETrying%20out%20Event%20Hub%20Capture%20to%20Synapse%3C%2FA%3E.%20It%20covers%20almost%20all%20the%20things%20that%20I%20was%20interested%20in%20so%20you%20can%20also%20go%20there%20and%20follow%20it%2C%20and%20then%20come%20here%20and%20review%20some%20topics%20that%20you%20can%20add%20in%20your%20learning.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSTRONG%3EEvent%20Hub%3C%2FSTRONG%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EWith%20the%20capture%20feature%20you%20can%20easily%20configure%20the%20way%20and%20the%20place%20where%20you%20need%20your%20data.%20Some%20important%20aspects%20that%20you%20should%20consider%20are%3A%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3EDefine%20your%20path%20and%20convention%20names%20to%20avoid%20large%20paths%20in%20your%20data%20lake%3C%2FLI%3E%0A%3CLI%3EDefine%20from%20the%20beginning%20the%20partitions%20of%20your%20data%20lake%3C%2FLI%3E%0A%3CLI%3EDefine%20your%20compression%20format%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3ETo%20extend%20all%20this%20information%20visit%3A%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fevent-hubs%2Fevent-hubs-capture-overview%3FWT.mc_id%3DPortal-Microsoft_Azure_EventHub%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3ECapture%20events%20through%20Azure%20Event%20Hubs%20in%20Azure%20Blob%20Storage%20or%20Azure%20Data%20Lake%20Storage%3C%2FA%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSTRONG%3EBlob%20Permissions%3C%2FSTRONG%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThis%20is%20one%20of%20the%20most%20interesting%20points%20that%20you%20want%20to%20know%20about%20in%20detail%20because%20it%20was%20the%20most%20complex%20task%20that%20I%20had%20to%20deal%20with.%3C%2FP%3E%0A%3CP%3EThe%20table%20shows%20us%20all%20the%20users%20and%20access%20that%20you%20must%20add%20into%20your%20blob%20storage%2C%20the%20one%20that%20is%20working%20as%20Data%20Lake.%20the%20is%20the%20user%20that%20is%20going%20to%20run%20scripts%20manually%2C%20or%20for%20unit%20test%2C%20then%20the%20managed%20identity%20that%20we%20created.%20Then%20the%20name%20of%20your%20synapse%20workspace%20creates%20a%20Service%20Principal%20which%20you%20also%20have%20to%20add%20in%20the%20permissions%20because%20is%20the%20one%20that%20executes%20the%20script%20once%20is%20published%2C%20and%20last%20one%20the%20Azure%20service%20principal%20for%20Azure%20Data%20Factory%20to%20run%20triggers.%3C%2FP%3E%0A%3CP%3EWhy%20Azure%20Data%20Factory%3F%20Because%20inside%20Azure%20Synapse%20engine%20the%20ADF%20engine%20is%20involved%20and%20is%20the%20one%20that%20will%20execute%20the%20scripts%20when%20they%20are%20launched%20with%20the%20trigger%20event.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CTABLE%3E%0A%3CTBODY%3E%0A%3CTR%3E%0A%3CTD%20width%3D%22208%22%3E%3CP%3E%3CSTRONG%3Erole%2Fuser%3C%2FSTRONG%3E%3C%2FP%3E%0A%3C%2FTD%3E%0A%3CTD%20width%3D%22208%22%3E%3CP%3E%3CSTRONG%3Edescription%3C%2FSTRONG%3E%3C%2FP%3E%0A%3C%2FTD%3E%0A%3CTD%20width%3D%22208%22%3E%3CP%3E%3CSTRONG%3Epurpose%3C%2FSTRONG%3E%3C%2FP%3E%0A%3C%2FTD%3E%0A%3C%2FTR%3E%0A%3CTR%3E%0A%3CTD%20width%3D%22208%22%3E%3CP%3Eyour%20user%3C%2FP%3E%0A%3C%2FTD%3E%0A%3CTD%20width%3D%22208%22%3E%3CP%3Ethis%20is%20for%20the%20user%20logged%3C%2FP%3E%0A%3C%2FTD%3E%0A%3CTD%20width%3D%22208%22%3E%3CP%3EWhen%20you%20are%20running%20scripts%20manually%3C%2FP%3E%0A%3C%2FTD%3E%0A%3C%2FTR%3E%0A%3CTR%3E%0A%3CTD%20width%3D%22208%22%3E%3CP%3Emanaged%20identity%3C%2FP%3E%0A%3C%2FTD%3E%0A%3CTD%20width%3D%22208%22%3E%3CP%3Ethe%20managed%20identity%20we%20created%3C%2FP%3E%0A%3C%2FTD%3E%0A%3CTD%20width%3D%22208%22%3E%3CP%3EBecause%20it%20will%20be%20assigned%20as%20administrator%3C%2FP%3E%0A%3C%2FTD%3E%0A%3C%2FTR%3E%0A%3CTR%3E%0A%3CTD%20width%3D%22208%22%3E%3CP%3Ewssynapse%20(synapse%20workspace%20name)%3C%2FP%3E%0A%3C%2FTD%3E%0A%3CTD%20width%3D%22208%22%3E%3CP%3Ethe%20workspace%20synapse%20resource%3C%2FP%3E%0A%3C%2FTD%3E%0A%3CTD%20width%3D%22208%22%3E%3CP%3EFor%20pipelines%3C%2FP%3E%0A%3C%2FTD%3E%0A%3C%2FTR%3E%0A%3CTR%3E%0A%3CTD%20width%3D%22208%22%3E%3CP%3EADF%3C%2FP%3E%0A%3C%2FTD%3E%0A%3CTD%20width%3D%22208%22%3E%3CP%3EADF%20Service%20Principal%3C%2FP%3E%0A%3C%2FTD%3E%0A%3CTD%20width%3D%22208%22%3E%3CP%3EFor%20triggers%3C%2FP%3E%0A%3C%2FTD%3E%0A%3C%2FTR%3E%0A%3C%2FTBODY%3E%0A%3C%2FTABLE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EFor%20deeper%20explanations%20please%20visit%3A%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fdata-factory%2Fhow-to-create-event-trigger%3Ftabs%3Ddata-factory%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3ECreate%20a%20trigger%20that%20runs%20a%20pipeline%20in%20response%20to%20a%20storage%20event%3C%2FA%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSTRONG%3EGit%20Integration%3C%2FSTRONG%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThere%20are%20many%20branch%20strategies%20that%20you%20can%20follow%20in%20your%20application%20and%20infrastructure%20as%20well%20depending%20on%20several%20factors%20but%20there%20are%20important%20points%20that%20you%20should%20consider.%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3EDefine%20a%20branch%20strategy.%20Sometimes%20it%20is%20common%20that%20the%20project%20needs%20to%20be%20done%20quickly%20and%20just%20create%20branches%20and%20branches%20and%20at%20some%20point%2C%20there%E2%80%99s%20a%20mess%20and%20you%20and%20your%20team%20will%20need%20to%20invest%20more%20time%20fixing%20that.%3C%2FLI%3E%0A%3CLI%3EYou%20can%20only%20publish%20from%20your%20collaboration%20branch%3C%2FLI%3E%0A%3CLI%3EWhen%20you%20publish%20your%20changes%20the%20branch%20that%20you%20set%20in%20your%20configuration%20as%20Publish%20branch%20will%20create%20and%20only%20have%20the%20ARM%20with%20all%20the%20artifacts%20in%20your%20workspace%20(except%20for%20pools)%3C%2FLI%3E%0A%3CLI%3EHighly%20recommended%20to%20protect%20your%20collaboration%20branch%20with%20PR%20so%20you%20avoid%20works%20losses.%3C%2FLI%3E%0A%3CLI%3EBefore%20going%20further%20in%20your%20development%20read%20this%20articles%20that%20will%20help%20you%20to%20understand%20more%20about%20this%20topic%3A%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fdata-factory%2Fcontinuous-integration-delivery%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3EContinuous%20integration%20and%20delivery%20in%20Azure%20Data%20Factory%3C%2FA%3E%2C%20%3CA%20href%3D%22https%3A%2F%2Fbuddy.works%2Fblog%2F5-types-of-git-workflows%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3E5%20types%20of%20Git%20workflows%20that%20will%20help%20you%20deliver%20better%20code%3C%2FA%3E%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSTRONG%3EPython%20Client%3C%2FSTRONG%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThis%20Python%20client%20is%20useful%20to%20send%20information%20to%20your%20Event%20Hubs.%20The%20unique%20goal%20of%20this%20piece%20is%20to%20simulate%20a%20device%20which%20sends%20all%20data%20to%20Event%20hubs%2C%20and%20ideal%20architecture%20looks%20like%20the%20image%20below%2C%20where%20you%20have%20several%20sources%20to%20capture%20the%20info%20needed%20for%20your%20solution.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Vanessa_Segovia_1-1655414310579.jpeg%22%20style%3D%22width%3A%20400px%3B%22%3E%3Cspan%20class%3D%22lia-inline-image-display-wrapper%22%20image-alt%3D%22Vanessa_Segovia_1-1655414310579.jpeg%22%20style%3D%22width%3A%20400px%3B%22%3E%3Cimg%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F381078iA36DFA79FF9B82A6%2Fimage-size%2Fmedium%3Fv%3Dv2%26amp%3Bpx%3D400%22%20role%3D%22button%22%20title%3D%22Vanessa_Segovia_1-1655414310579.jpeg%22%20alt%3D%22Vanessa_Segovia_1-1655414310579.jpeg%22%20%2F%3E%3C%2Fspan%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%3CSTRONG%3EChallenge%3C%2FSTRONG%3E%3C%2FP%3E%0A%3CP%3EWhen%20you%20finish%20the%20steps%20in%20the%20repo%20and%20complete%20the%20challenge%2C%20please%20come%20back%20and%20let%20me%20know%20in%20the%20comments%20how%20was%20it%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSTRONG%3ENext%20Steps%3A%3C%2FSTRONG%3E%3C%2FP%3E%0A%3CP%3EI%20will%20continue%20with%20my%20end-to-end%20scenario%20so%20keep%20in%20touch%20to%20see%20part%202%20when%20we%E2%80%99ll%20add%20Machine%20Learning%20to%20analyze%20this%20information%2C%20meanwhile%20you%20can%20learn%20more%20about%20streaming%20and%20synapse%20let%20you%20here%20some%20useful%20links%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fevent-hubs%2Fevent-hubs-capture-python%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3ECapture%20Event%20Hubs%20data%20in%20Azure%20Storage%20and%20read%20it%20by%20using%20Python%3C%2FA%3E%3C%2FP%3E%0A%3CP%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Farchitecture%2Fexample-scenario%2Fdata%2Fstream-ingestion-synapse%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3EHigh%20throughput%20stream%20ingestion%20to%20Azure%20Synapse%3C%2FA%3E%3C%2FP%3E%0A%3CP%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Farchitecture%2Fexample-scenario%2Fdataplate2e%2Fdata-platform-end-to-end%3Ftabs%3Dportal%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3EAnalytics%20end-to-end%20with%20Azure%20Synapse%3C%2FA%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-TEASER%20id%3D%22lingo-teaser-3518504%22%20slang%3D%22en-US%22%3E%3CP%3EReal-time%20ingestion%20with%20Event%20Hub%20and%20Synapse%3C%2FP%3E%3C%2FLINGO-TEASER%3E%3CLINGO-LABS%20id%3D%22lingo-labs-3518504%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EData%20%26amp%3B%20AI%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E