Event Hubs:
Azure Event Hubs is a highly scalable publish-subscribe service that can ingest millions of events per second
and stream them into multiple applications. This lets you process and analyze the massive amounts of
data produced by your connected devices and applications.
https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-features
- Namespace: Network endpoints and a range of access control and network integration management
- Event retention: Default value and shortest possible retention period is 1 day (24 hours)
- Publisher policy: Enables granular control over event publishers through publisher policies
- Capture: Blob storage account, or an Azure Data Lake Storage account
- Partitions: Organizes sequences of events sent to an event hub into one or more partitions, FIFO
- Event consumers: Any entity that reads event data from an event hub is an event consumer (AMQP 1.0)
Event Hubs frequently asked questions
- What is an Event Hubs namespace?
- When do I create a new namespace vs. use an existing namespace?
- What is the difference between Event Hubs Basic and Standard tiers?
- Where is Azure Event Hubs available?
- Can I use a single Advanced Message Queuing Protocol (AMQP) connection to send and receive from multiple event hubs?
- What is the maximum retention period for events?
- How do I monitor my Event Hubs?
- Where does Azure Event Hubs store data?
- What ports do I need to open on the firewall?
- What IP addresses do I need to allow?
Frequently asked questions – Azure Event Hubs | Microsoft Docs
Quickstart: Create an event hub using Azure portal
Azure Quickstart – Create an event hub using the Azure portal – Azure Event Hubs | Microsoft Docs
- On the Create namespace page, take the following steps:
- Select the subscription in which you want to create the namespace.
- Select the resource group you created in the previous step.
- Enter a name for the namespace. The system immediately checks to see if the name is available.
- Select a location for the namespace.
- Choose Basic for the pricing tier. To learn about differences between tiers, see Quotas and limits, Event Hubs Premium, and Event Hubs Dedicated articles.
- Leave the throughput units (for standard tier) or processing units (for premium tier) settings as it is. To learn about throughput units or processing units: Event Hubs scalability.
Tiers and there is also premium
standard: 20,1000, ie, ret 7, schema reg, storage account capture
Create an Event Hubs namespace
testit4eventhub
Pricing for each Throughput Unit per month
Capacity allocations throughput units (TUs) or processing units (PUs)) are billed at the namespace level.
Create an event hub
Create an event hub within the namespace
- On the Event Hubs Namespace page, select Event Hubs in the left menu.
- At the top of the window, select + Event Hub.
- Type a name for your event hub, then select Create.
- The partition count setting allows you to parallelize consumption across many consumers. For more information, see Partitions.
- The message retention setting specifies how long the Event Hubs service keeps data. For more information, see Event retention.
- You can check the status of the event hub creation in alerts. After the event hub is created, you see it in the list of event hubs.
Connection string for a specific event hub in a namespace
Get connection string – Azure Event Hubs – Azure Event Hubs | Microsoft Docs
- On the Event Hubs Namespace page, select the event hub in the bottom pane.
- On the Event Hubs instance page, select Shared access policies on the left menu.
- There’s no default policy created for an event hub. Create a policy with Manage, **Send, or Listen access.
- Select the policy from the list.
- Select the copy button next to the Connection string-primary key field.
Test send
send.py
Well we sendt some requests, but there seems to be a issue…, Authentication Put-Token failed. Retries exhausted in between:
# WE ARE NOW USING THE testit4hub (testit4eventhub/testit4hub) | Shared access policies ->AdminPolicy->Connection string–primary key
Authentication Put-Token failed. Retries exhausted.
Authentication Put-Token failed. Retries exhausted.
Authentication Put-Token failed. Retries exhausted.
Authentication Put-Token failed. Retries exhausted.
[…]
azure.eventhub.exceptions.AuthenticationError: The messaging entity xxxx could not be found
TrackingId: xxxxxx Timestamp:2022-01-29T15:12:00
CBS Token authentication failed.
Status code: 404
Description: The messaging entity xxxxx could not be found, xxx Timestamp:2022-01-29T15:12:00
hm….is this because of the rg in West Europe?
Try Norway Region, rm testit4eventhub and re-create it in Norway
Now the Resourcegroup, eventhubnamespace and eventhub is in Norway East and we create a policy in the SAS eventhub named MyAdminPolicyNorway.
What is the timestamp now? The time in Norway is 16:30
Authentication Put-Token failed. Retries exhausted.
Authentication Put-Token failed. Retries exhausted.
Authentication Put-Token failed. Retries exhausted.
Authentication Put-Token failed. Retries exhausted.
Timestamp:2022-01-29T15:29:50
MY BAD, MY BAD NOTE!!!
The Python code was wrong:
- conn_str = testit4hub (testit4eventhub/testit4hub) | Shared access policies-> SAS-> MyAdminPolicyNorway->Connection string–primary key
- eventhub_name = testit4hub (the instance, this was set to testit4eventhub)
Lets look at the result after sending 9 msg:
Python:
PS C:\giti2021\azure-arm-104\8_extra_tutorials\python_sdk_eventhub> python .\send.py
Messages sent to event hub success
Portal for testit4eventhub/testit4hub should be 9 incoming success and no outgoing yet
Cool, let rm the rg testit4 ro rm all resources we created.
code for send
TODO:
View msg
Where is Azure Event Hub messages stored? – Stack Overflow
You don’t have direct access to the transient storage used for EventHub messages, but you could write a consumer that reads from the EventHub continuously and persist the messages
Capture msg in storage account TDB
Receive events TBD
Create an event hub (within the namespace)
Partition count setting allows you to parallelize consumption across many consumers
Message retention setting specifies how long the Event Hubs service keeps data
(Capture on, enable auto delivery to blob or lake, no admin cost for it.)
Container
testit4container