wkaczurba

Eventgrid + EventHub

Eventgrid

eventgrid schemas deliovery durability control access to events receive events by using webhooks filter events

Exercise - route custom events to web endpoint by using Azure CLI

Tutorial source

Registering a provider:

The code below will setup eventgrid + create ARM-based webapp that gets the events. Example event to be send:

[
  {
    "id": "2601",
    "eventType": "recordInserted",
    "subject": "myapp/vehicles/motorcycles",
    "eventTime": "2023-06-01T04:40:46+0200",
    "data": {
      "make": "Contoso",
      "model": "Monster"
    },
    "dataVersion": "1.0"
  }
]

GIST source: https://gist.github.com/wkaczurba/e9940d689e6a764c56777e31c1b77211

#!/bin/bash

# Login if not created:
#az login

RANDOM_SUFFIX="4def88"

RESOURCE_GROUP=eventgrid-demo-${RANDOM_SUFFIX}-rg
MY_LOCATION=westeurope
MY_TOPIC_NAME=eg-topic-${RANDOM_SUFFIX}
MY_SITE_NAME=eg-site-${RANDOM_SUFFIX}
MY_SITE_URL="https://${MY_SITE_NAME}.azurewebsites.net"

if [ $(az group exists --name $RESOURCE_GROUP) = false ]; then
    echo Creating resource group $RESOURCE_GROUP:
    az group create --resource-group $RESOURCE_GROUP --location $MY_LOCATION
else
    echo Resource group $RESOURCE_GROUP exists. Not creating.
fi

# Enabling eventgrid resource provider
# NOTE: Needed only on subscription w/o Microosft.EventGrid enabled - go in portal: subscription -> resource-providers

az provider register --namespace Microsoft.EventGrid

# Creating a custom topic.

az eventgrid topic create --name $MY_TOPIC_NAME --location $MY_LOCATION --resource-group $RESOURCE_GROUP

# Creating a webapp - endpoint (using Azure's samples)

az deployment group create --resource-group $RESOURCE_GROUP \
    --template-uri "https://raw.githubusercontent.com/Azure-Samples/azure-event-grid-viewer/main/azuredeploy.json" \
    --parameters siteName=$MY_SITE_NAME hostingPlanName=viewerhost

echo "The web app URL: ${MY_SITE_URL}"

# Subscribe to a custom topic:

ENDPOINT="${MY_SITE_URL}/api/updates"
SUB_ID=$(az account show --subscription "" | jq -r '.id')

az eventgrid event-subscription create \
    --source-resource-id "/subscriptions/$SUB_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.EventGrid/topics/$MY_TOPIC_NAME" \
    --name az2204viewerSub \
    --endpoint $ENDPOINT

# Send an event:

# Retrieving URL+key for custom topic:
TOPIC_ENDPOINT=$(az eventgrid topic show --name $MY_TOPIC_NAME -g $RESOURCE_GROUP --query "endpoint" --output tsv)
KEY=$(az eventgrid topic key list --name $MY_TOPIC_NAME -g $RESOURCE_GROUP --query "key1" --output tsv)

EVENT='[ {"id": "'"$RANDOM"'", "eventType": "recordInserted", "subject": "myapp/vehicles/motorcycles", "eventTime": "'`date +%Y-%m-%dT%H:%M:%S%z`'", "data":{ "make": "Contoso", "model": "Monster"},"dataVersion": "1.0"} ]'

curl -X POST -H "aeg-sas-key: $KEY" -d "$EVENT" $TOPIC_ENDPOINT

Eventhub

Event Hubs Capture tutorial - eventhubs capture

Scale your processing app tutorial

  1. Idea of scaling application throuhg partitoned consumers: EventProcessorClient .NET/Java, EventHubConsumerClient for Python/Javascript
  2. Discusses Scaling/Load balancing/Seamless resume on failures/Consumption of messages
  3. Some similarities to Kinesis Client Library (AWS) but KCL/Kinesis did not have partitions/separate topics as such - KCL subscribed to shards.

More on the Client itself: https://learn.microsoft.com/en-us/dotnet/api/azure.messaging.eventhubs.eventprocessorclient?view=azure-dotnet

Control access to events tutorial

Access - as for other services:

Authorization using:

Perform common operations with the event hubs client library

Describes:

Message-based solutions

Service bus vs Queues:

Service bus

Basics:

Tiers:

Advanced features:

Service bus queues, topics and subscriptions

ref: tutorial

Service bus message payloads and serialization

Tutorial

Routing/correlation:

NOTE: sender owns queue for responses. Responses contain correlationID

Payload serialziation:

Send+receive message from a Service Bus queue by using .NET

Ref: tutorial

#!/bin/bash

LOCATION=westeurope
MY_NAMESPACE_NAME=servicebus-demo-123abc0
RESOURCE_GROUP=servicebus-demo-rg
SERVICE_BUS_NAME=service-queue

az group create --name $RESOURCE_GROUP --location $LOCATION

# Create service bus:
az servicebus namespace create \
    --resource-group $RESOURCE_GROUP \
    --name $MY_NAMESPACE_NAME \
    --location $LOCATION

# Create service bus queue:
az servicebus queue create --resource-group $RESOURCE_GROUP \
    --namespace-name $MY_NAMESPACE_NAME \
    --name $SERVICE_BUS_NAME

Then:

Result - peeking in Service Bus Explorer:

messages

properties

Queues

Basics

Tutorial

Create + manage Azure Queue Storage and messages by using .NET

Tutorial

TODO: Write a demo program for it.:

C#: