Tag Archives: Azure

Azure API Management

Azure API Management provides a set of services that can help users to manage API Lifecycle, i.e. Design. Mock, Deploy, Policy Management, Explore, Consume, and Monitor APIs.

Diagram showing key components of Azure API Management.
https://learn.microsoft.com/en-us/azure/api-management/api-management-key-concepts

we can see there are three core components here. Developer portal helping consumers to discover, try out and onboard to services. Management Plane helps providers manage the API policies and monitor them. The gateway is the interface between consumer clients and provider applications.

API Gateway

The API gateway acts as a facade to the backend services, allowing API providers to abstract API implementations and evolve backend architecture without impacting API consumers. The gateway enables consistent configuration of routing, security, throttling, caching, and observability.

https://learn.microsoft.com/en-us/azure/api-management/api-management-key-concepts#api-gateway

To create API Gateway, you will need to go to Azure Portal -> API Management Service -> Create

Management Plane

API providers interact with the service through the management plane, which provides full access to the API Management service capabilities. Customers interact with the management plane through Azure tools including the Azure portal, Azure PowerShell, Azure CLI, a Visual Studio Code extension, or client SDKs in several popular programming languages.

https://learn.microsoft.com/en-us/azure/api-management/api-management-key-concepts#management-plane

If Gateway was about implementing policies in real-time, Management plane is about helping developers set these policies and interact with analytics dashboards via portal VC Code extension or other Azure interfaces.

Developer Portal

App developers use the open-source developer portal to discover the APIs, onboard to use them, and learn how to consume them in applications.

https://learn.microsoft.com/en-us/azure/api-management/api-management-key-concepts#developer-portal

The developer portal allows consumers to search for APIs, explore them, consume them and view analytics from the consumer side.

Azure: Region pairs, Billing and Subscription hierarchy

Some old notes from Azure

Screenshot of the hierarchy for objects in Azure.
https://learn.microsoft.com/

Azure region pairs

Availability zones are created by using one or more datacenters. There’s a minimum of three zones within a single region. It’s possible that a large disaster could cause an outage big enough to affect even two datacenters. That’s why Azure also creates region pairs.

What is a region pair?

Each Azure region is always paired with another region within the same geography (such as US, Europe, or Asia) at least 300 miles away. This approach allows for the replication of resources (such as VM storage) across a geography that helps reduce the likelihood of interruptions because of events such as natural disasters, civil unrest, power outages, or physical network outages that affect both regions at once. If a region in a pair was affected by a natural disaster, for instance, services would automatically failover to the other region in its region pair.

Customize billing to meet your needs

If you have multiple subscriptions, you can organize them into invoice sections. Each invoice section is a line item on the invoice that shows the charges incurred that month. For example, you might need a single invoice for your organization but want to organize charges by department, team, or project.

Depending on your needs, you can set up multiple invoices within the same billing account. To do this, create additional billing profiles. Each billing profile has its own monthly invoice and payment method.

The following diagram shows an overview of how billing is structured. If you’ve previously signed up for Azure or if your organization has an Enterprise Agreement, your billing might be set up differently.

Flowchart-style diagram showing an example of setting up a billing structure where different groups like marketing or development have their own Azure subscription that rolls up into a larger company-paid Azure billing account.
https://learn.microsoft.com/en-us/training/modules/azure-architecture-fundamentals/management-groups-subscriptions

Hierarchy of management groups and subscriptions

You can build a flexible structure of management groups and subscriptions to organize your resources into a hierarchy for unified policy and access management. The following diagram shows an example of creating a hierarchy for governance by using management groups.

Diagram showing an example of a management group hierarchy tree.
https://learn.microsoft.com/en-us/training/modules/azure-architecture-fundamentals/management-groups-subscriptions

You can create a hierarchy that applies a policy. For example, you could limit VM locations to the US West Region in a group called Production. This policy will inherit onto all the Enterprise Agreement subscriptions that are descendants of that management group and will apply to all VMs under those subscriptions. This security policy can’t be altered by the resource or subscription owner, which allows for improved governance.

Azure: Designing Data Flows

Data Flow stages can be defined at a high level as

Ingest -> Transform -> Store -> Analyze.

Data can be processed as Batch Process, which is not a real time processing of data, for example, one collects sales data for whole day and run analytics at the end of the day. Stream processing of data is near real time, where data is processed as it is recieved.

ELT vs ETL: Extract Load and Transform vs Extract Trandorm and Load. As the terms suggests, in ELT, data is loaded first to storage and than transformed. whereas in ETL data is transofrmed first and than loaded. In case of large amount of data, ETL will be difficult as processing large data in real time will be difficult, so ELT might be preffered.

Data Management in Cloud: Azure provides multiple solution for data flow. When choosing a solution one needs to take care of following aspects, Security, Storage Type (IaaS vs PaaS, Blob, File, Database, etc), Performance, Cost, redunancy, availabillity, etc.

Let’s take a look at some important solutions by Azure

Azure Data Lake Storage: Azure Data Lake is a scalable data storage and analytics service. 

Azure Data Factory: Azure Data Factory is Azure’s cloud ETL service for scale-out serverless data integration and data transformation.

Azure Database Services: Azure provides various options for RDBMS and No-SQL database storage.

Azure HDInsight: an open-source analytics service that runs Hadoop, Spark, Kafka and more.

Azure DataBricks: Azure Databricks is a fast, easy and collaborative Apache Spark-based big data analytics service designed for data science and data engineering.

Azure Synapse Analytics: Azure Synapse Analytics is a limitless analytics service that brings together data integration, enterprise data warehousing and big data analytics. It gives you the freedom to query data on your terms, using either serverless or dedicated options—at scale.

Here is how a typical data flow look in Azure

Power BI azure synapse architecture
https://powerbi.tips/2020/12/power-bi-architecture-in-a-data-solution/

Azure Networking – 2

User-defined routes
You can use a user-defined route to override the default system routes so traffic can be routed through firewalls or NVAs.

For example, you might have a network with two subnets and want to add a virtual machine in the perimeter network to be used as a firewall. You can create a user-defined route so that traffic passes through the firewall and doesn’t go directly between the subnets.

When creating user-defined routes, you can specify these next hop types:

Virtual appliance: A virtual appliance is typically a firewall device used to analyze or filter traffic that is entering or leaving your network. You can specify the private IP address of a NIC attached to a virtual machine so that IP forwarding can be enabled. Or you can provide the private IP address of an internal load balancer.
Virtual network gateway: Use to indicate when you want routes for a specific address to be routed to a virtual network gateway. The virtual network gateway is specified as a VPN for the next hop type.
Virtual network: Use to override the default system route within a virtual network.
Internet: Use to route traffic to a specified address prefix that is routed to the internet.
None: Use to drop traffic sent to a specified address prefix.

If there are multiple routes with the same address prefix, Azure selects the route based on the type in the following order of priority:

  • User-defined routes
  • BGP routes
  • System routes

A network virtual appliance (NVA) is a virtual appliance that consists of various layers like:

  • a firewall
  • a WAN optimizer
  • application-delivery controllers
  • routers
  • load balancers
  • IDS/IPS
  • proxies

Azure Networking

VNet Peering: Virtual network peering enables you to seamlessly connect two Azure virtual networks. Once peered, the virtual networks appear as one, for connectivity purposes. There are two types of VNet peering.

Regional VNet peering connects Azure virtual networks in the same region.
Global VNet peering connects Azure virtual networks in different regions.

A VPN gateway is a specific type of virtual network gateway that is used to send encrypted traffic between an Azure virtual network and an on-premises location over the public Internet. You also use a VPN gateway to send encrypted traffic between Azure virtual networks over the Microsoft network.

Site-to-site connections connect on-premises datacenters to Azure virtual networks
VNet-to-VNet connections connect Azure virtual networks (custom)
Point-to-site (User VPN) connections connect individual devices to Azure virtual networks

There are two types of load balancers: public and internal.

A public load balancer maps the public IP address and port number of incoming traffic to the private IP address and port number of the VM. Mapping is also provided for the response traffic from the VM. By applying load-balancing rules, you can distribute specific types of traffic across multiple VMs or services. For example, you can spread the load of incoming web request traffic across multiple web servers.

An internal load balancer directs traffic to resources that are inside a virtual network or that use a VPN to access Azure infrastructure.

Application gateway: There are two primary methods of routing traffic, path-based routing, and multiple site routing.

path: /images, /videos
site: kamalmeet.com, bizt.com

Gateway transit
You can connect to your on-premises network from a peered virtual network if you enable gateways transit from a virtual network that has a VPN gateway. Using gateway transit, you can enable on-premises connectivity without deploying virtual network gateways to all your virtual networks.

Overlapping address spaces
IP address spaces of connected networks within Azure, between Azure and your on-premises network, can’t overlap. This is also true for peered virtual networks.

A is the host record and is the most common type of DNS record. It maps the domain or hostname to the IP address.
CNAME is a Canonical Name record that’s used to create an alias from one domain name to another domain name.
MX is the mail exchange record. It maps mail requests to your mail server, whether hosted on-premises or in the cloud.
TXT is the text record. It’s used to associate text strings with a domain name. Azure and Microsoft 365 use TXT records to verify domain ownership.

Azure Application Insights

Application Insights is aimed at the development team, to help you understand how your app is performing and how it’s being used. It monitors:

Request rates, response times, and failure rates – Find out which pages are most popular, at what times of day, and where your users are. See which pages perform best. If your response times and failure rates go high when there are more requests, then perhaps you have a resourcing problem.
Dependency rates, response times, and failure rates – Find out whether external services are slowing you down.
Exceptions – Analyze the aggregated statistics, or pick specific instances and drill into the stack trace and related requests. Both server and browser exceptions are reported.
Performance counters from your Windows or Linux server machines, such as CPU, memory, and network usage.
Diagnostic trace logs from your app – so that you can correlate trace events with requests.
Custom events and metrics that you write yourself in the client or server code, to track business events such as items sold or games won.

Azure Role Based Access Control

Role-based access control (RBAC) helps you manage who has access to Azure resources, what they can do with those resources, and what areas they have access to.

Security principal (who). An object that represents something that is requesting access to resources. Examples: user, group, service principal, managed identity
Role definition (what). Collection of permissions that lists the operations that can be performed. Examples: Reader, Contributor, Owner, User Access Administrator
Scope (where). Boundary for the level of access that is requested. Examples: management group, subscription, resource group, resource
Assignment. Attaching a role definition to a security principal at a particular scope. Users can grant access described in a role definition by creating an assignment. Deny assignments are currently read-only and can only be set by Azure.

You want the external team to collaborate with the internal developer team in a process that’s easy and secure. With Azure Active Directory (Azure AD) business-to-business (B2B), you can add people from other companies to your Azure AD tenant as guest users.

Why use Azure AD B2B instead of the federation?
With Azure AD B2B, you don’t take on the responsibility of managing and authenticating the credentials and identities of partners. Giving access to external users is much easier than in a federation. You don’t need an AD administrator to create and manage external user accounts.

Azure Cost Management

Azure Cost Management helps you plan for and control your costs, the services include- Cost analysis, budgets, recommendations, and exporting cost management data.

Cost analysis. You use cost analysis to explore and analyze your organizational costs. You can view aggregated costs by the organization to understand where costs are accrued and to identify spending trends.
Budgets. Budgets help you plan for and meet financial accountability in your organization. They help prevent cost thresholds or limits from being surpassed. Budgets can also help you inform others about their spending to proactively manage costs.
Recommendations. Recommendations show how you can optimize and improve efficiency by identifying idle and underutilized resources. Or, they can show less expensive resource options.
Exporting cost management data. If you use external systems to access or review cost management data, you can easily export the data from Azure.

You can apply tags to your Azure resources to logically organize them by categories. Each tag consists of a name and a value. For example, you can apply the name Environment and the value Production or Development to your resources. After creating your tags, you associate them with the appropriate resources.

If your organization has several subscriptions, you may need a way to efficiently manage access, policies, and compliance for those subscriptions. Azure management groups provide a level of scope above subscriptions. You organize subscriptions into containers called management groups and apply your governance conditions to the management groups.

Azure Policy is a service in Azure that you use to create, assign, and manage policies. These policies enforce different rules over your resources, so those resources stay compliant with your corporate standards and service level agreements.

Azure Virtual Machines

A virtual machine is the basic compute service provided by any cloud service provider. It falls under Infrastructure as a service, where you create a Windows or Linux Machine and manage it.

Sample AZ CLI command

az vm create \
--resource-group resource-group-for-testing \
--location westus \
--name SampleVM \
--image UbuntuLTS \
--admin-username azureuser \
--generate-ssh-keys \
--verbose

Unmanaged vs. Managed disks
The final storage choice you’ll make is whether to use unmanaged or managed disks.

With unmanaged disks, you are responsible for the storage accounts that are used to hold the VHDs corresponding to your VM disks. You pay the storage account rates for the amount of space you use.

Managed disks are the newer (and recommended) disk-storage model. They elegantly solve the complexity of unmanaged disks by putting the burden of managing the storage accounts onto Azure.

Increased reliability: Azure ensures that VHDs associated with high-reliability VMs will be placed in different parts of Azure storage to provide similar levels of resilience.
Better security: This means they can use role-based access control to restrict who can work with the VHD data.
Snapshot support: You can use snapshots to create a read-only copy of a VHD.
Backup support: You can automatically back up managed disks to different regions for disaster recovery with Azure Backup, all without affecting the service of the VM.

What is a Network Security Group?
Virtual networks (VNets) are the foundation of the Azure networking model and provide isolation and protection. Network Security Groups (NSGs) are the main tool you use to enforce and control network traffic rules at the networking level.

Security group rules
NSGs use rules to allow or deny traffic moving through the network. Each rule identifies the source and destination address (or range), protocol, port (or range), direction (inbound or outbound), a numeric priority, and whether to allow or deny the traffic that matches the rule. The following illustration shows NSG rules applied at the subnet and network-interface levels.

Keep in mind that security groups are optional at both levels. If no security group is applied, then all traffic is allowed by Azure. If the VM has a public IP, this could be a serious risk, particularly if the OS doesn’t provide some sort of firewall.

Bastion connections
The Azure Bastion service is a new fully platform-managed PaaS service that you provision inside your virtual network. It provides secure and seamless RDP/SSH connectivity to your virtual machines directly in the Azure portal over SSL. When you connect via Azure Bastion, your virtual machines do not need a public IP address.

Azure Active Directory

Azure AD (Active Directory) is a directory and identity management solution.

Concepts for AAD

  • Identity. An object that can get authenticated. An identity can be a user with a username and password. Identities also include applications or other servers that might require authentication through secret keys or certificates.
  • Account. An identity that has data associated with it. You can’t have an account without an identity.
  • Azure AD Account. An identity is created through Azure AD or another Microsoft cloud service, such as Microsoft 365. Identities are stored in Azure AD and accessible to your organization’s cloud service subscriptions. This account is also sometimes called a Work or school account.
  • Azure subscription. Used to pay for Azure cloud services. You can have many subscriptions and they’re linked to a credit card.
  • Azure tenant/directory. A dedicated and trusted instance of Azure AD, a Tenant is automatically created when your organization signs up for a Microsoft cloud service subscription.

Azure Active Directory has free as well as premium versions

  • Free: Provides user and group management, on-premises directory synchronization, basic reports, and single sign-on across Azure, Microsoft 365, and many popular SaaS apps.
  • Premium P1: In addition to the Free features, P1 also lets your hybrid users access both on-premises and cloud resources. It also supports advanced administration, such as dynamic groups, self-service group management, Microsoft Identity Manager (an on-premises identity and access management suite), and cloud write-back capabilities, which allow self-service password reset for your on-premises users.
  • Premium P2: In addition to the Free and P1 features, P2 also offers Azure Active Directory Identity Protection to help provide risk-based Conditional Access to your apps and critical company data. Privileged Identity Management is included to help discover, restrict, and monitor administrators and their access to resources and to provide just-in-time access when needed.