Tag Archives: Azure

Azure ARM Templates

There are times when you need to replicate a deployment environment. A very common example is that a team usually has multiple environments for development, testing, staging, and production. Also at times, you need a secondary production environment for resiliency. Manually creating environments can be a lengthy and error-prone process. A simple configuration miss can be hard to detect and cause major issues.

ARM or Azure Resource Manager templates comes to help here. ARM provides us to implement Infrastructure as Code.

ARM templates are JavaScript Object Notation (JSON) files that define the infrastructure and configuration for your deployment. The template uses a declarative syntax. The declarative syntax is a way of building the structure and elements that outline what resources will look like without describing its control flow. Declarative syntax is different than imperative syntax, which uses commands for the computer to perform. Imperative scripting focuses on specifying each step in deploying the resources.

https://docs.microsoft.com/en-gb/learn/modules/create-azure-resource-manager-template-vs-code/2-explore-template-structure?tabs=azure-cli

ARM templates are idempotent, which means you can deploy the same template many times and get the same resource types in the same state.

Element Description

  • schema: A required section that defines the location of the JSON schema file that describes the structure of JSON data.
  • contentVersion: A required section that defines the version of your template (such as 1.0.0.0).
  • apiProfile: An optional section that defines a collection of API versions for resource types.
  • parameters: An optional section where you define values that are provided during deployment.
  • variables: An optional section where you define values that are used to simplify template language expressions.
  • resources: A required section that defines the actual items you want to deploy or update in a resource group or a subscription.
  • output: An optional section where you specify the values that will be returned at the end of the deployment.

Example

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.1",
  "apiProfile": "",
  "parameters": {},
  "variables": {},
  "functions": [],
  "resources": [
    {
      "type": "Microsoft.Storage/storageAccounts",
      "apiVersion": "2019-06-01",
      "name": "learntemplatestorage123",
      "location": "westus",
      "sku": {
        "name": "Standard_LRS"
      },
      "kind": "StorageV2",
      "properties": {
        "supportsHttpsTrafficOnly": true
      }
    }
  ],
  "outputs": {}
}

Azure Kubernetes Service

AKS or Azure Kubernetes Service is a managed Kubernetes service by Azure. It helps developers to focus on development and let Azure take care of providing Kuebernets and in turn manage containers.

Before getting into AKS, Azure also provides Azure Container Instance to create and manage containers individually.

Azure Container Instances offers the fastest and simplest way to run a container in Azure, without having to manage any virtual machines and without having to adopt a higher-level service. Azure Container Instances is a great solution for any scenario that can operate in isolated containers, including simple applications, task automation, and build jobs.

AKS Terminology

  • Pools are groups of nodes with identical configurations.
  • Nodes are individual virtual machines running containerized applications.
  • Pods are a single instance of an application. A pod can contain multiple containers.
  • Container is a lightweight and portable executable image that contains software and all of its dependencies.
  • Deployment has one or more identical pods managed by Kubernetes.
  • Manifest is the YAML file describing a deployment.

https://docs.microsoft.com/en-us/learn/modules/configure-azure-kubernetes-service/3-kubernetes-clusters

The kubelet is the Kubernetes agent that processes the orchestration requests from the Azure-managed node, and scheduling of running the requested containers.

Virtual networking is handled by the kube-proxy on each node. The proxy routes network traffic and manages IP addressing for services and pods.

The container runtime is the component that allows containerized applications to run and interact with additional resources such as the virtual network and storage.

https://docs.microsoft.com/en-us/learn/modules/configure-azure-kubernetes-service/3-kubernetes-clusters

Azure Compute

Azure provides a set of compute options, ranging from Virtual Machines, Container-based deployment support (includes AKS or Azure Kubernetes Services), Azure Web apps, Azure functions, etc.

Virtual Machines: Give you control over infrastructure as well as Operation systems.

When one is provisioning VMs, be aware of two types of downtimes that can occur.

Planned Maintenance: When Microsoft makes an update to the Azure platform, for example, security patches being deployed.

Unplanned Maintenance: There can be issues like power failures, which will cause the downtime

To handle these issues we have various options

Availability Sets: Availability sets allows you to manage update domain and fault domains.

An update domain is a group of virtual machines and underlying physical hardware that can be rebooted at the same time. When planned maintenance is performed on the Azure platform, only one update domain is rebooted at a time. This ensures that all VMs and associated hardware are not taken down at the same time.

A fault domain is a group of virtual machines that shares a common power source and a common network switch. When virtual machines are added to an availability set, they are distributed across up to three different fault domains in resource manager deployments, or across two fault domains in classic deployments.

Availability Zones: It is physically separate zones within an Azure region. There are usually three zones in one region. To increase the availability of your application, you should deploy it in more than one zone.

Multi-Region Deployment: Once can further deploy to multiple regions to avoid region failures.

Virtual Machine ScaleSet: Azure virtual machine scale sets let you create and manage a group of load-balanced VMs. You can set rules for scaling as CPU percent goes beyond 60%. In addition, one can set Load Balancer which helps manage traffic at layer 4, or Application Gateway which helps distribute traffic at layer 7.

Azure Functions: This helps serverless computing, one can use a consumption-based plan where you pay only for the time the function was running. But it has limitations like the function can run for a max of 10 minutes. One can upgrade to an App service plan to avoid these limitations.

Azure Logic Apps: You can create workflows, to create low code/ no code applications.

Azure Kubernetes Service: Azure implementation for managing Kubernetes and Container based deployments.

Azure Service Fabric: Another offering to manage container-based deployments, supports both synchronous and asynchronous applications.

Azure App Service: Works on the model where you get your code and directly deploy it on Azure, without worrying about deployment details. This supports deployment slots, for example, you can create stage and prod deployment slots, once you deploy and test on stage successfully, you can just swap with prod deployment.

Azure App Service

Azure App Service is a fully managed web application hosting platform. This platform as a service (PaaS) allows developers to focus on designing and building applications while Azure takes care of the infrastructure to run and scale.

Deployment slots: You can create a staging deployment slot where you can push your code to test on Azure. Once you are happy with your code, you can easily swap the staging deployment slot with the production slot.

Advantages of Azure App Service

  • Continuous integration/deployment support
  • Integrated Visual Studio publishing and FTP publishing
  • Built-in auto-scale support (automatic scale-out based on real-world load)

You use Azure Automation State Configuration to make sure that the virtual machines (VMs) in a cluster are in a consistent state, with the same software installed and the same configurations.

PowerShell Desired State Configuration (DSC): PowerShell DSC is a declarative management platform that Azure Automation State Configuration uses to configure, deploy, and control systems. A declarative programming language separates intent (what you want to do) from execution (how you want to do it). You specify the desired state and let DSC do the work to get there. You don’t have to know how to implement or deploy a feature when a DSC resource is available. Instead, you focus on the structure of your deployment.

Local configuration manager (LCM): LCM is a component of the Windows Management Framework (WMF) on a Windows operating system. The LCM is responsible for updating the state of a node, like a VM, to match the desired state.

Azure Backup

Azure Backup is Microsoft’s cloud-based backup solution. It requires no additional infrastructure setup. The solution allows doing a one-of backup as well as a scheduled backup through backup policies. One can backup, SQL databases, Disks, VMs, Files, and Blobs.

Data Redundancy

Locally Redundant Storage or LRS: protects data from server rack or drive failures. This replicates data three times within a single data center.

Zone Redundant Storage or ZRS: Replicates backup data in availability zones, to generate data residency and resiliency in the same region.

Geo Redundant Storage or GRS: Protects data from region-wide outages by replicating data to a secondary region.

MARS Agent: MARS or Microsoft Azure Recovery Service helps backup on-prem Windows machines.

Azure Backup Center: To view and manage backups for various resources from one place.

Backup Policies: You can create policies for resources like VMs, SQL, file share, etc. Backup policies are created on the backup-vault basis. One can set rules like when to take a backup along with retention details like how long each backup should be retained.

Azure: Analyze Resource Usage

For any cloud, you need to keep a watch on your resource usage from security, health, and cost perspective. Azure provides multiple tools for this purpose.

Azure Monitor: helps facilitate logging and collection of logs.

Three types of logs in azure:

  • Activity logs provide a record of operations executed against a resource.
  • Diagnostic logs record change details against a resource. Diagnostic logs schema has the following required fields.
    • time
    • resource id
    • tenant id
    • operation name (format Microsoft.<providerName>/<ResourceType>/<Subtype>/<Write/Read/Delete/Action>)
  • Application logs are logs generated by an application

Log Analytics: this is a log search and analytics tool that collects logs from various sources and correlates the data.

Alerting: Azure monitor provides support for alerting. There are two types of alerts – Metric Alert (e.g. CPU, Memory, disk, and network usage) and Log Alerts

Alert has three parts- Target (Resource), Criteria (Logic or Condition), and Action (Action Group)

Action Group is a reusable set of notifications, such as email, voice call, message, or push to an application such as creating a ticket.

Azure Advisor: Azure has some built-in algorithms to help identify cost savings. The advisor makes recommendations on the last 14 days and targets low utilization systems.

Monitoring cost: One way to monitor cost is by resource tagging. This helps view cost based on a tag. Tags can be enforced via Azure policies.

Azure Cost Management: A reporting tool for helping to analyze and understand Azure spending. You can apply a budget and generate alerts to monitor the spend.

Azure Storage

As we know, storage is one of the important features of a cloud service provider. Azure provides various options for storage.

The following data services are provided by Azure

Blob Storage: It is a scalable data store for text and binary data.

Blob storage provides for the storage of 3 types of Blobs. Page, Block, and Append. Page blobs are a collection of 512-byte pages that are optimized for read and write operations. A common use for page blobs is for virtual hard disks used by Azure virtual machines. Block Blobs are for the efficient storage of large Blobs up to 4.57 terabytes. The blocks within a Blob can be updated or deleted as needed. Append Blobs are a form of blob storage that is optimized for appending operations. Modified append Blobs write data blocks only to the end of the blob. Existing blocks cannot be deleted or updated. 

File Storage: As the name suggests, this storage is optimized for storing files.

Queue Storage: Datastore for reliable messaging between application components.

Table Storage: This is a NoSQL Datastore by Azure. It does not enforce schema but stores data in form of name-value pairs. As it is nonrelational data, no complex joins are supported.
The hierarchy is Storage Account>> Tables >> Entities
For example, Kamal’s Account >> Customers >> {Key Values, name: Kamal, Address: }. As this is a NoSQL datastore, different entities can have a different set of properties, one customer has an address another does not.

Disk Storage: Azure Managed Disks that are associated with VMs. The following type of disks are supported-
Ultra Disks
Premium SSD
Standard SSD
Standard HDD

Type of Storage Account

  • General purpose V2 (Recommended)
  • General purpose V1 (Legacy- use V2)
  • Block Bob Storage account (premium – faster access)
  • Files (Premium- For files)
  • Blob Storage (Legacy- use V2)

Storage Sync Service is the top-level resource for Azure File Sync. It manages the relationships between Azure Files file shares and Windows Server endpoints.
A sync group is a grouping of a file share and servers that keep a set of files in sync.

Cloud Basics: Azure

Cloud is a set of services that helps you with your software development lifecycle, from development to build to deploy to secure to monitor. Any cloud service provider will provide multiple services, basic ones would be Compute, Storage, and Networking.

Azure Provides the following set of services.

Compute

  • VMs (IaaS)
  • Azure App Service (PaaS)
  • Containers
  • AKS (Azure Kubernetes Service)
  • Azure Functions (Serverless, Pay as you use)

Storage

  • Blob storage (Object Storage) – Hot Tier/ Archive Tier
  • File Storage
  • Data Lake Storage – Hadoop Compatible
  • SQL Database
  • Azure DB for open source
  • Azure Cosmos DB (No-SQL)
  • Azure Cache for Redis

Networking

  • VNet
  • Subnets
  • VNet Peering
  • Azure VPN
  • Azure Expres Route

There are three popular ways provided by Azure to interact with and manage services

The Azure portal: the portal is a Graphical User Interface (GUI) that makes it convenient to locate the resource you need and execute any required changes.

The Azure CLI: The Azure CLI is a cross-platform command-line program to connect to Azure and execute administrative commands on Azure resources.

az vm create \
--resource-group CrmTestingResourceGroup \
--name CrmUnitTests \
--image UbuntuLTS
…

Azure PowerShell: Azure PowerShell is a module you add to PowerShell to let you connect to your Azure subscription and manage resources.

New-AzVm -ResourceGroupName "CrmTestingResourceGroup"
-Name "CrmUnitTests" `
-Image "UbuntuLTS"
…

Further Reading: https://docs.microsoft.com/en-us/azure/architecture/solution-ideas/articles/cloud-native-apps

Azure Subscription and Resource Groups

When onboarding to the cloud, the first thing you will create is a subscription. Under a Subscription, you will create one or more resource groups. Resource groups are logical containers. This is used to group a collection of resources such as VMs, VNets, storage accounts, etc. This logical grouping helps in provisioning, monitoring, and access control.

Resources in a Resource Group can span across multiple regions. One way to create Resource Groups is thinking of the deployment stage, for example, dev resources, test resources, and production resources. This will help group and manage resources based on criticality.

Another important aspect one needs to be aware of Role Based Access Controls (RBAC). There are four key roles.

  • Contributor: Manage resources but not user accounts
  • Owner: Manage Resources and can assign user roles/ access
  • Reader: View only access
  • User access Administrator: Can manage access for users but cannot manage resources

Azure Security Center: Creates a default security policy automatically for each azure subscription.

Example Policies

  • System Updates (any updates missing for a system e.g. a VM)
  • Security Configurations
  • Endpoint Protection
  • Disk Encryption
  • Network Security Groups
  • Web Application Firewall
  • Next-Generation Firewall
  • SQL auditing and Threat Detection
  • SQL Encryption
  • Vulnerability assessment
  • Storage Encryption
  • JIT Network Access

Resource Policies: Resource policies can be set for a subscription, for example. only a certain size of VMs are allowed or resources can be created in a specific location.

Resource Locks: Two type of locks can be applied to resources, CannotDelete and ReadOnly.

Tagging: Another powerful mechanism to logically group and organize the resources. This helps in billing and management, for example, you can create a billing report based on the tag “XYZ” as Project Name.

Note that one can also move resources between Resource Groups. During the movement, both target and source groups will be locked. While the movement of resources, the location of the resource is not changed, only the Resource Group is updated.

Azure Network Watcher

Network watcher provides a set of tools to help monitor various aspects of the network.

IP Flow Verify: It is a tool under network watcher, that helps check if access is allowed from a source to the destination address. It also highlights the rule responsible for allowing or blocking access.

Next Hop: shows which will be the next hop when the source device will connect to the destination device.

Connection Troubleshoot: One can deploy a “network watcher agent” on the VM, this helps to check the connection between the VM and destination. It shows the complete route with the time taken at each hop, hence helping debug performance or rule blocking the connection.

VPN troubleshoot tool: As the name suggests, helps troubleshoot connection problems between two VPNs.

Connection Monitor: between a VM and endpoint, monitors the connectivity, e.g. between two VMs, shows a report with round trip latency based on the time period selected. You can further set alerts for example alert if the latency goes beyond 200ms.

Traffic Analytics: gives a report on a high level, for virtual networks or subnetworks, it shows traffic flow details. If there are malicious flows, they will be highlighted in the report. shows complete traffic details, sources, and destinations, and also highlights data on the port level, so you can figure out which ports are getting the most malicious traffic.

Network performance monitor: As the name suggests, helps monitor performance for the network, i.e. time taken in terms of latency.