Tag Archives: Cloud computing

Azure Load Balancer

A load balancer is a tool that helps us manage traffic coming to a web application. In the simplest form, let’s say the application is deployed on two or more machines, so the role of load balancer here would be to make sure that incoming requests load is evenly distributed on all the machines. Also if one of the servers is down or not responding, the load balancer will be responsible for detecting this failure and redirect the traffic to healthy machines.

To see the load balancer in action, let’s bring up two (or more as per convenience) VM’s in Azure and install the IIS server.

Create a Resource -> Add Virtual Machine -> Choose “Windows Server 2016 Datacenter” image-> Add access for RDP (3089) and HTTP (80) ports.

Make sure both the machines are part of the same Availability Set (or Virtual Scale Set).

RDP to the machines, you will see Server manager (or bring it up)

Choose the option to Add roles and Features, and go ahead and add the IIS server.

Finally, make sure that the windows firewall allows traffic on port 80. Go to “Windows firewall and advanced security options” -> Inbound Rules -> Add New Rule ->Type port ->number 80.

Once the above steps are done, you can access the IIS server default page when you will hit the IP address of these VMs. To distinguish between the two webpages, you can make some modifications to either of them.

Go to C:\inetpub\wwwroot -> update html or image.

The next step is to set up the load balancer. Add a new resource -> Load balancer. First thing you will need to provide backend pool, for which you will choose the availability set in which both the VMs are available (or Virtual Machine Scale Set), next you will need to set up Health probe, as both our VMs are listening on port 80, you can simply set the port 80 for health probe. If the load balancer senses some problem with a machine based on the interval (seconds after which the load balancer ping the health probe) and unhealthy threshold (number of failures occurred after which load balancer treats the node as failure), the load balancer will stop sending traffic to that node.

Finally, you will set up a Load balancing rule, where all you need to provide is an incoming port on which traffic is expected, backend pool and health probe which we had already setup. Once this is set up, you can hit the load balancing URL and see that traffic is directed to the IIS page we set up earlier. If you will refresh the page multiple times you will be able to see traffic is going to both the server randomly. If one of the servers is shut down, the load balancer keeps on working fine with traffic redirected to the second server.

In addition to load balancer rule, one can also set up NAT rules, which are usually used for forwarding traffic on a port to a specific VM. Here is a good reference for that

https://rasmusg.net/2017/11/20/part-1-of-2-port-forwarding-in-azure/

https://rasmusg.azurewebsites.net/2017/11/20/part-2-of-2-port-forwarding-in-azure/

While we are on the topic of load balancers, it is important to note that there are two other ways in which we can control the traffic in Azure. These are Application Gateway and Traffic Manager. Here is a good comparison of different options for load balancing and which to prefer when https://devblogs.microsoft.com/premier-developer/azure-load-balancing-solutions-a-guide-to-help-you-choose-the-correct-option/

Azure Messaging Services

Another important factor in software development is messaging. With the popularity of Microservices and Serverless applications for scalable design, Message-based communication has received special focus.

Azure does provide us with multiple ways for message-based communication.

Azure Storage Queue: This is a simple form of messaging where one can create a queue under Azure Storage service, send and receive messages from the queue.

Azure Service Bus Queue: If you need more sophisticated queues with more control on data retention, create topics with publisher-subscriber pattern, dead letter queue support etc, Azure Service Bus queue is an option for you.

Azure Storage queue vs Service Bus queue: By this point, it is obvious to ask the question when should one use the Storage queue and when to use the Service queue. Let’s look at some important points to consider

  • Storage queue uses storage infrastructure to provide simple GET/ PUT/ PEEK operations on queues, whereas Storage bus uses proper message-based infrastructure, with that message can be received without constant polling by subscribing to queues and topics.
  • Storage bus provides features like FIFO, duplication detection, “At most once” delivery, etc.
  • Storage queue provides point to point communication whereas service bus can be used for multiple publishers – subscribers design.
  • Service Bus has a limit on queue size as 80GB, which is not there in Storage queue

More Detailed comparison- https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-azure-and-service-bus-queues-compared-contrasted

Azure Relay Service: If you want to expose a service on your local network to cloud, you can use Azure Relay service without too much of a hassle. This uses socket-based communication and you will need not open a firewall port or get into network-level communications by setting a VPN gateway.

More- https://docs.microsoft.com/en-us/azure/service-bus-relay/relay-what-is-it

Azure Event Grid: Sometimes we want to message an application or send an alert based on some event happening, for example, your CPU usage for a Virtual machine is more than 80%, you would like to alert the admin or trigger an Azure function to take some action.

Azure Event Hub: Event hub is more relevant for processing larger amount of data like telemetry or streaming data. A good example of Event hub usage is Azure Application insights which showcase important information about applications using telemetry data.

Azure Notifications Hub: Azure Notification hub is a solution that provides you with the functionality of sending messages to mobile applications and devices. You can send push notifications to millions of devices in one go using the notification hub.

Azure Virtual Networks

What is a virtual network?
Often, an application cannot be deployed in isolation on a single machine. There will be multiple servers interacting with each other. There might be multiple backend servers, frontend servers and databases involved. Often it is a requirement that these resources work together for an application to work smoothly. Virtual Network provides a virtual boundary inside which these resources can exist and communicate with each other, at the same time being isolated from the rest of the world.

Creating a Virtual Network

Creating a Virtual Network is pretty straightforward in Azure. You can select Virtual network resource and Add a new one. But while creation you will need to take care of two things – Address space and Subnet.

Address Space is a range of internal IP addresses that can be used for the Virtual Network, hence determines how many resources can be added to the Virtual Network. The address space used is defined in terms of CIDR (Classless Inter-Domain Routing or Supernetting). One needs to be careful while giving address space range specially if we are planning to use multiple Virtual networks that need to connect as we should keep address ranges unique in that case to avoid overlapping.

Subnet is setting up smaller network ranges within a Virtual network. This is particularly useful when you would like to subgroup elements within a network, for example setting up a different subnet for frontend servers and backend servers.

Communicating with on-premise resources

Point to Site
There are times when a user wants to connect to a network, for example, accessing an office network from a personal laptop to access emails. Point to Site Connectivity through a VPN client to VPN Server is the best option in this case.

Site to site
We saw that we used a point to site case when we need to provide a single point to communicate with the VPN. Similarly, whole a particular location or an office needs access to a virtual network we can create a site to Site Connection with Virtual Network Gateway.

Expressroute
Expressroute is a dedicated private connection from the source to the VPN. Microsoft provided a set of locations to which users can connect using a dedicated private line and get onboard to Expressroute.

Communicate among VPNs

There will be cases when resources in one VPN needs to communicate to resources in another VPN on Azure. The best way to achieve this is by using VPN Peering.

“Virtual network peering enables you to seamlessly connect Azure virtual networks. Once peered, the virtual networks appear as one, for connectivity purposes. The traffic between virtual machines in the peered virtual networks is routed through the Microsoft backbone infrastructure, much like traffic is routed between virtual machines in the same virtual network, through private IP addresses only. Azure supports:
VNet peering – connecting VNets within the same Azure region
Global VNet peering – connecting VNets across Azure regions”
https://docs.microsoft.com/en-us/azure/virtual-network/virtual-network-peering-overview

Jumpbox Pattern
When accessing and managing Virtual network resources from outside, jumpbox pattern is a common mechanism. Basically one machine in virtual network is designated as jumpbox, this jumpbox is accessible from outside word but no other resources can be accessed. Once the administrator is on jumpbox machine, he will be able to manage other resources through jumpbox in a controlled manner.

Azure Storage

Storage is one of the most important aspects provided by any cloud service provider. At the end of the day, you need a good storage solution for managing your data, code, backups, executables and basically everything. You would need a different type of solutions to manage different types of data, like data which you access frequently vs one which is used once in a month, data which has sensitive information vs the one which all users should be able to access, data which should be stored in a relational database vs the data which should be stored on NoSQL database, the list goes on.

Azure has a solution to all your needs. Let’s take a look at different storage options provided by Azure.

Location: You would like to choose a location nearest to your access point for better performance.

Performance: Standard performance is cheaper and will save your data on magnetic drives whereas Premium storage will save on solid-state drives and is good for data that need high performance.

Account kind: Storage V2 and V1 are general-purpose storage accounts where V2 will give you an option of Cool or Hot Access Tier, which can be selected based on how frequently the data is used. Another Account kind is blog storage, specializes in data storage in blob form.

Replication:
Locally Redundant Storage – Replicated across different racks in single data storage. This will manage hardware failure.
Zone Redundant Storage – Replicates data in different zones in a region. This makes sure even if a data center is down, you don’t lose the data.
Geo-Redundant Storage – Data is replicated across geographies. GRS replicates your data to another data center in a secondary region, but that data is available to be read-only if Microsoft initiates a failover from the primary to a secondary region.
Read-access geo-redundant storage (RA-GRS): is based on GRS. RA-GRS replicates your data to another data center in a secondary region and also provides you with the option to read from the secondary region. With RA-GRS, you can read from the secondary region regardless of whether Microsoft initiates a failover from the primary to a secondary region.

Now with Azure storage, we can use one of the following services

Blobs: Blobs are Binary Large OBjects. Blob storage also known as Object Storage, is perfect for storing binary and text data. Medial files, images, documents, application installers etc are the best fit for this type of storage. The maximum file size that can be stored is 4.77 TerraBytes. Azure Data Lake storage works on top of Blob Storage.

Files: As the name suggests this type of storage is best when dealing with files. This also gives us SMB 3.0 protocol support which means you can directly be mounted on local or remote machines. File storage can be attached to VMs and accessed.

Tables: The solution should be considered when we need to store data in tabular form.

Queues: We can set up queues for message-based communications. Messages can be published and read from these queues.

Accessing Storage Accounts
Azure provides us two core mechanisms to access objects in storage, one by using the storage access keys, where 2 secured keys are provided and secondly by using a shared access signature which is used for temporary and limited access.

Securing Data
Data at Rest – One can use encryption for securing data at rest. Azure provides transparent data encryption by default (can be turned off) for databases (master database is not encrypted).

Data in transit- Data in transit can be secured by using https and smb 3.0 protocols.

Data in execution- Azure provides TEE or Trusted Execution Environment and Confidential computing with DC series virtual machines.

Cloud Computing – an Introduction

When I started my career, analyzing and finalizing hardware needs for deployments was a major task and had to be taken up months before actual production deployments. Hardware was costly. Though we had providers which would provide machine virtually, you would need to decide the requirements beforehand as once you procured a machine, you had to pay at least a month’s rent. And if you decide to upgrade or downgrade the server machine, it was a painful manual task.

Just imagine what a nightmare it would have been to scale up during a surge in requests. You had to foresee it, plan for it, arrange hardware for it (monthly rents).

With the cloud, things have changed for the better. You have a pay as you go model, so you actually pay for the usage of hardware only. With autoscaling features inbuilt into the cloud infrastructure, it is easy to increase or decrease compute power without any human intervention. Setting up databases and scaling them is another area which the cloud takes care for us. Most of the cloud service providers support both relational and NoSQL databases in an easy to use manner.

Security, access management, monitoring, encryption, and storage are some of the other services which are provided by cloud services providers of the shelf. Another popular set of services off-late is serverless compute. This means one can write code directly which can be run as functions on the cloud, without worrying about the deployment details completely. Cloud provider is responsible for scaling and maintaining such functions. This is in sync with the microservice approach where each function can behave as an independent microservice.

With one’s mind taken away from hardware details, it is easier for software engineers to focus on building quality products. But it is important that we design our products in a manner which are capable of taking advantage of cloud services. For example, it will be easier for a microservices-based application to autoscale in a cloud than a monolith application. A stateless service is easier to be deployed and scaled on cloud than stateful service. One still needs to take care of the fact which services are exposed on the internet and which should be exposed only to internal service. With the ease of deployment, it might be easier to mess up a running service, so proper automated and manual checks are required to be implemented.

Cloud, though makes things easier, but one needs to be cautious of using its capabilities and designing the system in a manner to make maximum use of services being provided

Spring Cloud

With more and more applications moving to the cloud, it is important for developers and architects to start thinking in terms of applications which are ready for cloud deployment. Spring cloud in an umbrella which consists of a set of spring projects that can help us create applications which are cloud ready. The advantage of using a spring cloud is that you get a solution to most of the challenges you face when you deploy on the cloud at one place.

Let’s look at some of the core tools provided by spring cloud.

Spring Config: An important challenge distributed applications poses is how to manage your configurations. Each application and service need to manage some configuration properties. Spring config server provides a centralized way to manage configurations. It can read configuration from multiple sources, by default it would expect the configuration to be available at Git, but this can be modified. The most important factor of about using spring configuration server is that you can refresh properties or configuration in services without the need for redeployment or restarting the services.

Discovery service- Eureka: Spring cloud borrows a lot of services from Netflix open sourced projects. Eureka is one such service from Netflix, which provides a discovery mechanism for services deployed, Spring provides seamless integration to Eureka. With a design supporting deployment and autoscaling fo microservices, this becomes important the services can be deployed and removed on the fly without manual interaction. A challenge in this approach is how will the calling services know about the location or URL for the services to be called. A discovery service mechanism provided by Eureka comes to help. Any new service getting added to the system will register itself with Eureka server. A service that needs to access this remote service will communicate with the Eureka server and ask for the service location. Eureka server returns the location of the service, which is then accessed by the client service.

Fault Tolerance with Hytrix: Another important factor one needs to be worried about is how failure is managed in an application. In a distributed system, where multiple services are calling each other, will the failure in a service bring down the whole system? Or is there a way we can contain the failure to a single service. Spring integrates with hystrix, which again is a project by Netflix, that provides us mechanism like a circuit breaker to manage failures in an application. For example, if a remote service is down or not responding, Hystrix helps us configure an alternate service or method which can take over. Also, we can add settings that after N number of failures, client service will stop calling remote service (the circuit is open), and set a timer when next retry happens (if the service has healed itself, operations continue as usual).

Routing with Zuul: With multiple services being part of the system, it becomes difficult for client service to keep a track of all the remote services. A routing gateway in such an environment is like a front gate, behind which all the services are available. A client system need not be aware of details of deployment of each service, and it will just request routing server to get the required information, which in turn will communicate with remote service and returns a response. The advantage of this approach is we can implement security, logging, auditing of incoming requests in a centralized manner and each individual service need not worry about these factors.

Load Balancing with Ribbon: Load balancing is an important feature in order to manage autoscaling of the services. If there are 10 instances of a service, we want to make sure each of the instances manages almost equal load to take advantage of scale. Load balancing can be implemented at the server level for which various cloud service provider already give us a certain option. But there can be cases when you want to implement Load balancing at the client side. For this, Spring integrates with Ribbon project from Netflix and provide us seamless load balancing with the help of annotations like @LoadBalance and @RibbonClient.

Additional Spring Cloud projects: As already mentioned spring cloud is an umbrella which contains a number of projects that help us make our application cloud ready. The number of projects under this umbrella keeps on increasing with an increase in the popularity of cloud-based deployment and distributed applications. For a detailed view of all the projects under spring cloud, one can visit and explore projects under https://spring.io/projects/spring-cloud

Designing a Solution with AWS

When one goes for a Cloud based solution with solution provider like Amazon AWS, there are 2 things which are important. One, you need to have a clarity on what you are trying to achieve, and second is understanding of the services being provided by the provider.

Both the aspects are equally important. AWS provides plethora of services which can amuse at the same time confuse one. You might be tempted to use services which might not be required for your project and unnecessarily adds to the cost. At the same time if services not used with proper understanding, can backfire in terms of output and cost. For example, in one of my projects, incorrect implementation of autoscaling ended up running unused servers adding to cost instead of saving it.

Additionally, one need to be aware of all the capabilities of the service provider, for example, what all database and backup services we can use, can we use caching services, monitoring services provided by the service provider. Otherwise you will end up putting in unnecessary effort in rebuilding the wheel.

Here is a good starting point for AWS usage –

AWS CloudFormation

When you are setting up an environment on AWS cloud, you need to go through many steps, like creation of IAM roles, Security groups, Databases, EC2 instances, load balancers etc. Often one resource is dependent on other and hence you have to create components one by one which can be time consuming. With Cloudformation scripts one can easily get the deployment steps automated. And most importantly, the script is reusable any number of times. So if I want to replicate a stage setup on production or another setup in another region, it is easily possible.

One can create template in JSON or YML formats. The template is submitted to cloud formation which executes the template and create the stack which is actual environment with all the mentioned components.

Another important thing is that you can not only create infrastructure, but also do required settings. For example, I needed to get setup for application done on EC2, which I was easily able to do with UserData section.

Here is an example

Resources: 
    AppNode1: 
        Type: AWS::EC2::Instance
        Properties:
            InstanceType: XXXXX # type here
            ImageId: ami-XXXX # any ami here
            KeyName: XXXX # name of the key if already exising or create a new one
            IamInstanceProfile: !Ref InstanceProfile
            NetworkInterfaces:
            - AssociatePublicIpAddress: true
              DeleteOnTermination: true
              Description: ENI for bastion host
              DeviceIndex: '0'
              SubnetId: subnet-XXXXX
              GroupSet:
              - !Ref AppNodeSG
            UserData:  
              "Fn::Base64":
                "Fn::Sub": |
                  #!/bin/bash
                  cd /root/
                  apt-get update
                  apt-get -y install awscli
                  aws s3 cp s3://XXXX/XXXX.XXX ~/some location
                  #One can install servers, download wars and deploy at runtime
    AppNode2: 
        Type: AWS::EC2::Instance
        Properties:
            # create another instance
    AppNodeSG: 
        # Security group to give access to ssh and port 80
        Type: AWS::EC2::SecurityGroup
        Properties: 
            GroupDescription: SecurityGroup for new AppNode
            VpcId: vpc-XXXXX
            SecurityGroupIngress:
            - IpProtocol: tcp
              FromPort: 80
              ToPort: 80
              CidrIp: 0.0.0.0/0
            - IpProtocol: tcp
              FromPort: 22
              ToPort: 22
              CidrIp: 0.0.0.0/0
    InstanceProfile:
        Type: AWS::IAM::InstanceProfile
        Properties: 
            Path: /
            Roles: [S3FullAccess] # S3FullAccess Role created Manually, so that my EC2 instance can access S3.

Clustering

A cluster in simple terms is group of similar things, in this case computers or servers. A more refined explanation would be that a cluster is group of computers, working together in such a way that for end user it is one single machine. This is close to what I discussed about implementation of virtualization, so yes clustering is a form of virtualization.

But when we are strictly talking about software architecture, we are actually talking about using cluster for load balancing or handling failover. For a simple web application, this would mean creating 2 or more similar server machines, which are maintained in a cluster. There is a single point of entry which dictates which server from cluster should fulfill incoming request. This is load balancing. The server at the entry point can use any algorithm like round robin or check the actual load on a server to assign a request to one of the servers in the cluster. At the same time if one of the machine goes down in cluster for some reason, other servers can share the load and end user will never know about the problem occurred at backend.

http://msdn.microsoft.com/en-us/library/ff649250.aspx

http://en.wikipedia.org/wiki/Cluster_%28computing%29

Virtualization

Virtual, as the word hint, is something that is not real, but gives feeling of being real. In computer world, my first interaction with virtual machine was at the very beginning in school days, when we were made to work on dumb terminals, which had only monitor and keyboard. The terminal used to be attached to a central powerful machine which would provide processing power and memory requirements.

That is a very crude example of virtualization. With hardware cost going down in last few years, the need of dumb terminals has evaporated. But with cloud computing coming into picture, virtualization in IT industry has reached to new scales. With cloud computing it is convenient and important to manage virtual machines based on requirement or load on the application.

Virtualization goes beyond a virtual machine (hardware virtualization, software virtualization, storage virtualization, network virtualization are key), but for sake of this posts simplicity, I will stick to virtual machines. A virtual machine is simply a machine which does not exist in real, but can be used as a real machine. The advantage of this type of arrangement is maximum usage of hardware and hence cost efficiency.

Hypervisor is the key software component which helps running multiple operating systems (and hence machines) on one system. Read here on hypervisors.

Suggested reads

http://en.wikipedia.org/wiki/Virtualization
http://en.wikipedia.org/wiki/Hypervisor
http://en.wikipedia.org/wiki/Virtual_machine
http://www.zdnet.com/blog/btl/virtualization-software-revives-dumb-terminals-cuts-it-costs/9808
http://www.webopedia.com/TERM/V/virtualization.html