Category Archives: Architecture

Enterprise Architecture- Building the core model

One challenge often faced by organizations is that IT is often reactive rather than guiding the operations. You bring in IT once you face a problem and build an IT solution. Once solution is build, struggle starts to integrate with other parts/ applications. A lot of time is than spent on making different pieces work together, which can be avoided if proper EA practices are in place.

Ideally, IT should look at the system and come up with opportunities of improving existing system like automating ordering system and adding new services like moving to mobile platforms.

IT and Business can than prioritize the solutions/ projects based on value addition. Having a big picture in front, it will be easier to take decisions and less time will be spent in making things communicate and work with each other.

Building a core model is important so that newer services can easily gets integrated. For example if centralized data handling and services to share data securely are already in place, getting a new mobile app to market is much easier than in a scenario where we do not have any such centralized solution in place already.

Creating the core IT model is not easy. You need to take a call what to keep as core and what should be customizable. As a rule of thumb, identify what is fixed and what can be changed/ customized in your business. Based on this information, we need to design which part of design is fixed and which is flexible. For example, in a particular business, product information might be centralized but sales can be customized.

In addition, core model need to take care of that fact which processes can be standardized and what data will be centralized and shared. More- http://kamalmeet.com/architecture/enterprise-architecture-manage-your-data-and-processes/

Enterprise Architecture- Manage your data and processes

Any Enterprise architecture needs to take care of two important things, processes and data. A standardized process makes sure certain operations are done in a certain way no matter who is performing them. Data ofcourse is a very important part for any organization, it helps in every aspect of business from fulfilling sales orders, maintaining inventory, making decisions for future etc. So it is important that data is shared across units in effective and secured manner.

Based on the business, need for data sharing and process standardization might vary. We know a standard process adds to predictability (less flexible = agile) but might not work in cases where innovation and flexibility is needed, for example in sales or research. So if we are looking at a business like McDonald’s, we know that each unit needs to follow similar process. Hence process will be part of my core architecture. But in case we are dealing with a Insurance sales business, where each unit might need a different strategy, we will not be be standardizing the process to detail level.

Similarly, decisions needs to taken on centralization of data. For example in a car manufacturing and sales unit, it is important to keep data on inventory, sales, production in sync. Whereas, for an insurance company, it might need flexibility of keeping car insurance and personal insurance data separately. Though you might need to keep products available information at common place. Nonetheless, the data definitions should be strict through out the organization, a completed sales mean same in all aspects.

Based on decisions made as per above analysis, we will be able to create our core architecture effectively. We will know what to add to core and what to keep flexible. The good core design will help business in maintainability and scalability. Adding a new business unit (or a new product or service) and integrating to existing business will depend on readiness of architecture we have finalized.

Steps for Designing a system from scratch

You are into a new project, and provided with some requirements. How would you go about designing the new system

We are in this situation a lot of times. So here I am trying to create a step by step guide from taking a requirement doc to finalizing a system architecture.

Stage 0: Before getting started with the design process, we need to make sure about the following.

– Do we have clear understanding of requirements?
– Are we creating something from scratch or enhancing and existing system. In later case we will have design and technology constraints from previous system?
– Have we identified non functional requirements- security, performance, availability etc.
– Have we identified all stakeholders and their role?
– Have we identified key players that will help in creating architecture (architects, Business Analysts, Product owners)
– Have we decided on time/ money to spent on design activities?
– Have we identified reference material? Do we have artifacts for similar design problems, from either inhouse or external sources?
– How are we going to maintain the design artifacts- wiki, git, svn, confluence etc. We will need to maintain versioning?
– Have we identified any guiding principles for the design- we will use open source softwares and tools, or we will be using linux system of deployment etc.
– Are there any constraints- say client wants to use any specific third party tools or technologies, any specific compliance required by law (multilingual support), service availability grantee
– Have we identified all third party systems with which our system will interact and how the interaction will be done?
– Are we creating the system in one go or will it be a phased delivery. Have we identified the value add provided by various components being built and prioritized the delivery?
– In case of phased delivery, we need to identify scope of each phase?
– Have we identified risks involved and mitigated them?
– If we are modifying or enhancing an existing system, we need to understand what areas can be reused, enhanced and built from scratch?
– Better to create a formal document to identify what all design artifacts are required.
– Define KPIs (Key performance indicators) and SLAs (Service Level Agreements)
– Have we defined acceptance criteria for the design?

Stage 1: Now we need to understand the business and what changes do we need.

– Have we understood organization structure?
– Have we identify business goals and objectives for the organization and what changes are required?
– Identify all business requirements, for example customer should be able to return a product is a business requirement.
– Identify and design current business processes (How current business work, does it fulfill all the business requirements or not, if yes, do we need to change or enhance the way it is being done right now, for example current purchase process is manual and we want to provide online options.)
– Identify changes or modification required in business processes

– Design artifacts to be delivered in this stage
Catalogs
— Organizations
— Actors
— Goals
— Roles
— Business Services
— Locations
— Process / Products
Matrices
— Business interaction
— Actor/ Role
Diagrams
— Business Services
— Functional Decomposition
— Product/ Process lifecycle
— Goal/ Service diagram
— Business Use cases
— Process Flow
— Event diagram

Stage 2: Focus on Data used

– What data is being used in the application? how it is originated and used?
– How the data is shared securely in enterprise
– Create common vocabulary and data definitions
– Identify security measures to be taken

– Design artifacts to be delivered in this stage
Catalogs
– Data Entities

Matrices
– Data Entity/ Business function
– Data Entity/ Application matrix

Diagrams
– Conceptual Data Diagram
– Logical Data Diagram
– Physical Data diagram
– Data lifecycle diagram
– Data Security diagram
– Data migration diagram

Stage 3: What all Applications are available? Changes required and new ones to be created

Application- Core parameters
– Platform independence
– Easy to use
– Identify existing applications and newly ones to be created at logical level and than map to physical level

– Design artifacts to be delivered in this stage
Catalogs
– Application portfolio
– Interface catalog

Matrices
– Application/ Organization
– Role/ Application
– Application/ Function
– Application/ Interactions

Diagrams
— Application communication
— Application and user location
— Application use case
— Application details – components/ modules and services
— Application details – Layered architecture if used

Stage 4: Understand the technology working behind the scenes

Control technical diversity: Minimizes cost of expertise.

Catalogs
— Technology portfolio

Matrices
— Application/ Technology

Diagrams
— Deployment diagram
— Environments and locations
— Communication engineering diagram (firewalls)

Stage 5: Lets consider Non Functional Requirements
— Security
— Performance
— Availability
— Disaster recover
— Data backups
— Others (Project specific)

Stage 6: Post Design phase:

– Did we identify reusable artifacts and services which can be used by other projects?
– Have we conducted periodic validation that design and product being build are in sync?
– Does the design change due to any change requests? Has that been reflected in design?
– Have we met all the acceptance criteria that were set initially?

Understanding Enterprise Architecture- basics

What is an Architecture?
An architecture helps in identifying components and their relationship. It provides basic guidelines for representing the components. It helps in understanding that how system can evolve and enhanced.

What is Enterprise Architecture (EA)?
Before getting into EA, we need to understand what an enterprise is? An enterprise organization is a set of companies with a common goal.
Now an enterprise can have different applications, solution components at different levels. EA helps getting the bigger picture by putting all the elements together. It helps understand how these applications interact with each other. How different processes are dependent and related.

Why an organization needs an EA?
This helps us understand impact of change at one part on whole enterprise, and hence helps in decision making, lowering down cost of operations, sharing of resources and capabilities, manage security, change management, helps make – buy- outsource decision etc. In addition their can be regulatory drivers in some govt and non-govt organizations which need the entity to maintain EA.

How do Architecture Frameworks help?
An Architecture framework provides common vocabulary so that every stakeholder’s understanding is same. It provides a set of tools and building blocks that can be used to create final architecture. In addition it provide a list of standards so that everybody involved can follow similar strategy.

What are different Architecture domains?
Common frameworks like TOGAF provide 4 domains
Business Architecture: Understanding of business processes.
Data Architecture: Structure of logical and physical data.
Application Architecture: Design of application systems to be created and deployed, their interactions and mapping to core business processes.
Technology Architecture: Details on infrastructure, middleware, deployments, communications etc.

Open-Closed principle Revisited

Reference: http://kamalmeet.com/system-design-and-documentation/understanding-openclosed-principle/

Open closed principle states that your classes should be open for extension but closed for modification. One way to look at it is that when you provide a library or a jar file to a system, you can ofcourse use the classes or extend the classes, but you cannot get into the code and update it.

At a principle level, this means you should code in a manner that you never need to update your class once code. One major reason behind this principle is that you have a class which is reviewed and Unit tested, you would not like someone to modify and possibly corrupt the code.

How do I make sure that my class follow open closed principle?

Let’s look at a design of this MyPizza class

public class MyPizza {
public void createPizza(Pizza pizza)
{
if(pizza.type.equals("Cheese"))
{
//create a cheese pizza
}
else if(pizza.type.equals("Veg"))
{
//create a veg pizza
}
}
}

Following pizza type classes use this

class Pizza
{
String type;
}

class CheesePizza extends Pizza{
CheesePizza()
{
this.type=”Cheese”;
}
}

class VegPizza extends Pizza{
VegPizza()
{
this.type=”Veg”;
}
}

The above design clearly violates the open closed principle. What if I need to add a double cheese pizza here. I will have to go to MyPizza class and update it, which is not following “closed for modification” rule.

How can fix this design?

public class MyPizza {
public void createPizza(Pizza pizza)
{
pizza.create();
}
}

class CheesePizza extends Pizza{
CheesePizza()
{
this.type="Cheese";
}

public void create()
{
//do the creation here
}
}

With this simple modification we are making sure that we will need not change the code in MyPizza class even when we will add new types of pizza, as actual responsibility of creation would be with the new class being created (DoubleCheese).

Reverse Engineering: MySQL WorkBench

In last post I talked about creating sequence diagrams using MaintainJ. Another important aspect you would want to understand for a Project is the database schema design. How many tables are there? How do they interact with each other? etc.

For understanding this design the best way is to look into ER or Entity Relationship diagram. Ideally one would create the ER diagram first and then implement database.

In case we do not have a ER diagram available we can create using Reverse Engineering the database to ER diagram.

For MySQL, we can use MySQL WorkBench tool to create one.

Download the installer from https://www.mysql.com/products/workbench/

Once installed, you can connect to you mysql database in workbench. Then in Database Tab at the top, select Reverse Engineer option, and select the schema you want to reverse engineer.

Reverse Engineering: MaintainJ

The best way to analyze the code with hundred of Java classes is to look into the documentation, class diagrams, sequence diagrams etc to understand the flow and usage. Unfortunately there are times when you would not be provided with any such documentation.

Reverse Engineering tools can be of help upto some level. MaintainJ is one such tool to help you with Java.

So if you have a working codebase for a web application, which you need to analyze, here are the steps to go ahead.

1. Download the MaintainJ war file from http://maintainj.com/userGuide.jsp?param=install
2. Add the war file to the server where main application (to be analyzed is available), for example if you project war file is added to tomcat – tomcat/webapps, add the MaintainJ.war
3. Now if you will visit the link to server like http://localhost:8080/MaintainJ/, it will let you provide the package to be traced and directory where output file to be added.
4. It will provide simple settings to be added to catalina.sh (or other server config),
5. Once all settings done, restart the server.
6. Go to MaintainJ link and start tracing.
7. Now browse through the actual app, MaintainJ will create sequence diagrams to the directory where you have provided the path.

You can view the ser file created by MaintainJ in eclipse by adding MaintainJ plugin to eclipse. Create a new project of MaintainJ trace type and copy generated ser files into this project in a folder.

A good overall demo is provided – http://maintainj.com/userGuide.jsp?param=overviewDemo

Shared Nothing vs Shared Everything

In database cluster implementation we can have multiple ways to make sure how different nodes will communicate with each other.

Shared nothing approach: None of the nodes will use others memory or storage. This is best suited for the solutions where inter node communication is not required, i.e. a node can come up with a solution on its own.

Shared Memory: In this approach memory is shared, i.e. each node/ processor is working with same memory. This is used when we need nodes to share solutions/ calculations done by other nodes and are available in memory.

Shared Everything: In this approach nodes share memory plus storage. This makes sense when nodes are working on problem where calculations and data created/ used by node is dependent on others.

Further Reads:
https://www.quora.com/What-are-the-differences-between-shared-nothing-shared-memory-and-shared-storage-architectures-in-the-context-of-scalable-computing-analytics

https://en.wikipedia.org/wiki/Shared_nothing_architecture

Message Oriented Middleware

In last post I talked about what is middleware, I will focus on message implementation of same today. Message oriented middleware or MOM mostly uses message queues to send and receive data between two systems.

In simple terms, a message is anything that is being sent from one system to another. Mostly MOM uses XML formats, sometimes SOAP based requests or plain texts. An example MOM system will send message to a  message queue or MQ, from where the receiver will pick up the message.

Advantages of Message Oriented Middleware

  1. Persistence: In normal client-server architecture, we will need to make sure both the systems to be available to have a successful communication. Whereas if we are using MQs, one system can still send messages even if the second is down.
  2. Support for Synchronous and Asynchronous communication: by default the communication is asynchronous but we can implement a synchronous system where a request message sender will wait for the response from other party.
  3. Messages can be consumed at will: If one system is busy when messages are received (which do not need immediate response), it can consume the messages when load is less. For example, a lot of systems are designed to consume the messages at non business hours.
  4. Reliability: As messages are persistent, threat of losing information is low even if one or more systems are not available. Additional security mechanism can be implemented in MQ layer.
  5. Decoupling of systems: Both client and server work independently, and often do not have knowledge for other end. System A creates a message and adds to message queue, without concerning who will pick it up as long as it gets the response message (if required). So one system can be written in Java and other can be in Dot Net.
  6. Scalability: As both machines involved in interaction are independent of each other, it is easier to add resources at either end without impacting the whole system.
  7. Group communication: Sender can send message to multiple queues or same queue can have multiple listeners. In addition Publisher- Subscriber approach can help broadcast a message.

Types of Messaging:

Point to Point: This is a simple messaging architecture where a sender will directly send a message to receiver through message queue.

Publisher-Subscriber (Pub-Sub): This type of communication is required when sender wants to send messages to multiple receivers. Topics are defined to which subscriber can subscribe and receive requests based on same. For example, say a core banking system can trigger messages on various events like new account open, a withdrawal is made, interest rate changed etc. For an event, multiple other systems might want that information to take an action, so say for all withdrawal events, systems like fraud detection, mobile messaging system, daily reporting system, account maintenance system subscribe. Whenever, publisher publishes the message to “Withdrawal” topic, all of these systems will receive the message and take appropriate action.