Edge Computing

Edge Computing takes distributed computing close to the information source, rather than relying on centralized data centers. This approach is relatively popular for systems that involve IOT devices.

edge-computing-use-cases-iot
https://innovationatwork.ieee.org/real-life-edge-computing-use-cases/

The idea is to keep computation near to source, to reduce latency. Decisions can be made faster as data does not need to be sent to long distance. This also reduce the amount of data sent to central servers as some level of data filtering and analysis is pre-processed at edge locations.

Edge Computing architecture usually contains following components

  • Edge devices: Devices that collect data from sensors, cameras, and other sources. Examples include IoT devices, cameras, and industrial equipment.
  • Edge gateway: An edge gateway acts as a bridge between the edge devices and the back-end systems.
  • Edge server: This is a server located at the edge of the network that is responsible for processing and analyzing data. It can run applications and services that are optimized for low-latency and high-performance requirements.
  • Fog nodes: These are intermediate devices that sit between the edge devices and the cloud or data center. They are responsible for processing and analyzing data, similar to edge servers, but they are typically more powerful and capable of running more complex applications and services.
  • Cloud/Data center: The data that is processed at the edge is then sent to a cloud or data center for further analysis, storage and sharing.
  • Management and orchestration platform: This is a platform that manages and monitors the edge devices, gateways and servers, and allows for the deployment, configuration, and management of edge applications and services.

Clean Code: Error Handling, Testing, and Classes

In the clean code series, I will cover Error Jandling, Testing and Clean classes in this post.

Error Handling

  • Catch only Exceptions meant to be caught, for example, checked exceptions in Java
  • Log as much information as possible when an error or exception occurs for analysis
  • Null objects should not be returned, instead, return empty objects

Testing

  • Code Coverage targets should be set and achieved
  • TDD helps write testable code and reduce the number of issues
  • Use the F.I.R.S.T rule for testing:
    • The test is fast-running.
    • The tests are independent of others.
    • The test is repeatable in various environments.
    • The test is self-validating.
    • The test is timely.

Clean Class

  • Single Responsibility
  • Open for extension and closed for Modifications
  • Readability: self-documenting class names, function names, and variable names

Clean Code: Comments, Formatting, and Objects

In continuation with the clean code series, here I am covering Comments, Formatting, and Objects and data structures.

Comments

  • Code should be self-explanatory, the purpose of the code is that humans should be able to understand and not that computer is able to execute it
  • Comment only where logic is complex
  • Private API should not have comments
  • Use comments when you want to caution other developers, for example, why List instead of Queue was chosen and should not be modified
  • Comments should answer why (was a decision made) and not how the code works

Formatting

  • Formatting is a way to communicate with fellow developers
  • Readability = Maintainability = Extensibility
  • Verticle Alignment: Keep connected functions together for better readability
  • Horizontal Alignment: one should never need to scroll right
  • Team Formatting Rules: everyone should follow the same rules, braces, ident size, spaces/tabs

Objects and Data Structures

  • Follow OOPS Principles, e.g. parameters and behavior should be encapsulated
  • Law of Demeter:  M method of an object O can only consume services of the following types of objects:
    • The object itself, O.
    • The M parameters.
    • Any object created or instantiated by M.
    • Direct components of O.
  • Avoid Static methods wherever possible

Clean Code: Naming and Functions

Inspired by Clean Code by Robert C Martin, trying to summarize coding best practices. Starting with Naming and Functions best practices in this post.

Naming

  • Names should encode the intent, for example, studentBirthYear.
  • Use Good distinction: Do not use list1, list2, etc
  • Use Pronounceable name: dobmmyy vs dateOfBirthInMonthAndYear
  • use searchable names: int i, j, when you will try to search you will find a lot of them in code
  • Do not add type: phoneString, name String, name and phone should be sufficient
  • Avoid unclear prefixes: m_name vs manager_name
  • nouns for names and verbs for functions: employee for class and paySalary for function
  • Use Consistent concept: controller vs manager
  • Don’t use the same name twice to mean 2 different things. paymentInfo at one place returns bank details and another place user payment
  • Use Domain specific names
  • Avoid too long or too short names: Long is fine if it conveys better information, but not too long that makes it difficult to pronounce

Functions

  • Write Small functions, functions larger than 20 lines should be avoided
  • Make sure the function does only one thing
  • Use minimum arguments: max 2, if the function takes too many arguments, it is doing too much
  • DRY, Do not Repeat yourself: IF you are doing the same thing in multiple functions, move it to commonplace
  • Don’t use flag element, parameters of the Boolean type as a parameter already clearly state that it does more than one thing.

Behavioral Design Patterns

After discussing creational and strutural design patterns, lets go over behavirol design patterns.

Template: allows one to create a template of code or algorithm which will provide guidelines for different implementations.

Strategy Pattern: helps selection of an algorithm at runtime.

Observer: A pattern that helps manage event-based communication (publish-subscribe). Objects being observed notifies observers about the event, which can then take required actions.

https://en.wikipedia.org/wiki/Observer_pattern

Memento: A pattern is used to restore the state of an object to a previous state.

Example: https://www.geeksforgeeks.org/memento-design-pattern/

Chain of Responsibility: At runtime, command objects are selected to execute responsibilities. One common example is FilterChain used with servlets where multiple filter commands are executed on receiving a Request.

The chain-of-responsibility pattern is structurally nearly identical to the decorator pattern, the difference being that for the decorator, all classes handle the request, while for the chain of responsibility, exactly one of the classes in the chain handles the request. This is a strict definition of the Responsibility concept in the GoF book. However, many implementations (such as loggers below, or UI event handling, or servlet filters in Java, etc.) allow several elements in the chain to take responsibility.

https://en.wikipedia.org/wiki/Chain-of-responsibility_pattern

Command Pattern: creates objects that encapsulate actions and parameters

interface Command
{
    public void execute();
}

Example: https://www.geeksforgeeks.org/command-pattern/

Iterator: Iterator helps accesses the elements of an object sequentially without exposing its underlying representation.

interface Iterator
{
    boolean hasNext();
    Object next();
}

Example: https://www.geeksforgeeks.org/iterator-pattern/

Interpreter: A pattern that helps you parse and evaluate sentences in a language.

Example: https://www.geeksforgeeks.org/interpreter-design-pattern/

Mediator: Two objects are decoupled as they do not communicate to each other directly, but via a mediator.

https://en.wikipedia.org/wiki/Mediator_pattern

Unlike adapter where the communication is one way and caller is aware of provider, in mediator a middle layer helps both parties to communicate.

Visitor: Implemented using accept() and visit() methods, the component being visited needs to implement accept method and visiting component implements visit method.

Visitor Pattern UML Diagram
https://www.tutorialspoint.com/design_pattern/visitor_pattern.htm

State: An object can manage its behavior based on state. For example, a mobilealert object can have state as ring, silent, vibrate and based on state its bevior is changing. Here is a good example implementation https://www.geeksforgeeks.org/state-design-pattern/

https://en.wikipedia.org/wiki/State_pattern

Structural Design Patterns

After discussing creational design patterns in the last post, let’s go over structural design patterns.

Adapter: One of the most common structural patterns that helps two incompatible components talk to each other. For example, for a caller that communicates in JSON, but the provider service only understands XML, an adapter helps in this situation.

Composite: To manage the cases where there can be a cyclic relationship between objects, for example, a Manager himself is an Employee, and at the same time Manager has a relationship (manages) other employees.

Facade: Hide complex implementation details from the caller, which calls the facade. The facade layer then communicates with various other implementing components and returns the final response back.

Proxy: A common pattern hiding the actual implementation from the caller client. The client calls the proxy class or proxy layer, which in turn communicates with implementing component.

Bridge: The pattern that helps decouple an abstraction from its implementation.

Flyweight: The flyweight pattern helps reuse objects where ever possible and only creates a new object if the object is not available.

It sounds a bit overlapping with the object pool pattern, but there are significant differences. An object pool is a creational pattern whereas flyweight is a structural pattern. Object pool makes sure objects of the same type are pooled, whereas flyweight takes care of different object implementations, stored as a singleton.

Example: https://www.tutorialspoint.com/design_pattern/flyweight_pattern.htm

Decorator: The core component is decorated or enhanced by additional decorator features on the fly.

Example: https://www.tutorialspoint.com/design_pattern/decorator_pattern.htm

https://en.wikipedia.org/wiki/Decorator_pattern

Creational Design Patters

Let us revisit creational design patterns.

Singleton pattern: Most commonly used creational pattern where we want to make sure a single object is maintained for a class.

Factory Pattern: Representing a real-world scenario, one can let the factory create objects of multiple types inheriting similar features, for example, a car factory building sedans as well as hatchbacks.

Abstract Factory Pattern: Additional layer above the factory pattern, for example, a vehicle factory has two sections a car factory and a bike factory, and the car factory further produces multiple types of cars.

Prototype Pattern: Object prototypes are created in advance, which is clones on demand and a new object is returned.

Builder Pattern: A commonly used pattern (DocumentBuilder and StringBuilder in Java), that breaks object creation activity into multiple steps.

Object Pool Pattern: The creation of an object can be an expensive affair for complex objects, a very common example is database connections. In such cases, the object pool pattern helps in reusing the objects once created, hence improving the performance of the system.

The object pool pattern is a software creational design pattern that uses a set of initialized objects kept ready to use – a “pool” – rather than allocating and destroying them on demand. A client of the pool will request an object from the pool and perform operations on the returned object. When the client has finished, it returns the object to the pool rather than destroying it.

https://en.wikipedia.org/wiki/Object_pool_pattern

Design: Twitter

In the series of exploring designs for popular systems, I will try to come up with Twitter’s system design today.

Functional Requirements

  • User should be able to Tweet
  • User should be able to follow and view tweets of others on their timeline
  • User should be able to search for tweets
  • User should be able to view current trends

Non Functional requirements

  • Availability
  • Performance
  • Eventual Consistency
  • Scale 150 million users with 500 million tweets per day or ~5500 Tweets/Second

30000 view of Architecture

30000 view of Twitter Architecture

There are one or two aspects of the above design which are very interesting. The first one we can see is the user timeline. This can be a complicated piece, whenever a user login into the app, he should see his timeline, which will show all the tweets from people he is following. The user might be following hundreds of accounts, it will not be feasible to calculate tweets from all these accounts at runtime and create timeline data. So a passive approach makes sense here, where we can keep the user timeline data in a cache beforehand.

Say user A is following user B, and user B publishes a new tweet, at that time itself, user A timeline will be updated with a new timeline getting added to user A timeline data. Similarly, if 100 users are following user B, all the timelines get updated (Fanout the tweet and update all timelines).

It can get tricky if user B has millions of followers. A different approach can be used in this case. Assuming there are not many such popular users, we can create a separate bucket for handling these popular users. Say user A is following user C (celebrity), so instead of updating the timeline for C beforehand, tweets for all such celebrity users can be pulled in real-time.

Another important aspect is hashtagging and trends exploration. For all the tweets coming in, the text can be tokenized and tokens can be analyzed for most usage. For example, when a cricket match is going on in India, many people might tweet with the term match or cricket. Again these trends might be geo-location-based as this particular trend is a country-specific one.