Previous Articles >> Network Storage: Optimizing Solutions Network Servers: Changing Dynamics

Related Articles >> Advantages of Grid and SOA

Need for a grid
Science has advanced by leaps and bounds and has grown more dependent on computational power for research and analysis. While a powerful machine was enough to analyze or compute whatever data, say a Pharma researcher had a decade ago; things have changed a lot. Specifically in areas such as medical research, nuclear physics, molecular studies, etc. For example, the amount of data that scientists download from satellite monitoring activities in outer layers of atmosphere goes up to approx 200 GB daily. Now you might realize the kind of giant processing power you would need to consume data recorded over say a week and perform computations on it. It has to be huge and powerful. This is one of the reasons scientists demanded a system powerful enough and with near infinite storage that could easily perform computation on the kind of data they accumulate. It is scenarios like this which lead to the need for Computational Grid. Rest as they say is history.

Grid architecture
Much like the Electric Grid from where the idea of Computational Grid came, the architecture is a layered one. Thus we have grid applications as the top most layer that might be scientific, engineering, and commercial or even web portals. The next layer is that of the grid environment and tools. This layer provides the libraries, runtime interfaces, even compilers and most importantly parallelization tools. Next comes the layer which is rather a vendor specific implementation, the Grid Middleware. This layer is in-charge of all the resource management, scheduling services, job submission, storage access, and info services across the entire grid. The middleware can further be segregated as a layer comprising two sub layers. Some conceptualize two different layers. The User-level middleware which takes care of the first two of all the tasks we mentioned for middleware. The second one, Core Grid Middleware that handles the latter four. Now since the grid will be using Internet as the communication, computation and in-fact storage infrastructure and will be communicating or connecting to clusters/grids across geographies; a Security Layer becomes indispensible. Also referred to as the Security infrastructure, this layer provides authentication and secure communication. The bottom most layer is the 'Grid Fabric' which is nothing but the existing 'network of networks' and its components, clusters running on various OS, storage devices, databases and even specific devices such as sensors.

How it works
At the heart of the Grid is what we call the broker. We can describe the working of the Grid at a rather abstract level as follows. Once a job is submitted for operation in a Grid, the broker discovers resources that the user can access through 'Grid Information Servers.' It then negotiates with grid-enabled resources or their 'Agents' using middleware or middleware services, maps these to the resources (also known as scheduling in Grid context) and then stages the data for processing or application to be run. This last step is referred to as 'Deployment' in Grid context. The broker finally collects results. It monitors the application's execution progress also. It also takes care of changes in the Grid structure and resource failures.

Grid Vs Cluster computing

In a grid environment, we have a loosely coupled architecture of systems connected majorly over a Wide Area Network or an Internet. The job is more or less the same as is done by a Computational Cluster, which is to harness resources of multiple ideal machines. But in case of a Grid it's not necessary that it will only leverage the processing power of all the machines. You can instead create a Data Grid which actually creates and manages distributed data storage and is also called a Grid.

The other key feature of a Grid which actually differentiates it from a Cluster is its de-centralized model, where you generally don't have a controller in place and each and every node works independently. In this case the nodes can also be heterogeneous in terms of Operating Systems hardware architecture.

One example of grid computing is the infamous SETI@home project to search for extraterrestrial intelligence. There is a centralized telescope which captures radio signals from space and then transfers the data captured in small packets to several million computers connected to the Internet. The nodes then process these packets of data in their idle time and return the results back to a data center. This way high processing power is
obtained, utilizing the idle time of several computers spanning across the globe.

In this example you can clearly see that the architecture is completely de-centralized and loosely coupled. And is also very highly heterogeneous because over the Internet one can't control which OS or architecture will a node be using.

Clusters on the other hand use a single server or controller to manage and distribute/aggregate the processes and one or more client nodes connected via a tightly coupled environment such as a high speed LAN or some specialized high speed interconnect such as Myranet, etc. But, unlike grid computing, where each client computer can run its own OS, this one is controlled and managed by a single OS running across the computers in the cluster, making it highly homogeneous in nature. The server provides various files to clients for execution. Applications are run on clients using parallel processing algorithms.

The clients are just dumb terminals, with no display in most of the cases or input devices connected to them. The server is the single interface for the entire system, where all input and output takes place. To the user the entire setup appears as a single system. These formations of clusters are commonly known as SSI or Single System Image.

Beowulf clusters, which are built from commodity 'off the shelf' computer parts running free OSes like Linux, are an example of such a kind of cluster. They provide very cost-effective parallel processing.