Distributed systems form a rapidly changing field of computer science Tanenbaum and Steen 2006. These systems, which consist of multiple nodes connected to each other, leverage the idea of parallelism and concurrency based on a collaborate effort with a large number of nodes. Parallelism is part of distributed systems and is the heart of parallel computing. Parallelism is the simultaneous use of multiple compute resources to solve a computational problem.
Parallelism has been employed for many years but interest in it has changed due to physical constraints that prevent frequency scaling Snir 2009. Parallelism is often used in HPC environments. Some SIMD models (Single Instruction Multiple Data) parallel computers can be traced back to the 1970s.
The motivation behind early SIMD computers was to amortize the propagation delay of the processor’s control unit over multiple instructions Hennessy and Patterson 2011. Distributed systems involve solving the same problem that you can solve on a single computer using multiple computers. A distributed system is composed of a nodes, these nodes are linked by a network and the size can be from a Local Area Networks (10’s of nodes) to a Wide Area Networks (1000’s or 1,000,000’s of nodes). Distributed systems make use of multiple nodes spread across a network to to achieve a common goalOne of the most important design goals of a distributed system is the ability to handle large amounts of load, the size of the system is just one aspect of scale. This is called scalability. Scalability is how much traffic can it handle to how many more transactions can be processed. Scalability is very important in order to avoid denial of service which occurs when multiple systems flood the bandwidth or resources of the system.
Without scalability websites like Facebook, Google, and Twitter would crash. Researchers are putting ideas into effect and are developing schemes that fulfill the requirements of a distributed system. In this paper, I present a survey of protocols and algorithms proposed for distributed systems. The remainder of this text is organized as follows: Section 2 will describe potential applications for distributed systems. Section 3 will discuss the execution of different classes of computer systems. Section 4 and 5 will introduce readers to parallel and distributed systems concepts. Section 6 will discuss the problem of task management.
Section 7 will provide a detailed investigation of algorithms. I conclude the text in Section 8.