Since parallel computation offers a way to scale simulation problems, there has always been a strong desire to do so. Inspired by common sense, the naive might think that two computers could solve one problem within half of the time that is required on a single machine, and ten computers could even reduce the amount of time to a tenth of that, and so on. However, it turns out that for many problems these projections do not hold. Depending on the problem and the algorithm in use, this imagination of scalability only holds for low grades of parallelization due to an inherent communication overhead.
Although computation itself takes place within less time, an additional amount of time has to be spent on the communication between separated or even dislocated parts of a parallel simulation, as long as the parts are not independent from each other. Faster computing parts will have to wait for slower ones, because the exchange of the simulation state has to take place at certain simulation times and computation on the involved heterogenous workstations will take different amounts of real time. This means that the slowest part determines the performance of the whole simulation.
One way to improve scalability is to carefully select the level at which parallelization takes place. There might be several levels which allow to split up a simulation problem into several smaller ones which can compute in parallel. This strongly depends on the nature of the problem under consideration and the algorithms that are utilized. Nevertheless, we discuss two extreme cases which should be applicable to a wide range of problems.