Synchronous versus asynchronous execution

The difference between synchronous and asynchronous execution becomes important when dealing with many independent processes which are trying to coordinate their actions / behaviour by interacting with each other. Such coordination is a necessary component of decentralized and distributed systems. Human social systems are a special case of a distributed system. Latest trends in IT industry clearly indicates, that in a few years, the amount of Internet-connected individual devices will exceed the Internet-connected humans by orders of magnitude (see IoT data governance challenges discussed at Dreamforce from TechTarget). Which implies, that most of the communication will happen not between humans and humans, or humans and devices, but between devices with devices. The socio-technological system will not any more be able to operate on the basis of humans deciding what is needed to be done and then instructing devices to carry on certain actions. Devices will have to decide on their behaviours by coordinating with other devices, occasionally asking for the help from humans and / or providing already processed and reduced information to them.

Within the domain of computer science, namely message-passing framework, synchrony and asynchrony are clearly defined2.

  • Synchrony. A communication among processes is considered synchronous when every process observes the same order of messages within the system. In the same manner, the execution is considered synchronous, when every individual process in the system observes the same total order of all the processes which happen within it.

  • Asynchrony is a kind of a reverse of synchrony. A communication among processes are considered asynchronous, when every communicating process can have a different observation of the order of the messages being exchanged (i.e. they can receive the same message at different times, etc.). Likewise, the execution is considered asynchronous when there is no one established way for processes to determine the single total order of the execution of all processes within the system.

These definitions connect to the 'Conceptual perspective' in an interesting manner -- in the case of synchrony, the global process / omniscient observer is not necessary, but what is necessary, is that every process agrees on the total ordering of all events in the system -- an equivalent of the "single truth". Recall also that synchrony is always build on asynchrony6 -- synchronous system can be described by asynchronous system with a set of constraints. Synchronization (section below) can be understood in terms of adding these constraints; decentralized / distributed consensus algorithms are therefore mechanisms of self-organizing constraints.

Further, the trade-off between synchrony and asynchrony is applicable and helpful for reasoning about general distributed systems. It is quite clear that we are surrounded by natural or constructed asynchronous systems. Nature itself can be considered an asynchronous/decentralized system. E.g. the same astronomical events can be observed in reverse order by two independent observers if each of these observers are closer to the different astronomical event (see the theory of relativity) and if there is no notion of global time, which is accessible to both observers. It is also clear that without any level of synchrony within the system (global or local) it is quite difficult to reason about it.

The diagram below (inspired and partially adopted from the post on Evolving terminology with evolving technology: decentralized vs distributed by ioptio@madesafe) illustrates how centralized -> distributed -> decentralized -> peer-to-peer systems map within each other and map to the continuum between completely synchronous and completely asynchronous systems. Eyes on the right illustrate approaches two approaches synchronization mechanisms. Top-down approach means designing a centralized system from the beginning and then thinking how to deal with asynchrony / parallelism. Bottom-up approach starts from complete mess (asynchrony) and tries to figure out how can it achieve complete or partial synchrony.

Note, that 'complete synchrony <-> complete asynchrony' axis does not order indicated system types but rather provides an additional axis for thinking about them: e.g. a peer-to-peer system could in principle be completely synchronous, but the rationale of designing such system would be not clear. It is somewhat more difficult to imagine centralized and asynchronous system, though.