Ecological Metaphors for Distributed Computing

In an age of big data, vast communication systems, Artificial Intelligence (AI) and the Blockchain, enormously complex tasks are divided across a network of computers.

Yet, we are only just getting to grips with the notions of asynchronous processes.

Further questions around distributed computing remain. How should distributed computing advance?

Can it still advance?

One exciting idea is to study distributed computation under the lens of Ecology.

Ecology has traditionally involved small, individual studies, which examine how organisms interact with their natural surroundings.

This sparked an interest in ant colony optimisations conducted by Feinerman & Korman.

An attempt was made to create algorithms to “simulate how ants lay, follow, and modify pheromone trails to find efficient paths from their hives to food sources”.

This has led to much research comparing the behaviour of ant colonies to distributed computation.

Researchers are learning how to control distributed computational systems to function like swarms of bees or colonies of ants. Each individual “party” operates in response to the same basic set of instructions.

But when the swarm comes together, its members can carry out complex behaviours without any centralised direction.

Distributed computing is a unique space and is often described as the process of many computers or processes aiming to achieve specific goals (joint or separate) in a way that necessitates interaction and cooperation.

Now, of course, this includes server farms and data centres, supercomputers, sensor networks, networks in general, the Internet of things (IoT), the Internet itself, even a single laptop, robotics and even a self-driving car.

You can see why the ecological approaches towards researching distributed computing are very well suited.

One of the most complicated areas of distributed computing was (and to some extent still is) the idea of an asynchronous computation.

Event though computation may be broken down into floating parts (a topology) distributed among some vast space, none of the parts (or parties) involved cannot assume there to be a “common clock” for keeping time and coordinating the sequential progress of computation.

The information one expects from another party (even if some form of a contract is in place) may be indefinitely delayed for reasons where a party is simply slow or simply inefficient (where weak relationships between parties increase costs and decrease reliability).

Thus, even when all messages from other parties eventually arrive, “their order of arrival may be arbitrary”.

Asynchronous problems often arise from computational time differences or heavy memory footprints between parties and the properties of their underlying communication.

In short, all of this evolves from the concept of time.

As research has advanced and technology has improved, we have seen many improvements, but two significant problems persist.

They are the combination of “coordination” and “cooperation” and the forever difficulty of computers “sharing resources”.

Issues like these have been tackled from many different angles (I have already mentioned ant colonies), particularly algebraic topology, which has proven to be seamlessly connected to distributed computing problems.

These problems were exacerbated further in modern environments due to the rise of Big Data and Machine Learning (ML).

Both Big Data and Machine Learning involve many data stored in some distributed network or the cloud. Thus, at times, it takes even longer to move the data back and forth. Depending on bandwidth capacity, this may be significantly slower or faster.

Furthermore, when data becomes too large for a single machine, a more distributed execution comes into the picture.

Data is partitioned and stored into bits and distributed across n* agents that perform computations on each bit of data.

In addition, such a computation requires a significant amount of energy.

With all this in mind, can such foundational conflicts of asynchronous computing and distributed computing be put to bed?

Will we ever be prosperous within the complex topologies of distributed computing?

These questions are to be treated delicately and command vast sources. However, what can be said is that the concepts and problems have softened over the years.

Over the last ten years, as we have seen from the advances of ML and AI, distributed computing and the notions of asynchronous behaviour extend into cross-disciplinary research, such as Quantum Computation.

This is an area of research that physicists, leveraged by computer scientists, speculated and eventually led to a remarkably intense and productive collaboration between the two groups.

Since the 1950s, such a collaboration opened up further research & created new disciplines of Quantum Mechanics and Quantum Information theory.

From a Quantum point of view, the general idea for tackling the problems of asynchronous and distributed computing starts by listing probabilities.

It’s a vast new approach, but its objective is to help identify the many pathways (if any) in which asynchronous behaviour or states can be transformed into one another and the probability of the outputs given specific inputs.