Federated Cloud Computing and Consensus Algorithms

In his 1956 book entitled An Introduction to Cybernetics[i], William Ross Ashby argues that interconnectability is a fundamental feature of machines. In his view, ‘two or more whole machines can be coupled to form one machine; and any one machine can be regarded as formed by the coupling of its parts, which can themselves be thought of as small, sub-machines’ (Ashby ,1956, p. 48). In 1962, Ashby predicts that these kinds of interconnections between computers or intelligent mechanisms will increasingly become normal and make up the core of self-organisation by computers. He argues[ii] that the core of this self-organisation is made up of the conditions under which the interconnections will be created.

In 2011[iii], the US National Institute of Standards and Technology (NIST) states that ‘cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services)’ (NIST, 2011, p. 3). This definition of cloud computing teaches us that computers that are interconnected in networks can make their joint capacity available to users based on prior hardware and software configurations. In a document published in 2021[iv], NIST broaches the possibility of incorporating these individual cloud configurations into a federated association or, in their words, ‘getting two or more cloud providers to interact or collaborate’, thus coining the term ‘federated cloud’.

Federation

The term ‘federation’ has a long history that revolves primarily around the development of political or social entities. In general, ‘federation’ denotes collaboration between individual, partially self-governing organisations, states, regions or other entities with a view to creating a new and collaborating whole without the constituent entities losing control or command of their own internal affairs. In most cases, the federation as an organisational form is under centralised governance that does not involve itself in the composition or functioning of the individual entities. Therefore, a federation as a whole needs to be governed based on agreements on how the federation should function. These agreements must be such that the participating parties can trust the functioning of the federation as a whole. According to NIST, the term ‘federated cloud’ refers to the coordination and organisation of collaboration between individual cloud providers. In this context, coordination and organisation concern the correlation between the separate cloud parties without wanting to intervene in the configuration or composition of the underlying networks or the specific configurations of the hardware and software used by the various cloud providers. In this way, the federated cloud can be seen as a ‘permissioned network’ of interconnected cloud providers. Cachin and Vukolic[v] defined permissioned network as a network ‘operated by known entities, such as in consortium blockchains, where members of a consortium or stakeholders in a given business context operate a permissioned blockchain network’. (2017, p. 2). In these networks, participating nodes such as individual cloud parties are not only identifiable beforehand, but these interconnected nodes themselves are also able to check their shared status and, if necessary, update it. A core task of these interconnected nodes is to determine in consensus which node can perform a specific information transaction with third parties outside the network.

Consensus

The 2021 NIST document focuses specifically on the topic of trust and governance of the collaboration within a federation of interconnected cloud parties. According to NIST: ‘While much trust and governance may be established out-of-band, we recognize that there are tools for establishing trust in an otherwise untrusted environment that are relevant for federated systems.’ (NIST, 2020, p. 45). According to NIST, these tools include consensus algorithms such as Paxos and Raft. What is interesting to note is that NIST[vi] previously (NIST, 2018) claimed that the development of these consensus algorithms started in the late 1980s. They considered the Paxos[vii] algorithm published by Leslie Lamport in 1998 to be the basis for the further development of consensus algorithms. About this publication NIST says: ‘The paper describes a consensus model for reaching agreement on a result in a network of computers where the computers or network itself may be unreliable.’ A contemporary example of a consensus network is Microsoft’s Confidential Consortium Framework, which is available on an open-source basis. This Coco Framework is based on consensus algorithms such as Paxos and Raft, creating possibilities for secure interconnection of local nodes or virtual machines in a closed network. The Coco Framework also offers ways to securely and reliably exchange and share information between groups of identified nodes. Based on the information shared, these nodes are able to autonomously make decisions on the performance of information transactions between nodes in the network and beyond. Given that there is no trusted third party in the network, it is up to the nodes to autonomously store the data and information they use in their own decentralised ledgers. The first experiments with this framework run by Centric show that it offers many possibilities for further development and application. However, these experiments also show that the thinking about development and application of these closed and secure networks within which nodes can autonomously make decisions requires different and new knowledge. New experiments that are currently being prepared will explore whether this development can also be applied in the context of, for example, multi-party computation where different parties’ networked computers can reach a shared outcome without any data and information being transferred.

European Federated Cloud Infrastructure

In 2020, the European Commission[viii] launched a European strategy for European data. This European strategy is based partly on new technological possibilities such as combinations of blockchain technology/consensus algorithms and digital infrastructures such as a European Federated Cloud Infrastructure, and therefore seems to be running in sync with the developments identified by NIST. In October 2020, the Dutch government signed the EU Declaration entitled ‘Building the next generation cloud for businesses and the public sector in the EU[ix]. In this declaration, the EU member states agree to pursue a European Federated Cloud to create next-generation secure, energy-efficient, and interoperable cloud infrastructure for Europe as a whole. The EU member states furthermore agreed to jointly ‘cooperate towards one set of common technical rules and norms (future EU Cloud Rulebook) and the deployment of interconnected cloud capacities across the EU, including common marketplaces’. Whether and, if so, how consensus algorithms will play a role in the organisation of the European Federated Cloud is (as yet) unclear. The Franco-German GAIA-X initiative also explores whether and, if so, how a European Federated Cloud infrastructure can be organised and implemented to gain greater control of data produced and used in Europe. The first document published by GAIA-X[x] in 2019 states as follows: ‘We understand data infrastructure as a federated technical infrastructure, consisting of components and services that make it possible to access data and to store, exchange and use it according to predefined rules’ (2019, p.2). The development towards a European Federated Cloud infrastructure for greater control and better governance of European data and information can therefore be considered a given.

Conclusion

The whole of a federation of interconnected individual cloud configurations can be considered a form of self-organisation by machines as once described by Ashby. Based on algorithms and software, an autonomous whole can be created that is able to autonomously make consensus-based decisions. These joint and consensus-based decisions serve as the basis for the autonomous execution and settlement of information transactions. This form of self-organisation can be the foundation for the creation of a sovereign European data space. The emphasis in the development of the European Federated Cloud infrastructure will inevitably have to be on the development of new conventions and rules for organisation and governance by this federated infrastructure. The rules for this federation will, however, only work if they can be implemented by the federated participants based on new and transparent algorithms, protocols, and software that underpin the European Federated Cloud infrastructure. The perspective in this development thus gradually shifts away from human action towards independent and autonomous action by hardware and software configurations that collaborate and make decisions within a federated association. Hopefully, this development will include sufficient focus on European values and the history on which it is based.

[i] Ashby, W.R. (1957) An Introduction to Cybernetics. London, Chapman Hall LTD. Second impression

[ii] Ashby, W.R. (1962) ‘Principles of the Self-Organizing System’. In: Heinz Von Foerster and George W. Zopf, Jr. (eds.), Principles of Self-Organization (Sponsored by Information Systems Branch, U.S. Office of Naval Research). Republished as a PDF in Emergence: Complexity and Organization (E:CO) Special Double Issue Vol. 6, Nos. 1-2 2004, pp. 102–126.

[iii] Mell P. and Grance T. (2011) The NIST Definition of Cloud Computing. NIST Special Publication 800-145

[iv] Lee C.A., Bohm R.B. and Michel M. (2020) NIST Federated Cloud Reference Architecture. NIST Special Publication 500-332., February 2020

[v] Cachin C. and Vukolic M. (2017) ‘Blockchain Consensus Protocols in the Wild’, 31st International Symposium on Distributed Computing (DISC 2017), Vienna, Austria, 16-20 October 2017

[vi] Yaga D., Mell P., Roby N., and Scarfone K. (2018) NIST Blockchain Technology Overview, NISTIR 8202, October 2018

[vii] Lamport L. (1998) ‘The Part-Time Parliament’. In: ACM Transactions on Computer Systems 16, 2 (May 1998), pp. 133-169.

[viii] European Commission (2020) European strategy for data. Brussels, 19.2.2020 COM(2020) 66 final: https://ec.europa.eu/info/sites/info/files/communication-european-strategy-data-19feb2020_en.pdf

[ix] Joint declaration: Building the next generation cloud for businesses and the public sector in the EU; https://ec.europa.eu/digital-single-market/en/news/towards-next-generation-cloud-europe

[x] German Federal Ministry for Economic Affairs and Energy. (2019) Project GAIA-X. A Federated Data Infrastructure as the Cradle of a Vibrant European Ecosystem. (October 2019): https://www.bmwi.de/Redaktion/EN/Publikationen/Digitale-Welt/project-gaia-x.pdf?__blob=publicationFile&v=4

Ben van Lier, Director Strategy & Innovation

Meer weten?

Ben van Lier, Director Strategy & Innovation

Voor meer informatie neem je contact op met Ben van Lier via +31 6 51 10 94 39 of per mail ben.van.lier@centric.eu.