Digital Government: Design, Development, and Evolution of Estonia’s X-Road.
Integration and interoperability are one of the biggest hurdles in the way digital transformation of governments; as Estonia has shown us with “X-Road”, it is one of the most critical technological pillar and enabler.
Interoperability platform enables easy integration with various government agencies and private enterprises, which in turn results in enablement of services, products, and new businesses. Moreover, It also results in cost efficiencies, agility, and innovation.
Design and development of such a platform requires leadership support, sound policies, alignment of stakeholders, and modern technology architecture.
Since I have never worked on “X-Road” to provide first-hand information, so instead I will share some very informative and useful links about the platform (and since I am lazy, — so this post is more of an aggregation).
What is X-Road?
X-Road is a centrally managed distributed data exchange layer between information systems that provides a standardized and secure way to produce and consume services. X-Road ensures confidentiality, integrity and interoperability between data exchange parties. The data is always exchanged directly between a service consumer and a service provider, and no third parties have access to it.
X-Road is a data exchange layer between service consumers and business services provided by various information systems owned by different organisations. The services available via X-Road are independently deployable and loosely coupled, and they communicate with each other using language-agnostic APIs. Each service can be developed, deployed and scaled independently without affecting other services as long as the API remains unchanged.
Just like microservice architecture is enabler for building scalable, fault-tolerant and highly-available systems. The real value comes from services that are built on top of the technical infrastructure and the content that they provide to users.
X-road is the backbone of communication and data exchange in Estonia. Over 900 organizations (public and private) daily use x-road for consuming and delivering over 45 Million e-services per month. Originally X-Road was simply used to send queries to different databases. Now it has developed into a tool that can also write to multiple databases, transmit large data sets and perform searches across several databases simultaneously. X-Road can be scaled up as new e-services and new platforms come online.
During data exchange, X-Road ensures its parties with:
· Autonomy — an X-Road member defines, which data services it wishes to render and who gains access rights to the services
· Confidentiality — information reaches only the authorized parties
· Evidential value — using a digital signature enables proving the source of received data
· Interoperability — all X-Road members speak the same language, regardless of the technology or architecture a member is using.
X-Road implements a set of common features to support and facilitate data exchange. IT provides the following features out of the box:
address management
message routing
access rights management
organization level authentication
machine level authentication
transportation layer encryption
time-stamping
digital signature of messages
logging
error handling.
Architecture: Is it a service mesh? an API Gateway? … no its X-Road
X-Road is best suited for external data exchange over the public Internet. The most common use case is data exchange between two organisations, but a single organisation may have information systems that are hosted in different locations and communicate with each other over the Internet too. In this case X-Road is a good fit for internal data exchange as well.
At first sight X-Road may seem like a service mesh as the architecture and feature sets have many similarities — both provide secure and standardized connections, service-to-service authentication, logging, reporting etc. In addition, both are based on an architecture model that implements service level communication through a proxy component. However, X-Road is not a service mesh as service mesh is the connection layer between different services in microservices architecture. In other words, service mesh is used as an internal connection layer within an application or between multiple applications of a single organisation whereas X-Road is used as a connection layer between different organisations and information systems.
How about X-Road and an API gateway then — are they mutually exclusive or can they be used side by side? X-Road and an API gateway are both used to publish services to external clients. Their architecture and feature sets are different even though they have features in common too, e.g. publish APIs to external clients, service-to-service authentication, authorization, logging, metrics. The major difference between X-Road and API gateway is that X-Road requires that the Security Server is used on both service consumer and provider side whereas API gateway enables client connections directly without any additional components on the client side.
Overall, an API gateway provides more flexibility and API management related features compared to X-Road, but when the same client communicates with multiple API gateways the client must adapt to different requirements and configurations of multiple service providers. Instead, X-Road provides a single communication channel between multiple service providers and services that all share the same configuration that is automatically distributed and applied by X-Road. In addition, X-Road guarantees that both service consumer and service provider meet the same security requirements, and non-repudiation of all the processed messages by signing, time-stamping and logging every processed message on the consumer and provider side. The logs can be used in a court proceeding as evidence. These features make X-Road ideal solution for secure, reliable and auditable data exchange.
Connecting to the X-Road ecosystem
The essence of the X-road is a pipe transport system, or, if you prefer a stellar stargate system, to ensure connectivity among participants. Thus, the first step to engaging with the X-Road is to hook into the pipe system, i.e. get membership of the X-Road Club. After your building has been outfitted with piping, you can make requests of the pipe system’s other constituent members. Still there is a limitation: you only can request data from those with whom you have a usage agreement.
In other words your organizations’s membership in the X-Road will give you access to the pipeline transmission system. But only via agreements with each particular data provider will your organization be able to access the other member’s data. Further, and more importantly, you will be able to access the data of another X-Road member only in accordance with the terms of your agreement with them.
The identity of each organization and technical entry point (Security Server) is verified using certificates that are issued by a trusted Certification Authority (CA) when an organization joins an X-Road ecosystem. The identities are maintained centrally, but all the data is exchanged directly between a consumer and provider. Message routing is based on organization and service level identifiers that are mapped to physical network locations of the services by X-Road. All the evidence regarding the data exchange is stored locally by the data exchange parties, and no third parties have access to the data. Time-stamping and digital signature together guarantee non-repudiation of the data sent via X-Road.
An X-Road ecosystem is a community of organizations using the same instance of the X-Road software for producing and consuming services. The owner of the ecosystem, the governing authority, controls who’s allowed to join the community, and the owner defines regulations and practices that the ecosystem must follow. The ecosystem may be nationwide, like in Estonia and Finland, or it may be limited to organizations matching certain criteria, e.g. clients of a commercial service provider. Technically, the X-Road software does not set any limitations to the size of the ecosystem or to the member organizations.
Two X-Road ecosystems can be joined together, federated. Federation is a one to one relationship between two ecosystems. Members of the federated ecosystems can publish and consume services with each other as if they were members of the same ecosystem. It is possible to create federation connections with multiple ecosystems, but transitive federation relationships are not supported. Ecosystem does not have a federation relationship with another ecosystem that it’s not directly federated with.
Evolution: NIIS Begins X-Road Core Software Development
Estonia and Finland developed the X-Road core together from 2015 until June 2018 when the development was handed over to NIIS [Nordic Institute for Interoperability Solutions]. The handover caused changes in the locations of source code repositories and the joint development model was updated as well.
Since June 2018 NIIS has been managing the X-Road core technology and finalizing the preparations regarding the beginning of the actual development activities.
The environments are hosted on Amazon Web Services (AWS) cloud platform and the process for setting them up is highly automated. However, also automation scripts and templates need to be cleaned up and refactored now and then, so the transfer did not mean just moving all the existing environments as-is, but updating, optimizing and refactoring the environments too.
True open innovation model
Estonia’s digitization began with the rejection of Finland’s old analog telephone exchange. Now, the roles have been reversed. In 2015, Finland introduced a partial version of X-Road with the aim of moving towards a more digitized society, with support from Estonia. The Estonian and Finnish governments are also embarking on cross-border data exchange.
Any country today can set up their own X-Road system. The X-Road technology is now open-sourced and readily available for any country to employ. Central components of its source codes were published openly under an MIT license on October 3, 2016. In fact, Azerbaijan and Namibia are considering deploying their own digital systems. Azerbaijan, for instance, has enlisted the help of the company that developed X-Road to create an “updated version of Estonia’s X-Road.” Namibia is similarly deploying the same technology, with help of Estonian companies like Cybernetica and e-Governance Academy.
Estonia today is a model of digitization. It points to the road forward, and with its open-source technology, it has now paved the road for those who want to follow.