The iADAATPA pan-European consortium has built a dynamic router that can smoothly switch between domain-specific neural machine translation engines, and produce near-human machine translations.
The iADAATPA consortium became MT-Hub, an open-source framework for Public Administrations to manage and route machine translation requests.
Speedy cross-border communications are always a challenge for industry, services and public administrations, and more so within the European Union, a political entity with 24 official languages and a myriad of co-official and minority languages, from Catalan-Valencian, Galician and Basque in Spain to Sami in Finland, Occitan in France or Ruzyn in the Czech Republic, Hungary and Slovakia. Most EU Member States have machine translation systems in place at some levels of Public Administration − however many are still rule-based or not offering the latest industrial functions and quality. Many National Anchor Points (NAPs) and national authorities feel it is high time to replace old systems with 21st century technology, particularly new neural machine translation.
Back in 2017, our mother company Pangeanic had the novel idea of putting together a European consortium to build a scalable and secure platform, aligned with the objectives of the European Commission’s “Connecting Europe Facility” for the provision of a single point of connection to automated translation services from several companies. In actual fact, this meant building a deployable open-source national infrastructure to allow public administrations to run any machine system from a single or multiple companies without beign locked in to a company’s solution. The idea was called iADAATPA (Intelligent, Automatic Domain Adapted Automated Translation for Public Administrations) and it would work as a translation engine router with intelligent language and domain detection mechanisms. The idea was to provide a national infrastructure that would support and work, in a way, like eTranslation, detecting the language and domain of a given document and routing it to the translation engine best suited to translate content. The consortium went further, improving the granularity of the selection to page or even paragraph level.
In short, one of the main attractive points of iADAATPA is to switch between machine translation engines within a document. Different parts of a document, can be translated by different engines, as soon a change in language or domain is detected. Thus, public administrations are free to offer framework contracts to one or several technological companies to run machine translation within their national boundaries with a ready-made infrastructure that has been funded by the “Connecting Europe Facility” program.
Partners in the project
The project started in September 2017 and ran for 1,5 years. The iADAATPA consortium was made up of Pangeanic (Spain), SEAD – the Secretary of State for Digital Advancement (Spain), Everis (Spain), Prompsit (Spain), Segittur (Spain), Tilde (Latvia), KantanMT (Ireland) and the Dublin City University (Ireland).
An integration was made from iADAATPA to the EC’s eTranslation, which provided valuable language resources and MT engines in seven EU domains. eDelivery (AS4 Domibus) was used to ensure the secure and reliable exchange of the documents to be translated. This feature is now part of the platform to guarantee encryption and security in all data transmissions. To further complement the capabilities of eTranslation, consortium members also made additional integrations to their own MT engines to cover domains not provided by eTranslation.
iADAATPA makes use of sophisticated algorithms applied to neural networks based on artificial intelligence, instead of the traditional statistical methods based word sequence patterns, for example. Consortium partners and the European Commission agreed to go for the neural-networks based approach early on in the project. The platform was implemented and tested in the following use cases:
- Generalitat Valenciana, Spain – The Open Data portal of the regional government’s transparency department
- SEGITTUR Spain – Digital services provided by the agency for innovation and tourism technology based in Madrid
- Lithuanian Parliament – Translation of parliamentary proceedings in Lithuanian Parliament
- Dublin City College website
- Deutsche Welle
Results and benefits
iADAATPA’s language and domain detection capabilities are very attractive to achieve a better machine translation experience. Offering a vendor and technology neutral platform will increase machine translation efficiency and competition and lead to a higher use of the technology by public administrations, cost savings, more multilingual content available, better communication between European public administrations and competitive translation prices on the market. By enhancing access to different MT providers, it will also help realise the Digital Single Market in Europe.
The platform has been proven ready for use by public administrations, and the Consortium is working closely with Spanish authorities to have it deployed as a national infrastructure in Spain. Based on the positive outcome of the project, Spanish authorities have decided to adopt iADAATPA, now called MT-hub platform, as a core component of their national translation infrastructure.
Results also surpassed all expectations of the Lithuanian Parliament. Lithuania had had bad experiences with statistical machine translation technologies in the past. Their language is highly inflective and morphologically rich and thus difficult to translate with traditional statistical translation methods. With iADAATPA, productivity and MT quality significantly increased. The consortium reported productivity increases of around 40 % in achieving final human quality translations.
All members of the consortium had complementary skills and assets, which made collaboration very fruitful. Professor Andy Way from the Dublin City University rejoiced that “the iADAATPA consortium partners proved such a high level of professionalism that I’d love to work with all of them again. This is the proof of a well-run consortium.”
The consortium is now focusing on a part 2 of the project with the objective to make the “new-age machine translation” a reality and have more Public Administrations adopting the tool as a new service. This service will help public administrations to run a no “lock-in” policy to manage their own machine translation affairs, with the possibility of running their own “MT Hubs” at different levels.