Confirmed speakers (12 August update)
Asma Alamri and Gabriela Saldanha
Google Translate and Children’s Literature Translation
Khaled Ben Milad
The TM matching of the associated source-and-target pairs is estimated through computing a score for the similarity of TM sources to input segments. In order to filter out less similar segments, translators can set a minimum match threshold, 70% for example (Bloodgood and Strauss, 2014), to limit TM suggestions to those for which scores are equal to or above the threshold value, on the assumption that these suggestions will be the most useful. However, this begs the question as to what sort of qualities the TM sources need to display to be considered highly similar to the input segment. In fact, the answer is not as simple as might be supposed since useful target segments in the TM dataset might exist but fail to be selected because their associated source segment contains a move (re-ordering) operation. This may be due to the fact that the TM matching procedure attempts to search for similarity using the same word order as the whole input segment: For instance, if [abcd] is given to a translator as a source text but their TM source is [bacd], would the TM matching algorithms accurately compute the high similarity? And if not, why not?
The word order of Arabic is flexible so verbal sentences, which are more common, can begin using the subject (Habash, 2010). In such a scenario, an experiment set out to answer the question above. Our hypothesis is that current TM algorithms, which are based largely on Levenshtein edit-distance calculations (Simard and Fujita 2012), will not provide appropriate fuzzy-matching scores if a TM source includes a move (reordering) operation.
Data and Methodology
The evaluation method treated the TM as a ‘black-box’ component; a test suite was used as an instrument to evaluate TM recall. In order to run the test, 85 verbal sentences in Arabic, which ranged from three to ten words in length, were extracted from the Arabic – English MeedanMemory. The subject of each test segment contained a different type of sub-segment, a single or multiple-word unit. Having extracted the segments, we applied a move operation – Verb place (1) was exchanged to Subject unit position (2). As a result, the test segments were SVO while the word order of TM sources was VSO, bearing in mind that the meaning of the two sentences was identical. After processing the test segments, we then submitted these as a document for translation to the five CAT tools – namely, Déjà Vu X3; OmegaT 5.2; memoQ 9.0; Memsource Cloud and SDL Trados Studio 2019. The document was uploaded as a file for translation, MeedanMemory attached as a TM and 70% was set as the translation threshold in the CAT tools. Then, those target segments that were matched at the translation threshold or higher were presented in the proposal window, while lower matches were not.
The results show that the fuzzy match values of segment retrieval that included a move operation reduced as the length of the segments decreased. However, TM matching metrics of the five CAT tools used different routines for handling such moves. The matching scores when retrieving short segments that included a move operation were lower than for long segments, despite the high usability of these segments, regardless of the sentence length.
The paper concludes that only longer sentences which include a move operation are likely to be presented as TM proposals; short sentences which include a move operation do not benefit from the use of TM. A possible explanation for the production of low-scoring matches is that the TM systems’ matching metrics did not recognise the move intervention, in which a procedure of calculating strings of surface forms was used.
Khaled Ben Milad
The evolution of neural machine translation (NMT) has shown effectiveness and promising results in some European languages (Sennrich et al., 2016), but its success is limited to language pairs with availability of large amounts of parallel data; the standard encoder-decoder models show poor performance in low-resource languages (Koehn and Knowles, 2017). For Arabic, as a morphologically rich and low-resourced language, the afore-mentioned approach has its own limitations that make it not effective enough for translation quality. A number of MT systems that have switched to NMT and offer a service of Arabic <> English translation were evaluated in terms of their translation quality. Abdelaal and Alazzawie (2020) investigated the quality of Google’s output when translating from Arabic to English using human ranking. The study found that the system’s output produced high semantic adequacy, but some errors were found regarding fluency. A similar study, Al-mahasees (2020) tested two NMT systems – Google, Bing, in addition to the Sakhr hybrid MT system – using adequacy and fluency ratings. The study, which was conducted twice, in 2016 and 2017, to compare the development of the systems, found that Google outperformed the other systems regarding the production of adequate and fluent output over the two years in both directions. The current study will evaluate translation quality of a set of NMT systems – free and commercial, in both directions using human judgment and automatic metrics.
Data and Methodology
For the purpose of the study, ten texts in Arabic and ten in English were randomly extracted from an Arabic-English corpus – LDC2004T18, each text consisting of several sentences – test one consists of 117 sentences in Arabic, while test two includes 109 sentences in English. Having extracted sentences, the two test suites were translated using, in addition to the free systems – Bing NMT; Google NMT and Yandex NMT, the commercial interactive and adaptive Lilt. The reason behind involving Lilt, which has only recently supported Arabic, was to compare its translation quality against the free systems’ output. The evaluation method employed in the study was based on procedures of human judgment and automatic evaluation. In terms of human judgment, a subset of four source sentences each paired with its four MT systems’ output in each direction was distributed in an online questionnaire, in which participants were asked to rate adequacy and fluency on a four-point Likert scale, according to TAUS quality criteria. For automatic evaluation, BLEU was applied.
In terms of adequacy and fluency ratings, 11 respondents answered the questionnaire. The participants’ preference varied from system to system. However, the overall mean score reveals that the systems’ output expressed a high level of meaning rather than producing fluent translation, which suggests that heavier edit-operations were needed for post-editing fluency. A further observation is that the quality of Arabic to English translation was assigned relatively higher scores than when translating from English to Arabic. The very common error noticed in the systems’ output when Arabic was a target was untranslated words – proper nouns, which led to a reduction in translation quality. However, the outcome of human evaluation reveals that the most adequate and fluent MT system was Google, followed by Bing, with the other two systems behind, in both translation directions. Regarding the automatic scores, the BLEU 3 metric was run on the same questionnaire translation pairs. The result showed that Bing NMT was the best, followed by Google NMT, in both directions. A comparison of the human judgment results against BLEU scores shows that although the system producing the best quality translation was different between the two quality evaluation procedures, translation from morphologically rich languages was easier for NMT systems than translating into morphologically rich languages. Additional information is that we also ran the tests on ModernMT during the period of the free translation service offered due to COVID-19 – in May 2020. The system’s output received a better BLEU score than Lilt and Yandex in terms of Arabic to English translation, while it scored the best regarding English to Arabic translation, in which proper names were translated.
This study investigated translation quality of a set of NMT systems in terms of Arabic<>English translation using human evaluation and automatic metrics. The results reveal that NMT systems produced more adequate translation than fluent translation in both directions. Further, producing fluent translation into Arabic was more difficult than into English. Moreover, Google NMT was rated by evaluators the most adequate and fluent system in both translation directions, while Bing NMT achieved the best BLEU score.
Denis Dechandon and Maria Recort Ruiz
Integration of Semantics and Metadata
Denis Dechandon and Maria Recort Ruiz
Terminology: Towards a Systematic Integration of Semantics and Metadata
The European Parliament works in 24 languages and is committed to ensuring the highest possible degree of resource-efficient multilingualism. In the European Parliament, all parliamentary documents are published in all of the EU’s official languages, which are considered equally important.
The right of each Member of the Parliament to read and write parliamentary documents, follow debates and speak in his or her own official language is expressly recognised in the Parliament’s Rules of Procedure.
Multilingualism also makes the European institutions more accessible and transparent for all citizens of the Union, which is essential for the success of EU democracy. Europeans are entitled to follow the Parliament’s work, ask questions and receive replies in their own language, under European legislation.
In order to produce the different language versions of its written documents and communicate with EU citizens in all the official languages, the European Parliament maintains an in-house translation service able to meet its quality requirements and work to the tight deadlines imposed by parliamentary procedures. Interpreting services are provided for multilingual meetings organised by the official bodies of the institution.
Despite the efforts, deaf and hard of hearing people cannot currently follow the European Parliament debates in real time. Subtitling by human transcribers, with the high degree of multilingualism required, is a highly resource-intensive task. Automatic live transcription of parliamentary debates would make them accessible to people with disabilities and thus improve the services the Parliament is offering to the citizens.
The European Parliament already uses technology as a support to the translation process. Offline automatic speech recognition is used to facilitate the production of the verbatim transcript of plenary sessions.
The European Parliament has begun to explore further the potential of online automatic speech recognition and machine translation technologies for the 24 languages in managing multilingualism efficiently and providing better, cost-efficient services for cross-lingual communication for its Members and European citizens.
We will present how the European Parliament wishes to support multilingual innovation and the digitalisation of all EU official languages through targeted developments relying heavily on the use of AI technologies.
It is therefore aiming to enter into an Innovation Partnership in order to acquire a licence for a tool that is able to automatically transcribe and translate parliamentary multilingual debates in real time.
The tool will also be able to learn from corrections and supporting data as well as from user feedback, so as to enhance quality levels over time.
The objective is to provide a useful service for Members of the European Parliament in accessing debates on screen as well as to provide accessibility for deaf and hard of hearing people who currently have no direct access to the debates of the European Parliament.
The ultimate goal is to provide an automatic transcription and machine translation service for parliamentary debates covering the 24 official languages used by the institution.
NMT engines with bilingual glossary feature: does it really improve terminology accuracy and consistency?
In times of change we are looking for new ways to perform our regular tasks, like working from home without seeing our colleagues and having the chance to work with them face-to-face. Despite the distance, we need to cooperate with each other, discuss, guide and teach our workmates, learn from each other, make plans together and do our best to promote the team´s success.
But how can we boost our teamwork and keep distance at the same time? On the example of STAR Corporate Language Management, we will show you how to work with each other effectively. Thanks to our management tool, remote teams and individuals can manage the translation process in a well-planned way and improve their collaboration in teams. You will see how to use STAR Corporate Language Management and its benefits, how to coordinate the translation within a web-based platform and achieve your best result working together – completely virtually.
We will create a translation project simulating a normal workflow from its beginning up to completion. The audience will see how a workflow may be modified depending on the company´s structure and needs and how all the participants cooperate with each other within the created model.
Working with STAR Corporate Language Management reduces administrative duties and especially in these times, when all parties involved are working from different locations, it links all participants through defined processes. Be part of a fictive team and simulate a real translation order with us. Observe how the different roles interact with each other within the platform. The team members take action only when their respective step comes, and their expertise is needed. To achieve a smooth transition from one step to the other, it is important to model the right workflow and define the roles and tasks, automating the routine. In our example we will use the model Customer-TranslatorReviewer-Customer. However, workflow customisation is the key to success here. And since the processes can vary from company to company, we will give you an idea which roles and steps are primary and how they can be extended.
Another important topic of distance working is the way of task assignment: task pool or direct job assignment. Task pools have a lot of benefits, but also disadvantages compared to direct job assignments. We will have a look into them and analyze the role of the project manager in both scenarios. Project managers are the team leaders when it comes to translation orders. We will give valuable hints how to optimize their work and get the best results using web tools so that they are well equipped to handle project scheduling, standardization, resource optimization, reporting and controlling.
We are all facing the problems of remote teamwork, such as transparency and project security. We will show possibilities as to how to solve them in an efficient way, how to manage company-wide processes with one tool, virtually bringing people together.
Assessing cross-lingual word similarities with the use of neural networks
What is the relationship between these three activities you will say? Well, you can do all in a row if you are well organised. The first one to warm you, the second one to cool you and the last one to let you do your job relaxed and mentally well prepared.
But those activities have to be practiced at the same place, otherwise, one day will not be enough, especially if you have to take public transport or drive to the place where interpretation is requested.
Ever heard about the RSI – Remote Interpretation Services? No? But you know what the coronavirus is, surely? Well, there is a relationship between the two: in the past, hundreds of years ago, interpreters had to move to the place of interpretation (booth, exhibition, court, …) and would lose a lot of time and sometimes nerves, spend a little money for the night, food and show nice dress apparel on site. Then came a nasty virus which forced everyone to stay home for some weeks and a lot of events were cancelled. All of a sudden, event organisers, but also all companies in need of communication devices discovered that interpretation services can also be offered on-line with rather simple tools through the Internet.
Actually, RSI were created well before the COVID-19 crisis, but they were not very welcomed by parties on both sides: interpreters would refuse to use such a degrading device, arguing about the low quality of the equipment, the risk of connection interruption, the loss of quality of interpretation and criticise every colleague showing too much interest. No serious discussion could be started about RSI without hysteric recriminations by the “real professionals”.
And for the companies, though interested by the reduced cost of this service compared to on site interpretation, they doubted that the technique was reliable, that the interpreters were real professionals and that the audience would be satisfied and would appreciate such service.
Then, due to the fact that nearly anybody today in our Western societies has a smartphone that could support the download of a specific App and possibly use earbuds, RSI is today accessible nearly for free for the audience and at a reduced price for the customers.
How could such a change occur and what are the future prospects?
Digitalisation is applied in every domain, including in translation and interpretation businesses. Solutions have existed for a long time, but were not really considered in the past. The main opposition came from the professionals themselves, either in translation or in interpretation. A translation agency is considered a kind of devil stealing the heart and essence of the profession to resell garbage, useless documents, thus hiring the worst translators and taking the most profitable customers away from the market.
RSI underwent the same criticism in terms of recruitment: only incompetent interpreters would work for such horrible employers, equipment would be of bad quality, and unpredicable events could occur during transmission. So, why do broadcasting programs show World Cup finals or any event if the quality of the devices were not good. Everyone should go to the stadium instead. Ah ah, but the match takes place on the other side of the world, so what is to be done to see the event?