Translation process research has developed tools to gather and analyse empirical data, but while a variety of measures have proved useful and reliable to measure post-edit machine translation effort (see e.g. Vieira 2016 : 42), translation processes are seldom considered when assessing the relevance of a given Machine translation post-editing scenario. Our study seeks to determine the impact of including MTPE in the evaluation process. We selected adequacy and fluency ratings. Based on two distinct experimental conditions, we then compared the ratings produced without performing PE and those produced immediately after a light PE process. Inter-rater reliability was assessed for each segment in each text (N=55) using Fleiss’ kappa for adequacy and fluency scores, and an intraclass correlation coefficient (Vieira 2016 : 52) for temporal measures. While the reliability of the measures collected without PE was low, the measures collected in PET were for the most part homogeneous. Qualitative analyses of the problematic segments, as evidenced by both kappa and intraclass correlation coefficients, showed strong Spearman’s correlations, whether positive or negative, between temporal measures and all the other metrics for NMT but weakest ones for SMT. Based on these results, we discuss the advantages and risks of NMTPE.

This presentation will give participants a quick introduction to translationQ and then an overview of the current use cases, both in universities and in big organisations. Specifically, the productivity features for integrating with Microsoft Word and existing translation tools (like SDL or Matecat) through XLIFF support will be discussed and illustrated.

The in-house Language Service at Swiss Post translates a wide variety of texts from and into German, French, Italian, and English. It has emerged from internal discussions over the years that the extensive hype around neural machine translation (NMT) and its improved fluency, in comparison with previous approaches, has led many of Swiss Post’s employees to turn to freely available, generic MT systems to obtain quick, raw translations.

Before introducing NMT into their production workflow, Swiss Post’s Language Service decided to carry out a study to assess whether their customers (Swiss Post employees, who are also end-users of translations produced by the Language Service) would rate post-edited NMT more highly than raw NMT for both a customized and a generic MT engine (DeepL). Most importantly, the study also assessed whether the customers would be willing to pay for post-edited texts when made aware of some production metadata, such as data security and cost. This latter aspect could help determine whether the customers would still value the human intervention or whether they would rather accept a lower quality translation and associated risks if it means they can save on costs.

Due to the long lasting presence on the translation market STAR can provide three digital assets. Translation memory, TermStar data bases and STAR MT come into action where they fit best. Their combination in several STAR products shows the seamless integration of three types of resources. The strength of each asset is brought together in order to support best linguistic results.

Having developed both STAR Transit (translation memory system) and STAR MT (machine translation platform), STAR has combined these into a single integrated solution. The MT engines are trained with domain-specific translations and terminology. In the translation process with Transit, files are imported and pretranslated with TM. All untranslated segments are MT-translated. These MT translations are clearly marked and provided to the translator as suggestions together with the fuzzy matches from the TM. With the concept of TM-validated MT suggestions translators gain even more from both technologies because the translations from TM and MT are combined into one match.
In general, the STAR TM-MT environment provides numerous features that support the demanding requirements of legal documentation. First of all, the document structure for all translations is kept in the TM and can be searched for context information. Secondly, domain-specific dictionaries can easily be created and used via an integrated terminology component. In addition, context information to the terms can quickly be retrieved from the translations in the TM via “dynamic linking”. And finally, numerous sophisticated QA functions, including specific options to check MT-based translations, support high quality.

The OECD (Organisation for Economic Co-operation and Development) has launched a Digital Strategy in order to maximize the business benefits of digital initiatives. In 2018, one of the Translation Division’s contributions to this program was the development of a Neural Machine Translation (NMT) system in collaboration with the WIPO (World Intellectual Property Organization). The objectives of this Pilot project was to explore new sources of efficiency gains and to extend the coverage of OECD content in both official languages (English and French).

This workshop will explain what has been achieved with this project. It will describe the methodology chosen to assess the quality of the sentences translated automatically and share the results and the lessons learned. Hopefully, it should help the participants decide how to take advantage of this new technology.

Post-editing has become a regular part of the machine translation process, and many translators are already specialised in providing this service. Post-editing is, however, not the only way to improve the outcome of machine translation. An alternative is the pre-editing of a text prior to the utilisation of machine translation software. The poster describes the results of an independent study project in a course for translation students working with the language pairs English-German and English-Spanish. The task was to analyse specific problems of MT with regard to text type and terminology, with a view to discovering how pre-editing a text can improve the result of MT. The participants first produced a human translation of a text they had chosen themselves. They then carried out a translation using DeepL and assessed the problems of the resulting texts. In order to reduce mistakes and to achieve a better machine-translated version, they pre-edited the source texts and compared the revised outcome to the previous versions.

Overall, it can be said that in many cases the translation by DeepL was surprisingly good and could be further improved by pre-editing. Some problems could easily be solved by pre-editing. This was achieved in particular by replacing terms that could have several meanings by terms which only imply the desired meaning, and that sentences in which the references were not clear reformulated so that the references were clear. This was in line with the general rules for pre-editing, that ambiguous terms should be avoided, references should be clearly identifiable, and complex sentences should be simplified. The translation results that were achieved by pre-editing the texts were surprisingly good in some cases, but there are certain problems that cannot be solved without post-editing. The inconsistent use of terms, for example, is an almost unmanageable problem. A human translator is therefore still essential for a good translation.

In a study done in 2018 it was reported that few professors teach technology in few translation courses in Mexico. Some reasons for this were that instructors had not been well trained in their academic programs when they were students, or they lacked a more comprehensive knowledge of these technologies (Peña, 2018). Effective training was not possible for most of these instructors as students and they seem to be reproducing similar learning insufficiencies with future translators. Because of this, another survey-based project was devised to identify the use that professionals who graduated from Mexican translation programs are making of translation technologies.

How has their educational background affected their disposition towards the use of translation technologies? Some results indicate that professional translators do not resort to the use of “core” translation technologies very often, but do use other electronic resources useful for accomplishing their tasks. One in two translators thinks their income has increased due to their technology knowledge, and they learn about these technologies on their own. Professional translators think they could have learned about Translation Environment Tools ((TEnTs) at university (and they wished they had), but university instructors are still not teaching these technologies as much. So there is a need reported by a few professionals, but not being dealt by some university programs.

Keywords: translation technologies, translation training, TEnTs, translation environment tools, computer-aided translation, CAT, Mexico

Peña, A. (2018). Use of technologies in Mexican translation programs. Unpublished manuscript.

This article presents a platform, named ACCOLÉ, for the collaborative annotation of translation errors.

ACCOLÉ offers a range of services that allow simplified management of corpora and typologies of errors, annotation of effective errors, collaboration during annotation, and finally different kinds of search in corpora. ACCOLÉ allows the annotation of translation errors according to built-in error typologies, Vilar’s typology or DQF-MQM or uploaded ones, on several corpora of different texts, translated by different Statistical or Neural Machine Translation systems, as well as processing the annotated corpora created in order to look for typical error models and patterns, related to a specific MT system.

The collaboration feature also gives the possibility to detect any misleading interpretation of an error type among the annotators. ACCOLÉ currently provides 15 corpora, 7 projects of 201,474 words and 18,301 annotations that we will describe in the final paper. Eventually, we will implement the semi-automatic propagation of found patterns on other corpora to enlarge the scope of linguistic studies, thus providing to the community a wide range of error annotated bilingual parallel corpora.

Evaluating the performance of an MT system with new content – that is, MT performance prediction – is one of the most challenging aspects of MT, mainly because lack of reference translations does not allow using automatic metrics. Additionally, human evaluation can be expensive and time-consuming. As an LSP, at CPSL we deal with hundreds of translation requests daily and must choose the most appropriate workflow for our customers in a timely manner. That’s why we needed a fast, reliable and cost-effective solution allowing us to find out if a given MT system is suitable for specific content. After trying different methods and tools, we chose the solution provided by ContentQuo, a translation quality management platform, based on the widely accepted Adequacy-Fluency methodology for MT evaluation. In our presentation we will introduce you to the challenges of MT quality evaluation and how we address them with ContentQuo.