This poster presents a follow up to our teaching case study described in ASLIB 2015. The subject of the present paper is how do we integrate the new ISO 25000 series (ISO/IEC 2014) to update the Eagles 7-steps recipe1 , which is one of the deliverables of the Evaluation of Natural Language Processing Systems project (EAGLES I and II) dating back to the 90’s, based on the ISO 9126 software evaluation series.
As mentioned in Starlander and Morado Vazquez (2013), the main objectives of the methodology taught within the Computer Assisted Translation MA course at the Faculty of Translation and Interpreting of the University of Geneva, is to provide our students with a functional evaluation methodology and the necessary knowledge to fulfil a task that they often have to face at the start of their carrier as a translator, or hence as freshly baked CAT tool “experts”.
The given assignment did not change radically from what was described in previous work. They still have to apply the imposed method in order to compare two translation memory systems acquired depending on a specific professional profile ranging from a freelance to a experimented CAT tool expert running a translators support in an international organization.
What is new in the present case study is that the students need to move away from the “classic EAGLES 7-steps” through the integration of the new quality characteristics contained in the ISO 2500 standards series. The main changes in the latter compared to the 9126 is the clarification of terminology used (Abran et al, 2005) and in the quality characterisitcs.
The present poster paper will focus on the results obtained by the students, in order to give a flavour of the achieved work within only several weeks of our MA course. The aim of this paper is thus to provide a ready-made methodology to evaluate CAT tools, that can be reused not only in the academic field by contributing to include such knowledge into “basic” translator’s training but also by freelancers willing to evaluate several tools before making their choice.
It must be noted that although the “ability to evaluate the suitability of a tool in relation to technical needs and price” was identified by Pym (2012) as one of the necessary skills that translation students should acquire, this skill is not yet usually included into classical translation’s training, not even during CAT tool classes.
The proposed methodology is based on a yet another simplification of the EAGLES methodology while integrating a quality model based on the new ISO 25000 series (ISO/IEC 2014). The proposed methodology will be tested by our 2015-2016 MA students. We will thus provide an overview of the proposed evaluation methodology which will be accompanied by a feedback questionnaire. The latter will inform the reader on how this methodology was perceived by the students groups while applying it during their assignment.
1 EAGLES Evaluation Working Group (1999): The EAGLES 7-step recipe, available at http://www.issco.unige.ch/en/research/projects/eagles/ewg99/7steps.html
———
References:
Abran, A., Al-Qutaish, R. E., & Desharnais, J. M. (2005). Harmonization issues in the updating of the ISO standards on software product quality. Metrics News, 10(2), 35-44.
EAGLES Evaluation Working Group (1999): The EAGLES 7-step recipe, available at http://www.issco.unige.ch/en/research/projects/eagles/ewg99/7steps.html
ISO/IEC 9126-1:2001 Software engineering — Product quality — Part 1: Quality model
ISO/IEC 25010 (2011) : Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — System and software quality models
ISO/IEC (2014). ISO/IEC 25000:2014(en) Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — Guide to SQuaRE. Geneva, International Organization for Standardization / International Electrotechnical Commission.
Pym, A. (2012) ‘Translation skill-sets in a machine-translation age’, [online], available: http://usuaris.tinet.cat/apym/on-line/training/2012_competence_pym.pdf [accessed 6 Nov 2013].
Starlander, M. and L. Morado Vazquez (2013). Training translation students to evaluate CAT tools using Eagles: a case study. Aslib: Translating and the Computer 35. Londres, Aslib.
