Download or read book Exploring Prediction Uncertainty in Machine Translation Quality Estimation written by Daniel Beck and published by . This book was released on 2016 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Quality Estimation for Machine Translation written by Lucia Specia and published by Springer Nature. This book was released on 2022-05-31 with total page 148 pages. Available in PDF, EPUB and Kindle. Book excerpt: Many applications within natural language processing involve performing text-to-text transformations, i.e., given a text in natural language as input, systems are required to produce a version of this text (e.g., a translation), also in natural language, as output. Automatically evaluating the output of such systems is an important component in developing text-to-text applications. Two approaches have been proposed for this problem: (i) to compare the system outputs against one or more reference outputs using string matching-based evaluation metrics and (ii) to build models based on human feedback to predict the quality of system outputs without reference texts. Despite their popularity, reference-based evaluation metrics are faced with the challenge that multiple good (and bad) quality outputs can be produced by text-to-text approaches for the same input. This variation is very hard to capture, even with multiple reference texts. In addition, reference-based metrics cannot be used in production (e.g., online machine translation systems), when systems are expected to produce outputs for any unseen input. In this book, we focus on the second set of metrics, so-called Quality Estimation (QE) metrics, where the goal is to provide an estimate on how good or reliable the texts produced by an application are without access to gold-standard outputs. QE enables different types of evaluation that can target different types of users and applications. Machine learning techniques are used to build QE models with various types of quality labels and explicit features or learnt representations, which can then predict the quality of unseen system outputs. This book describes the topic of QE for text-to-text applications, covering quality labels, features, algorithms, evaluation, uses, and state-of-the-art approaches. It focuses on machine translation as application, since this represents most of the QE work done to date. It also briefly describes QE for several other applications, including text simplification, text summarization, grammatical error correction, and natural language generation.

Download or read book Investigating Continuous Space Language Models for Machine Translation Quality Estimation written by Kashif Shah and published by . This book was released on 2015 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Word Confidence Estimation and Its Applications in Statistical Machine Translation written by Ngoc Quang Luong and published by . This book was released on 2014 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Machine Translation (MT) systems, which generate automatically the translation of a target language for each source sentence, have achieved impressive gains during the recent decades and are now becoming the effective language assistances for the entire community in a globalized world. Nonetheless, due to various factors, MT quality is still not perfect in general, and the end users therefore expect to know how much should they trust a specific translation. Building a method that is capable of pointing out the correct parts, detecting the translation errors and concluding the overall quality of each MT hypothesis is definitely beneficial for not only the end users, but also for the translators, post-editors, and MT systems themselves. Such method is widely known under the name Confidence Estimation (CE) or Quality Estimation (QE). The motivations of building such automatic estimation methods originate from the actual drawbacks of assessing manually the MT quality: this task is time consuming, effort costly, and sometimes impossible in case where the readers have little or no knowledge of the source language. This thesis mostly focuses on the CE methods at word level (WCE). The WCE classifier tags each word in the MT output a quality label. The WCE working mechanism is straightforward: a classifier trained beforehand by a number of features using ML methods computes the confidence score of each label for each MT output word, then tag this word with highest score label. Nowadays, WCE shows an increasing importance in many aspects of MT. Firstly, it assists the post-editors to quickly identify the translation errors, hence improve their productivity. Secondly, it informs readers of portions of sentence that are not reliable to avoid the misunderstanding about the sentence's content. Thirdly, it selects the best translation among options from multiple MT systems. Last but not least, WCE scores can help to improve the MT quality via some scenarios: N-best list re-ranking, Search Graph Re-decoding, etc. In this thesis, we aim at building and optimizing our baseline WCE system, then exploiting it to improve MT and Sentence Confidence Estimation (SCE). Compare to the previous approaches, our novel contributions spread of these following main points. Firstly, we integrate various types of prediction indicators: system-based features extracted from the MT system, together with lexical, syntactic and semantic features to build the baseline WCE systems. We also apply multiple Machine Learning (ML) models on the entire feature set and then compare their performances to select the optimal one to optimize. Secondly, the usefulness of all features is deeper investigated using a greedy feature selection algorithm. Thirdly, we propose a solution that exploits Boosting algorithm as a learning method in order to strengthen the contribution of dominant feature subsets to the system, thus improve of the system's prediction capability. Lastly, we explore the contributions of WCE in improving MT quality via some scenarios. In N-best list re-ranking, we synthesize scores from WCE outputs and integrate them with decoder scores to calculate again the objective function value, then to re-order the N-best list to choose a better candidate. In the decoder's search graph re-decoding, the proposition is to apply WCE score directly to the nodes containing each word to update its cost regarding on the word quality. Furthermore, WCE scores are used to build useful features, which can enhance the performance of the Sentence Confidence Estimation system. In total, our work brings the insightful and multidimensional picture of word quality prediction and its positive impact on various sectors for Machine Translation. The promising results open up a big avenue where WCE can play its role, such as WCE for Automatic Speech Recognition (ASR) System, WCE for multiple MT selection, and WCE for re-trainable and self-learning MT systems.

Download or read book Translation Quality Assessment written by Joss Moorkens and published by Springer. This book was released on 2018-07-13 with total page 292 pages. Available in PDF, EPUB and Kindle. Book excerpt: This is the first volume that brings together research and practice from academic and industry settings and a combination of human and machine translation evaluation. Its comprehensive collection of papers by leading experts in human and machine translation quality and evaluation who situate current developments and chart future trends fills a clear gap in the literature. This is critical to the successful integration of translation technologies in the industry today, where the lines between human and machine are becoming increasingly blurred by technology: this affects the whole translation landscape, from students and trainers to project managers and professionals, including in-house and freelance translators, as well as, of course, translation scholars and researchers. The editors have broad experience in translation quality evaluation research, including investigations into professional practice with qualitative and quantitative studies, and the contributors are leading experts in their respective fields, providing a unique set of complementary perspectives on human and machine translation quality and evaluation, combining theoretical and applied approaches.

Download or read book Deep Interactive Text Prediction and Quality Estimation in Translation Interfaces written by Christopher M. Hokamp and published by . This book was released on 2018 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: The output of automatic translation systems is usually destined for human consumption. In most cases, translators use machine translation (MT) as the first step in the process of creating a fluent translation in a target language given a text in a source language. However, there are many possible ways for translators to interact with MT. The goal of this thesis is to investigate new interactive designs and interfaces for translation. In the first part of the thesis, we present pilot studies which investigate aspects of the interactive translation process, building upon insights from Human-Computer Interaction (HCI) and Translation Studies. We developed HandyCAT, an open-source platform for translation process research, which was used to conduct two user studies: an investigation into interactive machine translation and evaluation of a novel component for post-editing. We then propose new models for quality estimation (QE) of MT, and new models for es- timating the confidence of prefix-based neural interactive MT (IMT) systems. We present a series of experiments using neural sequence models for QE and IMT. We focus upon token-level QE models, which can be used as standalone components or integrated into post-editing pipelines, guiding users in selecting phrases to edit. We introduce a strong recurrent baseline for neural QE, and show how state of the art automatic post-editing (APE) models can be re-purposed for word-level QE. We also propose an auxiliary con- fidence model, which can be attached to (I)-MT systems to use the model's internal state to estimate confidence about the model's predictions. The third part of the thesis introduces lexically constrained decoding using grid beam search (GBS), a means of expanding prefix-based interactive translation to general lexical constraints. By integrating lexically constrained decoding with word-level QE, we then suggest a novel interactive design for translation interfaces, and test our hypotheses using simulated editing. The final section focuses upon designing an interface for interactive post-editing, incorporating both GBS and QE. We design components which introduce a new way of interacting with translation models, and test these components in a user-study.

Download or read book Document level Machine Translation Quality Estimation written by Carolina Scarton and published by . This book was released on 2016 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Natural Language Processing and Chinese Computing written by Fei Liu and published by Springer Nature. This book was released on 2023-10-07 with total page 897 pages. Available in PDF, EPUB and Kindle. Book excerpt: This three-volume set constitutes the refereed proceedings of the 12th National CCF Conference on Natural Language Processing and Chinese Computing, NLPCC 2023, held in Foshan, China, during October 12–15, 2023. The 143 regular papers included in these proceedings were carefully reviewed and selected from 478 submissions. They were organized in topical sections as follows: dialogue systems; fundamentals of NLP; information extraction and knowledge graph; machine learning for NLP; machine translation and multilinguality; multimodality and explainability; NLP applications and text mining; question answering; large language models; summarization and generation; student workshop; and evaluation workshop.

Download or read book Neural Machine Translation written by Philipp Koehn and published by Cambridge University Press. This book was released on 2020-06-18 with total page 409 pages. Available in PDF, EPUB and Kindle. Book excerpt: Learn how to build machine translation systems with deep learning from the ground up, from basic concepts to cutting-edge research.

Download or read book Comparative Quality Estimation for Machine Translation written by Eleftherios Avramidis and published by . This book was released on 2019 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Machine Translation written by Junhui Li and published by Springer Nature. This book was released on 2021-01-13 with total page 154 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book constitutes the refereed proceedings of the 16th China Conference on Machine Translation, CCMT 2020, held in Hohhot, China, in October 2020. The 13 papers presented in this volume were carefully reviewed and selected from 78 submissions and focus on all aspects of machine translation, including preprocessing, neural machine translation models, hybrid model, evaluation method, and post-editing.

Download or read book SHEF NN Translation Quality Estimation with Neural Networks written by Kashif Shah and published by . This book was released on 2015 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book New Directions in Empirical Translation Process Research written by Michael Carl and published by Springer. This book was released on 2015-08-13 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: This volume provides a comprehensive introduction to the Translation Process Research Database (TPR-DB), which was compiled by the Centre for Research and Innovation in Translation and Technologies (CRITT). The TPR-DB is a unique resource featuring more than 500 hours of recorded translation process data, augmented with over 200 different rich annotations. Twelve chapters describe the diverse research directions this data can support, including the computational, statistical and psycholinguistic modeling of human translation processes. In the first chapters of this book, the reader is introduced to the CRITT TPR-DB. This is followed by two main parts, the first of which focuses on usability issues and details of implementing interactive machine translation. It also discusses the use of external resources and translator-information interaction. The second part addresses the cognitive and statistical modeling of human translation processes, including co-activation at the lexical, syntactic and discourse levels, translation literality, and various annotation schemata for the data.

Download or read book Enhancing Deep Learning with Bayesian Inference written by Matt Benatan and published by Packt Publishing Ltd. This book was released on 2023-06-30 with total page 386 pages. Available in PDF, EPUB and Kindle. Book excerpt: Develop Bayesian Deep Learning models to help make your own applications more robust. Key Features Gain insights into the limitations of typical neural networks Acquire the skill to cultivate neural networks capable of estimating uncertainty Discover how to leverage uncertainty to develop more robust machine learning systems Book Description Deep learning has an increasingly significant impact on our lives, from suggesting content to playing a key role in mission- and safety-critical applications. As the influence of these algorithms grows, so does the concern for the safety and robustness of the systems which rely on them. Simply put, typical deep learning methods do not know when they don't know. The field of Bayesian Deep Learning contains a range of methods for approximate Bayesian inference with deep networks. These methods help to improve the robustness of deep learning systems as they tell us how confident they are in their predictions, allowing us to take more care in how we incorporate model predictions within our applications. Through this book, you will be introduced to the rapidly growing field of uncertainty-aware deep learning, developing an understanding of the importance of uncertainty estimation in robust machine learning systems. You will learn about a variety of popular Bayesian Deep Learning methods, and how to implement these through practical Python examples covering a range of application scenarios. By the end of the book, you will have a good understanding of Bayesian Deep Learning and its advantages, and you will be able to develop Bayesian Deep Learning models for safer, more robust deep learning systems. What you will learn Understand advantages and disadvantages of Bayesian inference and deep learning Understand the fundamentals of Bayesian Neural Networks Understand the differences between key BNN implementations/approximations Understand the advantages of probabilistic DNNs in production contexts How to implement a variety of BDL methods in Python code How to apply BDL methods to real-world problems Understand how to evaluate BDL methods and choose the best method for a given task Learn how to deal with unexpected data in real-world deep learning applications Who this book is for This book will cater to researchers and developers looking for ways to develop more robust deep learning models through probabilistic deep learning. You're expected to have a solid understanding of the fundamentals of machine learning and probability, along with prior experience working with machine learning and deep learning models.

Download or read book Exploration and Exploitation of Multilingual Data for Statistical Machine Translation written by and published by . This book was released on 2012 with total page 179 pages. Available in PDF, EPUB and Kindle. Book excerpt: "Shortly after the birth of computer science, researchers realised the importance of machine translation as a task worth of concentrated effort, but it is only recently that algorithms are able to provide automatic translations usable by the masses. Modern translation systems are dependent on bilingual corpora, a modern Rosetta Stone, from which the learn cross-lingual relationships that can be used to translate sentences which are not in the training corpus. This data is crucial. If it is insufficient, or out-of-domain, then translation quality degrades. To improve quality, we need to both perfect methods that extract usable translation from additional multilingual resources, and improve the constituent models of a translation system to better exploit existing multilingual data sets. In this thesis, we focus on these dual problems. Our approach is two-fold, and the thesis is structures accordingly. In part I we study the problem of extracting translations from the web, with a focus on exploiting the growing predominance of microblog platforms. We present novel methods for the language identification of microblog posts, and conduct a thorough analysis of existing methods that explore these microblog posts for new translations. In part II we study the orthogonal problem of improving language models for the tasks of reranking and source side morphological analysis. We begin by analysing a plethora of syntactic features for reranking n-best lists output from an automatic translation system. We then present a novel algorithm that allows for exact inference from high-order hidden Markov models, which we use to segment source text input. In this way, the thesis gives insight into the retrieval of relevant training data, and introduces novel methods that better utilise existing multilingual corpora."--Omslag.

Download or read book Conformal Prediction for Reliable Machine Learning written by Vineeth Balasubramanian and published by Newnes. This book was released on 2014-04-23 with total page 323 pages. Available in PDF, EPUB and Kindle. Book excerpt: The conformal predictions framework is a recent development in machine learning that can associate a reliable measure of confidence with a prediction in any real-world pattern recognition application, including risk-sensitive applications such as medical diagnosis, face recognition, and financial risk prediction. Conformal Predictions for Reliable Machine Learning: Theory, Adaptations and Applications captures the basic theory of the framework, demonstrates how to apply it to real-world problems, and presents several adaptations, including active learning, change detection, and anomaly detection. As practitioners and researchers around the world apply and adapt the framework, this edited volume brings together these bodies of work, providing a springboard for further research as well as a handbook for application in real-world problems. - Understand the theoretical foundations of this important framework that can provide a reliable measure of confidence with predictions in machine learning - Be able to apply this framework to real-world problems in different machine learning settings, including classification, regression, and clustering - Learn effective ways of adapting the framework to newer problem settings, such as active learning, model selection, or change detection

Download or read book The Role of Syntax and Semantics in Machine Translation and Quality Estimation of Machine translated User generated Content written by Rasoul Samad Zadeh Kaljahi and published by . This book was released on 2015 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: