journal article translation

• My Invoices
• Coupons/Discounts
• Referral Rewards
• English Language Editing
• Scientific Editing

Academic Translation

• Manuscript Formatting
• Research Data Support
• For Organizations
• Get a Price Quote
• Get Started

This Academic Translation service is brought to you by American Journal Experts (AJE). Get academic documents translated into English from Simplified Chinese, Portuguese, and Spanish. Upload your document for Academic Translation

Translation into polished English is available for documents in Simplified Chinese, Portuguese, and Spanish. You are matched with translators who are experts in your field of study and who will accurately translate field-specific terms. Once your document has been translated, it will be edited by AJE English Editing to ensure that the final document is polished and sounds natural in English.

Testimonials

I am very satisfied with AJE translation service. It is quick and efficient and return a high quality revision. Referee´s complains ended since I begun the revision of my papers with AJE.

Horacio Dottori, Brazil

American Journal Experts are very quick, have a very good price and do an excellent job. I have proved many companies of translation and this is one of the best. I think the journals reviewers will never say us more that the English of our manuscripts should be revised for a native!

David González-Cutre, Spain

Our Guarantee

All complete manuscripts submitted for the Academic Translation service are eligible to receive an AJE Editing Certificate.

Academic Translation includes unlimited free re-editing of your translated text and one additional free translation of up to 1,500 new words related to the original document. AJE guarantees that if you are not satisfied with the translation, or if a journal says that the English in your paper needs improvement, your document will be re-edited for free until English is no longer a barrier to publication.

Do you have translators in my field?

We have translators in every major field. If you don’t see your area of study on our list , please contact us, and we will let you know if any qualified translators are available.

How are AJE translation services different from traditional translators?

Unlike traditional translation companies, we use translators who are active researchers in your area of study. This ensures that they know the jargon of your field in both English and your native language, guaranteeing the most accurate translation possible. AJE also includes a quality review for every translation by one of our experienced Academic Translation Advisors. We believe this additional step is critical to providing the most accurate translation of your important research. Moreover, our translation service includes English language editing by fluent speakers in your field, so you can be assured of the quality of the language in your final paper. Finally, our translation service includes unlimited English re-editing, so if you need to make changes to your paper later, we will re-edit it for free.

How long will it take to have my paper re-edited?

Because a Translation re-edit may involve a variety of steps (e.g., Translation, Editing, and/or Formatting), we cannot provide a specific timetable that applies to all re-edits. Once you submit your paper for re-editing, we will determine what steps are needed and contact you with your actual deadline within 2 business days.

See All Frequently Asked Questions

• Nature China
• Nature Japan
• Nature Korea
• Nature India
• Nature Middle East
• Nature Research
• Palgrave Macmillan

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 01 September 2020

Transforming machine translation: a deep learning system reaches news translation quality comparable to human professionals

• Martin Popel   ORCID: orcid.org/0000-0002-3628-8419 1   na1 ,
• Marketa Tomkova   ORCID: orcid.org/0000-0001-9094-2365 2   na1 ,
• Jakub Tomek   ORCID: orcid.org/0000-0002-0157-4386 3   na1 ,
• Łukasz Kaiser   ORCID: orcid.org/0000-0003-1092-6010 4 ,
• Jakob Uszkoreit   ORCID: orcid.org/0000-0001-5066-7530 4 ,
• Ondřej Bojar   ORCID: orcid.org/0000-0002-0606-0050 1 &
• Zdeněk Žabokrtský   ORCID: orcid.org/0000-0001-8149-4054 1  

Nature Communications volume  11 , Article number:  4381 ( 2020 ) Cite this article

51k Accesses

103 Citations

173 Altmetric

Metrics details

• Communication
• Computer science

The quality of human translation was long thought to be unattainable for computer translation systems. In this study, we present a deep-learning system, CUBBITT, which challenges this view. In a context-aware blind evaluation by human judges, CUBBITT significantly outperformed professional-agency English-to-Czech news translation in preserving text meaning (translation adequacy). While human translation is still rated as more fluent, CUBBITT is shown to be substantially more fluent than previous state-of-the-art systems. Moreover, most participants of a Translation Turing test struggle to distinguish CUBBITT translations from human translations. This work approaches the quality of human translation and even surpasses it in adequacy in certain circumstances.This suggests that deep learning may have the potential to replace humans in applications where conservation of meaning is the primary aim.

Similar content being viewed by others

Highly accurate protein structure prediction with AlphaFold

John Jumper, Richard Evans, … Demis Hassabis

Natural language instructions induce compositional generalization in networks of neurons

Reidar Riveland & Alexandre Pouget

A visual-language foundation model for computational pathology

Ming Y. Lu, Bowen Chen, … Faisal Mahmood

Introduction

The idea of using computers for translation of natural languages is as old as computers themselves 1 . However, achieving major success remained elusive, in spite of the unwavering efforts of the machine translation (MT) research over the last 70 years. The main challenges faced by MT systems are correct resolution of the inherent ambiguity of language in the source text, and adequately expressing its intended meaning in the target language (translation adequacy) in a well-formed and fluent way (translation fluency). Among key complications is the rich morphology in the source and especially in the target language 2 . For these reasons, the level of human translation has been thought to be the upper bound of the achievable performance 3 . There are also other challenges in recent MT research such as gender bias 4 or unsupervised MT 5 , which are mostly orthogonal to the present work.

Deep learning transformed multiple fields in the recent years, ranging from computer vision 6 to artificial intelligence in games 7 . In line with these advances, the field of MT has shifted to the use of deep-learning neural-based methods 8 , 9 , 10 , 11 , which replaced previous approaches, such as rule-based systems 12 or statistical phrase-based methods 13 , 14 . Relying on the vast amounts of training data and unprecedented computing power, neural MT (NMT) models can now afford to access the complete information available anywhere in the source sentence and automatically learn which piece is useful at which stage of producing the output text. This removal of past independence assumptions is the key reason behind the dramatic improvement of translation quality. As a result, neural translation even managed to considerably narrow the gap to human-translation quality on isolated sentences 15 , 16 .

In this work, we present a neural-based translation system CUBBITT (Charles University Block-Backtranslation-Improved Transformer Translation), which significantly outperformed professional translators on isolated sentences in a prestigious competition WMT 2018, namely the English–Czech News Translation Task 17 . We perform a new study with conditions that are more representative and far more challenging for MT, showing that CUBBITT conveys meaning of news articles significantly better than human translators even when the cross-sentence context is taken into account. In addition, we validate the methodological improvements using an automatic metric on English↔French and English↔Polish news articles. Finally, we provide insights into the principles underlying CUBBITT’s key technological advancement and how it improves the translation quality.

Deep-learning framework transformer

Our CUBBITT system (Methods 1) follows the basic Transformer encoder-decoder architecture introduced by Vaswani et al. 18 . The encoder represents subwords 19 in the source-language sentence by a list of vectors, automatically extracting features describing relevant aspects and relationships in the sentence, creating a deep representation of the original sentence. Subsequently, the decoder converts the deep representation to a new sentence in the target language (Fig.  1a , Supplementary Fig.  1 ).

a The input sentence is converted to a numerical representation and encoded into a deep representation by a six-layer encoder, which is subsequently decoded by a six-layer decoder into the translation in the target language. Layers of the encoder and decoder consist of self-attention and feed-forward layers and the decoder also contains an encoder-decoder attention layer, with an input of the deep representation created by the last layer of encoder. b Visualization of encoder self-attention between the first two layers (one attention head shown, focusing on “magazine” and “her”). The strong attention link between ‘magazine’ and ‘gun’ suggests why CUBBITT ultimately correctly translates “magazine” as “zásobník” (gun magazine), rather than “časopis” (e.g., news magazine). The attention link between ‘woman’ and ‘her’ illustrates how the system internally learns coreference. c Encoder-decoder attention on the second layer of the decoder. Two heads are shown in different colors, each focusing on a different translation aspect which is described in italic. We note that the attention weights were learned spontaneously by the network, not inputted a priori.

A critical feature of the encoder and decoder is self-attention, which allows identification and representation of relationships between sentence elements. While the encoder attention captures the relationship between the elements in the input sentence (Fig.  1b ), the encoder-decoder attention learns the relationship between elements in the deep representation of the input sentence and elements in the translation (Fig.  1c ). In particular, our system utilizes the so-called multi-head attention, where several independent attention functions are trained at once, allowing representation of multiple linguistic phenomena. These functions may facilitate, for example, the translation of ambiguous words or coreference resolution.

Utilizing monolingual data via backtranslation

The success of NMT depends heavily on the quantity and quality of the training parallel sentences (i.e., pairs of sentences in the source and target language). Thanks to long-term efforts of researchers, large parallel corpora have been created for several language pairs, e.g., the Czech-English corpus CzEng 20 or the multi-lingual corpus Opus 21 . Although millions of parallel sentences became freely available in this way, this is still not sufficient. However, the parallel data can be complemented by monolingual target-language data, which are usually available in much larger amounts than the parallel data. CUBBITT leverages the monolingual data using a technique termed backtranslation, where the monolingual target-language data are machine translated to the source language, and the resulting sentence pairs are used as additional (synthetic) parallel training data 19 . Since the target side in backtranslation are authentic sentences originally written in the target language, backtranslation can improve fluency (and sometimes even adequacy) of the final translations by naturally learning the language model of the target language.

CUBBITT is trained with backtranslation data in a novel block regime (block-BT), where the training data are presented to the neural network in blocks of authentic parallel data alternated with blocks of synthetic data. We compared our block regime to backtranslation using the traditional mixed regime (mix-BT), where all synthetic and authentic sentences are mixed together in random order, and evaluated the learning curves using BLEU, an automatic measure, which compares the similarity of an MT output to human reference translations (Methods 2–13). While training with mix-BT led to a gradually increasing learning curve, block-BT showed further improved performance in the authentic training phases, alternated with reduced performance in the synthetic ones (Fig.  2a , thin lines). In the authentic training phases, block-BT was better than mix-BT, suggesting that a model extracted at the authentic-data phase might perform better than mix-BT trained model.

a The effect of averaging eight last checkpoints with block-BT and mix-BT on the translation quality as measured by BLEU on the development set WMT13 newstest. The callouts (pointing to the initial and final peaks of the block-BT + avg8 curve) illustrate the 8 averaged checkpoints (synth-trained ones as brown circles, auth-trained ones as violet circles). b Diagram of iterated backtranslation: the system MT1 trained only on authentic parallel data is used to translate monolingual Czech data into English, which are used to train system MT2; this step can be iterated one or more times to obtain MT3, MT4, etc. The block-BT + avg8 model shown in a is the MT2 model in (B) and in Supplementary Fig.  2 . c BLEU results on WMT17 test-set relative to the WMT17 winner UEdin2017. All five systems use checkpoint averaging.

CUBBITT combines block-BT with checkpoint averaging, where networks in the eight last checkpoints are merged together using arithmetic average, which is a very efficient approach to gain better stability, and by that improve the model performance 18 . Importantly, we observed that checkpoint averaging works in synergy with the block-BT. The BLEU improvement when using this combination is clearly higher than the sum of BLEU improvements by the two methods in separation (Fig.  2a ). The best performance was gained when averaging authentic-trained model and synthetic-trained models in the ratio of 6:2; interestingly, the same ratio turned out to be optimal across several occasions in training. This also points out an advantage of block-BT combined with checkpoint averaging: the method automatically finds the optimal ratio of the two types of synthetic/authentic-trained models, as it evaluates all the ratios during training (Fig.  2a ).

The final CUBBITT system was trained using iterated block-BT (Fig.  2b , Supplementary Fig.  2 ). This was accompanied by other steps, such as data filtering, translationese tuning, and simple regex postprocessing (Methods 11). Evaluating the individual components of CUBBITT automatically on a previously unseen test-set from WMT17 showed a significant improvement in BLEU over UEdin2017, the state-of-the-art system from 2017 (Fig.  2c ).

Evaluation: CUBBITT versus a professional agency translation

In 2018, CUBBITT won the English→Czech and Czech→English news translation task in WMT18 17 , surpassing not only its machine competitors, but it was also the only MT system, which significantly outperformed the reference human translation by a professional agency in WMT18 English→Czech news translation task (other language pairs were not evaluated in such a way to allow comparison with the human reference) (Fig.  3a ). Since this result is highly surprising, we decided to investigate it in greater detail, evaluating potential confounding factors and focusing at how it can be explained and interpreted. We first confirmed that the results are not due to the original language of the reference sentences being English in half of the evaluated sentences and Czech in the other half of the test dataset (Supplementary Fig.  4 ; Methods 13), which was proposed to be a potential confounding factor by the WMT organizers 17 and others 22 , 23 .

a Results from context-unaware evaluation in WMT18, showing distributions of source-based direct assessment (SrcDA) of five MT systems and human reference translation, sorted by average score. CUBBITT was submitted under the name CUNI-Transformer. Online G, A, and B are three anonymized online MT systems. b Translations by CUBBITT and human reference were scored by six non-professionals in the terms of adequacy, fluency and overall quality in a context-aware evaluation. The evaluation was blind, i.e., no information was provided on whether the translations are human or machine translated. The scores (0–10) are shown as violin plots with boxplots (median + interquartile range), while the boxes below represent the percentage of sentences scored better in reference (orange), CUBBITT (blue), or the same (gray); the star symbol marks the ratio of orange vs. blue, ignoring gray. Sign test was used to evaluate difference between human and machine translation. c As in a , but evaluation by six professional translators. *** P  < 0.001; ** P  < 0.01; * P  < 0.05.

An important drawback in the WMT18 evaluation was the lack of cross-sentence context, as sentences were evaluated in random order and without document context. While the participating MT systems translated individual sentences independently, the human reference was created as a translation of the entire documents (news articles). The absence of cross-sentence context in the evaluation was recently shown to cause an overestimation of the quality of MT translations compared to human reference 22 , 23 . For example, evaluators will miss MT errors that would be evident only from the cross-sentence context, such as gender mismatch or incorrect translation of an ambiguous expression. On the other hand, independent evaluation of sentences translated considering cross-sentence context might unfairly penalize reference translations for moving pieces of information across sentences boundaries, as this will appear as an omission of meaning in one sentence and an addition in another.

We therefore conducted a new evaluation, using the same English→Czech test dataset of source documents, CUBBITT translations, and human reference translations, but presenting the evaluators with not only the evaluated sentences but also the document context (Methods 14–18; Supplementary Fig.  5 ). In order to gain further insight into the results, we asked the evaluators to assess the translations in terms of adequacy (the degree to which the meaning of the source sentence is preserved in the translation), fluency (how fluent the sentence sounds in the target language), as well as the overall quality of the translations. Inspired by a recent discussion of the translation proficiency of evaluators 22 , we recruited two groups of evaluators: six professional translators (native in the target language) and seven non-professionals (with excellent command of the source language and native in the target language). An additional exploratory group of three translation theoreticians was also recruited. In total, 15 out of the 16 evaluators passed a quality control check, giving 7824 sentence-level scores on 53 documents in total. See Methods 13–18 for further technical details of the study.

Focusing first at evaluations by non-professionals as in WMT18, but in our context-aware assessment, CUBBITT was evaluated to be significantly better than the human reference in adequacy ( P  = 4.6e-8, sign test) with 52% of sentences scored better and only 26% of sentences scored worse (Fig.  3b ). On the other hand, the evaluators found human reference to be more fluent ( P  = 2.1e-6, sign test), evaluating CUBBITT better in 26% and worse in 48% (Fig.  3b ). In the overall quality, CUBBITT nonsignificantly outperformed human reference ( P  = 0.6, sign test, 41% better than reference, 38% worse; Fig.  3b ).

In the evaluation by professional translators, CUBBITT remained significantly better in adequacy than human reference ( P  = 7.1e-4, sign test, 49% better, 33% worse; Fig.  3c ), albeit it scored worse in both fluency ( P  = 3.3e-19, sign test, 23% better, 64% worse) and overall quality ( P  = 3.0e-7, sign test, 32% better, 56% worse; Fig.  3c ). Fitting a linear model of weighting adequacy and fluency in the overall quality suggests that professional translators value fluency more than non-professionals; this pattern was also observed in the exploratory group of translation theoreticians (Supplementary Fig.  6 ). Finally, when scores from all 15 evaluators were pooled together, the previous results were confirmed: CUBBITT outperformed the human reference in adequacy, whereas the reference was scored better in fluency and overall quality (Supplementary Fig.  7 ). Surprisingly, we observed a weak, but significant effect of sentence length, showing that CUBBITT’s performance is more favorable compared to human in longer sentences with regards to adequacy, fluency, and overall quality (Supplementary Fig.  8 , including an example of a well-translated complex sentence).

We next decided to perform additional evaluation that would allow us to better understand where and why our machine translations are better or worse than the human translations. We asked three professional translators and three non-professionals to add annotations of types of errors in the two translations (Methods 19). In addition, the evaluators were asked to indicate whether the translation was wrong because of cross-sentence context.

CUBBITT made significantly fewer errors in addition of meaning, omission of meaning, shift of meaning, other adequacy errors, grammar, and spelling (Fig.  4a , example in Fig.  5a–c , Supplementary Data  1 ). On the other hand, reference performed better in error classes other fluency errors and ambiguous words (Fig.  4a , Supplementary Fig.  9 , examples in Fig.  5d, e , Supplementary Data  1 ). As expected, CUBBITT made significantly more errors due to cross-sentence context (11.7% compared to 5.2% in reference, P  = 1.2e-10, sign test, Fig.  4a ), confirming the importance of context-aware evaluation of translation quality. Interestingly, when only sentences without context errors are taken into account, not only adequacy, but also the overall quality is significantly better in CUBBITT compared to reference in ratings by non-professionals ( P  = 0.001, sign test, 49% better, 29% worse; Supplementary Fig.  10 ), in line with the context-unaware evaluation in WMT18.

a Percentages of sentences with various types of errors are shown for translations by human reference and CUBBITT. Errors in 405 sentences were evaluated by six evaluators (three professional translators and three non-professionals). Sign test was used to evaluate difference between human and machine translation. b Translations by five machine translation systems were scored by five professional translators in the terms of adequacy and fluency in a blind context-aware evaluation. The systems are sorted according to the mean performance, and the scores (0–10) for individual systems are shown as violin plots with boxplots (median + interquartile range). For each pair of neighboring systems, the box in between them represents the percentage of sentences scored as better in one, the other, or the same in both (gray). The star symbol marks the ratio when ties are ignored. Sign test was used to evaluate difference between the pairs of MT systems. *** P  < 0.001; ** P  < 0.01; * P  < 0.05.

The Czech translations by the human reference and CUBBITT, as well as the values of the manual evaluation for the individual sentences, are shown in Supplementary Data  1 .

We observed that the type of document, e.g., business vs. sports articles, can also affect the quality of machine translation when compared to human translation (Methods 18). The number of evaluated documents (53) does not allow for any strong and significant conclusions at the level of whole documents, but the document-level evaluations nevertheless suggest that CUBBITT performs best in news articles about business and politics (Supplementary Fig.  11A-B ). Conversely, it performed worst in entertainment/art (both in adequacy and fluency) and in news articles about sport (in fluency). Similar results can be observed also in sentence-level evaluations across document types (Supplementary Fig.  11C–D ).

The fact that translation adequacy is the main strength of CUBBITT is surprising, as NMT was shown to improve primarily fluency over the previous approaches 24 . We were therefore interested in comparison of fluency of translations made by CUBBITT and previous state-of-the-art MT systems (Methods 20). We performed an evaluation of CUBBITT in a side-by-side direct comparison with Google Translate 15 (an established benchmark for MT) and UEdin 25 (the winning system in WMT2017 and a runner-up in WMT 2018). Moreover, we included a version of basic Transformer with one iteration of mix-BT, and another version of basic Transformer with block-BT (but without iterated block-BT), providing human rating of different approaches to backtranslation. The evaluators were asked to evaluate adequacy and fluency of the five presented translations (again in a blind setting and taking cross-sentence context into account).

In the context-aware evaluation of the five MT systems, CUBBITT significantly outperformed Google Translate and UEdin both in adequacy (mean increase by 2.4 and 1.2 points, respectively) and fluency (mean increase by 2.1 and 1.2 points, respectively) (Fig.  4b ). The evaluation also shows that this increase of performance stems from inclusion of several components of CUBBITT: the Transformer model and basic (mix-BT) backtranslation, replacement of mix-BT with block-BT (adequacy: mean increase by 0.4, P  = 3.9e-5; fluency: mean increase by 0.3, P  = 1.4e-4, sign test), and to a lesser extent also other features in the final CUBBITT system, such as iterated backtranslation or data filtering (adequacy: mean increase by 0.2, P  = 0.054; fluency: mean increase by 0.1, P  = 0.233, sign test).

Finally, we were interested to see whether CUBBITT translations are distinguishable from human translations. We therefore conducted a sentence-level Translation Turing test, in which participants were asked to judge whether a translation of a sentence was performed by a machine or a human on 100 independent sentences (the source sentence and a single translation was shown; Methods 21). A group of 16 participants were given machine translations by Google Translate system mixed in a 1:1 ratio with reference translations. In this group, only one participant (with accuracy of 61%) failed to significantly distinguish between machine and human translations, while the other 15 participants recognized human translations in the test (with accuracy reaching as high as 88%; Fig.  6 ). In a group of different 15 participants, who were presented machine translations by CUBBITT mixed (again in the 1:1 ratio) with reference translations, nine participants did not reach the significance threshold of the test (with the lowest accuracy being 56%; Fig.  6 ). Interestingly, CUBBITT was not significantly distinguished from human translations by three professional translators, three MT researchers, and three other participants. One potential contributor to human-likeness of CUBBITT could be the fact that it is capable of restructuring translated sentences where the English structure would sound unnatural in Czech (see an example in Fig.  5f , Supplementary Data  1 ).

a Accuracy of individual participants in distinguishing machine from human translations is shown in a bar graph. Fisher test was used to assess whether the participant significantly distinguished human and machine translations and Benjamini–Hochberg method was used to correct for multiple testing. Participants with a Q value below 0.05 were considered to have significantly distinguished between human and machine translations. b Percentage of participants, who significantly distinguished human and machine translations for CUBBITT (top, blue) and for Google Translate (bottom, green).

Generality of block backtranslation

Block-BT with checkpoint averaging clearly improves English→Czech news translation quality. To demonstrate that the benefits of our approach are not limited to this language pair, we trained English→French, French→English, English→Polish, and Polish→English versions of CUBBITT (Methods 4, 5, 12) and evaluated them using BLEU as in Fig.  2a . The results are consistent with the behavior on the English→Czech language pair, showing a synergistic benefit of block-BT with checkpoint averaging (Fig.  2a , Supplementary Figs.  3 , 14 ).

How block backtranslation improves translation

Subsequently, we sought to investigate the synergy between block-BT and checkpoint averaging, trying to get an insight into the mechanism of how this improves translation on the English→Czech language pair. We first tested a simple hypothesis that the only benefit of block regime and checkpoint averaging is an automatic detection of the optimal ratio of authentic and synthetic data, given that in block-BT the averaging window explores various ratios of networks trained on authentic and synthetic data. Throughout our experiments, the optimal ratio of authentic and synthetic blocks was ca. 3:1, so we hypothesized that mixed-BT would benefit from authentic and synthetic data mixed in the same ratio. However, this hypothesis was not supported by additional explorations (Supplementary Fig.  15 ), which suggests that a more profound mechanism underlies the synergy.

We next hypothesized that training the system in the block regime compared to the mix regime might aid the network to better focus at the two types of blocks (authentic and synthetic), one at a time. This would allow the networks to more thoroughly learn the properties and benefits of the two blocks, leading to a better exploration of space of networks, ultimately yielding greater translation diversity during training. We measured translation diversity of a single sentence as the number of all unique translations produced by the MT system at hourly checkpoints during training. Comparing translation diversity between block-BT and mix-BT on the WMT13 newstest, we observed block-BT to have greater translation diversity in 78% sentences, smaller in 18% sentences, and equal in the remaining 4% sentences (Methods 22–23), supporting the hypothesis of greater translation diversity of block-BT compared to mix-BT.

The increased diversity could be leveraged by checkpoint averaging by multiple means. In theory, this can be as simple as selecting the most frequent sentence translation among the eight averaged checkpoints. At the same time, checkpoint averaging can generate sentences that were not present as the preferred translation in any of the eight averaged checkpoints (termed novel Avg8 translation), potentially combining the checkpoints’ best translation properties. This may involve producing a combination of phrase translations seen in the averaged checkpoints (Fig.  7a , Supplementary Fig.  17 ), or creation of a sentence with phrases not seen in any of the averaged checkpoints (Fig.  7b ). The fact that even phrase translations with low frequency in the eight averaged checkpoints can be chosen by checkpoint averaging stems from the way the confidence of the networks in their translations is taken into account (Supplementary Fig.  18 ).

a A case where the translation resulting from checkpoint averaging is a crossover of translations present in AUTH and SYNTH blocks. All the mentioned translations are shown in Supplementary Fig.  17 . b A case where the translation resulting from checkpoint averaging contains a phrase that is not the preferred translation in any of the averaged checkpoints.

Comparing the translations produced by models with and without averaging, we observed that averaging generated at least one translation never seen without averaging (termed novel Avg∞ ) in 60% sentences in block-BT and in 31.6% sentences in mix-BT (Methods 23). Moreover, averaging generated more novel Avg∞ translations in block-BT than mix-BT in 55% sentences, fewer in only 6%, and equal in 39%.

We next sought to explore what is the mechanism of the greater translation diversity and more novel Avg translations in block-BT compared to mix-BT. We therefore computed how translation diversity and novel Avg8 translations develop over time during training and what is their temporal relationship to blocks of authentic and synthetic data (Methods 24). In order to be able to track these features over time, we computed diversity and novel Avg8 using the last eight checkpoints (the width of the averaging window) for each checkpoint during training. While mix-BT gradually and smoothly decreased in both metrics over time, block-BT showed a striking difference between the alternating blocks of authentic and synthetic data (Fig.  8a , Supplementary Fig.  16 ). The novel Avg8 translations in block-BT were most frequent in checkpoints where the eight averaged checkpoints contained both the authentic- and synthetic-trained blocks (Fig.  8a ). Interestingly, also the translation diversity of the octuples of checkpoints in block-BT (without averaging) was highest at the borders of the blocks (Supplementary Fig.  16 ). This suggests that it is the alternation of the blocks that increases the diversity of the translations and generation of novel translations by averaging in block-BT.

a Percentage of WMT13 newstest sentences with novel Avg8 translation (not seen in the previous eight checkpoints without averaging) over time, shown separately for block-BT (red) and mix-BT (blue). The checkpoints trained in AUTH blocks are denoted by magenta background and letter A, while the SYNTH blocks are shown in yellow background and letter S. b Evaluation of translation quality by BLEU on WMT13 newstest set for four different versions of block-BT (left) and mix-BT (right), exploring the importance of novel Avg8 sentences created by checkpoint averaging. The general approach is to take the best system using checkpoint averaging (Avg), and substitute translations of novel Avg8 and not-novel Avg8 sentences with translations produced by the best system without checkpoint averaging (noAvg), observing the effect on BLEU. In blue is the BLEU achieved by the model with checkpoint averaging, while in purple is the BLEU achieved by the model without checkpoint averaging. In red is the BLEU of a system, which used checkpoint averaging, but where the translations that are not novel Avg8 were replaced by the translations produced by the system without checkpoint averaging. Conversely, yellow bars show BLEU of a system, which uses checkpoint averaging, but where the novel Avg8 translations were replaced by the version without checkpoint averaging.

Finally, we tested whether the generation of novel translations by averaging contributes to the synergy between block regime and checkpoint averaging as measured by BLEU (Methods 25). We took the best model in block-BT with checkpoint averaging (block-BT-Avg; BLEU 28.24) and in block-BT without averaging (block-BT-NoAvg; BLEU 27.54). We next identified 988 sentences where the averaging in block-BT-Avg generated a novel Avg8 translation, unseen in the eight previous checkpoints without averaging. As we wanted to know what role do the novel Avg8 sentences play in the improved BLEU of block-BT-Avg compared to block-BT-NoAvg (Fig.  2a ), we next computed BLEU of block-BT-Avg translations, where the translations of 988 novel Avg8 sentences were replaced with the block-BT-NoAvg translations. Such replacement led to decrease of BLEU almost to the level of block-BT-NoAvg (BLEU 27.65, Fig.  8b ). Conversely, replacement of the 2012 not-novel Avg8 sentences resulted in only a small decrease (BLEU 28.13, Fig.  8b ), supporting the importance of novel translations in the success of block-BT with checkpoint averaging. For a comparison, we repeated the same analysis with mix-BT and observed that replacement of novel Avg8 sentences in mix-BT showed a negligible effect on the improvement of mix-BT-Avg over mix-BT-NoAvg (Fig.  8b ).

Altogether, our analysis shows that generation of novel sentences is an important mechanism of how checkpoint averaging combined with block-BT lead to synergistically improved performance. Specifically, averaging at the interface between authentic and synthetic blocks leads to the highest diversity and generation of novel translations, allowing combining the best features of the diverse translations in the two block types (examples in Fig.  7 , Supplementary Fig.  17 ).

In this work, we have shown that the deep-learning framework CUBBITT outperforms a professional human-translation agency in adequacy of English→Czech news translations. In particular, this is achieved by making fewer errors in adding, omitting, or shifting meaning of the translated sentences. At the same time, CUBBITT considerably narrowed the gap in translation fluency to human, markedly outperforming previous state-of-the-art translation systems. The fact that the main advantage of CUBBITT is improved adequacy could be viewed as surprising, as it was thought that the main strength of NMT was increased fluency 24 . However, our results are in line with the study of Läubli et al. 23 , who observed the deficit of NMT to human to be smaller in adequacy than in fluency. The improvement in translation quality is corroborated by a Translation Turing test, where most participants failed to reliably discern CUBBITT translations from human.

Critically, our evaluation of translation quality was carried out in a fully context-aware evaluation setting. As discussed in this work and in other recent articles on this topic 22 , 23 , the previous standard approach of combining context-aware reference translation with context-free assessment gives an unfair advantage to machine translation. Consequently, this study is also an important contribution to MT evaluation practices and points out that the relevance of future evaluations in MT competitions such as WMT will be increased when cross-sentence context is included. In addition, our design where fluency and adequacy are assessed separately, and by professional translators and non-professionals, brings interesting insight into evaluator priorities. The professional translators were observed to be more sensitive to errors in fluency than non-professionals and to have a stronger preference for fluency when rating overall quality of a translation. Such difference in preference is an important factor in designing studies, which measure solely the overall translation quality. While in domains such as artistic writing, fluency is clearly of utmost importance, there are domains (e.g., factual news articles), where an improvement in preservation of meaning may be more important to a reader than a certain loss of fluency. Our robust context-aware evaluation with above-human performance in adequacy demonstrates that human translation is not necessarily an upper bound of translation quality, which was a long-standing dogma in the field.

Among key methodological advances of CUBBITT is the training regime termed block backtranslation, where blocks of authentic data alternate with blocks of synthetic data. Compared to traditional mixed backtranslation, where all the data are shuffled together, block regime offers markedly increased diversity of translations produced during training, suggesting a more explorative search for solutions to the translation problem. This increased diversity can be then greatly leveraged by the technique of checkpoint averaging, which is capable of finding consensus between networks trained on purely synthetic data and networks trained on authentic data, often combining the best of the two worlds. We speculate that such block-training regime of training may be beneficial also for other ways of data organization into blocks and may in theory be applicable beyond backtranslation, or even beyond the field of machine translation.

During reviews of this manuscript, the WMT19 competition took place 26 . The testing dataset was different, and evaluation methodology was innovated compared to WMT18, which is why the results are not directly comparable (e.g., the translation company was explicitly instructed to not add/remove information from the translated sentences, which was a major source of adequacy errors in this study (Fig.  4a )). Also based on discussions with our team’s members, the organizers of WMT19 implemented a context-aware evaluation. In this context-aware evaluation of English→Czech news task, CUBBITT was the winning MT system and reached overall quality score 95.3% of human translators (DA score 86.9 vs 91.2), which is similar to our study (94.8%, mean overall quality 7.4 vs 7.8, all annotators together). Given that WMT19 did not separate overall quality into adequacy and fluency, it is not possible to validate the potential super-human adequacy on their dataset.

Our study was performed on English→Czech news articles and we have also validated the methodological improvements of CUBBITT using automatic metric on English↔French and English↔Polish news articles. The generality of CUBBITT’s success with regards to other language pairs and domains remains to be evaluated. However, the recent results from WMT19 on English→German show that indeed also in other languages the human reference is not necessarily the upper bound of translation quality 26 .

The performance of machine translation is getting so close to human reference that the quality of the reference translation matters. Highly qualified human translators with infinite amount of time and resources will likely produce better translations than any MT system. However, many clients cannot afford the costs of such translators and instead use services of professional translation agencies, where the translators are under certain time pressure. Our results show that the quality of professional-agency translations is not unreachable by MT, at least in certain aspects, domains, and languages. Nevertheless, we suggest that in the future MT competitions and evaluations, it may be important to sample multiple human references (from multiple agencies and ideally also prices).

We stress out that CUBBITT is the result of years of open scientific collaboration and is a culmination of the transformation of the field. It started with the MT competitions that provided open data and ideas and continued through the open community of deep learning, which provided open-source code. The Transformer architecture significantly lowered the hardware requirements for training MT models (from months on multi-GPU clusters to days on a single machine 18 ). More effective utilization of monolingual data via iterated block backtranslation with checkpoint averaging presented in this study allows generating large amount of high-quality synthetic parallel data to complement existing parallel datasets at little cost. Together, these techniques allow CUBBITT to be trained by the broad community and to considerably extend the reach of MT.

1 CUBBITT model

Our CUBBITT translation system follows the Transformer architecture (Fig.  1 , Supplementary Fig.  1 ) introduced in Vaswani et al. 18 . Transformer has an encoder-decoder structure where the encoder maps an input sequence of tokens (words or subword units) to a sequence of continuous deep representations z . Given z , the decoder generates an output sequence of tokens one element at a time. The decoder is autoregressive, i.e., consuming the previously generated symbols as additional input when generating the next token.

The encoder is composed of a stack of identical layers, with each layer having two sublayers. The first is a multi-head self-attention mechanism, and the second is a simple, position-wise fully connected feed-forward network. We employ a residual connection around each of the two sublayers, followed by layer normalization. The decoder is also composed of a stack of identical layers. In addition to the two sublayers from the encoder, the decoder inserts a third sublayer, which performs multi-head attention over the output of the encoder stack. Similar to the encoder, we employ residual connections around each of the sublayers, followed by layer normalization.

The self-attention layer in the encoder and decoder performs multi-head dot-product attention, each head mapping matrices of queries ( Q ), keys ( K ), and values ( V ) to an output vector, which is a weighted sum of the values V :

where Q ∈ \({\Bbb R}^{n \times d_k}\) , K ∈ \({\Bbb R}^{n \times d_k}\) , V ∈ \({\Bbb R}^{n \times d_v}\) , n is the sentence length, d v is the dimension of values, and d k is the dimension of the queries and keys. Attention weights are computed as a compatibility of the corresponding key and query and represent the relationship between deep representations of subwords in the input sentence (for encoder self-attention), output sentence (for decoder self-attention), or between the input and output sentence (for encoder-decoder attention). In encoder and decoder self-attention, all queries, keys and values come from the output of the previous layer, whereas is the encoder-decoder attention, keys and values come from the encoder’s topmost layer and queries come from the decoder’s previous layer. In the decoder, we modify the self-attention to prevent it from attending to following positions (i.e., rightward from the current position) by adding a mask, because the following positions will not be known in inference time.

2 English–Czech training data

Our training data are constrained to the data allowed in the WMT 2018 News translation shared task 17 ( www.statmt.org/wmt18/translation-task.html ). Parallel (authentic) data are: CzEng 1.7, Europarl v7, News Commentary v11 and CommonCrawl. Monolingual data for backtranslation are: English (EN) and Czech (CS) NewsCrawl articles. Data sizes (after filtering, see below) are reported in Supplementary Table  1 .

While all our monolingual data are news articles, only less than 1% of our parallel data are news (summing News Commentary v12 and the news portion of CzEng 1.7). The biggest sources of our parallel data are: movie subtitles (63% of sentences), EU legislation (16% of sentences), and Fiction (9% of sentences) 27 . Unfortunately, no finer-grained metadata specifying the exact training-data domains (such as politics, business, and sport) are available.

We filtered out ca. 3% of sentences in the monolingual data by restricting the length to 500 characters and in case of Czech NewsCrawl also by keeping only sentences containing at least one accented character (using a regular expression m/[ěščřžýáíéúůd’t’ň]/i). This simple heuristic is surprisingly effective for Czech; it filters out not only sentences in other languages than Czech, but also various non-linguistic content, such as lists of football or stock-market results.

We divided the Czech NewsCrawl (synthetic data) into two parts: years 2007–2016 (58,231 k sentences) and year 2017 (7152 k sentences). When training block-BT, we simply concatenated four blocks of training data: authentic, synthetic 2007–2016, authentic and synthetic 2017. The sentences within these four blocks were randomly shuffled; we only do not shuffle across the data blocks. When training mix-BT, we used exactly the same training sentences, but we shuffled them fully. This means we upsampled the authentic training data two times. The actual ratio of authentic and synthetic data (as measured by the number of subword tokens) in the mix-BT training data was approximately 1.2:1.

3 English–Czech development and test data

WMT shared task on news translation provides a new test-set (with ~3000 sentences) each year collected from recent news articles (WMT = Workshop on statistical Machine Translation. In 2016, WMT was renamed to Conference on Machine Translation, but keeping the legacy abbreviation WMT. For more information see the WMT 2018 website http://www.statmt.org/wmt18 .). The reference translations are created by professional translation agencies. All of the translations are done directly, and not via an intermediate language. Test sets from previous years are allowed to be used as development data in WMT shared tasks.

We used WMT13 (short name for WMT newstest2013) as the primary development set in our experiments (e.g., Figure  2a ). We used WMT17 as a test-set for measuring BLEU scores in Fig.  2c . We used WMT18 (more precisely, its subset WMT18-orig-en, see below) as our final manual-evaluation test-set. Data sizes are reported in Supplementary Table  2 .

In WMT test sets since 2014, half of the sentences for a language pair X-EN originate from English news servers (e.g., bbc.com) and the other half from X-language news servers. All WMT test sets include the server name for each document in metadata, so we were able to split our dev and test sets into two parts: originally Czech (orig-cs, for Czech-domain articles, i.e., documents with docid containing “.cz”) and originally English (orig-en, for non-Czech-domain articles. The WMT13-orig-en part of our WMT13 development set contains not only originally English articles, but also articles written originally in French, Spanish, German and Russian. However, the Czech reference translations were translated from English. In WMT18-orig-en, all the articles were originally written in English.).

According to Bojar et al. 17 , the Czech references in WMT18 were translated from English “by the professional level of service of Translated.net, preserving 1–1 segment translation and aiming for literal translation where possible. Each language combination included two different translators: the first translator took care of the translation, the second translator was asked to evaluate a representative part of the work to give a score to the first translator. All translators translate towards their mother tongue only and need to provide a proof or their education or professional experience, or to take a test; they are continuously evaluated to understand how they perform on the long term. The domain knowledge of the translators is ensured by matching translators and the documents using T-Rank, http://www.translated.net/en/T-Rank .”

Toral et al. 22 furthermore warned about post-edited MT used as human references. However, Translated.net confirmed that MT was completely deactivated during the process of creating WMT18 reference translations (personal communication).

4 English–French data

The English–French parallel training data were downloaded from WMT2014 ( http://statmt.org/wmt14/translation-task.html ). The monolingual data were downloaded from WMT 2018 (making sure there is no overlap with the development and test data). We filtered the data for being English/French using the langid toolkit ( http://pypi.org/project/langid/ ). Data sizes after filtering are reported in Supplementary Table  3 . When training English–French block-BT, we concatenated the French NewsCrawl2008–2014 (synthetic data) and authentic data, with no upsampling. When training French–English block-BT, we split the English NewsCrawl into three parts: 2011–2013, 2014–2015, and 2016–2017 and interleaved with three copies of the authentic training data, i.e., upsampling the authentic data three times. We always trained mix-BT on a fully shuffled version of the data used for the respective block-BT training.

Development and test data are reported in Supplementary Table  4 .

5 English–Polish data

The English–Polish training and development data were downloaded from WMT2020 ( http://statmt.org/wmt20/translation-task.html ). We filtered the data for being English/Polish using the FastText toolkit ( http://pypi.org/project/fasttext/ ). Data sizes after filtering are reported in Supplementary Table  5 . When training English–Polish block-BT, we upsampled the authentic data two times and concatenated with the Polish NewsCrawl2008–2019 (synthetic data) upsampled six times. When training Polish–English block-BT, we upsampled the authentic data two times and concatenated with English NewsCrawl2018 (synthetic data, with no upsampling). We always trained mix-BT on a fully shuffled version of the data used for the respective block-BT training.

Development and test data are reported in Supplementary Table  6 .

6 CUBBITT training: BLEU score

BLEU 28 is a popular automatic measure for MT evaluation and we use it for hyperparameter tuning. Similarly to most other automatic MT measures, BLEU estimates the similarity between the system translation and the reference translation. BLEU is based on n-gram (unigrams up to 4-grams) precision of the system translation relative to the reference translation and a brevity penalty to penalize too short translations. We report BLEU scaled to 0–100 as is usual in most papers (although BLEU was originally defined as 0–1 by Papineni et al. 28 ); the higher BLEU value, the better translation. We use the SacreBLEU implementation 29 with signature BLEU+case.mixed+lang.en-cs+numrefs.1+smooth.exp+tok.13a.

7 CUBBITT training: hyperparameters

We use the Transformer “big” model from the Tensor2Tensor framework v1.6.0 18 . We followed the training setup and tips of Popel and Bojar 30 and Popel et al. 31 , training our models with the Adafactor optimizer 32 instead of the default Adam optimizer. We use the following hyperparameters: learning_rate_schedule = rsqrt_decay, batch_size = 2900, learning_rate_warmup_steps = 8000, max_length = 150, layer_prepostprocess_dropout = 0, optimizer = Adafactor. For decoding, we use alpha = 1.0, beam_size = 4.

8 CUBBITT training: checkpoint averaging

A popular way of improving the translation quality in NMT is ensembling, where several independent models are trained and during inference (decoding, translation) each target token (word) is chosen according to an averaged probability distribution (using argmax in the case of greedy decoding) and used for further decisions in the autoregressive decoder of each model.

However, ensembling is expensive both in training and inference time. The training time can be decreased by using checkpoint ensembles 33 , where N last checkpoints of a single training run are used instead of N independently trained models. Checkpoint ensembles are usually worse than independent ensembles 33 , but allow to use more models in the ensemble thanks to shorter training time. The inference time can be decreased by using checkpoint averaging, where the weights (learned parameters of the network) in the N last checkpoints are element-wise averaged, creating a single averaged model.

Checkpoint averaging has been first used in NMT by Junczys-Dowmunt et al. 34 , who report that averaging four checkpoints is “not much worse than the actual ensemble” of the same four checkpoints and it is better than ensembles of two checkpoints. Averaging ten checkpoints “even slightly outperforms the real four-model ensemble”.

Checkpoint averaging has been popular in recent NMT systems because it has almost no additional cost (averaging takes only several minutes), the results of averaged models have lower variance in BLEU and are usually at least slightly better than models without averaging 30 .

In our experiments, we store checkpoints each hour and average the last 8 checkpoints.

9 CUBBITT training: Iterated backtranslation

For our initial experiments with backtranslation, we reused an existing CS → EN system UEdin (Nematus software trained by a team from the University of Edinburgh and submitted to WMT 2016 35 ). This system itself was trained using backtranslation. We decided to iterate the backtranslation process further by using our EN → CS Transformer to translate English monolingual data and use that for training a higher quality CS → EN Transformer, which was in turn used for translating Czech monolingual data and training our final EN → CS Transformer system called CUBBITT. Supplementary Fig.  2 illustrates this process and provides details about the training data and backtranslation variants (mix-BT in MT1 and block-BT in MT2–4) used.

Each training we did (MT3–5 in Supplementary Fig. 2) took ca. eight days on a single machine with eight GTX 1080 Ti GPUs. Translating the monolingual data with UEdin2016 (MT0) took ca. two weeks and with our Transformer models (MT1–3) it took ca. 5 days.

10 CUBBITT training: translationese tuning

It has been observed that text translated from language X into Y has different properties (such as lexical choice or syntactic structure) compared to text originally written in language Y 36 . Term translationese is used in translation studies (translatology) for this phenomenon (and sometimes also for the translated language itself).

We noticed that when training on synthetic data, the model performs much better on the WMT13-orig-cs dev set than on the WMT13-orig-en dev set. When trained on authentic data, it is the other way round. Intuitively, this makes sense: The target side of our synthetic data are original Czech sentences from Czech newspapers, similarly to the WMT13-orig-cs dataset. In our authentic parallel data, over 90% of sentences were originally written in English about non-Czech topics and translated into Czech (by human translators), similarly to the WMT13-orig-en dataset. There are two closely related phenomena: a question of domain (topics) in the training data and a question of so-called translationese effect, i.e., which side of the parallel training data (and test data) is the original and which is the translation.

Based on these observations, we prepared an orig-cs-tuned model and an orig-en-tuned model. Both models were trained in the same way; they differ only in the number of training steps. For the orig-cs-tuned model, we selected a checkpoint with the best performance on WMT13-orig-cs (Czech-origin portion of WMT newstest2013), which was at 774k steps. Similarly, for the orig-en-tuned model, we selected the checkpoint with the best performance on WMT13-orig-en, which was at 788k steps. Note that both the models were trained jointly in one experiment, just selecting checkpoints at two different moments. The WMT18-orig-en test-set was translated using the orig-en-tuned model and the WMT18-orig-cs part was translated using the orig-cs-tuned model.

11 CUBBITT training: regex postediting

We applied two simple post-processings to the translations, using regular expressions. First, we converted quotation symbols in the translations to the correct-Czech lower and upper quotes („ and “) using two regexes: s/(ˆ|[({[])(“|,,|”|“)/$1„/g and s/(“|”)($|[,.?!:;)}\]])/“$2/g. Second, we deleted phrases repeated more than twice (immediately following each other); we kept just the first occurrence. We considered phrases of one up to four words. This postprocessing affected less than 1% sentences in the dev set.

12 CUBBITT training: English–French and English–Polish

We trained English→French, French→English, English→Polish and Polish→English versions of CUBBITT, following the abovementioned English–Czech setup, but using the training data described in Supplementary Tables  3 and 5 and the training diagram in Supplementary Fig.  3 . All systems (including M1 and M2) were trained with Tensor2Tensor Transformer (no Nematus was involved). Iterated backtranslation was tried only for French→English. No translationese tuning was used (because we report just the BLEU training curve, but no experiments where the final checkpoint selection is needed). No regex post-diting was used.

13 Reanalysis of context-unaware evaluation in WMT18

We first reanalyzed results from the context-unaware evaluation of WMT 2018 English–Czech News Translation Task, provided to us by the WMT organizers ( http://statmt.org/wmt18/results.html ). The data shown in Fig.  3a were processed in the same way as by the WMT organizers: scores with BAD and REF types were first removed, a grouped score was computed as an average score for every triple language pair (“Pair”), MT system (“SystemID”), and sentence (“SegmentID”) was computed, and the systems were sorted by their average score. In Fig.  3a , we show distribution of the grouped scores for each of the MT systems, using paired two-tailed sign test to compare significance of differences of the subsequent systems.

We next assessed whether the results could be confounded by the original language of the source. Specifically, one half of the test-set sentences in WMT18 were originally English sentences translated to Czech by a professional agency, while the other half were English translations of originally Czech sentences. However, both types of sentences were used together for evaluation of both translation directions in the competition. Since the direction of translation could affect the evaluation, we first re-evaluated the MT systems in WMT18 by splitting the test-set according to the original language in which the source sentences were written.

Although the absolute values of source direct assessment were lower for all systems and reference translation in originally English source sentences compared to originally Czech sentences, CUBBITT significantly outperformed the human reference and other MT systems in both test sets (Supplementary Fig.  4 ). We checked that this was true also when comparing z-score normalized scores and using unpaired one-tail Mann–Whitney U test, as by the WMT organizers.

Any further evaluation in our study was performed only on documents with the source side as the original text, i.e., with originally English sentences in the English→Czech evaluations.

14 Context-aware evaluation: methodology

Three groups of paid evaluators were recruited: six professional translators, three translation theoreticians, and seven other evaluators (non-professionals). All 16 evaluators were native Czech speakers with excellent knowledge of the English language. The professional translators were required to have at least 8 years of professional translation experience and they were contacted via The Union of Interpreters and Translators ( http://www.jtpunion.org/ ). The translation theoreticians were from The Institute of Translation Studies, Charles University’s Faculty of Arts ( https://utrl.ff.cuni.cz/ ). Guidelines presented to the evaluators are given in Supplementary Methods 1.1.

For each source sentence, evaluators compared two translations: Translation T1 (the left column of the annotation interface) vs Translation T2 (the right column of the annotation interface). Within one document (news article), Translation T1 was always a reference and Translation T2 was always CUBBITT, or vice versa (i.e., each column within one document being purely reference translation or purely CUBBITT). However, evaluators did not know which system is which, nor that one of them is a human translation and the other one is a translation by MT system. The order of reference and CUBBITT was random in each document. Each evaluator encountered reference being Translation T1 in approximately one half of the documents.

Evaluators scored 10 consecutive sentences (or the entire document if shorter than 10 sentences) from a random section of the document (the same section was used in both T1 and T2 and by all evaluators scoring this document), but they had access to the source side of the entire document (Supplementary Fig.  5 ).

Every document was scored by at least two evaluators (2.55 ± 0.64 evaluators on average). The documents were assigned to evaluators in such a way that every evaluator scored nine different nonspam documents and most pairs of evaluators had at least one document in common. This maximized the diversity of annotator pairs in the computation of interannotator agreement. In total, 135 (53 unique) documents and 1304 (512 unique) sentences were evaluated by the 15 evaluators who passed quality control (see below).

15 Context-aware evaluation: quality control

The quality control check of evaluators was performed using a spam document, similarly as in Läubli et al. 23 and Kittur et al. 37 . In MT translations of the spam document, the middle words (i.e., except the first and last words in the sentence) were randomly shuffled in each of the middle six sentences of the document (i.e., the first and last two sentences were kept intact). We ascertained that the resulting spam translations made no sense.

The criterion for evaluators to pass the quality control was to score at least 90% of reference sentences better than all spam sentences (in each category: adequacy, fluency, overall). One non-professional evaluator did not pass the quality control, giving three spam sentences a higher score than 10% of the reference sentences. We excluded the evaluator from the analysis of the results (but the key results reported in this study would hold even when including the evaluator).

16 context-aware evaluation: interannotator agreement

We used two methods to compute interannotator agreement (IAA) on the paired scores (CUBBITT—reference difference) in adequacy, fluency, and overall quality for the 15 evaluators. First, for every evaluator, we computed Pearson and Spearman correlation of his/her scores on individual sentences with a consensus of scores from all other evaluators. This consensus was computed for every sentence as the mean of evaluations by other evaluators who scored this sentence. This correlation was significant after Benjamini–Hochberg correction for multiple testing for all evaluators in adequacy and fluency and overall quality. The median and interquartile range of the Spearman r of the 15 evaluators were 0.42 (0.33–0.49) for adequacy, 0.49 (0.35–0.55) for fluency, and 0.49 (0.43–0.54) for overall quality. The median and interquartile range of the Pearson r of the 15 evaluators were 0.42 (0.32–0.49) for adequacy, 0.47 (0.39–0.55) for fluency, and 0.46 (0.40–0.50) for overall quality.

Second, we computed Kappa in the same way as in WMT 2012–2016 38 , separately for adequacy, fluency, and overall quality (Supplementary Table  7 ).

17 Context-aware evaluation: statistical analysis

First, we computed the average score for every sentence from all evaluators who scored the sentence within the group (non-professionals, professionals, translation theoreticians for Fig.  3 and Supplementary Fig.  7B ) or within the entire cohort (for Supplementary Fig.  7A ). The difference between human reference and CUBBITT translations was assessed using paired two-tailed sign test (Matlab function sign test) and P values below 0.05 were considered statistically significant.

In the analysis of relative contribution of adequacy and fluency in the overall score (Supplementary Fig.  6 ), we fitted a linear model through scores in all sentences, separately for human reference translations and CUBBITT translations for every evaluator, using matlab function fitlm(tableScores,‘overall~adequacy+fluency’,‘RobustOpts’,‘on’, ‘Intercept’, false).

18 Context-aware evaluation: analysis of document types

For analysis of document types (Supplementary Fig.  11 ), we grouped the 53 documents (news articles) into seven classes: business (including economics), crime, entertainment (including art, film, one article about architecture), politics, scitech (science and technology), sport, and world. Then we compared the relative difference of human reference minus CUBBITT translation scores on the document-level scores and sentence-level scores and used sign test to assess the difference between the two translations.

19 Evaluation of error types in context-aware evaluation

Three non-professionals and three professional translator evaluators performed a follow-up evaluation of error types, after they finished the basic context-aware evaluation. Nine columns were added into the annotation sheets next to their evaluations of quality (adequacy, fluency, and overall quality) of each of the two translations. The evaluators were asked to classify all translation errors into one of eight error types and to identify sentences with an error due to cross-sentence context (see guidelines). In total, 54 (42 unique) documents and 523 (405 unique) sentences were evaluated by the six evaluators. Guidelines presented to the evaluators are given in Supplementary Methods 1.2.

Similarly to Section 5.4, we compute IAA Kappa scores for each error type, based on the CUBBITT—Reference difference (Supplementary Table  8 ).

When carrying out statistical analysis, we first grouped the scores of sentences with multiple evaluations by computing the average number of errors per sentence and error type from the scores of all evaluators who scored this sentence. Next, we compared the percentage of sentences with at least one error (Fig.  4a ) and the number of errors per sentence (Supplementary Fig.  9 ), using sign test to compare the difference between human reference and CUBITT translations.

20 Evaluation of five MT systems

Five professional-translator evaluators performed this follow-up evaluation after they finished the previous evaluations. For each source sentence, the evaluators compared five translations by five MT systems: Google Translate from 2018, UEdin from 2018, Transformer trained with one iteration of mix-BT (as MT2 in Supplementary Fig. 2, but with mix-BT instead of block-BT), Transformer trained with one iteration of block-BT (MT2 in Supplementary Fig. 2), and the final CUBBITT system. Within one document, the order of the five systems was fixed, but it was randomized between documents. Evaluators were not given any details about the five translations (such as whether they are human or MT translations or by which MT systems). Every evaluator was assigned only documents that he/she has not yet evaluated in the basic quality + error types evaluations. Guidelines presented to the evaluators are given in Supplementary Methods 1.3.

Evaluators scored 10 consecutive sentences (or the entire document if this was shorter than 10 sentences) from a random section of the document (the same for all five translations), but had access to the source side of the entire document. Every evaluator scored nine different documents. In total, 45 (33 unique) documents and 431 (336 unique) sentences were evaluated by the five evaluators.

When measuring interannotator agreement, in addition to reporting IAA Kappa scores for the evaluation of all five systems (as usual in WMT) in Supplementary Table  9 , we also provide IAA Kappa scores for each pair of systems in Supplementary Fig.  12 . This confirms the expectation that a higher interannotator agreement is achieved in comparisons of pairs of systems with a large difference in quality.

When carrying out statistical analysis, we first grouped the scores of sentences with multiple evaluations by computing the fluency and adequacy score per sentence and translation from the scores of all evaluators who scored this sentence. Next, we sorted the MT systems by the mean score, using sign test to compare the difference between the consecutive systems (for Fig.  4b ). Evaluation of the entire test-set (all originally English sentences) using BLEU for comparison is shown in Supplementary Fig.  13 .

21 Translation turing test

Participants of the Translation Turing test were unpaid volunteers. The participants were randomly assigned into four non-overlapping groups: A1, A2, B1, B2. Groups A1 and A2 were presented translations by both human reference and CUBBITT. Groups B1 and B2 were presented translations by both human reference and Google Translate (obtained from https://translate.google.cz/ on 13 August 2018). The source sentences in the four groups were identical. Guidelines presented to the evaluators are given in Supplementary Methods 1.4.

The evaluated sentences were taken from originally English part of the WMT18 evaluation test-set (i.e., WMT18-orig-en) and shuffled in a random order. For each source sentence, it was randomly assigned whether Reference translation will be presented to group A1 or A2; the other group was presented this sentence with the translation by CUBBITT. Similarly, for each source sentence, it was randomly assigned whether Reference translation will be presented to group B1 or B2; the other group was presented this sentence with the translation by Google Translate. Every participant was therefore presented human and machine translations approximately in a 1:1 ratio (but this information was intentionally concealed from them).

Each participant encountered each source sentence at most once (i.e., with only one translation), but each source sentence was evaluated for all the three systems. (Reference was evaluated twice, once in the A groups, once in the B groups.) Each participant was presented with 100 sentences. Only participants with more than 90 sentences evaluated were included in our study.

The Translation Turing test was performed as the first evaluation in this study (but after the WMT18 competition) and participants who overlapped with the evaluators of the context-aware evaluations were not shown results from the Turing test before they finished all the evaluations.

In total, 15 participants evaluated a mix of human and CUBBITT translations (five professional translators, six MT researchers, and four other), 16 participants evaluated a mix of human and Google Translate translations (eight professional translators, five MT researchers, and three other). A total of 3081 sentences were evaluated by all participants of the test.

When measuring interannotator agreement, we computed the IAA Kappas (Supplementary Table  10 ) using our own script, treating the task as a simple binary classification. While in the previous types of evaluations, we computed the IAA Kappa scores using the script from WMT 2016 38 , this was not possible in the Translation Turing test, which does not involve any ranking.

When carrying out statistical analysis, we computed the accuracy for each participant as the percentage of sentences with correctly identified MT or human translations (i.e., number of true positives + true negatives divided by the number of scored sentences) and the significance was assessed using the Fisher test on the contingency table. The resulting P -values were corrected for multiple testing with the Benjamini–Hochberg method using matlab function fdr_bh(pValues,0.05,‘dep’,‘yes’) 39 and participants with the resulting Q -value below 0.05 were considered to have significantly distinguished between human and machine translations.

22 Block-BT and checkpoint averaging synergy

In this analysis, the four systems from Fig.  2a were compared: block-BT vs mix-BT, both with (Avg) vs without (noAvg) checkpoint averaging. All four systems were trained with a single iteration of backtranslation only, i.e., corresponding to the MT2 system in Supplementary Fig.  2 . The WMT13 newstest (3000 sentences) was used to evaluate two properties of the systems over time: translation diversity and generation of novel translations by checkpoint averaging. These properties were analyzed over the time of the training (up to 1 million steps), during which checkpoints were saved every hour (up to 214 checkpoints).

23 Overall diversity and novel translation quantification

We first computed the overall diversity as the number of all the different translations produced by the 139 checkpoints between 350,000 and 1,000,000 steps. In particular, for every sentence in WMT13 newstest, the number of unique translations was computed in the hourly checkpoints, separately for block-BT-noAvg and mix-BT-noAvg. Comparing the two systems in every sentence, block-BT-noAvg produced more unique translations in 2334 (78%) sentences; mix-BT-noAvg produced more unique translations in 532 (18%) sentences; and the numbers of unique translations were equal in 134 (4%) sentences.

Next, in the same checkpoints and for every sentence, we compared translations produced by models with and without averaging and computed the number of checkpoints with a novel Avg∞ translation. These are defined as translations that were never produced by the same system without checkpoint averaging (by never we mean in none of the checkpoints between 350,000 and 1,000,000). In total, there were 1801 (60%) sentences with at least one checkpoint with novel Avg∞ translation in block-BT and 949 (32%) in mix-BT. When comparing the number of novel Avg∞ translations in block-BT vs mix-BT in individual sentences, there were 1644 (55%) sentences with more checkpoints with novel Avg∞ translations in block-BT, 184 (6%) in mix-BT, and 1172 (39%) with equal values.

24 Diversity and novel translations over time

First, we evaluated development of translation diversity over time using moving window of octuples of checkpoints in the two systems without checkpoint averaging. In particular, for every checkpoint and every sentence, we computed the number of different unique translations in the last eight checkpoints. The average across sentences is shown in Supplementary Fig.  16 , separately for block-BT-noAvg and mix-BT-noAvg.

Second, we evaluated development of novel translations by checkpoint averaging over time. In particular, for every checkpoint and every sentence, we evaluated whether the Avg model created a novel Avg8 translation, i.e., whether the translation differed from all the translations of the last eight noAvg checkpoints. The percentage of sentences with a novel Avg8 translation in the given checkpoint is shown in Fig.  8a , separately for block-BT and mix-BT.

25 Effect of novel translations on evaluation by BLEU

We first identified the best model (checkpoint) for each of the systems according to BLEU: checkpoint 775178 in block-BT-Avg (BLEU 28.24), checkpoint 775178 in block-BT-NoAvg (BLEU 27.54), checkpoint 606797 in mix-BT-Avg (BLEU 27.18), and checkpoint 606797 in mix-BT-NoAvg (BLEU 26.92). We note that the Avg and NoAvg systems do not necessarily need to have the same checkpoint with the highest BLEU, however it was nevertheless the case in both block-BT and mix-BT systems here. We next identified which translations in block-BT-Avg and in mix-BT-Avg were novel Avg8 (i.e., not seen in the last eight NoAvg checkpoints). There were 988 novel Avg8 sentences in block-BT-Avg and 369 in mix-BT-Avg. Finally, we computed BLEU of Avg translations, in which either the novel Avg8 translations were replaced with the NoAvg versions (yellow bars in Fig.  8b ), or vice versa (orange bars in Fig.  8b ); separately for block-BT and mix-BT.

Reporting summary

Further information on research design is available in the  Nature Research Reporting Summary linked to this article.

Data availability

Data used for comparison of human and machine translations may be downloaded at http://hdl.handle.net/11234/1-3209 .

Code availability

The CUBBITT source code is available at https://github.com/tensorflow/tensor2tensor . Codes for analysis of human and machine translations were uploaded together with the analyzed data at http://hdl.handle.net/11234/1-3209 .

Hirschberg, J. & Manning, C. D. Advances in natural language processing. Science 349 , 261–266 (2015).

Article   ADS   MathSciNet   CAS   Google Scholar  

Bojar, O. Machine Translation. In Oxford Handbooks in Linguistics 323–347 (Oxford University Press, 2015).

Hajič, J. et al. Natural Language Generation in the Context of Machine Translation  (Center for Language and Speech Processing, Johns Hopkins University, 2004).

Vanmassenhove, E., Hardmeier, C. & Way, A. Getting gender right in neural machine translation. In Proc. 2018 Conference on Empirical Methods in Natural Language Processing 3003–3008 (Association for Computational Linguistics, 2018).

Artetxe, M., Labaka, G., Agirre, E. & Cho, K. Unsupervised neural machine translation. In 6th International Conference on Learning Representations, ICLR 2018—Conference Track Proceedings (2018).

Lecun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521 , 436–444 (2015).

Article   ADS   CAS   Google Scholar  

Moravčík, M. et al. DeepStack: Expert-level artificial intelligence in heads-up no-limit poker. Science 356 , 508–513 (2017).

Article   ADS   MathSciNet   Google Scholar  

Sutskever, I., Vinyals, O. & Le, Q. V. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems 27 (NIPS 2014) 3104–3112 (Curran Associates, Inc., 2014).

Bahdanau, D., Cho, K. & Bengio, Y. Neural machine translation by jointly learning to align and translate. In International Conference on Learning Representations (2014).

Luong, M.-T. & Manning, C. D. Stanford neural machine translation systems for spoken language domains. In Proceedings of The International Workshop on Spoken Language Translation (IWSLT) (2015).

Junczys-Dowmunt, M., Dwojak, T. & Hoang, H. Is neural machine translation ready for deployment? A case study on 30 translation directions. In Proceedings of the Ninth International Workshop on Spoken Language Translation (IWSLT) (2016).

Hutchins, W. J. & Somers, H. L. An introduction to machine translation . (Academic Press, 1992).

Brown, P. F., Della Pietra, S. A., Della Pietra, V. J. & Mercer, R. L. The mathematics of statistical machine translation. Comput. Linguist 19 , 263–311 (1993).

Google Scholar  

Koehn, P., Och, F. J. & Marcu, D. Statistical phrase-based translation. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - NAACL ’03 1, 48–54 (2003).

Wu, Y. et al. Google’s neural machine translation system: bridging the gap between human and machine translation. Preprint at http://arxiv.org/abs/1609.08144 (2016).

Hassan, H. et al. Achieving human parity on automatic Chinese to English news translation. Preprint at http://arxiv.org/abs/1803.05567 (2018).

Bojar, O. et al. Findings of the 2018 conference on machine translation (WMT18). In Proceedings of the Third Conference on Machine Translation (WMT) 2, 272–307 (2018).

Vaswani, A. et al. Attention is all you need. In Advances in Neural Information Processing Systems (Curran Associates, Inc., 2017).

Sennrich, R., Haddow, B. & Birch, A. Neural machine translation of rare words with subword units. 54th Annu. Meet. Assoc. Comput. Linguist . https://doi.org/10.18653/v1/P16-1162 (2015).

Bojar, O. et al. CzEng 1.6: Enlarged Czech-English parallel corpus with processing tools dockered. In Text, Speech, and Dialogue: 19th International Conference, TSD 2016 231–238 (2016).

Tiedemann, J. OPUS—-parallel corpora for everyone. In Proceedings of the 19th Annual Conference of the European Association of Machine Translation (EAMT) 384 (2016).

Toral, A., Castilho, S., Hu, K. & Way, A. Attaining the unattainable? Reassessing claims of human parity in neural machine translation. In Proceedings of the Third Conference on Machine Translation (WMT) 113–123 (2018).

Läubli, S., Sennrich, R. & Volk, M. Has machine translation achieved human parity? A case for document-level evaluation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP) 4791–4796 (2018).

Castilho, S. et al. Is neural machine translation the new state of the art? Prague Bull. Math. Linguist 108 , 109–120 (2017).

Article   Google Scholar  

Haddow, B. et al. The University of Edinburgh’s submissions to the WMT18 news translation Task. In Proc. Third Conference on Machine Translation 2, 403–413 (2018).

Barrault, L. et al. Findings of the 2019 conference on machine translation (WMT19). In Proc. Fourth Conference on Machine Translation (WMT) 1–61 (2019).

Bojar, O. et al. The joy of parallelism with CzEng 1.0. In Proc. Eighth International Language Resources and Evaluation Conference (LREC’12) 3921–3928 (2012).

Papineni, K., Roukos, S., Ward, T. & Zhu, W.-J. BLEU: a method for automatic evaluation of machine translation. In Proc. 40th Annual Meeting on Association for Computational Linguistics—ACL ’02 311–318 (Association for Computational Linguistics, 2002).

Post, M. A Call for clarity in reporting BLEU scores. In Proc. Third Conference on Machine Translation (WMT) 186–191 (2018).

Popel, M. & Bojar, O. Training tps for the transformer model. Prague Bull. Math. Linguist . https://doi.org/10.2478/pralin-2018-0002 (2018).

Popel, M. CUNI transformer neural MT system for WMT18. In Proc. Third Conference on Machine Translation (WMT) 482–487 (Association for Computational Linguistics, 2019).

Shazeer, N. & Stern, M. Adafactor: Adaptive learning rates with sublinear memory cost. In Proc. 35th International Conference on Machine Learning, ICML 2018 4603–4611 (2018).

Sennrich, R. et al. The University of Edinburgh’s neural MT systems for WMT17. In Proc. Second Conference on Machine Translation (WMT) 2 , 389–399 (2017).

Junczys-Dowmunt, M., Dwojak, T. & Sennrich, R. The AMU-UEDIN submission to the WMT16 news translation task: attention-based NMT models as feature functions in phrase-based SMT. In Proc. First Conference on Machine Translation (WMT) 319–325 (2016).

Sennrich, R., Haddow, B. & Birch, A. Edinburgh neural machine translation systems for WMT 16. In Proc. First Conference on Machine Translation (WMT) 371–376 (2016).

Gellerstam, M. Translationese in Swedish novels translated from English. In Translation Studies in Scandinavia: Proceedings from the Scandinavian Symposium on Translation Theory (SSOTT) 88–95 (1986).

Kittur, A., Chi, E. H. & Suh, B. Crowdsourcing user studies with Mechanical Turk. In Proc. 26th Annual CHI Conference on Human Factors in Computing Systems (CHI ’08) 453–456 (ACM Press, 2008).

Bojar, O. et al. Findings of the 2016 conference on machine translation. In Proc. First Conference on Machine Translation: Volume 2, Shared Task Papers 131–198 (2016).

Groppe, D. fdr_bh MATLAB central file exchange. https://www.mathworks.com/matlabcentral/fileexchange/27418-fdr_bh . (2020).

Download references

Acknowledgements

We thank the volunteers who participated in the Translation Turing test, Jack Toner for consultation of written English, and the WMT 2018 organizers for providing us with the data for the re-evaluation of translation quality. This work has been partially supported by the grants 645452 (QT21) of the European Commission, GX19-26934X (NEUREM3) and GX20-16819X (LUSyD) of the Grant Agency of the Czech Republic. The work has been using language resources developed and distributed by the LINDAT/CLARIAH-CZ project of the Ministry of Education, Youth and Sports of the Czech Republic (project LM2018101).

Author information

These authors contributed equally: Martin Popel, Marketa Tomkova, Jakub Tomek.

Authors and Affiliations

Faculty of Mathematics and Physics, Charles University, Prague, 121 16, Czech Republic

Martin Popel, Ondřej Bojar & Zdeněk Žabokrtský

Ludwig Cancer Research Oxford, University of Oxford, Oxford, OX1 2JD, UK

Marketa Tomkova

Department of Computer Science, University of Oxford, Oxford, OX1 3QD, UK

Jakub Tomek

Google Brain, Mountain View, California, CA, 94043, USA

Łukasz Kaiser & Jakob Uszkoreit

You can also search for this author in PubMed   Google Scholar

Contributions

M.P. initiated the project. L.K. and J.U. designed and implemented the Transformer model. M.P. designed and implemented training of the translation system. J.T., M.T., and M.P. with contributions from O.B. and Z.Ž. designed the evaluation. M.T., J.T., and M.P. conducted the evaluation. M.T. and J.T. analyzed the results. M.T., J.T., and M.P. wrote the initial draft; all other authors critically reviewed and edited the manuscript.

Corresponding author

Correspondence to Martin Popel .

Ethics declarations

Competing interests.

J.U. and L.K. are employed by and hold equity in Google, which funded the development of Transformer. The remaining authors (M.P., M.T., J.T., O.B., Z.Ž.) declare no competing interests.

Additional information

Peer review information Nature Communications thanks Alexandra Birch and Marcin Junczys-Dowmunt for their contribution to the peer review of this work. Peer reviewer reports are available.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information, peer review file, description of additional supplementary files, supplementary data 1, reporting summary, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Popel, M., Tomkova, M., Tomek, J. et al. Transforming machine translation: a deep learning system reaches news translation quality comparable to human professionals. Nat Commun 11 , 4381 (2020). https://doi.org/10.1038/s41467-020-18073-9

Download citation

Received : 16 September 2019

Accepted : 24 July 2020

Published : 01 September 2020

DOI : https://doi.org/10.1038/s41467-020-18073-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

A neural network to identify requests, decisions, and arguments in court rulings on custody.

• José Félix Muñoz-Soro
• Rafael del Hoyo Alonso
• Francisco Lacueva

Artificial Intelligence and Law (2024)

A novel two-way rebalancing strategy for identifying carbonylation sites

• Linjun Chen
• Xiao-Yuan Jing

BMC Bioinformatics (2023)

Predicting corrosion inhibition efficiencies of small organic molecules using data-driven techniques

• Bahram Vaghefinazari
• Christian Feiler

npj Materials Degradation (2023)

Design of intelligent module design for humanoid translation robot by combining the deep learning with blockchain technology

Scientific Reports (2023)

Synthetic electronic health records generated with variational graph autoencoders

• Giannis Nikolentzos
• Michalis Vazirgiannis
• Erik G. Brandt

npj Digital Medicine (2023)

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

• Search Menu
• Advance articles
• Editor's Choice
• Special Collections
• Author Guidelines
• Submission Site
• Open Access
• Reasons to submit
• About BioScience
• Journals Career Network
• Editorial Board
• Advertising and Corporate Services
• Self-Archiving Policy
• Potentially Offensive Content
• Terms and Conditions
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

Short-term actions: translation and the promotion of multilingual science, the role of academic institutions in promoting translation efforts, long-term vision: from a language hub to a language network, acknowledgments, references cited.

• < Previous

Overcoming Language Barriers in Academia: Machine Translation Tools and a Vision for a Multilingual Future

• Article contents
• Figures & tables
• Supplementary Data

Emma Steigerwald, Valeria Ramírez-Castañeda, Débora Y C Brandt, András Báldi, Julie Teresa Shapiro, Lynne Bowker, Rebecca D Tarvin, Overcoming Language Barriers in Academia: Machine Translation Tools and a Vision for a Multilingual Future, BioScience , Volume 72, Issue 10, October 2022, Pages 988–998, https://doi.org/10.1093/biosci/biac062

• Permissions Icon Permissions

Having a central scientific language remains crucial for advancing and globally sharing science. Nevertheless, maintaining one dominant language also creates barriers to accessing scientific careers and knowledge. From an interdisciplinary perspective, we describe how, when, and why to make scientific literature more readily available in multiple languages through the practice of translation. We broadly review the advantages and limitations of neural machine translation systems and propose that translation can serve as both a short- and a long-term solution for making science more resilient, accessible, globally representative, and impactful beyond the academy. We outline actions that individuals and institutions can take to support multilingual science and scientists, including structural changes that encourage and value translating scientific literature. In the long term, improvements to machine translation technologies and collective efforts to change academic norms can transform a monolingual scientific hub into a multilingual scientific network. Translations are available in the supplemental material.

The language in which science is primarily   communicated has varied through time and space, cycling through Chinese, Sumerian, Egyptian, Persian, Greek, Latin, Arabic, German, and French, to name a few (von Gizycki 1973 , Montgomery and Crystal 2013 ). The use of English as the scientific lingua franca began only 400 years ago, alongside Great Britain's growing colonial empire. After the World Wars, it continued to expand with the increasing military, economic, and technological clout of the United States (Canagarajah 2002 , Gordin 2015 ). Since then, English dominance has extended across the entire globe, as no language has previously done. Today, 98% of peer-reviewed scientific publishing is in English ( Ammon 2012 , Liu 2017 ), and English is the official language of most scientific events and international and indexed academic journals.

Having a common language benefits science by facilitating international scientific communication and creating a monolingual repository for publications and data (Montgomery and Crystal 2013 ). The maintenance of a common scientific language is also useful for the dissemination and recognition of research performed by scientists whose primary language is not widely spoken, as well as for facilitating communication between such scientists and the wider scientific community. Having a shared scientific language also facilitates international mobility and limits the number of additional languages required for international collaboration. However, despite the benefits of a common language, maintaining a single universal scientific language creates barriers by requiring the majority of researchers in the world to become proficient in an additional language prior to engaging with the global academic community. Through its “Recommendation on Open Science,” UNESCO has called on scientific institutions to foster global, multilingual, and cross-disciplinary research programs in order to provide more equitable access to scientific knowledge and careers (UNESCO 2021 ).

In the present article, we summarize the costs of a single universal language in science and provide a set of practical approaches that individuals, academic societies, and institutions can take to help break down language barriers, focusing on machine translation tools for written sources and structural change that would better support a multilingual academy. Although the suggestions contained in the present article are built from and sometimes particularly pertinent to our research experiences in ecology, evolution, and conservation, these ideas may be useful to a broader scientific audience.

The costs of a single universal language in science

Although maintaining a central language has its benefits (see above), it also stymies the advancement of science, creates barriers within academia, and complicates applying scientific evidence to decision-making outside of academia. For example, because academic knowledge is mostly communicated in English, scientists and other members of society often overlook knowledge generated in other languages. One concrete manifestation of that is using keywords exclusively in English during literature searches (Pabón Escobar and da Costa 2006 , Kirchik et al. 2012 , Liang et al. 2013 , Neimann Rasmussen and Montgomery 2018 , Amano et al. 2021a ). This effect can be amplified by language biases in search engines (Rovira et al. 2021 ). Overlooking non-English studies can result in large gaps within global databases, which affects policy, management, and decision-making (Amano and Sutherland 2013 , Amano et al. 2016 , 2021a , Konno et al. 2020 , Angulo et al. 2021 , Kirpotin et al. 2021 ). For example, the exclusion of the many studies on conservation interventions published in languages other than English can reduce the evidence being considered during decision-making processes and lead to less-optimal natural resource management (Amano et al. 2021a ). In addition, non-English-language literature could expand both the geographical and the taxonomic coverage in biodiversity studies (Khelifa and Mahdjoub 2022 ). Biases in who contributes to science and makes these management decisions also reduce the credibility and global buy-in to these management practices (Baldi and Palotas 2021 ).

English proficiency also influences who participates in science at a global scale, which is detrimental to science, because a diversity of perspectives bolsters the construction of robust and innovative scientific knowledge (Bennett 2013 , AlShebli et al. 2018 , Hofstra et al. 2020 ). Proficiency in English is often a requirement for professional advancement, such as publishing in high-impact journals, receiving international grants, and participating in international conferences (Hwang 2005 , Clavero 2010 , Huttner-Koros and Perera 2016 , Ramírez-Castañeda 2020 ). Non-Anglophones are therefore under constant pressure to improve their English language skills (Tardy 2004 , Lindsey and Crusan 2011 , Corcoran 2015, Suzina 2021 ), which can be a source of anxiety and an emotional burden (Ramírez-Castañeda 2020 , Amano et al. 2021b ). Moreover, this challenge is not experienced equally across English learners but, rather, weighs particularly heavily on learners whose dominant language is highly divergent from English and on learners from regions in which English-language instruction or media are not widely available, two issues that are not mutually exclusive. Language barriers can impose a severe financial burden on individuals, who may pay for English classes, proofreading, and translation services, reinforcing socioeconomic inequity in science, especially because these burdens are experienced to a greater extent by those in countries with a lower gross domestic product (Schofield and Mamuna 2003 , Kieffer 2010 , González Mellado et al. 2020 , Ramírez-Castañeda 2020 , Khelifa and Mahdjoub 2022 ). Biases during peer review may lead non-Anglophones to publish in lesser-known journals or in regional journals that publish in other languages, making their research less discoverable (Mur Dueñas 2012 ). These burdens intensify the dependence of many non-Anglophone scientists on scientists with high English proficiency (Ordóñez-Matamoros et al. 2011 ). Ultimately, these barriers can impede non-Anglophones from obtaining jobs, tenure, or promotion (Moreno 2010 ).

Constraining diverse points of view to fit within the structure and vocabulary of a single language impoverishes scholarly discourse and observations of nature. For instance, language shapes how we perceive color (Siok et al. 2009 ), our understanding and memory of events (Fausey et al. 2010 ), and our ability to gauge the awareness or knowledge of others (Jara-Ettinger and Rubio-Fernandez 2021 ). When we write only in English, we limit our way of describing the relationships between ideas—a type of loss that has been analogized to the creation of an epistemological monoculture (Martin 2009 , Bennett 2013 , Aguilar Gil 2020 ). Moreover, constraining global scientific discussions to a single language can limit who builds, has access to, and communicates scientific knowledge to the broader public (Canagarajah 2002 , Tardy 2004 , Huttner-Koros and Perera 2016 , O'Neil 2018 ), profoundly affecting the relationship between science and society. Scientific monolingualism may reduce the dissemination of science to non–English-speaking institutions and communities, which can leave new knowledge inaccessible to the people for whom it is most relevant, such as those living near study sites, local public media, and regional policymakers (Márquez and Porras 2020 ). This is likely particularly impactful for people in countries with low English proficiency, who have reduced access to knowledge communicated exclusively in English (Amano et al. 2016 , Saha et al. 2019 ), sometimes even to studies that feature these regions (Barath 2019 ). Although the disconnection between science and society is unfortunate for any scientific field, the cost is particularly high for applied sciences and crisis disciplines such as climate science, epidemiology, and conservation (Meadow et al. 2015 , Saha et al. 2019 , Amano et al. 2021a ), where the rapid dissemination of new results makes a material difference to urgent decisions that must be made despite incomplete evidence.

The existence of a single universal language of science may currently serve to share new knowledge broadly and practically. However, those who bear the costs of a single language also tend to face additional barriers—for example, those associated with colonialism, because the language that an individual speaks is tied to the history of their country and culture. Therefore, maintaining a single language in science without providing adequate support to people who do not speak that language will continue to perpetuate historical imbalances. Attempts to create a more accessible centralized language (e.g., Esperanto) have not gained traction (Tonkin 1987 ), and although English may present some linguistic advantages (e.g., relatively simple and genderless grammar), it is not the only language with these attributes, and its dominance can be attributed to the historical factors mentioned above. Therefore, we propose that science would benefit from integrating multiple languages. Multilingual science will also benefit our community by creating support systems that can facilitate potential future transitions, because although it may feel unlikely, history has shown that dominant languages are likely to continue changing over time.

Science benefits from diverse viewpoints, and language is one of many axes of diversity (AlShebli et al. 2018 , Hofstra et al. 2020 ). However, little structural support exists in the present to help non-Anglophones publish and advance professionally in English. Recently, Amano and collaborators ( 2021b ) highlighted some practical tips to overcome language barriers, such as promoting multilingual activities, being empathetic with those who face language barriers, providing an English proofing network for preprints (Khelifa et al. 2022 ), and translating scientific literature ( Amano et al. 2016 , 2021b , Márquez and Porras 2020 , Ramírez-Castañeda 2020 ). Multilingual publishing is another mechanism that actively promotes and places value on contributions in different languages. Machine translation tools can help scientists take concrete steps toward publishing in multiple languages, including in English. In the present article, we largely focus on machine translation tools for written sources; however, we highlight that similar efforts can be extended to the spoken word.

An overview of machine translation tools and how to improve them for scientific literature

The earliest approaches to machine translation used painstakingly programmed linguistic rules and very large dictionaries, but they had limited success because language is full of ambiguity and computers had no access to the type of real-world knowledge and social interactions that people use to interpret language (Way 2020 ). Following the introduction of the Internet and the increasing trend of producing texts in digital form, machine translation researchers moved away from linguistic approaches and toward data-driven machine translation, which capitalized on the strengths of computers (e.g., pattern matching, rapid calculations). Around the turn of the millennium, statistical machine translation systems began to appear, including early free online tools, such as Google Translate. In statistical machine translation, the developers fed the computer with vast quantities of previously translated texts, and the system used these examples to calculate the probability that a given phrase should be translated in a certain way in a future text (Way 2020 ). Statistical machine translation tools produced better-quality output than linguistic approaches, but there was still considerable room for improvement. Another data-driven approach, known as neural machine translation , appeared in late 2016, and it has presented another leap forward in terms of translation quality. Today, the majority of machine translation tools use artificial neural networks in combination with artificial intelligence–based techniques such as machine learning (Forcada 2017 ). These techniques require developers to provide the machine translation system with many training examples of original source texts and their translations for the system to learn. Therefore, translation tools are more easily tuned to widely used languages or languages with more of these examples. Although they are not perfect, neural machine translation systems provide a more viable starting point than older machine translation systems, which relied on linguistic or statistical approaches. The results of neural machine translation systems can be used for basic knowledge acquisition or as a first draft that can then be improved (e.g., for academic writing; Parra Escartín and Goulet 2020 ). Increasing numbers of people are using neural machine translation tools because of their ease of use and free online availability (e.g., DeepL and Google Translate; Bowker 2021 ).

However, using machine translation tools still requires good judgment, which is why there is a need for machine translation literacy (Bowker and Ciro 2019 , Bowker 2021 ). Machine-learning technologies are very sensitive to the quantity and quality of their training data. To work well, machine translation systems need access not only to enormous quantities of previously translated texts and their corresponding original texts but also to good quality texts that are relevant to the focal topic (Chu and Wang 2020 ). For example, the language used in specialized fields contains many technical terms and constructions that are not part of everyday language. Therefore, for a machine translation system to accurately translate texts in the field of biology, it would need to be provided with millions of examples of previously translated texts specifically from this domain. Moreover, these examples would need to cover all the desired language combinations (e.g., English and French, Chinese and Hindi, English and Chinese). In some cases, when a particular language pair has relatively few translated texts available, the lack of training data can be overcome by using a widely spoken language as a pivot language (e.g., translating from Spanish to Chinese using English as an intermediary), although this approach may propagate errors (Kim et al. 2019 ). Similarly for spoken communication, the recent COVID-19 pandemic rapidly increased the need for and use of online communication platforms that provide closed-captioning in multiple languages. However, piping two imperfect technologies (machine translation and speech recognition) together can compound translation errors (Sulubacak et al. 2020 ), similar to problems arising from the use of pivot languages.

There are clear steps that scientists and machine translation tool developers can take to improve the implementation of technologies in scientific translation. A concerted effort toward providing open-access, human-verified, and high-quality translations of abstracts in scientific journals would significantly contribute toward generating the data necessary for training machine translation systems. At the moment, free online translation tools are trained mainly on general language data rather than on scientific jargon or specialized language. Researchers and tool developers could collaborate on open-access tools that train machine translation systems for specialized fields of research. Simultaneously, we could encourage scientists to develop or contribute to multilingual glossaries of specialized terminology, in part to help keep up with the constant generation of new scientific jargon (Nkomo and Madiba 2012 , Wild 2021 ). For instance, Wikipedia is an excellent open-access platform for finding multilingual translations of technical and scientific topics. However, it is currently underused by several scientific disciplines, and several languages with large numbers of speakers (such as Hindi and Turkish) are underrepresented (Kincaid et al. 2020 , Roy et al. 2021 ).

When, why, and how scientific literature can be translated

With the aid of translation tools, contributing translations of abstracts, keywords, and entire articles could become the norm for research programs that cross languages (figure  1 ; Amano et al. 2021b ). Indeed, translating scientific abstracts is already a common practice for some journals in bilingual or (primarily) non-Anglophone countries (e.g., the Canadian Journal of Forest Research , the Brazilian Journal of Biology ). Normalizing the practice of translation will increase access to scientific research for scientists, students, teachers, policymakers, journalists, and members of society at large. It could also shift the work of translation to be more equally shared between native English speakers and those who speak English as an additional language, because translations would not only happen from other languages to English (figure  2 a) but also from English to other languages (figure  2 b). When native English speakers cannot directly translate to a language they do not speak, they will need to find and pay for (or reciprocate) translation services, as is common practice for non-Anglophones. In addition, translating abstracts will help to substantially improve the accuracy of machine translation for scientific texts, as we described above.

An example decision tree that authors can use to decide when and how to translate their research output. Scientists whose research programs meet one or more of the listed circumstances may consider translating into languages relevant to those circumstances. Understanding that researchers are often limited by resources and time, we provide this diagram as a suggestion of when to prioritize translation, because translations may be useful in additional circumstances.

Two visual metaphors to describe breaking down language barriers and moving science toward multilingualism. (a) Today, English operates as a central hub for scientific communication, receiving much more input from speakers of other languages than vice versa. (Only languages with more than 230 million active speakers are shown.) Abbreviations: Ar, Arabic; Be, Bengali; Ch, Chinese; En, English; Fr, French; Hi, Hindi; Po, Portuguese; Ru, Russian; Sp, Spanish; Ur, Urdu. The numbers were estimated according to Eberhard and colleagues ( 2022 ). (b) In the short term, machine translation tools and efforts by scientific communities can help form secondary language hubs (see the main text) that create and disseminate scientific knowledge among all languages within each language family. For instance, Hindi may serve as a connector language; science translated into Hindi can then be more easily translated from Hindi into other Indo-Aryan languages. (c) As machine translation technologies improve, greater exchange across language families will indirectly benefit the speakers of languages with smaller numbers of active speakers (inset), who, owing to geography or history, often must learn a second language from one of these major families. For instance, the greater availability of texts translated to Italic languages will facilitate translation into languages historically and geographically associated with Spanish (i.e., indigenous languages of Iberia, South America, and Central America). (d) Currently, students must become proficient in English during or prior to their graduate studies if they wish to pursue science as a career, presenting a language barrier that may intersect with associated barriers. (e) In the short term, structural changes by institutions, actions by individuals, and machine translation tools can help students bridge the barrier. (f) In the long term, advanced translation technologies and a more multilingual scientific academy will help demolish language barriers. Under this more accessible paradigm, scientists may be able to advance their careers and their English proficiency in parallel, rather than needing English proficiency as a prerequisite for a career. Ultimately, a more multilingual scientific community will make science more accessible to the multilingual public.

We recommend that researchers consider translating articles (or, minimally, abstracts and keywords) when the work is conducted in a region or country in which the primary language is not English, when the team involves researchers whose primary language is not English, or when the research directly or indirectly affects a group of people whose primary language is not English (figure  1 ). Whenever research fits into the first or third of these categories, the teams should always work collaboratively with local researchers to avoid the practice of parachute science (Haelewaters et al. 2021 , de Vos et al. 2022 ). Authors who speak the language selected for translation may wish to create a first draft using DeepL Translator, Baidu Translate, Naver Pagago, Yandex.Translate, Google Translate, or a similar tool, which can then be manually edited. Authors who do not speak the additional language can work with journals to find reciprocal translation partners or other modes of support (supplemental table S1; Amano et al. 2021b ), or they can search for reciprocal translation opportunities through forums such as ResearchGate or preprint servers such as bioRxiv (Khelifa et al. 2022 ). Because some aspects of translation are subjective (e.g., specific vocabulary choices or idiomatic translations), it is critical to reference the person or software that was used and whether the machine translation (if one was done) was verified by a human (Croft 2021; see the translation note in the acknowledgments). Importantly, creating a byline for translation or language editing will normalize the acknowledgment of these critical services, provide scientists with an alternative option to exchanging authorship for editing assistance, and provide a language contact to whom translation questions can be directed.

Translations of previously published scientific articles (even by the authors themselves) often cannot be posted publicly because of copyright restrictions. Therefore, researchers may wish to include a translation of a full manuscript as part of the supplemental material when submitting their work for publication or as part of material stored on accessible web platforms such as the Open Science Framework ( osf.io ) or GitHub (GitHub.com), which can be updated at a later time point to include additional translations. In the future, we suggest that journals with copyright restrictions could implement fee waivers for translated versions (see table S1). Note that open access itself does not imply anything about copyright, but some journals publish open access articles with Creative Commons licenses such as CC BY, CC BY-SA, CC BY-NC, or CC BY-NC-SA, which allow translation without copyright infringement (BY means “Credit must be given to the creator,” SA means “Adaptations must be shared under the same terms,” and NC means “Only noncommercial uses of the work are permitted”; see creativecommons.org for more information). Creative Commons licenses with the ND term (“no derivatives”) would require written permission from the copyright holders to publicly post a translation, which is a type of derivative. If the authors wish to conduct a translation once a paper has been published and it is not published under one of these Creative Commons licenses, they do have a few options, including paying the copyright fee, obtaining a fee waiver (not easy, in our experience), requesting an erratum to append a document to the supplemental files, and choosing to publish a plain-language summary or reflection instead, perhaps as a blog or magazine article (table S1, figure  1 ). In the case of posting preprints before publication in a peer-reviewed journal, some servers permit authors to share information in languages additional to English (e.g., EcoEvoRxiv), although this is not the case for all (e.g., bioRxiv).

A contribution that all researchers and journals can make, regardless of their native language, is to prepare a plain language summary that is both reader friendly and translation friendly (Bowker and Ciro 2019 ). A text that is less structurally complex can still be rich in meaning, but it will be easier for readers to digest and for machine translation tools or human translators to translate. Because the goal of plain writing is simply to write as clearly as possible, the technique can be applied to any language. However, the specific approaches for reducing structural complexity or linguistic ambiguity may differ from one language to the next (see table  1 for examples that apply to English). More detailed information on how to write in an easy to translate style can be found in the plain language toolkit prepared for scientists by Evidence for Democracy (Qaiser 2021 ). One way that journals can help make papers better suited for machine translation and more accessible to readers with lower English proficiency is to soften word limits, because the methods to shorten sentence structure tend to introduce grammatical complexity and ambiguity. The advent of online-only journals has provided a great opportunity for journals to soften word limits without incurring production fees (table S1).

Plain language writing tips to reduce structural complexity and linguistic ambiguity in English, including ideas from Bowker and Ciro ( 2019 ).

Note: Recommended, free online tools that can suggest how to accomplish these goals for a given piece of writing can be found at sites such as https://hemingwayapp.com and https://datayze.com/?category=writing .

Journals and academic societies have the power to change norms, because they are important forums in which scientists engage with each other and are recognized for their work. Journals can actively contribute to addressing language barriers and supporting multilingual science by providing clear guidelines regarding when authors are expected to translate articles or abstracts (e.g., see figure  1 ), how translations can be included in published articles, how research in other languages should be cited (Fung 2008), and how to search for journal content in languages other than English (see table S1 for additional suggestions and supplemental table S2 for examples). Including multilingual scientists on boards and committees can help drive and facilitate these efforts (table S1). Societies or journals could also provide free translation services or promote mentorship within academic societies to provide English proofing (e.g., translatE 2020 , Khelifa et al. 2022 ). In addition, several recent papers have highlighted how individual scientists can reduce bias and improve the peer-review process for nonnative English speakers when acting as reviewers or editors (Romero-Olivares 2019 , Mavrogenis et al. 2020 , Amano et al. 2021b ).

The translation of titles, abstracts, keywords and full texts—which can greatly improve machine translation tools—could be facilitated if journals create a streamlined process for authors to add translations during or after publication and provide a clear statement of copyright policy regarding whether adding translations is subject to the same copyright restrictions as other use cases. Some journals already have systems in place for abstract translations (see table S2 for examples). Multilingual scientific literature and conference booklets would permit researchers and other members of society to use their primary language when scanning the literature or conference abstract books to find relevant articles and talks. Finally, author guidelines that encourage the inclusion of multilingual graphical abstracts (e.g., figure 1 in Chu et al. 2021 ) also increase accessibility, and plain-language abstracts or highlights have the additional benefit of being machine translation–friendly (see our long-term vision below; see Shailes 2017 for examples of plain-language summaries produced by journals, societies, and other organizations).

The actions cataloged above could incur at least two types of burden on journal staff and conference organizers: the financial burden of providing free translation services and the time needed to review translated texts. If a journal or conference cannot presently afford to freely translate their contributors’ science, they (or a consortium of journals) might consider creating a forum on their website or via existing preprint servers (Khelifa et al. 2022 ) where contributors can identify reciprocal or volunteer language editing and translation partners. For example, Cochrane, a United Kingdom–based charitable organization, has a network of volunteers that translate their systematic reviews of medical literature from English to various languages ( www.cochrane.org/join-cochrane/translate ). Although these solutions relieve the burden on the staff of journals and conferences, they demand free labor from researchers. To compensate, journals and conferences could create systems of incentive such as discounts in publication or registration fees. One alternative to overcome the time needed to review translations is to require authors to label translations with standardized disclaimers, such as “manually translated by a fluent speaker,” “machine translated,” or “machine translated and manually edited for accuracy.” Journals could simply note that these translations have not undergone peer review, as is already the case for most supplemental material (e.g., see the Molecular Ecology journal guidelines for abstract translation in table S2). Other actions outlined in table S1 might also come with additional cost (e.g., providing closed captioning in conference talks), but we see this cost as an important and worthwhile investment toward a more inclusive academic environment, which will benefit science, the individuals that participate in it, and society at large.

Universities can promote efforts to overcome language barriers through both their educational role and their role in shaping research program priorities. For example, they could emphasize or recognize the importance of publications in (non-English) national and regional journals for tenure and promotion files, contract renewals, or degree requirements. Faculty and students often feel pressure to publish in English-language journals, because this boosts the rankings and impact factor of their institution, but national and regional publications play an important role in disseminating knowledge (Moreno 2010 , Vaidyanathan 2019 ), which closely aligns with many university missions.

Importantly, because machine translated texts are imperfect, machine translation literacy is essential (Bowker 2021 ). Universities can develop cross-disciplinary courses to teach and enact the practice of scientific translation, which is itself part of a vast field of study (Munday 2016 ). Universities can make machine translation literacy training part of a standard STEM (science, technology, engineering, and mathematics) curriculum, so that new researchers are acquainted with the strengths and limitations of translation technologies (Bowker 2021 ). Students are already widely using these technologies, but perhaps without an appreciation for how to work around their limitations (Bowker 2021 ). In addition, students in the sciences could be encouraged to study foreign languages, as is common in the humanities (Kellsey and Knievel 2004 ), especially if conducting research in non-Anglophone regions.

Many other institutions can do their part in improving scientific standards, making science more accessible, and, therefore, ultimately more globally impactful (table S1). Public databases , such as GenBank or Online Mendelian Inheritance in Man, are critically important resources, and a multilingual approach to their online platforms, as is demonstrated by the International Union for the Conservation of Nature or Fonoteca Zoológica, would permit broader engagement with these resources. Funding agencies can include clauses that encourage or require researchers working internationally to include local scientists in their research and encourage budget items to support translating results and outreach that engages with local communities in their local languages. In addition, international funding agencies could permit the submission of grant or scholarship proposals in several languages, especially if these funds are focused on communities or students who do not necessarily speak English.

At present, scientific publishing is largely centered on the English language, with relatively few languages receiving substantial input from the English hub (figure  2 a; the estimated number of active speakers for each language displayed is from Eberhard et al. 2022 ). For this reason, nonnative English speakers must generally acquire English proficiency before or during their graduate career or else forgo participation in the international scientific community (figure  2 d). We envision that multilingualism is the outcome of a prolonged process of inclusion of languages brought about by improved translation technologies and changes in community norms. A first step that can be taken toward multilingual science is the creation of temporary secondary language hubs that can act as networking communities and knowledge centers for non-Anglophones (figure  2 b), supporting these scientists throughout the launch of their early careers (figure  2 e). For example, hubs for Mandarin, Hindi, or Spanish would establish information streams between sets of languages with many speakers, additional languages pertaining to each language family, and the central English node. Efforts to facilitate the creation of these secondary hubs in science are already happening through multilingual conference activities, bilingual journals, and regional academic societies (table S2; Márquez and Porras 2020 , Amano et al. 2021b ). In the future, tertiary hubs could be established until greater multilingualism is achieved (figure  2 c) and English proficiency is no longer requisite for participation in the international scientific community (figure  2 f). Geographic proximity, political history, and language origin can be some of the strategies used to define a tertiary hub.

Most of Western society has accepted that a universal language is integral to the scientific enterprise (Aguilar Gil 2020 ); therefore, we acknowledge how unreachable or unnecessary a multilingual future may appear. However, a multilingual vision encompasses more than academia; it also aligns with multidisciplinary and plurinational efforts to preserve languages, culture, and knowledge (UNESCO 2021 ; Endangered Languages Project 2022 ). To reach such a long-term goal, we envision that accurate and readily available translation technologies, as well as collective efforts supporting and integrating multilingual science, will play important roles. The ideas presented in this article are a starting point only and will require further discussion. They are not exclusive, universal, or definitive, and the community will require other changes to make scientific fora more inclusive. We encourage the creation of discussion groups on this topic to generate new and innovative ideas to help solve language barriers.

The authors thank many friends and collaborators who have engaged with us in discussion and development of these ideas, including Providence Akayezu, Rauri Bowie, Safa Fanaian, Paula Iturralde-Pólit, Xinyi Liu, Martin A. Nuñez, Katharine Owens, Diego Peralta, Daniel V. de Latorre, Noah Whiteman, Molly Womack, and Christine Zirneklis. We also thank the University of California, Berkeley, Museum of Vertebrate Zoology community, who have been pushing forward several initiatives to improve equity and inclusion in our field, and Tatsuya Amano, whose published articles and resources on the value of multilingual science have guided many of our discussions. This work was supported in part by the Zuckerman STEM Leadership Program (to JTS); by the Julius H Freitag Memorial Award, the Hannah M. and Frank Schwabacher Fund, and the David and Marvalee Wake Fund (to ES); by the National Research, Development, and Innovation Office of Hungary (contract no. ED_18–1–2018–0003 to AB); by UC Berkeley start-up funding (to RDT); and by the Social Sciences and Humanities Research Council of Canada (grant no. SSHRC 435–2020–0089 to LB). Finally, we want to acknowledge the unseen effort of all the people that work protecting endangered languages and researchers’ efforts to make science available in different languages and formats.

Translation acknowledgements: The Spanish, Portuguese, and Hungarian translations were conducted using DeepL Translator ( www.deepl.com/translator ) and were proofread and edited by VRC, DYCB, and Brigitta Palotás, respectively. The French translation was conducted by Thibault de Poyferré who manually corrected translations that were initially generated with Google Translate and DeepL Translator. Additional translations are acknowledged in the repository linked in the supplemental material.

This article was the product of an interdisciplinary collaboration between scholars of evolution, ecology, and conservation (DYCB, VRC, RDT, ES, JTS, and AB) and a scholar of translation studies (LB). DYCB and VRC learned English as a second language while pursuing basic and higher education in Brazil and Colombia. They moved to the United States for graduate school, where they are constantly confronted with language barriers and recognize the additional burden that English dominance causes to their colleagues in South America and other regions. ES and RDT grew up speaking English; learned Spanish in high school and while conducting fieldwork, respectively; they acknowledge that advancing their research goals has been facilitated both by having English as a first language and by having collaborators who speak English. LB grew up speaking English, learned French and Spanish as part of her training to become a translator, and recognizes the advantages of being able to work in a native language in comparison to an additional language. JTS grew up as a native English speaker in the United States. She has learned multiple languages, which has facilitated working in countries in which English is not the primary language. However, she has found that being a native English speaker has provided many opportunities she otherwise would not have had. AB learned English as a second language during higher education in communist Hungary, where little importance was placed on the English language. When he started his career, he was handicapped by the lack of an English-speaking environment, in addition to the difficulties inherent in the fact that English is not closely related to Hungarian.

Author Biographical

Emma Steigerwald ( [email protected] ) is a doctoral candidate in the Museum of Vertebrate Zoology and the Department of Environmental Science, Policy, and Management, at the University of California, Berkeley, in Berkeley, California, in the United States. Valeria Ramírez-Castañeda is a doctoral candidate in the Museum of Vertebrate Zoology and the Department of Integrative Biology at the University of California, Berkeley, in Berkeley, California, in the United States. Débora Y. C. Brandt is a doctoral candidate in the Department of Integrative Biology at the University of California, Berkeley, in Berkeley, California, in the United States. András Báldi is a scientific advisor at the Institute of Ecology and Botany, at the Centre for Ecological Research, in Vácrátót, Hungary. Julie Teresa Shapiro is Zuckerman postdoctoral fellow in the Department of Life Sciences at Ben-Gurion University of the Negev, in Be'er Sheva, Israel. Lynne Bowker ( [email protected] ) is a professor in the School of Translation and Interpretation at the University of Ottawa, in Ottawa, Ontario, Canada. Rebecca D. Tarvin ( [email protected] ) is a professor at the Museum of Vertebrate Zoology and in the Department of Integrative Biology at the University of California, Berkeley, in Berkeley, California, in the United States.

Aguilar Gil YE . 2020 . Aguilar-Guevara A , Bravo Varela J , Ogarrio Bodillo G , Quaresma Rodríguez V , eds. Ää: Manifiestos sobre la Diversidad Lingüística . Almadía .

AlShebli BK , Rahwan T , Woon WL. 2018 . The preeminence of ethnic diversity in scientific collaboration . Nature Communications 9 : 5163 .

Google Scholar

Amano T , Sutherland WJ. 2013 . Four barriers to the global understanding of biodiversity conservation: Wealth, language, geographical location and security . Proceedings of the Royal Society B 280 : 20122649 .

Amano T , González-Varo JP , Sutherland WJ. 2016 . Languages are still a major barrier to global science . PLOS Biology 14 : e2000933 .

Amano T , et al . 2021a . Tapping into non-English-language science for the conservation of global biodiversity . PLOS Biology 19 : e3001296 .

Amano T , C Rios Rojas , Boum II Y , Calvo M , Misra BB . 2021 . Ten tips for overcoming language barriers in science . Nature Human Behaviour 5 : 1119 – 1122 .

Ammon U. 2012 . Linguistic inequality and its effects on participation in scientific discourse and on global knowledge accumulation: With a closer look at the problems of the second-rank language communities . Applied Linguistics Review 3 : 333 – 355 .

Angulo E , et al . 2021 . Non-English languages enrich scientific knowledge: The example of economic costs of biological invasions . Science of the Total Environment 775 : 144441 .

Baldi A , Palotas B. 2021 . How to diminish the geographical bias in IPBES and related science? Conservation Letters 14 : e12786 .

Barath H. 2019 . Indian initiatives aim to break science's language barrier . Nature 571 : 289 – 290 .

Bennett K. 2013 . English as a linguafranca in academia: Combating epistemicide through translator training . Interpreter and Translator Trainer 7 : 169 – 193 .

Bowker L. 2021 . Promoting linguistic diversity and inclusion: Incorporating machine translation literacy into information literacy instruction for undergraduate students . International Journal of Information, Diversity, and Inclusion 5 : 127 – 151 .

Bowker L , Ciro JB. 2019 . Machine Translation and Global Research: Towards Improved Machine Translation Literacy in the Scholarly Community . Emerald .

Google Preview

Canagarajah AS. 2002 . A Geopolitics of Academic Writing . University of Pittsburgh Press .

Chu C. , Wang R. 2020 . A survey of domain adaptation for machine translation . Journal of Information Processing . 28 : 413 – 426 .

Chu C , et al . 2021 . Conceptual and theoretical dimensions of biodiversity research in China: Examples from plants . National Science Review 8 : nwab060 .

Clavero M. 2010 . “ Awkward wording. Rephrase”: Linguistic injustice in ecological journals . Trends in Ecology and Evolution 25 : 552 – 553 .

Corcoran JN. 2015 . English as the International Language of Science: A Case Study of Mexican Scientists’ Writing for Publication . PhD dissertation . University of Toronto , Toronto, Canada .

Croft J. 2021 . Why translators should be named on book covers. Guardian (10 September 21) .

De Vos A. 2022 . Stowing parachutes, strengthening science . Conservation Science and Practice 4 : e12709 .

Eberhard DM , Simon GF , Fennig CD. 2022 . Ethnologue: Languages of the World 25th ed. Ethnologue . www.ethnologue.com .

Endangered Languages Project . 2022 . Endangered Languages Project: Supporting and celebrating global linguistic diversity . Endangered Languages Project. www.endangeredlanguages.com .

Fausey CM , Long BL , Inamori A , Boroditsky L. 2010 . Constructing agency: The role of language . Frontiers in Psychology 1 : 162 .

Forcada ML. 2017 . Making sense of neural machine translation . Translation Spaces 6 : 291 – 309 .

Fung IC. 2008 . Citation of non-English peer review publications: Some Chinese examples . Emerging Themes in Epidemiology 5 : 12 .

González Mellado CC , Lara Pineda FI , Lotito Bustos FC . 2020 . Aprendizaje del Idioma Inglés en Establecimientos Educacionales y su Relación con Diferentes Contextos Socioculturales . PhD dissertation . Viña del Mar University, Viña del Mar, Chile .

Gordin MD. 2015 . Scientific Babel: How Science Was Done before and after Global English . University of Chicago Press .

Haelewaters D , Hofmann TA , Romero-Olivares AL. 2021 . Ten simple rules for global north researchers to stop perpetuating helicopter research in the global south . PLOS Computational Biology 17 : e1009277 .

Hofstra B , Kulkarni VV , Munoz-Najar Galvez S , He B , Jurafsky D , McFarland DA . 2020 . The diversity–innovation paradox in science . Proceedings of the National Academy of Sciences 117 : 9284 – 9291 .

Huttner-Koros A , Perera S. 2016 . Communicating science in English: A preliminary exploration into the professional self-perceptions of Australian scientists from language backgrounds other than English . Journal of Science Communication 15 : A03 .

Hwang K. 2005 . The inferior science and the dominant use of English in knowledge production: A case study of Korean science and technology . Science Communication 26 : 390 – 427 .

Jara-Ettinger J , Rubio-Fernandez P. 2021 . Quantitative mental state attributions in language understanding . Science Advances 7 : eabj0970 .

Kellsey C , Knievel JE. 2004 . Global English in the humanities? A longitudinal citation study of foreign-language use by humanities scholars . College and Research Libraries 65 : 194 – 204 .

Khelifa R , Mahdjoub H. 2022 . Integrate geographic scales in equity, diversity, and inclusion . Nature Ecology and Evolution 6 : 4 – 5 .

Khelifa R , Amano T , Nuñez MA. 2022 . A solution for breaking the language barrier . Trends in Ecology and Evolution 37 : 109 – 112 .

Kieffer MJ. 2010 . Socioeconomic status, English proficiency, and late-emerging reading difficulties . Educational Researcher 39 : 484 – 486 .

Kim Y , Petrov P , Petrushkov P , Khadivi S , Ney H. 2019 . Pivot-based transfer learning for neural machine translation between non-English languages . Pages 866 – 876 in Inui K , Jiang J , Ng V , Wan X , eds. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Association for Computational Linguistics. .

Kincaid DW , Beck WS , Brandt JE , Mars Brisbin M , Farrell KJ , Hondula KL , Larson EI , Shogren AJ . 2020 . Wikipedia can help resolve information inequality in the aquatic sciences . Limnology and Oceanography Letters 6 : 18 – 23 .

Kirchik O , Gingras Y , Larivière V. 2012 . Changes in publication languages and citation practices and their effect on the scientific impact of Russian science (1993–2010) . Journal of the American Society for Information Science and Technology 63 : 1411 – 1419 .

Kirpotin SN , et al . 2021 . Impacts of environmental change on biodiversity and vegetation dynamics in Siberia . Ambio 50 : 1926 – 1952 .

Konno K , Akasaka M , Koshida C , Katayama N , Osada N , Spake R , Amano T. 2020 . Ignoring non-English-language studies may bias ecological meta-analyses . Ecology and Evolution 10 : 6373 – 6384 .

Liang L , Rousseau R , Zhong Z. 2013 . Non-English journals and papers in physics and chemistry: Bias in citations? Scientometrics 95 : 333 – 350 .

Lindsey P , Crusan DJ. 2011 . How faculty attitudes and expectations toward student nationality affect writing assessment . Across the Disciplines 8 : 1 – 19 .

Liu W. 2017 . The changing role of non-English papers in scholarly communication: Evidence from Web of Science's three journal citation indexes . Learned Publishing 30 : 115 – 123 .

Márquez MC , Porras AM. 2020 . Science communication in multiple languages is critical to its effectiveness . Frontiers in Communication 5 : 31 .

Martin P. 2009 . “ They have lost their identity but not gained a British one”: Non-traditional multilingual students in higher education in the United Kingdom . Language and Education 24 : 9 – 20 .

Mavrogenis AF , Quaile A , Scarlat MM. 2020 . The good, the bad and the rude peer-review . International Orthopaedics 44 : 413 – 415 .

Meadow AM , Ferguson DB , Guido Z , Horangic A , Owen G , Wall T. 2015 . Moving toward the deliberate coproduction of climate science knowledge . Weather, Climate, and Society 7 : 179 – 191 .

Montgomery SL , Crystal D . 2013 . Does Science Need a Global Language? English and the Future of Research . University of Chicago Press .

Moreno A. 2010 . Researching into English for research publication purposes from an applied intercultural perspective . Pages 59 – 73 in Ruiz-Garrido MF , Palmer-Silveira JC , Fortanet-Gómez I , eds. English for Professional and Academic Purposes. Utrecht Studies in Language and Communication , vol. 22 . Rodopi .

Munday J. 2016 . Introducing Translation Studies: Theories and Applications , 4th ed. Routledge .

Mur Dueñas P . 2012 . Getting research published internationally in English: An ethnographic account of a team of finance Spanish scholars’ struggles . Iberica 24 : 139 – 156 .

Neimann Rasmussen L , Montgomery P . 2018 . The prevalence of and factors associated with inclusion of non-English language studies in Campbell systematic reviews: A survey and meta-epidemiological study . Systematic Reviews 7 : 129 .

Nkomo D , Madiba M. 2012 . The compilation of multilingual concept literacy glossaries at the university of cape town: A lexicographical function theoretical approach . Lexikos 21 : 144 – 168 .

O'Neil D. 2018 . English as the lingua franca of international publishing . World Englishes 37 : 146 – 165 .

Ordóñez-Matamoros G , Cozzens SE , Garcia-Luque M. 2011 . North-South and South-South research collaboration: What differences does it make for developing countries? The case of Colombia . Pages 1 – 11 in Melkers J , Monroe-White T , Cozzens SE , eds. 2011 Atlanta Conference on Science and Innovation Policy . IEEE . doi:10.1109/ACSIP.2011.6064479

Pabón Escobar SC , da Costa MC . 2006 . Visibility of Latin American scientific publications: The example of Bolivia . Journal of Science Communication 05 : A01 .

Parra Escartín C , Goulet M-J. 2020 . When the post-editor is not a translator: Can machine translation be post-edited by academics to prepare their publications in English? Page 89 – 106 in Koponen M , Mossop B , Robert IS , Scocchera G , eds. Translation, Revision and Post-editing . Routledge .

Qaiser F. 2021 . Let's be clear(er): The language of science needs to be plain and simple. Evidence for Democracy (27 July 2021) . https://evidencefordemocracy.ca/en/content/lets-be-clearer-language-science-needs-be-plain-and-simple .

Ramírez-Castañeda V. 2020 . Disadvantages in preparing and publishing scientific papers caused by the dominance of the English language in science: The case of Colombian researchers in biological sciences . PLOS ONE 15 : e0238372 .

Romero-Olivares AL. 2019 . Reviewers, don't be rude to nonnative English speakers . Science (3 October 2019) .

Rovira C , Codina L , Lopezosa C. 2021 . Language bias in the Google Scholar ranking algorithm . Future Internet 13 : 31 .

Roy D , Bhatia S , Jain P. 2021 . Information asymmetry in Wikipedia across different languages: A statistical analysis . Journal of the Association for Information Science and Technology 73 : 347 – 361 .

Saha S , Afrad MMH , Saha S , Saha SK. 2019 . Towards making global health research truly global . Lancet Global Health 7 : e1175 .

Schofield P , Mamuna G. 2003 . The relationship of socio-economic status and length/medium of English instruction with individual differences and English proficiency in Pakistan . Journal of Research 3 : 1 – 28 .

Shailes S. 2017 . Plain-language summaries of research: Something for everyone . Elife 6 : e25411 .

Siok WT , Kay P , Wang WSY , Chan AHD , Chen L , Luke K-K , Tan L. H. 2009 . Language regions of brain are operative in color perception . Proceedings of the National Academy of Sciences 106 : 8140 – 8145 .

Sulubacak U , Caglayan O , Gronroos S-A , Rouhe A , Elliot D , Specia L , Tiedemann J. 2020 . Multimodal machine translation through visuals and speech . Machine Translation 34 : 97 – 147 .

Suzina A. 2021 . English as lingua franca . Or the sterilisation of scientific work . Media, Culture, and Society 43 : 171 – 179 .

Tardy C. 2004 . The role of English in scientific communication: Lingua franca or Tyrannosaurus rex? Journal of English for Academic Purposes 3 : 247 – 269 .

Tonkin H. 1987 . One hundred years of Esperanto: A survey . Language Problems and Language Planning 11 : 264 – 282 .

translatE . 2020 . List of non-English-language journals in ecology and conservation . translatE (17 November 2020). https://translatesciences.com/list-of-non-english-language-journals-in-ecology-and-conservation-uploaded .

[ UNESCO] United Nations Educational, Scientific, and Cultural Organization . 2021 . UNESCO recommendation on open science. UNESCO . https://en.unesco.org/science-sustainable-future/open-science/recommendation .

Vaidyanathan G. 2019 . No paper, no PhD? India rethinks graduate student policy . Nature (31 May 2019). www.nature.com/articles/d41586-019-01692-8 .

von Gizycki R. 1973 . Centre and periphery in the international scientific community: Germany, France and Great Britain in the 19th century . Minerva 11 : 474 – 494 .

Way A. 2020 . Machine translation: Where are we at today? Pages 311 – 332 in Angelone E , Ehrensberger-Dow M , Massey G , eds. The Bloomsbury Companion to Language Industry Studies . Bloomsbury .

Wild S. 2021 . African languages to get more bespoke scientific terms . Nature 596 : 469 – 470 .

Supplementary data

Email alerts, citing articles via.

• Recommend to your Library

Affiliations

• Online ISSN 1525-3244
• Copyright © 2024 American Institute of Biological Sciences
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

• January 2019
• January 2018
• October 2018
• January 2017
• October 2017
• January 2016
• October 2016
• January 2015
• October 2015
• July 2014 Issue
• October 2014 Issue
• Previous issues
• Translators
• Video Blogs
• Press Releases
• Translation Agencies Interested in listing in the Translation Directory? Click this link for more info.
• Product & Service Providers
• Freelance Translators / Interpreters

Featured Articles

• Book Recommendation

FREE e-Books

• Events Calendar
• Industry News
• Google Hangouts JOIN US!!
• Google Review Handout
• Conference News
• General Inquiries
• Submit an Article Call for Papers and Editorial Policies
• Get to Know Our Community Members
• Tell Us About Yourself

Translation Journal

Digital Online Journal - Established 1997

A digital online journal for translators, interpreters , and interested friends of the translation industry.

I am a Translator   Read the Journal

Join Translation Journal

Click on the Subscribe button below to receive regular updates.

Featured Article

5 reasons you may not hire bilingual person instead of professional interpreter.

A bilingual person is the one who knows both the languages involved in the conversation and is able to understand as well as translate the same. So, the question will come to anybody’s mind that why such bilingual person cannot work as the interpreter! The logic is correct but practically a bilingual person would not be the right alternative for professional interpreter. Read More

Industry Knowledge - Conferences Present Brilliant Opportunities

Contribute to TJ

Featured Contributors

Click on the image below to see the works of featured contributor

Download, Read & Enjoy!

Get e-Books

Our Recommendations

Products & Services

for Translators & Interpreters

Course Corner

e-Learning courses

*NEW Courses

Our Community

Translators & Interpreters

The Translation Journal is in an online journal for translators and interpreters and friends of the industry. The articles are written by translators, interpreters and industry experts and has been published online for over 17 years! It is platform to spotlight the translators talents and achievements and this platform aims to be a source of high quality materials specifically dedicated to help translators and interpreters more productive, to make more money, and to provide more service to their clients.

Be Part of this Community

Submit an article, or subscribe to Translation Journal for free or share your knowledge with your colleagues or post a question in the Forum. .

How-To Guides

Want to succeed focus on systems rather than goals +.

How are you coming along with setting goals for 2016? Did you block out some time in your calendar this past week to plan how you want your business and personal life to look at the end of 2016? I hope you are well on your way!

How to Get (and Use) Testimonials +

Testimonials can work magic for your business by building trust and credibility with prospective clients. But like any magic, they must be used responsibly.

How to Send Large Files +

Have you ever had trouble sending a file attachment by email and wondered why? Chances are the file was too large. Most mail servers have size limitations for both incoming and outgoing files, with a maximum of about 20 MB.

Translators and interpreters must often create and send large files to clients, such as InDesign files that include high-resolution photos and videos. Because of their size, simply sending these files through an email service provider is not an option.

The good news is that there are many different file-sharing solutions available on the Internet that allow anyone – with or without technical expertise – to transfer files of any size with ease.

9 Free Tools to Convert Documents to PDFs +

An added feature is that most PDF programs also allow you to put on extra security measures such as watermarks, passwords, or data encryption.

How to Set Your Rate as a Freelance Translator +

It’s the most common question I hear when talking with translators and interpreters: How much should I charge? The question comes from everywhere in the industry, from the recent graduate just starting out in a career in translation and interpreting to the seasoned professional who needs to renegotiate his or her rates with a client.

Testimonials: What to Include and How to Display +

Testimonials give you the opportunity to provide proof about your services. Make it easy for potential clients to see what others are saying about you and your company by adding testimonials to your website and marketing materials.

How to Get Premium Rates for Your Services +

To get top dollar for your services, you need to be strategic about where you market yourself. You may be able to offer your usual services to a different audience and double or triple your income – without working more! There are, however, a few things to consider when strategically targeting clients.

October 2018 Issue - NEW!!

• FICTION TRANSLATION PROBLEMS
• Reading Comprehension: Blog Exposure in the English as a Foreign Language (EFL) Context of Iran
• Book Review of Éric Poirier and Daniel Gallego-Hernández’s Business and Institutional Translation: New Insights and Reflections
• RABINDRANATH TAGORE AND TRANSLATION STUDIES
• A Reflexion on the Murky Difference between Adaptation and Translation

January 2019 Issue - NEW!!

• ADEQUACY PROBLEMS IN TRANSLATION
• Translation and Censorship
• Professionalità in International and Comparative Research: An Untranslatable Concept?
• Review of “Quand le pou éternuera”* (Quand les poules auront des dents, en Ukraine): Expressions des peuples. Génie des Langues, authored by Muriel Gilbert
• Igbo-English Fictional Translation and Igbo-English Bilingualism: Animals, Fish and Plants Related Terms in Bell-Gam Translated By P.J. Ezeh

P.S. You look really good today. 

Remember me

• Forgot login?

• University of Michigan Library
• Research Guides
• Translation Studies
• Translation Studies journals
• Literature in translation
• Works about translation
• Translation resources online
• Dictionaries

Journals in Translation Studies

To find Translation Studies journals available through the library, search Library Catalog Search for the subject Translating and Interpreting Periodicals " . To locate a more exhaustive list of such journals, you may use the same subject to search the WorldCat .

Important peer-reviewed journals:

• Babel: Revue international de la traduction
• Exchanges: A Journal of Literary Translations
• The Journal of Specialised Translation
• Meta: Translators' Journal
• New Voices in Translation Studies
• Perspectives: Studies in Translation Theory and Practice
• International journal of translation and interpreting research
• Translation and Interpreting Studies (TIS)
• Translation and Literature
• Translation Journal: TJ
• Translation Studies Abstracts / Bibliography of Translation Studies
• The Translator
• Zhongguo fan yi = Chinese Translators Journal

Translation journals published at the University of Michigan:

• Absinthe: World Literature in Translation Publishes foreign literature in English translation, with a particular focus on previously untranslated contemporary fiction, poetry and creative non-fiction by living authors; edited by graduate students in the Department of Comparative Literature.
• Canon Translation Journal Online literary magazine dedicated to promoting and publishing undergraduate and graduate translation.

• Focus and Scope
• Publication Ethics
• Article Guidelines
• Book Review Guidelines
• Peer Review Process
• Editorial Team
• Online Submission
• Screening for Plagiarism
• Copyright Notice
• Journal History
• Indexing & Repository
• Register Existing Account

CITATION ANALYSIS

• Scopus Citation (Last 5 Years)
• Scopus Citation
• Google Scholar
• Author Guidelines
• Review Form for Reviewer
• Revision Matrix for Author
• Review Flowchart
• Accreditation Documents
• Announcements
• Upcoming Issues

The Influence of Translation Techniques on the Accuracy and Acceptability of Translated Utterances that Flout the Maxim of Quality

Translating the implied meanings in utterances is one of the trickiest situations translators may confront. One example is in translating utterances that flout the maxim of quality. When a speaker flouts the maxim of quality, they are implying further information that is not represented in the utterance. Translators use various translation techniques in order to convey the meaning of the original text in the most appropriate and acceptable form in the target text. This study investigates how translation techniques may affect the quality of a translation. The approach implemented in this study is pragmatics in translation. This study belongs to the field of descriptive qualitative research with an embedded case study. For data, we look at all the utterances which may be said to be flouting the maxim of quality in the source text and its translation. Content analysis and focus group discussion were applied as the methods to collect and analyze the data. A focus group discussion was used to assess translation quality. The majority of the data was classified as accurate and acceptable, while the rest was considered less accurate and inaccurate due to the application of the translation technique amplification (addition), discursive creation and literal translation. Some data was also found to be less acceptable due to literal translation and pure borrowing.

Abdeelah, A. S. (2004). The translators dilemma – Implicatures and the role of the translator. Translation Journal, 8 (2), 54-63.

Akhiroh, N. S. (2013). The influence of translation technique on the quality of the translation of international news in Seputar Indonesia daily. Language Circle Journal of Language and Literature, 7 (2), 41-51.

Black, E. (2006). Pragmatic Stylistic. Edinburgh: Edinburgh University Press Ltd.

Brown, P., & Stephen, L. (1987). Politeness: Some universals in language usage. Cambridge: Basil Blackwell.

Cutting, J. (2002). Pragmatics and discourse: A resource for students. New York: Routledge.

Grice, H. P. (1975). Logic and conversation. New York: Oxford University Press.

Hatim, B., & Jeremy, M. (2004). Translation: An advanced resource book. UK: Psychology Press.

Larson, M. L. (1984). Meaning-based translation: A Guide to cross-language Equivalence . New York: University Press of America.

Molina, L., & Albir, A. (2002). ‘Translation techniques revisited: A dynamic and functionalist approach’. Meta: Translators’ Journal, 47 (4), 499-512.

Moyes, J. (2012). Me before you, [PDF]. https://bookwormbyheart.blogspot.co.id/2016/03/jojo-moyes-book-collection-kindle-epub.html, accessed on November 17 2016.

Moyes, J. (2016). Me Before You. (translated from English into Indonesian by Lesmana, T. Sebelum mengenalmu). Jakarta: PT Gramedia Pustaka Utama.

Nababan, M. R. (2003). Teori menerjemah bahasa inggris. Yogyakarta: Pustaka Pelajar.

Nababan, M. R., Nuraeni, A., & Sumardiono. (2012). ‘Pengembangan model penilaian kualitas terjemahan’. Kajian linguistik dan sastra, 24 (1), 39-57.

Newmark, P. (1988). A textbook of translation. London: Prentice Hall.

Nida, E. A. (1964). Towards a Science of Translating. Leiden: E. J. Brill.

Nugraha, S. K. (2016). Kajian terjemahan kalimat yang merepresentasikan tuturan pelanggaran maksim pada subtitle film ‘The Queen’ (Kajian terjemahan dengan pendekatan pragmatik) . Thesis . Sebelas Maret University: Surakarta.

Nuraeni, Ardiana., Kusumawati, F., & Nababan, M. R. (2016). A Translation Study on School Signboards in Surakarta: Types, Functions, and Quality. Humaniora, 28 (2), 198-207.

Paltridge, B. (2006). Discourse Analysis. London: MPG Books Ltd.

Putri, R. (2016). Terjemahan turn yang mengakomodasi flouting maksim prinsip kerjasama dalam novel The Cairo Affair karya Olen Steinhauer . Thesis . Sebelas Maret University: Surakarta.

Sudaryanto. (1986). Metodologi penelitian linguistik, ke arah memahami metode linguistik. Yogyakarta: Gajah Mada University Press.

Sumardiono. (2012). Kajian terjemahan ujaran yang menagandung implikatur pada novel The Da Vinci Code (Sebuah tinjauan pragmatik pada penerjemahan) . Thesis . Sebelas Maret University: Surakarta.

Spradley, J. P. (1980). Participant observation . New York: Holt, Reiinheart and Wiinson.

Thomas, J. (1995). Meaning in Interaction: an Introduction to Pragmatics. England: Longman.

Wijana, I. D. P., & Muhammad, R. (2011). Analisis wacana pragmatik : kajian teori dan analisis. Surakarta: Yuma Pressindo.

Zhonggang, S. (2006). A Relevance Theory Perspective on Translating the Implicit Information in Literary Texts. Journal of Translation, 2 (2), 41-60.

Article Metrics

• There are currently no refbacks.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Proc Natl Acad Sci U S A
• v.118(21); 2021 May 25

The intricate relationship between transcription and translation

Michael w. webster.

a Department of Integrated Structural Biology, Institut de Génétique et de Biologie Moléculaire et Cellulaire, F-67404 Illkirch Cedex, France;

b Université de Strasbourg, F-67081 Strasbourg Cedex, France;

c CNRS UMR 7104, F-67404 Illkirch Cedex, France;

d INSERM U1258, F-67404 Illkirch Cedex, France

Albert Weixlbaumer

Author contributions: M.W.W. and A.W. wrote the paper; and M.W.W. made Fig. 1 .

Two conserved processes express the genetic information of all organisms. First, DNA is transcribed into a messenger RNA (mRNA) by the multisubunit enzyme RNA polymerase (RNAP). Second, the mRNA directs protein synthesis, when the ribosome translates its nucleotide sequence to amino acids using the genetic code. Because these two processes are so fundamental, a multitude of regulatory processes have evolved to regulate them. Most examples involve regulation of either transcription or translation. In PNAS, Chatterjee et al. ( 1 ) instead describe a complex and intricate regulatory process in which transcription and translation are concurrently regulated by each other.

Transcription and translation are commonly viewed as separate. In eukaryotes, their respective confinement to the nucleus and cytoplasm enforces this. Yet, prokaryotes have no such barrier, and newly synthesized mRNAs are translated while they are still being transcribed. RNAP and the trailing ribosome are therefore in close spatial proximity, allowing each to influence the activity of the other. The possibility of a physical connection that could support functional coupling was proposed in 1964 by Marshall Nirenberg’s laboratory based on biochemical experiments ( 2 ). They highlighted the potential importance of regulatory processes that simultaneously affect both transcription and translation. Electron micrographs of ruptured Escherichia coli cells, commonly termed “Miller spreads,” confirmed the close proximity between RNAP and the trailing ribosome ( 3 ).

The role and mechanism of coupling have received renewed interest over the past 10 y. Biochemical and structural approaches alongside new measurements of gene expression rates in vivo have clarified several important aspects. Early studies had demonstrated that translation can release RNAP from regulatory pauses ( 4 ). This mechanism, part of a process known as attenuation, had been described in the context of the leader sequences of specific operons. Yet more recent evidence points to additional genome-wide mechanisms of translation promoting transcription: the trailing ribosome pushing RNAP forward along the gene ( 5 , 6 ). RNAP pauses regularly when it encounters specific DNA sequences and can slide backward. A forward translocating ribosome could thus minimize the formation and aid in the release of transcriptional pauses. This could explain the synchronization of transcription and translation rates observed in E. coli ( 5 ). It is also essential to fitness, as translation maintains genome stability by releasing arrested transcription complexes that would otherwise interfere with DNA replication ( 7 ).

The molecular architecture that occurs during pause release likely resembles recent structures of ribosome–RNAP complexes determined with short intervening mRNAs ( 8 – 10 ). This supramolecular assembly has been termed the “expressome.” The expressome is dynamic and adopts a distinct arrangement when the transcription factor NusG is present ( 9 , 10 ). By simultaneously binding the ribosome and RNAP, NusG acts as a physical bridge. This minimizes the formation of mRNA secondary structures that could inhibit transcription and translation and also sequesters a NusG domain that promotes transcription termination. Impairment of the NusG–ribosome interaction impacts coupling in vivo ( 11 ). RfaH, another member of the NusG family, also has the ability to bind both RNAP and the ribosome, but the consequences are less-well-understood ( 12 ).

In PNAS, Chatterjee et al. ( 1 ) shed further light with an analysis of factors that drive the establishment of coupled translation at an early stage of transcription. Using biochemical and single-molecule fluorescence analyses, the interplay between ribosome recruitment and rates of transcription and translation are examined. Further, how each of these is modulated by a regulatory riboswitch is tested. Previous work has focused on the regulation of RNAP by translation elongation, and here a link is demonstrated between RNAP and translation initiation: an interesting aspect of transcription–translation coupling.

Transcription of the mRNA studied by Chatterjee et al. ( 1 ) undergoes a programmed pause after ∼100 nucleotides have been synthesized, a common feature of E. coli transcripts ( 13 ). RNAP waits for translation initiation to occur and coupling to be established. This is analogous to the pause sequences upstream of attenuation sites, where coupling controls transcription termination ( 14 ). Chatterjee et al. reveal that recruitment of the small ribosomal subunit to the mRNA is stimulated by the paused RNAP and further promoted by NusG. This extends the regulatory role of NusG beyond transcription to translation initiation. Chatterjee et al. also present evidence that the translating ribosome then stimulates RNAP release from the pause and increases transcription rates. This is consistent with the notion that the transcriptional pause allows time for translation to commence ( Fig. 1 ) and supports observations made in vivo ( 5 , 15 ).

Model of the establishment of transcription–translation coupling. Recruitment of the ribosome 30S subunit is enhanced by paused RNAP and further promoted by coupling factors NusG or RfaH that bind both complexes. Translation then prompts RNAP to resume transcription with an increased rate.

The contribution of RNAP and NusG or RfaH relative to the mRNA ribosome binding site is examined by Chatterjee et al. ( 1 ) using a riboswitch. Riboswitches are structured regulatory RNA elements involved in sensing pH, temperature, and the presence of metabolites. Riboswitches can affect transcription and translation as a function of their ligands ( 16 – 18 ). The Bacillus subtilis riboswitch used by Chatterjee et al. senses the concentration of a nucleotide precursor preQ 1 . Upon preQ 1 -induced folding, the riboswitch masks the ribosome binding site. This reduces translation and, in part as a consequence of the role of translation in stimulating transcription, reduces transcription. Comparisons of the unfolded and folded riboswitch also revealed that the role of NusG in recruiting the small ribosomal subunit depends on the ribosome binding site of the mRNA’s being accessible. Interestingly, recruitment of the small subunit by the NusG paralog RfaH did not depend on this. This is suggestive of an unexpected mechanistic difference between the factors.

The results presented by Chatterjee et al. ( 1 ) add another layer to the intricate relationship between RNAP and the ribosome. The relationship evidently extends beyond a simple cooperative forward push of the two machineries along the gene. Many exciting questions arise. How does RNAP promote ribosome recruitment? An interaction between RNAP and the small ribosomal subunit was previously structurally characterized ( 19 ) and could be involved. The importance of the transcription factors NusG, which was investigated here, and others such as NusA, which was not, remains to be elucidated. How the ribosome releases RNAP from paused states remains an important outstanding question. Transcription pausing occurs in at least three ways: inhibitory mRNA hairpins such as those located upstream of attenuators, RNAP back-tracking, and thermodynamically driven “consensus” pauses. The ribosome’s ability to release RNAP from hairpin-induced and back-tracked pauses has previously been demonstrated ( 4 , 6 ). Chatterjee et al. now present evidence of release from a pause sequence resembling the consensus. It remains to be tested whether the ribosome releases RNAP from each pause type by shared or distinct mechanisms.

Physical coupling of RNAP to the ribosome is not the only mechanism of functional coordination between them. For example, reduced global translation rates direct reduced transcription via the alarmone (p)ppGpp ( 20 , 21 ). The contribution of different types of coupling in different contexts is an important question. The ubiquity of transcription–translation coupling across species also remains unresolved. Concurrent transcription–translation was observed in archaea ( 22 ), suggesting similarities across prokaryotic domains, yet not all bacteria display physical coupling. In B. subtilis , transcription outpaces the trailing ribosome ( 23 ). Expressome formation in Mycoplasma pneumonia appears to predominantly involve the transcription factor NusA and not NusG, as it does in E. coli ( 24 ). Addressing these and further questions will require complementary approaches. Classical biochemistry and single-molecule studies will need to be combined with high-resolution structural biology and genome-wide approaches to study the effects of coupling and the effect of its disruption.

A reductionist approach has proved powerful to the study of gene expression mechanisms. Transcription and translation have been studied in great detail for several decades in isolation. However, many key enzymes in gene expression and beyond are organized in multicomponent, supramolecular assemblies. Here new functions emerge, which are not necessarily predictable from the sum of their parts. It is exciting to see that we no longer neglect this aspect and begin to address the next level of structural organization to learn about the biological roles and functional implications of these higher-order assemblies.

Acknowledgments

M.W.W. and A.W.’s research is supported by an EMBO (European Molecular Biology Organisation) long-term fellowship to M.W.W., by the European Research Council starting grant TRANSREG (679734) to A.W., and by the Agence Nationale de la Recherche grant PrTxConf (ANR-17-CE11-0042). We are also supported by the French Infrastructure for Integrated Structural Biology (FRISBI ANR-10-INBS-05, Instruct-ERIC, and grant ANR-10-LABX-0030- INRT, a French State fund managed by the Agence Nationale de la Recherche under the program Investissements d’Avenir ANR-10- IDEX-0002-02).

The authors declare no competing interest.

See companion article, “A translational riboswitch coordinates nascent transcription–translation coupling,” 10.1073/pnas.2023426118 .