Understanding Translation Quality Score

Smartcat Translation Quality Score (TQS) is a measurement tool integrated into the Smartcat platform and will help you understand the quality of your AI translations. It helps you to understand which AI translations meet your quality requirements and which require your review and possible revision with editing (either you or one of your reviewers within your team.

When you are within your workspace, you will be able to see this feature within your Project Overview page in the form of a widget on the right-hand side. The widget will show you how many translation segments within your projects have received a good score. The segments that have a high score can be confirmed without review and editing.

Furthermore, you will be able to see which of the translation segments received a low score. These will require human review and possibly editing. Once editing is complete, you can confirm these segments. You can either complete the review and editing process yourself, invite a member of your team to do so, or hire a subject-matter-expert reviewer from Smartcat Marketplace with one-click best-match AI sourcing.

All edits made by the assigned reviewer will be stored in your translation memory and Smartcat AI will learn from these edits and see them to increase quality for all future translations.

To further automate the process of confirming segments with good and satisfactory translation quality, you set up a specific pre-translation rule, which we will describe further below in this article.

Translation quality score is a number ranging from 0 to 100. Essentially, it provides a quick overview of how good your translation is. 

All translation segments are classified in two different categories: “great quality” and “needs review”. This serves to provide a clear indication to project managers and reviewers on which action should be taken in respect to each segment:

• Segments with a quality score that is above the specific threshold are considered to be good and can be confirmed, either manually or automatically.

• Segments with a quality score below that threshold are recommended to be reviewed by a team member to improve the translation quality.

Currently, the translation quality score can only assess the quality of the AI translation. Translations that are auto-propagated from the translation memory as well as human-reviewed translations receive a maximum score (for proper statistic calculations).

You don't actually have to do anything to activate TQS. It appears automatically, for you to view on two separate areas of your translation workspace on Smartcat.

1. Project level widget. A side-screen widget that displays overall quality. This display is to provide fast, easy access to overall quality information.

2. Segment level. For a more detailed, micro view of each specific text segment inside a translation file.

Smartcat highlights your translation quality score for a given language pair or for a translation project overall. This acts as a means to understand the quality level of your translation and help you decide how to proceed. Both AI translation and manual translation may require adjustments.

Here are three steps to improve translation quality score.

Step 1: Address any immediate issues

Ensure the translation has no obvious mistakes. Smartcat highlights critical issues such as lack of formatting tags or placeholders, mistranslation of glossary terms, or punctuation issues.

Step 2: Proofread to compare source and target texts

Ensure the translation accurately reflects the source text with a proofreading step. 

Some issues to look out for include making a translation too short or too long, or over-translating – adding extra content or explanations that were not there in the source. Such issues, which are not faithful to the source text, result in a lower translation quality score.

Step 3: Ensure target formatting and punctuation is accurate

Proofreading doesn't just apply to language. It is essential to also ensure that punctuation and formatting mirrors the source language file.

For instance, providing a translation in upper case while the source is lower case negatively affects translation quality score.

Additionally, when proofreading, aim to guarantee translation fluency and that it uses proper grammatical gender.

Hire a language editor or proofreader from Smartcat Marketplace by clicking on the on-screen Invite editor button.

In your Smartcat workspace, you can review translations yourself or invite your colleagues to carry out this step. You can also hire a professional linguist from Smartcat Marketplace. All users can collaborate on the same files, inside the same private workspace.

In all cases, translation quality score is the ultimate measure of the quality of the translation work performed and the faithfulness of the translated target language to the original source language.

Here is how you can set up AI-driven editing workflows to reduce the review and editing workload for internal reviewers by setting up the automatic approval of translations with high-quality scores.

1. First, go to Projects from your workspace home page.

2. Then, select a project.

3. Click on Translation Rules from the left-hand menu.

4. Add a new rule.

5. Choose Quality-based automation.

6. Select at which stage of the translation workflow requires the automatic confirmation.

7. Choose the TQS (Translation Quality Score) Threshold. 

8. Select Save and Run to ensure the changes come into effect.

9. In Smartcat Editor, you can see that all translation segments with a translation quality score that were higher than the threshold were automatically confirmed by the system. The reviewer only needs to verify the remaining segments.

Click through the interactive demo: