# Finetuning T5 on translation task

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffinetuning-t5-on-translation-task%2F9878%3E%5C
**Category:** 🤗Transformers\
**Created:** [September 10, 2021, 9:57am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffinetuning-t5-on-translation-task%2F9878 "2021-09-10T09:57:28Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![pere](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/pere/32/40613_2.png) [@pere](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpere)%5C
**Post date:** [September 10, 2021, 9:57am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffinetuning-t5-on-translation-task%2F9878%2F1 "2021-09-10T09:57:28Z")

</div>

I am finetuning T5 on a translation task. I am using Flax.

The translation task is between two fairly similar dialects.

Watching how the quality of the translation progresses, I see that is starts with predicting “”. Then it calculates the loss between and , and then slowly after several iterations gets to something that is fairly similar to , and then starts improving after finally ending up with a translation. Even if I end up with something that is qualitatively decent (and a BLEU-score of above 80), it still seems slow and “unstable”.

Since this is a seq2seq model, I guess it already has a fairly good way of doing source-\>encode/decode-\>source, and then calculating loss based on [encoded-decoded] source vs target, instead of calculating the loss based on vs .

I am fairly new to these models. Does this make sense?