Round-trip errors, a throwaway head, three Slavic languages

Spent a chunk of this year on grammatical error correction for Russian, Ukrainian, and Slovene with a research partner. MultiGEC-2025 task. Three low-resource Slavic languages. The same wall in all three: not enough labeled errors. Two ideas did the work.

SeLex-RT: manufacture errors by translating in a circle. Take a clean sentence. Push it through a pivot language with NLLB-200. Translate it back. The round trip introduces realistic learner-style errors, and you get aligned (corrupted, clean) pairs for free. SimAlign over multilingual BERT does the token alignment. Corruption rate 15%.

The interesting knob is the pivot. We ran an ablation across close, medium, and distant pivots. For Russian: Bulgarian, English, Chinese. Pivot distance reshapes the error distribution. Closer pivots produce lexical and morphological errors. Distant pivots produce broader syntactic divergence. Different pivots are different teachers, and the close one isn’t always the best teacher.

Output: 2.8M synthetic (corrupted, clean) pairs across three languages and three pivot distances, at gabelev/selex-rt-outputs.

A detection head you throw away. While fine-tuning mBART-50 to correct text, we bolt an 11-class ERRANT-style token-level classifier onto the encoder. Both losses train jointly. The detection signal pulls the encoder toward representations that know where the errors are. At inference, drop the head and run vanilla mBART. Zero serving cost. Better dev ERRANT.

Numbers. Single A40 (48 GB). mBART-50 at 610M parameters. Seed 42 throughout. Scored with the official MultiGEC-2025 GLEU + ERRANT pipeline. Best dev ERRANT F0.5: 0.323 on Russian (+RT+Det λ=0.1), 0.333 on Slovene (distant pivot). No distributed training. No exotic hardware.

Both moves transfer. You can synthesize labeled data for any low-resource task by routing through a detour. The auxiliary-task trick borrows signal from a related labeling effort without paying for it at inference. The Slavic part of this project is the contingent part. The two moves are what travel.

Code, configs, result tables, and the long-form research log at MultiGEC-public.