Frequently Asked Questions
Questions about Kashmiri NLP, answered
Everything you need to know about Kashmiri NLP, machine translation, and how to participate in our research.
Q.What is Kashmiri machine translation?
Kashmiri machine translation is the automated conversion of text between Kashmiri and other languages, primarily English, using neural models. It is hard for three specific reasons: Kashmiri has rich verbal morphology and pronominal cliticisation, it is written in Perso-Arabic Nastaliq whose diacritics carry roughly sixteen vowel distinctions and are often omitted, and there is very little parallel data. Standard tokenizers trained on Latin-script text fragment it badly.
Q.Is Kashmiri supported by Google Translate?
No. As of 2026 Google Translate does not support Kashmiri. Kashmiri is covered by some multilingual research models — Meta's NLLB-200 and AI4Bharat's IndicTrans2 both include it — but it sits at the data-scarce tail of both. Among consumer services, Microsoft Translator lists Kashmiri, and Kashmir AI Research runs a free research-preview translator at kashmirairesearch.online/translate.
Q.Which model translates Kashmiri to English best?
On the ICRTC-2026 paper's test set (drawn from the 124,102-pair corpus), IndicTrans2-1B scores highest at 52.88 BLEU, followed by NLLB-200-distilled-600M at 31.57, then a QLoRA-fine-tuned Mistral-7B at 13.89. IndicTrans2's figure should be treated as a ceiling rather than a fair zero-shot result, because our corpus draws on AI4Bharat BPCC and likely overlaps its pre-training data. The practical finding is that a purpose-built 600M translation model beats a fine-tuned 7B general-purpose LLM on every metric.
Q.Why do large language models fail at Kashmiri?
The ICRTC-2026 paper pointed to tokenization: Mistral's BPE tokenizer produces 4.06 times more tokens per Kashmiri word than per English word (5.89 against 1.45), while NLLB-200's multilingual SentencePiece reaches near parity (2.50 against 1.40). Follow-up work (2026) is re-testing these results; the details will appear in a follow-up paper.
Q.How much parallel data does a language like Kashmiri need?
The ICRTC-2026 paper reported that a model trained on roughly 30K pairs scored 0.46 BLEU against 13.89 for 125K pairs, and read this as a threshold for cross-lingual transfer. Follow-up work (2026) is re-testing these results; the details will appear in a follow-up paper.
Q.Can LLM translation be improved without retraining the model?
The ICRTC-2026 paper reported that constraining generation length at decode time (capping output at twice the source word count, with a repetition penalty) gained 12.75 BLEU and cut over-generation from 97% to 39%, with no change to model weights. Over-generation means the model does not stop when the translation is complete. Follow-up work (2026) is re-testing these results; the details will appear in a follow-up paper.
Q.How big is the Kashmir AI Research parallel corpus?
124,102 Kashmiri–English sentence pairs. 97,920 come from AI4Bharat's BPCC kas_Arab collection (98,923 before filtering), which is web-crawled news and government text, and 26,182 from the SMUQamar corpus, which is colloquial and educational. It is deduplicated (exact matches), bounded to 5 to 1,000 characters per side, and requires more than 20% Perso-Arabic characters on the Kashmiri side; semantic similarity is measured on a sample as a quality check, not used as a filter. The split is 117,896 training and 6,206 evaluation pairs.
Q.Has this research been published?
Yes. "Anatomy of Decoder-Only LLM Failure in Low-Resource Machine Translation: Tokenizer Fertility, Data Thresholds, and Decoding Strategies for Kashmiri" by Faizan Ayoub and Neha Prerna Tigga received the Best Paper Award at ICRTC-2026, the 14th International Conference on Recent Trends in Computing, held 3–4 July 2026 at SRM Institute of Science and Technology, Delhi-NCR, in association with Springer.
Q.Where can I find Kashmiri language datasets and models?
The Kashmir AI Research resource hub catalogues 205 of them in one place — 84 datasets, 36 papers, 60 models, 17 tools and 8 research groups, spanning 1932 to 2026, filterable by task and year. It covers corpora, OCR and speech datasets, translation and text-to-speech models, and the institutions producing them, including AI4Bharat, the University of Kashmir, NIT Srinagar, CFILT and LDC-IL. Anyone can submit a missing resource.
Q.How does the Kashmir AI Research evaluation platform work?
Native Kashmiri speakers are shown a Kashmiri sentence and two anonymised English translations, and rate each for adequacy and fluency on a five-point scale before choosing an overall preference. Model identities are hidden and left-right placement is re-randomised for every sentence, so neither the model nor its screen position can bias a rating. Time per judgement is recorded so implausibly fast responses can be filtered, and inter-annotator agreement is tracked with Cohen's weighted kappa. To our knowledge it is the first evaluation tool built specifically for Kashmiri machine translation.
Q.Do I need technical skills to participate as an evaluator?
None. If you can read Kashmiri and understand English, you can take part, and you do not need an account with any institution. A session takes ten to fifteen minutes. Automated metrics like BLEU penalise valid Kashmiri paraphrases, which is exactly why native-speaker judgement is the part no model can supply.
Q.What is a low-resource language in NLP?
A language with very little digital text, few annotated datasets, and almost no language technology tooling. Kashmiri qualifies on all three counts: it is excluded from most NLP benchmarks, has no consumer translation product, and until recently had no standard evaluation infrastructure. Roughly half of the world's 7,000-plus languages sit in this category.
Q.What is LLM fine-tuning for Kashmiri?
In this research it means adapting Mistral-7B-Instruct-v0.2 to Kashmiri-to-English translation using QLoRA — 4-bit quantisation with low-rank adapters, about 23 million trainable parameters, trained for 1,000 steps on two NVIDIA T4 GPUs in roughly four hours. NLLB-200 and IndicTrans2 were not fine-tuned; they were evaluated zero-shot as baselines.
Q.Will the Kashmiri dataset be publicly available?
Not as a merged file: the sources are public, but the merged corpus is not ours to redistribute. The 26,182 pairs drawn from the SMUQamar corpus are licensed CC-BY-NC-SA-4.0, so any derivative containing them inherits non-commercial share-alike terms. Rather than relicense data we did not create, the plan is to publish the aggregation and filtering code, the source manifests and the split seed, so anyone can rebuild the exact 124,102-pair training set from the original AI4Bharat BPCC and SMUQamar releases. The filters and split are already documented on the dataset page.
Q.How can I contribute to Kashmiri NLP?
Two ways, none requiring a research background. Rate translations on the evaluation platform if you speak Kashmiri, or submit a dataset, model, paper or tool to the resource hub if you know of one that is missing. Tools for digitising legacy InPage files and handwritten manuscripts are planned but not built yet.
Q.Who founded Kashmir AI Research?
Faizan Ayoub, an AI researcher working on low-resource language technology. Kashmir AI Research is an independent research initiative on Kashmiri natural language processing and machine translation.
Q.What is Faizan Ayoub's contribution to Kashmiri NLP?
He designed and ran the experiments, built the evaluation platform, and wrote the ICRTC-2026 Best Paper, with Neha Prerna Tigga supervising the research. The specific contributions are the first quantified tokenizer fertility analysis for Kashmiri, the identification of a cross-lingual transfer threshold near 30,000 parallel pairs, a deployable decoding strategy worth 12.75 BLEU without retraining, and a human evaluation platform built for the language.
Still have questions?
Reach out, or take part in the study.