Kashmiri and large language models
Language models that cover or are adapted to Kashmiri, the text corpora used to pretrain them, and studies of how general-purpose LLMs handle the language.
- 35
- Resources2024–2026
- 10
- Datasets
- 16
- Models
- 6
- Papers
Filter this task in the full hub · Guide: Kashmiri machine translation
Datasets 10
KS-PRET-5M — 5M Word Kashmiri Pretraining Corpus
5 million words / 12 million tokens of Kashmiri text recovered from professionally typeset InPage Nastaliq archives. The largest and highest-fidelity Kashmiri pretraining corpus published to date, and the source corpus behind Koshur Pixel.
DCAD-2000
Multilingual pretraining corpus covering 2,000+ languages, cleaned by treating noise as anomaly detection (NeurIPS 2025 Datasets & Benchmarks). Includes Kashmiri (kas_Arab) files; licence inherits from its sources.
HPLT 3.0 — Kashmiri subsets
Third release of HPLT's web-crawled monolingual collection (Internet Archive and Common Crawl) in 198 languages, with more data, annotations and metadata than v2. Files are served from HPLT's own servers.
KS-LIT-3M — 3.1 Million Word Pretraining Dataset
Meticulously cleaned 3.1-million-word (16.4M characters) Kashmiri text corpus specifically designed for pretraining LLMs from scratch. The largest open Kashmiri text resource.
FineWeb-2 — Kashmiri subsets
Hugging Face's multilingual web pretraining corpus, with Kashmiri subsets in Perso-Arabic, Devanagari and Latin script. Language identification uses GlotLID.
HPLT 2.0 — Kashmiri subsets
The HPLT project's cleaned web corpus, with Kashmiri subsets in Perso-Arabic (949 documents) and Devanagari (106 documents) script.
IITB-IndicMonoDoc
Document-level monolingual corpora spanning 22 Indian languages including Kashmiri. Useful for pretraining and for document-level rather than sentence-level modelling.
Kashmiri Text Corpus (HNM)
A 15 MB plain-text Kashmiri corpus of about 1.69 million words (57K unique), released with a character and word statistics report. Access is granted on request.
MURI-IT
2.2M instruction–output pairs across 200 languages built with multilingual reverse instructions: outputs are human-written native text, and the instructions are generated for them. Includes a small Kashmiri portion.
Sangraha — Kashmiri subset
AI4Bharat's large pretraining corpus for Indian languages; the 'verified' Kashmiri subset is about 0.8 MB.
Models 16
Kong Posh — Gemma 3n Kashmiri Adapter
Rank-256 LoRA adapter continuing Google's Gemma 3n E4B on about 30 million tokens of Kashmiri-labelled text. A base-model continuation adapter for research, not a chat or instruction model.
ks_byte_lm — SpaceByte (HNM)
Byte-level SpaceByte-style causal language model trained from scratch on a Kashmiri pretraining corpus; Transformers-compatible release (trust_remote_code).
Param-2 17B (BharatGen)
Hybrid mixture-of-experts LLM pretrained from scratch with a focus on Indian languages; supports English, Hindi and 21 more Indian languages including Kashmiri. Released as an early post-training checkpoint.
Sarvam Kashmiri (fine-tuned) — Gaash Lab
A Sarvam language model fine-tuned for Kashmiri by Gaash Lab, NIT Srinagar.
Wikilangs — Kashmiri models
Tokenizers, n-gram models, Markov chains, vocabulary statistics and word embeddings trained on Kashmiri Wikipedia, with an evaluation report.
Kashmiri BERT (kashmiri-llm-bert-base)
BERT-base model trained specifically for the Kashmiri language. Useful for downstream tasks like text classification, NER, and feature extraction.
Sarvam-Translate
Document-level translation model for the 22 official Indian languages including Kashmiri, built on Gemma3-4B-IT, designed for long inputs and varied content formats.
Goldfish — Kashmiri (Devanagari)
Small monolingual language model for Kashmiri in Devanagari script, trained on 6 MB of text, from the Goldfish suite of monolingual models for 350 languages.
Kashur GPT v1 (HNM)
Small decoder-only transformer for generating Kashmiri text continuations from prompts, with a live demo on Spaces. Not intended for production use without further evaluation.
BLOOM-560M
Open multilingual LLM pretrained on ROOTS corpus (incl. Persian/Urdu). Used for zero-shot Kashmiri text classification. Effective for cross-lingual transfer to Kashmiri.
IndicBARTSS
Indic-focused variant of mBART, specifically designed for Indian language generation and translation tasks including Kashmiri. Optimized for Indic script families.
IndicBERTv2 (ai4bharat)
Multilingual BERT model pretrained on Indian languages including Kashmiri. Used as embedding backbone in news classification benchmarks (F1=0.98 with GRU).
Kashmiri Text Generation — Llama3 8B Instruct
Fine-tuned Llama 3 8B model for Kashmiri question answering and English→Kashmiri translation. Research-grade prototype for Kashmiri text generation.
kashurai — Kashmiri Dataset Collection
Curated community collection grouping Kashmiri-specific LLMs, text datasets, and ASR models in one browsable hub. Great starting point for discovery.
mBART-large-50
Multilingual sequence-to-sequence model pretrained with a denoising objective on 50 languages. Kashmiri is not among them: using it for Kashmiri means borrowing another language code (for example Urdu) and fine-tuning.
XLS-R 300M (Wav2Vec2)
Self-supervised speech model pretrained on 436K hours of audio in 128 languages. Fine-tunable for Kashmiri ASR tasks with limited labeled data.
Papers 6
Anatomy of Decoder-Only LLM Failure in Low-Resource Machine Translation
Best Paper at ICRTC-2026. First quantified tokenizer fertility analysis for Kashmiri (4.06x overhead), a sharp cross-lingual transfer threshold near 30K pairs, and a deployable source-constrained decoding strategy worth +12.75 BLEU without retraining. Compares QLoRA Mistral-7B against NLLB-200 and IndicTrans2 on a 124,102-pair corpus.
KS-LIT-3M: A Curated Kashmiri Text Dataset for LLM Pretraining
Introduces a 3.1-million-word curated Kashmiri text dataset designed for pretraining LLMs. Addresses the scarcity of high-quality Kashmiri training data with CC-BY-4.0 licensing.
KS-PRET-5M: A 5 Million Word, 12 Million Token Kashmiri Pretraining Dataset
Successor to KS-LIT-3M. A 5-million-word, 12-million-token Kashmiri pretraining corpus recovered largely from professionally typeset InPage archives in Nastaliq script — the highest-quality stratum currently available for Kashmiri NLP.
Dataset Creation & Benchmarking for Kashmiri News Snippet Classification
First manually labeled corpus of 15,036 Kashmiri news snippets across 10 domains. Benchmarks ParsBERT (F1=0.98), IndicBERTv2+GRU, BLOOM-560m, and Flan-T5 in zero-shot settings.
Enhancing Low-Resource Indian Language MT Using LLMs with Preference Optimization
RL-inspired framework using Direct Preference Optimization and Hypergeometric-Gamma Reward with mT5-Large model. Covers Kashmiri, Konkani, and Dogri translation.
EnsembleSenti-Kash: Stacked Ensemble for Kashmiri Sentiment Classification
A stacked ensemble learning system for Kashmiri sentiment analysis with manually labeled data. Compares SVM, Random Forest, XGBoost, mBERT, and LSTM architectures.
Tools 2
Kashmiri Tokenizers (HNM)
Kashmiri subword and word tokenizers (BPE, Unigram, WordPiece, word- and character-level) released by HNM.
KashmiriGPT — AI Language Assistant
AI-powered Kashmiri language assistant accepting Roman and Perso-Arabic input. Provides responses in Kashmiri script, Roman transliteration, and English for cultural preservation.