Paper digest: Creole & low-resource NLP

Anchor papers: Armstrong, Liu, Owoputi, Stewart — JamPatoisNLI: A Jamaican Patois Natural Language Inference Dataset (arXiv:2109.00514, 2021) · Lent et al. — CreoleVal: Multilingual Multitask Benchmarks for Creoles (arXiv:2310.19567, 2023) · NLLB Team — No Language Left Behind: Scaling Human-Centered Machine Translation (arXiv:2207.04672, 2022)
Tokenization spine: Sennrich, Haddow, Birch — Neural Machine Translation of Rare Words with Subword Units (arXiv:1508.07909, 2015) · Kudo & Richardson — SentencePiece (arXiv:1808.06226, 2018)

One line claim

For Jamaican, Trinidadian, and other Caribbean Creole-influenced language, the binding constraints are labeled data, orthographic variation, and tokenizers trained on someone else’s text — not the absence of a Transformer. Low-resource NLP papers show how to measure that gap honestly before anyone promises a Cultural LLM.

A Cultural LLM without a Creole eval suite is a press release. JamPatoisNLI and CreoleVal are what accountability looks like on paper.

Field frame

Why this literature exists

Mainstream NLP scaled on English and a short list of high-resource languages. Creoles — including Jamaican Patois / Patwa, Haitian Kreyòl, and the English-lexified Creole continuum around Trinidad and Tobago — sit in a different regime: large speaker populations, intense code-switching with standard English, contested spelling, scarce parallel text, and almost no commercial benchmark culture.

That mismatch matters in 2026 because Caribbean AI headlines now include lab cohorts and “Cultural LLM” ambitions. Our reporting on Amber Group’s Jamaica Cultural LLM plans is explicit: the programme and ownership language are real; the public model card is not. The research question underneath the politics is older and sharper. What data and metrics would make a Jamaican or Trinidadian speech-text system good, and which published datasets already exist so teams are not inventing metrics in the dark?

This digest is a paper map for that question. It is not a claim that any one lab has solved Caribbean NLP.

Method themes that transfer

1. Build task datasets even when pretraining data is thin.
JamPatoisNLI (Armstrong et al., 2021) constructs a natural language inference benchmark for Jamaican Patois, with premise/hypothesis pairs labeled for entailment-style reasoning. The methodological lesson is general: pick a task with clear labels, work with community-competent annotators, and publish splits that make zero-shot transfer from English models falsifiable. You do not need a trillion-token crawl to start measuring whether a multilingual model understands Patwa.

2. Multi-Creole, multitask evaluation.
CreoleVal (Lent et al., 2023) argues that Creoles should not be evaluated as one-off curiosities. It assembles multilingual, multitask benchmarks across Creole languages so methods can be compared without reinventing ad hoc test sets per paper. For Caribbean programmes that span Jamaican Patwa, Haitian Kreyòl, and related Atlantic Creoles, that framing is more honest than a single English MMLU screenshot.

3. Massively multilingual MT as infrastructure, not destiny.
No Language Left Behind (NLLB Team, 2022) scales translation to hundreds of languages with a human-centered data and evaluation story: document which languages are covered, how data was collected, and where quality still fails. Creoles and Caribbean Englishes appear inside a broader low-resource map. NLLB does not replace a Cultural LLM. It shows what “coverage” means when you take language lists seriously — and how much labor sits behind each added language.

4. Tokenization is policy.
BPE (Sennrich et al., 2015) and SentencePiece (Kudo & Richardson, 2018) are the quiet governors of low-resource performance. A tokenizer trained mostly on English news will shatter Creole morphology and spelling variants into ugly piece sequences, waste context length, and bias the model toward standard English completions. For JA/TT speech-text systems, tokenizer decisions — vocabulary size, whether to include Creole orthographies, how to handle code-switched lines — are first-order model choices, not preprocessing trivia.

5. Speech-text needs paired audio, not only chat logs.
Text-only Creole corpora help classification and MT. Product reality in Jamaica and Trinidad is voice notes, radio, and WhatsApp audio. ASR/TTS for Creole-influenced speech inherits every orthography problem plus acoustic variation. Low-resource speech papers repeatedly show the same bottleneck: licensed, consented, transcribed hours beat scraping. Pair this with modality unification research such as Omni-Diffusion only after you have a speech corpus plan; architecture will not invent transcripts.

Results that moved

Exact scores evolve with model generations; the structural results matter more for Caribbean builders.

English-centric models transfer unevenly. JamPatoisNLI-style evaluations typically show that strong English NLI systems degrade on Patwa without adaptation. That is the empirical core of the Cultural LLM pitch: off-the-shelf models are not automatically local. The paper’s value is giving you a number to cite when a vendor says “multilingual” and means “Western European plus a bit of Spanish.”

Multilingual models help, then plateau. NLLB and related massively multilingual Transformers improve low-resource translation versus English-only baselines, especially when parallel data and careful sampling exist. Gains are not uniform. Languages with less parallel text and more orthographic noise stay behind. CreoleVal-style benchmarks exist so those plateaus are visible instead of hidden inside an average BLEU.

Annotation quality dominates clever architectures. Across low-resource NLP, adding a few thousand clean labeled examples often beats another round of prompt engineering on a frozen English model. That is uncomfortable for demo culture and correct for programme design — including the Amber lab’s training-then-build cadence described in our Amber brief.

Code-switching is the Caribbean default. Many academic splits still treat monolingual Creole and monolingual English as separate worlds. Real JA/TT text mixes. Systems that cannot handle intra-sentential switching will look fine on tidy benchmarks and fail on WhatsApp. When you design corpora, label the mix deliberately.

JA and TT: what “speech-text” actually requires

Jamaica and Trinidad and Tobago share a practical stack even when the linguistics differ:

• Written standard Caribbean English for schools, government, and much formal business.
• English-lexified Creole / continuum speech in daily life, with community orthographies that may or may not match literary or academic conventions.
• Heavy code-switching in the channels that matter for products: WhatsApp, radio call-ins, market talk, campaign voiceovers.

A speech-text programme therefore needs at least four artifact types, not one “Cultural LLM” slogan:

1. ASR transcripts with time-aligned audio and explicit variety tags (more Patwa vs more acrolectal English, etc.).
2. TTS prompts and evaluated listening tests with local listeners — WER alone is not enough for brand voice.
3. Text task sets in the spirit of JamPatoisNLI (inference, classification, QA) so chat quality is measurable.
4. Parallel or comparable corpora for MT and for retrieval bilingual search (English query → Creole passage and the reverse).

Trinidad-specific public LLM-era benchmarks are thinner in the open literature than Jamaican Patwa resources. That is a research gap, not a reason to pretend generic “Caribbean English” evals cover Port of Spain court language or Tobago classroom speech. The honest programme move is to fund TT annotation modeled on JamPatoisNLI/CreoleVal discipline rather than to wait for a foreign lab to notice.

NLLB’s human-centered framing also warns against silent language omission. If a massively multilingual model card lists Haitian Kreyòl but not Jamaican Patwa, that is a coverage fact you can act on: either contribute data, build adapters, or stop claiming the API is locally fit.

Limits

Public Creole resources remain sparse relative to speaker counts. JamPatoisNLI and CreoleVal are anchors, not a complete Caribbean eval stack. Trinidad and Tobago English Creole / Trinidadian Vernacular has even less widely cited LLM-era benchmarking than Jamaican Patwa. Do not fake precision: cite what exists, and treat TT speech-text as a data creation problem first.

Orthography wars are not bugs in the repo. Community spelling varies; school standards, social media, and literary norms disagree. A lab that freezes one orthography without community process will ship linguistic politics as “preprocessing.”

Speech rights and privacy are sharper in small societies. Scraping radio or WhatsApp exports into a training set can be illegal, unethical, or both. Low-resource does not mean low-consent.

Transformer efficiency papers — FlashAttention, MoE — change training cost. They do not label Patwa NLI items. Keep the layers straight.

Who should care

Caribbean AI labs and universities. If you are building toward a Cultural LLM, these papers are the minimum reading list before architecture debates. Pair programme PR with an eval roadmap.

Product teams shipping chat, CS, or voice in JA/TT. Your users already code-switch. Measure that. Consider whether you need full pretraining, continued pretraining, or — often more rational — retrieval over a local corpus.

Funders and ministries. Ask for dataset cards and benchmark names, not only cohort headcount. JamPatoisNLI and CreoleVal give you vocabulary for diligence.

Global multilingual NLP researchers. Caribbean Creoles are a stress test for tokenization, code-switching, and community-valued evaluation — the same issues NLLB framed as human-centered MT.

Creative and CS teams in the region. Ad scripts, radio reads, and WhatsApp customer service already live in the continuum. If you fine-tune or prompt a foreign model for “Trinidad voice,” keep a held-out listening/readability panel. Paper benchmarks will not catch a bot that sounds like a US customer-support template with “nuh” sprinkled on top. Related product context on this blog includes Cyargpt / AutoTrustJA and the Amber Cultural LLM reporting linked above.

Tokenizer and data choices also interact with systems cost. Longer tokenized Creole sequences burn more context and more billable tokens even when the utterance is short in words — another reason SentencePiece/BPE policy is a product decision, and another reason FlashAttention-class efficiency and RAG corpus design belong in the same programme conversation as linguistics.

Bottom line

Creole and low-resource NLP say the quiet part: language technology fails as data and evaluation policy before it fails as attention math. JamPatoisNLI makes Jamaican Patois measurable on NLI; CreoleVal pushes multitask Creole comparison; NLLB shows what scaled multilingual coverage costs; BPE/SentencePiece decide how much of a Creole sentence survives into the model. For Jamaica and Trinidad speech-text ambitions — including Amber’s Cultural LLM north star — the research-aligned path is corpus, tokenizer, and evals first. Architecture second.

We report papers. Digests written at Yatishara Blog; not a language authority.