OPEN-SOURCE SCRIPT
Telah dikemas kini Structural Language Model
Overview
Structural Language Model treats price action as a language. Each bar is tokenised into one of five structural symbols, and a low-order Markov model learns the grammar — the probability of what comes next given the recent context. Instead of "match the nearest historical shape" (fragile, overfit-prone k-NN), it estimates P(next token | last k tokens): a nonparametric conditional-move model that proves or disproves itself, live, on your symbol. It is a research/forecast read, not a signal service.
The five-symbol grammar
Every bar becomes one token, built from robust intrabar primitives (gap-immune, no fragile sweep/FVG detection), with adaptive thresholds so the alphabet stays balanced across symbols and timeframes:
X− down impulse · d ordinary down · c compression / indecision · u ordinary up · X+ up impulse
The model then learns grammar like c → X+ (breakout), X+ → X− (reversal), runs of u/X+ (trend), X+ → c (exhaustion), using order-1 or order-2 transition counts with Laplace smoothing, updated online.
Why these parts are one tool
The tokeniser turns raw OHLC into a balanced, information-rich alphabet — without it the Markov counts are dominated by whatever token is most common.
The Markov model reads out, each bar, a directional bias (P up-ish − P down-ish), a predictability score (how peaked the next-token distribution is, via normalized entropy), a structural-surprise spike (−log P of the token that just printed — a grammar break), and the full next-bar probability ladder.
The harness is the part that makes it honest. It's prequential (predict-then-update: each transition is scored from counts that exclude its own outcome, so every score is out-of-sample), it runs a walk-forward in-sample vs out-of-sample split with Wilson 95% intervals, and it draws a reliability curve — binning OOS predictions by predicted P(up) and showing the realized up-rate per bin. A rising, significant curve = real calibrated information; a flat one = none. Remove any part and you can no longer answer "is this model actually calibrated on this market?"
How to use it
Read the directional bias line against its conviction bands as context, not a trigger, and check predictability for how peaked the forecast is. Then read the harness — the model is only worth trusting where the out-of-sample up-lean lift is above 1 and/or the reliability spread is positive and significant (✓sig). A flat or insignificant curve means there's no calibrated edge here; treat it as descriptive only, or try another symbol/timeframe. The dashboard has a Compact layout (default: forecast + the one calibration line that matters) and a Pro layout (the full ladder, in/out-of-sample lift, and the three-bin reliability curve). Bias-turn crosses are optionally mirrored on the price chart. It is never a standalone signal.
Non-repainting
Tokens and counts update only on confirmed bars, and the score for bar t uses counts as they stood before bar t's transition was added — nothing reads its own future. The live next-bar forecast naturally refines as the current bar forms (it's a forecast, not a settled statistic). All harness figures are out-of-sample by construction.
Honest limits
OHLCV only. A per-bar tokeniser maximises samples but is coarser than a swing/event grammar (a documented future extension). Any edge is typically modest and market/timeframe-dependent — directional forecasting on noisy price is hard, and no indicator has an inherent edge. That's exactly why the harness is built in: validate it before trusting it.
Outputs for other scripts
Generic EXP_* plots — bias, predictability, structural surprise, live P(next up-ish), and the OOS lift — are published to the Data Window for use from other scripts via input.source().
Concept credits
Markov chains / n-gram language models — A. Markov (1913); C. Shannon (1948)
Prequential (predict-then-update) evaluation — A. P. Dawid (1984)
Additive (Laplace) smoothing — P.-S. Laplace
Entropy — C. Shannon (1948)
Wilson score interval — E. B. Wilson (1927)
Synthesis and Pine implementation are the author's own; no third-party Pine code reused.
Disclaimer
Research and education only. Not financial advice, not a signal service, not a guarantee of future results. Validate with your own testing, apply realistic costs, and manage risk.
Structural Language Model treats price action as a language. Each bar is tokenised into one of five structural symbols, and a low-order Markov model learns the grammar — the probability of what comes next given the recent context. Instead of "match the nearest historical shape" (fragile, overfit-prone k-NN), it estimates P(next token | last k tokens): a nonparametric conditional-move model that proves or disproves itself, live, on your symbol. It is a research/forecast read, not a signal service.
The five-symbol grammar
Every bar becomes one token, built from robust intrabar primitives (gap-immune, no fragile sweep/FVG detection), with adaptive thresholds so the alphabet stays balanced across symbols and timeframes:
X− down impulse · d ordinary down · c compression / indecision · u ordinary up · X+ up impulse
The model then learns grammar like c → X+ (breakout), X+ → X− (reversal), runs of u/X+ (trend), X+ → c (exhaustion), using order-1 or order-2 transition counts with Laplace smoothing, updated online.
Why these parts are one tool
The tokeniser turns raw OHLC into a balanced, information-rich alphabet — without it the Markov counts are dominated by whatever token is most common.
The Markov model reads out, each bar, a directional bias (P up-ish − P down-ish), a predictability score (how peaked the next-token distribution is, via normalized entropy), a structural-surprise spike (−log P of the token that just printed — a grammar break), and the full next-bar probability ladder.
The harness is the part that makes it honest. It's prequential (predict-then-update: each transition is scored from counts that exclude its own outcome, so every score is out-of-sample), it runs a walk-forward in-sample vs out-of-sample split with Wilson 95% intervals, and it draws a reliability curve — binning OOS predictions by predicted P(up) and showing the realized up-rate per bin. A rising, significant curve = real calibrated information; a flat one = none. Remove any part and you can no longer answer "is this model actually calibrated on this market?"
How to use it
Read the directional bias line against its conviction bands as context, not a trigger, and check predictability for how peaked the forecast is. Then read the harness — the model is only worth trusting where the out-of-sample up-lean lift is above 1 and/or the reliability spread is positive and significant (✓sig). A flat or insignificant curve means there's no calibrated edge here; treat it as descriptive only, or try another symbol/timeframe. The dashboard has a Compact layout (default: forecast + the one calibration line that matters) and a Pro layout (the full ladder, in/out-of-sample lift, and the three-bin reliability curve). Bias-turn crosses are optionally mirrored on the price chart. It is never a standalone signal.
Non-repainting
Tokens and counts update only on confirmed bars, and the score for bar t uses counts as they stood before bar t's transition was added — nothing reads its own future. The live next-bar forecast naturally refines as the current bar forms (it's a forecast, not a settled statistic). All harness figures are out-of-sample by construction.
Honest limits
OHLCV only. A per-bar tokeniser maximises samples but is coarser than a swing/event grammar (a documented future extension). Any edge is typically modest and market/timeframe-dependent — directional forecasting on noisy price is hard, and no indicator has an inherent edge. That's exactly why the harness is built in: validate it before trusting it.
Outputs for other scripts
Generic EXP_* plots — bias, predictability, structural surprise, live P(next up-ish), and the OOS lift — are published to the Data Window for use from other scripts via input.source().
Concept credits
Markov chains / n-gram language models — A. Markov (1913); C. Shannon (1948)
Prequential (predict-then-update) evaluation — A. P. Dawid (1984)
Additive (Laplace) smoothing — P.-S. Laplace
Entropy — C. Shannon (1948)
Wilson score interval — E. B. Wilson (1927)
Synthesis and Pine implementation are the author's own; no third-party Pine code reused.
Disclaimer
Research and education only. Not financial advice, not a signal service, not a guarantee of future results. Validate with your own testing, apply realistic costs, and manage risk.
Nota Keluaran
v1.2 — Correctness + adaptive theme + cost model (no model change)- Cost model (for a forecaster, the cost question is "does trading the lean survive costs?"): a new
"Round-trip cost (bps)" input and a "Net edge OOS (bps/bar)" read — on high-conviction out-of-sample
bars, the average next-bar return in the model's leaned direction minus cost. Positive = the forecast
survives costs; negative = costs eat the (typically modest) lift. Exported as EXP_NetEdgeBps.
Non-repainting one-bar hold entered at the prior close; descriptive, not a live backtest.
- The dashboard theme is now background-adaptive (Auto / Dark / Light) instead of a dark-only toggle.
- Correctness: the "ignore first session bar" guard called ta.change from behind a short-circuiting
AND, so it was skipped when the toggle was off (a Pine warning). It now runs every bar; output same.
- No change to the tokeniser, the Markov model or the calibration statistics — everyday behaviour is
identical.
Research / forecast-calibration study, in-sample statistics, not investment advice.
Skrip sumber terbuka
Dalam semangat TradingView sebenar, pencipta skrip ini telah menjadikannya sumber terbuka, jadi pedagang boleh menilai dan mengesahkan kefungsiannya. Terima kasih kepada penulis! Walaupuan anda boleh menggunakan secara percuma, ingat bahawa penerbitan semula kod ini tertakluk kepada Peraturan Dalaman.
Penafian
Maklumat dan penerbitan adalah tidak bertujuan, dan tidak membentuk, nasihat atau cadangan kewangan, pelaburan, dagangan atau jenis lain yang diberikan atau disahkan oleh TradingView. Baca lebih dalam Terma Penggunaan.
Skrip sumber terbuka
Dalam semangat TradingView sebenar, pencipta skrip ini telah menjadikannya sumber terbuka, jadi pedagang boleh menilai dan mengesahkan kefungsiannya. Terima kasih kepada penulis! Walaupuan anda boleh menggunakan secara percuma, ingat bahawa penerbitan semula kod ini tertakluk kepada Peraturan Dalaman.
Penafian
Maklumat dan penerbitan adalah tidak bertujuan, dan tidak membentuk, nasihat atau cadangan kewangan, pelaburan, dagangan atau jenis lain yang diberikan atau disahkan oleh TradingView. Baca lebih dalam Terma Penggunaan.