OPEN-SOURCE SCRIPT
AetherEdge - Q-Learning Regime Agent
🖊️ Overview
AE-QRA is a reinforcement-learning agent that learns its policy from experience, right on your chart. It discretizes the market into 27 regimes (Trend × Momentum × Volatility) and learns which action to take in each — Short, Flat, or Long — by maximizing reward through tabular Q-learning. No instructions, no labels: the agent relies on reward alone, sharpening its judgment as it shifts from exploration to exploitation.
🔶 Key Features
A real Q-learning engine — the TD(0) update Q(s,a) ← Q(s,a) + α[r + γ·maxQ(s',a') − Q(s,a)] runs every bar, updating a value table over states × actions.
Reward function — reward = position × next-bar return − turnover cost. Profitable decisions are reinforced; needless trading is penalized.
Exploration–exploitation, made real — an ε-greedy policy whose ε decays over time, automatically shifting from random exploration to exploiting the learned policy.
Policy heatmap — the learned policy across 27 regimes, visualized as a 9-cell color grid at the live volatility slice; the cell for the current regime glows.
Regime coloring — bars and background are tinted by the agent's current stance, growing more saturated with conviction.
Action-flip markers — triangle markers mark switches to Long or Short.
Live statistics panel — action, conviction, exploration rate ε, cumulative reward (policy P&L), regime coverage, and the current state.
Fully deterministic, non-repainting design — the exploration RNG is seed-fixed; learning and decisions occur on confirmed bars only, on a one-bar lag.
🧠 Technical Architecture
The state space is built from three discrete features — ATR-unit trend deviation (↓/·/↑), RSI momentum (↓/·/↑), and volatility percentile rank (low/mid/high). Their product gives 27 states, with three actions (Short = −1 / Flat = 0 / Long = +1). The value table Q is persisted as a var matrix (27 × 3).
On each confirmed bar, the agent first observes the reward of its previous action (position × realized return − switching cost), then updates that value via TD(0) using the maximum Q at the next state. It then selects the next action ε-greedily. ε decays as ε = ε_min + (ε_start − ε_min)·exp(−step/decay) — exploratory early on (frequent random actions), exploiting the learned policy as it matures. Conviction is computed as the separation of the three Q-values at the current state and feeds both the background tint and the panel. Because the RNG is a deterministic LCG, the policy is reproducible under identical settings.
🎯 Three design choices stand out. First, embedding a switching cost in the reward suppresses over-trading at the learning level and makes Flat a meaningful choice. Second, confining learning and decisions to confirmed bars on a one-bar lag eliminates future leakage. Third, the deliberately compact 27-state design helps the policy converge even on shorter histories.
⚙️ Recommended Settings & Tuning Guide
As a crypto starting point — BTC/ETH (1D, 4H): α = 0.10, γ = 0.95, ε-Start = 0.90, ε-Decay = 500, switching cost 3 bps; this yields clean trend adaptation. High-volatility names (SOL, XRP): raise switching cost to 5–8 bps to curb trade frequency, and shorten ε-Decay toward 300 so the agent reaches the exploitation phase faster through the noise.
Per parameter: Learning rate α sets adaptation speed — 0.15–0.25 in fast-rotating regimes, 0.05–0.10 in stable ones. Discount γ sets foresight — higher weights long-run reward (0.95–0.99 for swings, 0.85–0.92 for scalps). ε-Decay sets the length of the exploration phase — larger explores longer and learns more cautiously. Switching cost doubles as a direct knob on trade frequency.
💡 How to Use in Practice
The core read is the agent's stance × conviction. A flip to Long (triangle marker) with high conviction and a rising cumulative reward reads as a tailwind for buying dips. The policy heatmap is a powerful context tool: by reading the colors of the cells adjacent to the current regime (the glowing cell), you can anticipate how the agent will act if the market shifts slightly. When ε is still high, the policy is undecided, so treat signals as informational only.
For multi-timeframe work, confirm the macro bias from the higher-timeframe stance (1D), then execute aligned flips on a lower timeframe (1H–4H). Layered over market structure (S/R, order blocks) or volume, it adds which regime, and with what conviction, the agent chooses to go long — useful confirmation context.
⚠️ Important Notes
Until the warmup period (default 300 bars) and the exploration phase pass, actions are largely exploratory (random). Reloading the indicator, or changing settings/seed, makes the agent relearn across the entire history from scratch — learning state is not persisted. The panel's "Policy P&L" is an in-sample, while-learning metric that does not fully account for switching cost or fill slippage; it is not a backtest or forward result. The 27-state tabular design is intentionally coarse — it learns a regime policy, not fine price structure. Decisions update on confirmed bars, and the agent's stance is not a direct buy/sell instruction.
🚨 Disclaimer
This indicator is for educational and informational purposes only and does not constitute financial or investment advice. Past performance is not indicative of future results. All trading involves risk. Use it alongside your own thorough testing and sound risk management; all trading decisions remain solely your own responsibility.
AE-QRA is a reinforcement-learning agent that learns its policy from experience, right on your chart. It discretizes the market into 27 regimes (Trend × Momentum × Volatility) and learns which action to take in each — Short, Flat, or Long — by maximizing reward through tabular Q-learning. No instructions, no labels: the agent relies on reward alone, sharpening its judgment as it shifts from exploration to exploitation.
🔶 Key Features
A real Q-learning engine — the TD(0) update Q(s,a) ← Q(s,a) + α[r + γ·maxQ(s',a') − Q(s,a)] runs every bar, updating a value table over states × actions.
Reward function — reward = position × next-bar return − turnover cost. Profitable decisions are reinforced; needless trading is penalized.
Exploration–exploitation, made real — an ε-greedy policy whose ε decays over time, automatically shifting from random exploration to exploiting the learned policy.
Policy heatmap — the learned policy across 27 regimes, visualized as a 9-cell color grid at the live volatility slice; the cell for the current regime glows.
Regime coloring — bars and background are tinted by the agent's current stance, growing more saturated with conviction.
Action-flip markers — triangle markers mark switches to Long or Short.
Live statistics panel — action, conviction, exploration rate ε, cumulative reward (policy P&L), regime coverage, and the current state.
Fully deterministic, non-repainting design — the exploration RNG is seed-fixed; learning and decisions occur on confirmed bars only, on a one-bar lag.
🧠 Technical Architecture
The state space is built from three discrete features — ATR-unit trend deviation (↓/·/↑), RSI momentum (↓/·/↑), and volatility percentile rank (low/mid/high). Their product gives 27 states, with three actions (Short = −1 / Flat = 0 / Long = +1). The value table Q is persisted as a var matrix (27 × 3).
On each confirmed bar, the agent first observes the reward of its previous action (position × realized return − switching cost), then updates that value via TD(0) using the maximum Q at the next state. It then selects the next action ε-greedily. ε decays as ε = ε_min + (ε_start − ε_min)·exp(−step/decay) — exploratory early on (frequent random actions), exploiting the learned policy as it matures. Conviction is computed as the separation of the three Q-values at the current state and feeds both the background tint and the panel. Because the RNG is a deterministic LCG, the policy is reproducible under identical settings.
🎯 Three design choices stand out. First, embedding a switching cost in the reward suppresses over-trading at the learning level and makes Flat a meaningful choice. Second, confining learning and decisions to confirmed bars on a one-bar lag eliminates future leakage. Third, the deliberately compact 27-state design helps the policy converge even on shorter histories.
⚙️ Recommended Settings & Tuning Guide
As a crypto starting point — BTC/ETH (1D, 4H): α = 0.10, γ = 0.95, ε-Start = 0.90, ε-Decay = 500, switching cost 3 bps; this yields clean trend adaptation. High-volatility names (SOL, XRP): raise switching cost to 5–8 bps to curb trade frequency, and shorten ε-Decay toward 300 so the agent reaches the exploitation phase faster through the noise.
Per parameter: Learning rate α sets adaptation speed — 0.15–0.25 in fast-rotating regimes, 0.05–0.10 in stable ones. Discount γ sets foresight — higher weights long-run reward (0.95–0.99 for swings, 0.85–0.92 for scalps). ε-Decay sets the length of the exploration phase — larger explores longer and learns more cautiously. Switching cost doubles as a direct knob on trade frequency.
💡 How to Use in Practice
The core read is the agent's stance × conviction. A flip to Long (triangle marker) with high conviction and a rising cumulative reward reads as a tailwind for buying dips. The policy heatmap is a powerful context tool: by reading the colors of the cells adjacent to the current regime (the glowing cell), you can anticipate how the agent will act if the market shifts slightly. When ε is still high, the policy is undecided, so treat signals as informational only.
For multi-timeframe work, confirm the macro bias from the higher-timeframe stance (1D), then execute aligned flips on a lower timeframe (1H–4H). Layered over market structure (S/R, order blocks) or volume, it adds which regime, and with what conviction, the agent chooses to go long — useful confirmation context.
⚠️ Important Notes
Until the warmup period (default 300 bars) and the exploration phase pass, actions are largely exploratory (random). Reloading the indicator, or changing settings/seed, makes the agent relearn across the entire history from scratch — learning state is not persisted. The panel's "Policy P&L" is an in-sample, while-learning metric that does not fully account for switching cost or fill slippage; it is not a backtest or forward result. The 27-state tabular design is intentionally coarse — it learns a regime policy, not fine price structure. Decisions update on confirmed bars, and the agent's stance is not a direct buy/sell instruction.
🚨 Disclaimer
This indicator is for educational and informational purposes only and does not constitute financial or investment advice. Past performance is not indicative of future results. All trading involves risk. Use it alongside your own thorough testing and sound risk management; all trading decisions remain solely your own responsibility.
Skrypt open-source
W zgodzie z duchem TradingView twórca tego skryptu udostępnił go jako open-source, aby użytkownicy mogli przejrzeć i zweryfikować jego działanie. Ukłony dla autora. Korzystanie jest bezpłatne, jednak ponowna publikacja kodu podlega naszym Zasadom serwisu.
Wyłączenie odpowiedzialności
Informacje i publikacje nie stanowią i nie powinny być traktowane jako porady finansowe, inwestycyjne, tradingowe ani jakiekolwiek inne rekomendacje dostarczane lub zatwierdzone przez TradingView. Więcej informacji znajduje się w Warunkach użytkowania.
Skrypt open-source
W zgodzie z duchem TradingView twórca tego skryptu udostępnił go jako open-source, aby użytkownicy mogli przejrzeć i zweryfikować jego działanie. Ukłony dla autora. Korzystanie jest bezpłatne, jednak ponowna publikacja kodu podlega naszym Zasadom serwisu.
Wyłączenie odpowiedzialności
Informacje i publikacje nie stanowią i nie powinny być traktowane jako porady finansowe, inwestycyjne, tradingowe ani jakiekolwiek inne rekomendacje dostarczane lub zatwierdzone przez TradingView. Więcej informacji znajduje się w Warunkach użytkowania.