# winkComposer — Full Documentation All pages from https://composer.winkjs.org/docs concatenated as one document for LLM ingestion. Nav order preserved; import statements stripped. --- # Documentation — winkComposer Source: https://composer.winkjs.org/docs > [!NOTE] > > winkComposer and this site are under active development as we transition to open source. Interfaces will break, details will go stale, and whole sections will reshape — please bear with us. --- # Playground — Build Something, See It Run Source: https://composer.winkjs.org/docs/playground --- # Hello Flow — Your First Pipeline Source: https://composer.winkjs.org/docs/playground/hello-flow A flow chains composable building blocks — called nodes — to process streaming data. Each node reads fields from the incoming message, computes something, and adds new fields for the next node to use. The message grows richer as it moves through the chain.} /> In this example we build a simple temperature monitor. The input is a stream of noisy sensor readings that fluctuate around 70°F — sometimes drifting above safe limits, sometimes dropping back. The flow uses four nodes to smooth the noise, detect when the temperature stays too high, and broadcast an alert. ## Step 1 — Smooth noisy data ```javascript .esMean('smooth', 'temperature', { mean: 'smoothTemp' }, { halfLife: 5 }) ``` Reads `temperature` from each message, computes an exponentially smoothed mean, and adds `smoothTemp` to the message. Noisy spikes flatten out; the underlying trend emerges. ## Step 2 — Detect when it's too hot ```javascript .threshold('check', 'smoothTemp', { active: 'overheating' }, { mode: 'above', threshold: 80 }) ``` Reads `smoothTemp` (added by step 1) and adds `overheating: true` when the value crosses above 80. ## Step 3 — Confirm it's not a fluke ```javascript .persistenceCheck('confirm', msg => msg.overheating === true, { persistenceConfirmed: 'confirmed' }, { minVotes: 2, outOfTotal: 3 }) ``` A single spike shouldn't trigger an alert. This node sets `confirmed: true` only if overheating persists in at least 2 of the last 3 messages. ## Step 4 — Broadcast the alert ```javascript .emitIf('alert', msg => msg.confirmed === true, { target: 'mqtt', insightType: 'tempAlert' }) ``` Sends a copy of the message to MQTT when confirmed. The message continues through the flow — `emitIf` is a side effect, not a stop. --- ## See it run Click each step below to add a node and watch the flow come alive — these are real winkComposer nodes running in your browser. --- ## Put it together ```javascript flow('temperature-monitor') .esMean('smooth', 'temperature', { mean: 'smoothTemp' }, { halfLife: 5 }) .threshold('check', 'smoothTemp', { active: 'overheating' }, { mode: 'above', threshold: 80 }) .persistenceCheck('confirm', msg => msg.overheating === true, { persistenceConfirmed: 'confirmed' }, { minVotes: 2, outOfTotal: 3 }) .emitIf('alert', msg => msg.confirmed === true, { target: 'mqtt', insightType: 'tempAlert' }) .run(); ``` Each node reads what previous nodes added and contributes its own fields — the message grows as it flows through: --- ## Next Steps - [Under the Hood](/docs/concepts/under-the-hood) — understand what happens inside the pipeline you just built - [Detecting Bearing Failure](/docs/use-cases/bearing-health) — watch a real failure detected in real time --- # Recipes — Streaming Intelligence Patterns Source: https://composer.winkjs.org/docs/playground/recipes ## Change detection Which approach fits depends on the signal. Gradual drift needs evidence accumulation — patience. Sudden shifts need immediate detection — speed. Subtle shifts need pattern recognition — sensitivity. Some real systems need all three. ## Model-vs-reality diagnostics When a sensor reports what it sees but cannot see everything, the model-vs-reality gap becomes the diagnostic signal. ## Signal quality Sometimes the problem is not that the signal *changed* — it is that it *stopped changing*. A frozen sensor reports stale data that looks perfectly valid to every other check. ## Subsampling & compression The opposite of detection: deciding which samples are not worth keeping. A high-rate sensor stream is mostly redundant — the trick is identifying the *informative* samples and dropping the rest without losing what matters. Read the deadband recipe first for the adaptive-gate pattern; the trajectory-aware recipe then adds a Kalman predictor and boundary anchoring for when reconstruction quality matters. --- # Gradual Drift Source: https://composer.winkjs.org/docs/playground/recipes/gradual-drift This recipe uses a fast/slow esMean (exponentially smoothed mean) crossover to surface the drift, a [Page-Hinkley](https://en.wikipedia.org/wiki/Page%27s_cumulative_sum_test) change-point test to accumulate statistical evidence, and a persistence check to confirm the detection is sustained. ```javascript flow('drift-detector') .sanitize('sane', 'temperature', { failureReason: 'failReason' }, { ranges: { temperature: { min: 0, max: 200 } } }) .median3('m3', 'temperature', { median3: 'm3' }) .esMean('fast', 'm3', { mean: 'fastMean' }, { halfLife: 1.35 }) .esMean('slow', 'm3', { mean: 'slowMean' }, { halfLife: 13.5 }) .diff('divergence', 'fastMean', 'slowMean', { diff: 'drift' }) .pageHinkley('ph', 'drift', { phShift: 'driftDetected', phTestStatistic: 'phStat' }, { delta: 0.1, lambda: 3 }) .persistenceCheck('confirm', msg => msg.driftDetected === true, { persistenceConfirmed: 'confirmed' }, { minVotes: 3, outOfTotal: 5 }) .run() ``` **Drag the slider** and watch the fast and slow averages diverge as drift builds. --- ## What You're Seeing The gray line is the raw temperature — noisy readings with a drift that starts at sample 200 but is invisible to the eye among the noise. The **cyan line** (fast esMean, halfLife=1.35) responds to recent values within a few samples. The **purple dashed line** (slow esMean, halfLife=13.5) represents the long-term baseline — it takes roughly 10x longer to move. During stable operation, both averages overlap. After sample 200, the fast esMean climbs with the drift while the slow esMean lags behind. Their divergence feeds the Page-Hinkley test, which accumulates evidence until the persistence check confirms. The **rose vertical** marks the confirmed detection. --- ## Where This Pattern Fits | Domain | What drifts | Why it's invisible | |--------|------------|-------------------| | Water treatment | Dissolved oxygen sensor calibration | Membrane fouling shifts the zero point by 0.1 mg/L per week | | Manufacturing | Cutting tool dimensions | Tool wear shifts the baseline by a few micrometres per batch | | HVAC | Zone temperature setpoint | Actuator degradation causes 0.5°F per month | | Chemical process | Catalyst conversion efficiency | Activity drops 0.2% per day | --- ## How It Works Two esMean nodes with different half-lives create a fast/slow crossover detector. The fast esMean (halfLife=1.35, α≈0.40) responds to recent changes within a few samples. The slow esMean (halfLife=13.5, α≈0.05) represents the long-term baseline. The diff node subtracts slow from fast. During stable operation, this difference fluctuates around zero. When drift begins, the fast esMean tracks the new level while the slow esMean lags, producing a sustained positive difference. The Page-Hinkley test accumulates this difference as statistical evidence. The delta parameter (0.1) sets the minimum magnitude worth tracking; lambda (3) sets the evidence threshold for declaring a change. The persistence check (3 of 5) confirms the detection is sustained — not a transient spike in the test statistic. esMean-based detection is robust to non-Gaussian noise — the ARL (Average Run Length before false alarm) drops only 5% under moderate skewness, compared to 56% for CUSUM (Lucas & Saccucci, 1990). This matters for real sensor data. --- ## References - Page, E.S. (1954). *Continuous inspection schemes.* Biometrika, 41(1/2), 100–115. [doi:10.2307/2333009](https://doi.org/10.2307/2333009) - Hinkley, D.V. (1971). *Inference about the change-point from cumulative sum tests.* Biometrika, 58(3), 509–523. [doi:10.1093/biomet/58.3.509](https://doi.org/10.1093/biomet/58.3.509) - Lucas, J.M. & Saccucci, M.S. (1990). *Exponentially weighted moving average control schemes: properties and enhancements.* Technometrics, 32(1), 1–12. [doi:10.1080/00401706.1990.10484583](https://doi.org/10.1080/00401706.1990.10484583) --- ## Next Steps - [Sudden Shifts](/docs/playground/recipes/sudden-shift) — the complementary recipe for abrupt step changes using Kalman innovation gating - [Frozen Sensors](/docs/playground/recipes/sensor-freeze) — detect when a sensor stops changing using running standard deviation collapse - [Under the Hood](/docs/concepts/under-the-hood) — understand what happens inside the pipeline --- # Directional Trend Source: https://composer.winkjs.org/docs/playground/recipes/directional-trend A sensor signal doesn't fail in one instant. It slips — a tenth of a dB, a millivolt, a milliwatt — too little per sample to trigger a magnitude alarm, too steady across samples to read as noise. By the time the magnitude crosses threshold, the rate has already told you where the signal was going.} /> This recipe converts the signal into its instantaneous slope with a Savitzky-Golay derivative kernel, then thresholds the slope in both directions and requires a short persistence window. Declines and rises become visible samples before the magnitude does. ```javascript flow('directional-trend') .sanitize('sane', 'x', { failureReason: 'failReason' }, { ranges: { x: { min: -10, max: 10 } } }) .median3('clean', 'x', { median3: 'xClean' }) .kernel('slope', 'xClean', { filtered: 'slope' }, { preset: 'trend5' }) .threshold('fallGate', 'slope', { active: 'trendingDown' }, { mode: 'below', threshold: -0.07 }) .threshold('riseGate', 'slope', { active: 'trendingUp' }, { mode: 'above', threshold: 0.07 }) .persistenceCheck('confirmFall', msg => msg.trendingDown === true, { persistenceConfirmed: 'confirmedDown' }, { minVotes: 2, outOfTotal: 3 }) .persistenceCheck('confirmRise', msg => msg.trendingUp === true, { persistenceConfirmed: 'confirmedUp' }, { minVotes: 2, outOfTotal: 3 }) .run() ``` **Drag the slider** through the signal and watch the slope catch both the decline and the recovery while the magnitude is still moving. Toggle the data source to compare the technique on a synthetic trace and on a real WiFi RSSI stream from Velocis Systems. --- ## What You're Seeing The **slate line** is the raw signal. In the synthetic mode it drops from 5.0 down to 1.0 over 25 samples, holds low for a short plateau, then climbs back over 15 samples. In the real-data mode, an enterprise client's RSSI falls sharply as it roams onto a distant AP, stays poor for about 35 samples, and snaps back when it roams home. The **cyan line** on the right axis is the slope from the trend kernel — the weighted local gradient of the signal. During steady operation it fluctuates around zero. It settles at a sustained negative value during declines and a sustained positive value during rises. The **orange bands** mark where the slope has crossed *below* the cutoff — a decline in progress. The **green bands** mark where the slope has crossed *above* the cutoff — a rise in progress. These are the raw outputs of the two `threshold` nodes and show the full duration of each directional window. The **rose verticals** are the confirmation events — where `persistenceCheck` has seen two of the last three trending ticks and fires its confirmed state. The small gap between the first orange (or green) band and the first rose vertical is the persistence delay — the price paid for noise rejection. Switch the kernel between `trend5`, `trend7`, and `trend11` to see the latency-smoothness tradeoff: shorter windows catch the transition earlier but react to more noise. --- ## Where This Pattern Fits | Domain | What drifts | Why the rate matters | |--------|------------|----------------------| | WiFi client | RSSI as a client moves or an AP loads | Act before quality crosses the glitch threshold | | Battery-powered device | Voltage or state of charge | Rate distinguishes normal draw from fault | | Server fleet | Memory consumption before OOM | Rate sets the time-to-remediation window | | Fuel or fluid systems | Tank level | An unexpected rate is a leak | | Process plant | Temperature before equipment failure | Rate signals bearing or insulation wear | --- ## How It Works The kernel node applies a weighted sum over a sliding window. Loaded with the Savitzky-Golay 1st-derivative coefficients `[−2, −1, 0, 1, 2] / 10` (the `trend5` preset), it performs a polynomial linear regression over five points and returns the slope. The output is the local rate of change in the same units as the signal per sample. Wider kernels trade latency for smoothness. `trend5` reacts within a few samples but carries the most noise. `trend11` produces a smoother slope at the cost of a later response. For the 5-minute polling cadence used in the real-data toggle, `trend5` is the sweet spot. Two threshold nodes fan out from the slope — one fires when the slope drops below `−cutoff` (a decline), the other when it rises above `+cutoff` (a rise). Each gate feeds its own persistence check. The cutoff is domain-chosen: `±0.07 /sample` on the synthetic signal, `±0.5 dBm/sample` on the real WiFi trace. Each persistence check requires at least two firings within a three-sample window, suppressing single-tick noise without adding much latency. ### Validation on real data On an anonymized client health export from Velocis Systems — five clients, four days at 5-minute polling — this detector caught 71% of the drops later confirmed by an `extremaRank` persistence check, with 84% precision and a median 20-minute lead. The slope fires on the downswing; `extremaRank` validates at the recovery peak. The two are complementary, not redundant. See the [WiFi AP Health use case](/docs/use-cases/wifi-ap-health) for the applied narrative. --- ## Choosing Between This and the `trend` Node Both expose a numeric slope, built on different models. The `trend` node takes the first difference `x[n] − x[n−1]` and applies exponential smoothing. It emits a state label (`learning / stable / rising / falling`), a confidence score, the smoothed slope (`rocMean`), and an optional acceleration hint — one primitive, bundled outputs. The kernel path fits a line over a fixed window (`trend5 / trend7 / trend11`) and returns its slope as a raw number. Threshold and persistence nodes decide what to do with it downstream. Reach for the **kernel path** when the cutoff must be readable in the signal's own units (`−0.5 dBm/sample`), when you need independent fall and rise booleans, or when the slope feeds downstream math. Reach for the **`trend` node** when the bundled state, confidence, or acceleration hint are what you want to consume. The kernel path is a **measurement instrument**. The `trend` node is a **prepared verdict**. --- ## References - Savitzky, A. & Golay, M.J.E. (1964). *Smoothing and differentiation of data by simplified least squares procedures.* Analytical Chemistry, 36(8), 1627–1639. [doi:10.1021/ac60214a047](https://doi.org/10.1021/ac60214a047) - Data for the Real WiFi toggle: anonymized Cisco DNA Center client health export from [Velocis Systems](https://velocis.in), used with permission. One client, 350 samples at 5-minute polling. PII removed upstream: MAC addresses, usernames, IP addresses, and location identifiers replaced with anonymized labels. --- ## Next Steps - [Gradual Drift](/docs/playground/recipes/gradual-drift) — the complementary recipe using Page-Hinkley change-point detection. Slope answers *how fast and which way*; Page-Hinkley answers *did it change*. - [Sudden Shifts](/docs/playground/recipes/sudden-shift) — for abrupt step changes using Kalman innovation gating. - [Under the Hood](/docs/concepts/under-the-hood) — how messages, state, and specs compose inside the pipeline. --- # Sudden Shifts Source: https://composer.winkjs.org/docs/playground/recipes/sudden-shift A tank level reads 850 litres every minute. Then it reads 680. Is the sensor wrong, or did something just happen? A threshold alarm would need to be set so tight it fires on noise, or so loose it misses real events. The Kalman filter sidesteps this trade-off — it knows what it predicted, and the chi-squared test fires when reality disagrees.} /> This recipe uses a single Kalman 1d node with innovation gating. Toggle between **follow mode** (track the jump to the new level) and **exclude mode** (reject it as an outlier and continue at the old level) — same node, one parameter. ```javascript flow('shift-detector') .sanitize('sane', 'level', { failureReason: 'failReason' }, { ranges: { level: { min: -50, max: 200 } } }) .median3('m3', 'level', { median3: 'm3' }) .kalman1d('kf', 'm3', { filtered: 'estimate', innovation: 'innovation', innovationGate: 'gateValue', variance: 'estVariance' }, { followMode: true, sensorVariance: 4, processVariance: 0.04, chi2Threshold: 6.63 }) .run() ``` **Drag the slider** and watch the filter track the signal — then see what happens at the outliers and the step change. --- ## What You're Seeing The gray line is the raw level signal — stable around 50 with Gaussian noise, two isolated outlier spikes (samples 80 and 150), and an abrupt step change to 62 at sample 250. The **emerald line** is the Kalman filtered estimate — it smoothly tracks the true level, rejecting noise. The **indigo curve** on the right axis is the innovation (prediction error) — flat near zero during normal operation, spiking when measurements violate the model's expectation. **Rose verticals** mark where the innovation gate exceeds the chi-squared threshold (6.63, corresponding to a 1% false alarm rate). The gate fires at both outlier spikes and at the step change. **Toggle the mode** to see the difference: - **Follow mode:** At the step change, the filter resets to the new level and tracks normally afterward. The gate fires once, then innovation drops back to zero. - **Exclude mode:** The filter rejects the step as an outlier and continues estimating at the old level. Innovation stays elevated because every subsequent measurement disagrees with the model. --- ## Where This Pattern Fits | Domain | What shifts | Why instant detection matters | |--------|-----------|-------------------------------| | GPS / aerospace | Satellite position estimate | Wrong position means wrong guidance | | Smart grid | Bus voltage | Equipment switching — the grid must rebalance | | Fuel monitoring | Tank level | Leak vs refill — the step direction matters | | Pipeline pressure | Line pressure | A burst — every minute counts | | IoT sensors | Any calibrated sensor | Sensor fault vs real event — exclude vs follow | --- ## How It Works The Kalman filter maintains a state estimate (the filtered level) and an uncertainty estimate (the covariance). Each new measurement produces an **innovation** — the difference between what the filter predicted and what it observed. The **innovation gate** normalises this surprise by the expected innovation variance (measurement noise + estimation uncertainty). The result is a chi-squared(1) statistic. At the default threshold of 6.63 (99th percentile), the gate fires when the innovation is larger than what the model can explain with 99% confidence. In **follow mode**, a gate exceedance resets the filter to the new measurement — the filter accepts the jump and tracks from the new level. In **exclude mode**, the filter ignores the measurement and continues with its prediction — useful for rejecting sensor glitches while maintaining the true estimate. The chi-squared threshold directly controls the false alarm rate: 3.84 = 5%, 6.63 = 1%, 10.84 = 0.1%. No tuning by trial and error — the statistics provide the guarantee (under Gaussian assumptions). --- ## References - Kalman, R.E. (1960). *A new approach to linear filtering and prediction problems.* Journal of Basic Engineering, 82(1), 35–45. [doi:10.1115/1.3662552](https://doi.org/10.1115/1.3662552) - Mehra, R.K. & Peschon, J. (1971). *An innovations approach to fault detection and diagnosis in dynamic systems.* Automatica, 7(5), 637–640. [doi:10.1016/0005-1098(71)90028-8](https://doi.org/10.1016/0005-1098(71)90028-8) - Basseville, M. & Nikiforov, I.V. (1993). *Detection of Abrupt Changes: Theory and Application.* Prentice Hall. [Free PDF](https://people.irisa.fr/Michele.Basseville/kniga/kniga.pdf) --- ## Next Steps - [Gradual Drift](/docs/playground/recipes/gradual-drift) — the complementary recipe for slow, invisible drift using fast/slow esMean crossover - [Frozen Sensors](/docs/playground/recipes/sensor-freeze) — detect when a sensor stops changing using running standard deviation collapse - [Under the Hood](/docs/concepts/under-the-hood) — understand what happens inside the pipeline --- # Hidden Parasitic Drain Source: https://composer.winkjs.org/docs/playground/recipes/hidden-load A `kalman1d` node with a control input predicts how much current *should* change the charge each tick. When the BMS measurement consistently falls below that prediction, the innovation drifts negative — the fingerprint of a leakage the sensor cannot see. `pageHinkley` accumulates statistical evidence of that shift and flags it. `sanitize` and `median3` clean the input upstream. ```javascript flow('parasitic-drain-detector') .sanitize('sane', 'soc', { failureReason: 'failReason' }, { ranges: { soc: { min: 0, max: 100 } } }) .median3('m3', 'soc', { median3: 'm3' }) .kalman1d('kf', 'm3', { filtered: 'estimate', innovation: 'innovation' }, { control: 'current', controlModel: 0.0167, sensorVariance: 0.01, processVariance: 0.0005 }) .pageHinkley('ph', 'innovation', { phShift: 'drainDetected', phTestStatistic: 'phStat' }, { delta: 0.005, lambda: 3, detectDrop: true, minWarmUpSamples: 30 }) .run() ``` **Drag the slider** past tick 200 and watch innovation develop a faint negative bias — the leakage's fingerprint. --- ## What You're Seeing The **gray line** is the raw BMS State of Charge — a noisy voltage-derived estimate that drops as the battery discharges. The **cyan line** is the Kalman-filtered estimate, smoother than the raw reading. The **purple curve** on the right axis is the innovation — the difference between what the filter predicted and what the BMS measured. For the first 200 ticks, the current sensor sees all the discharge. The filter predicts from current (coulomb counting), the BMS confirms, and innovation fluctuates around zero — the model matches reality. At tick 200 (the **amber vertical**), a faulty hydraulic valve controller on the auxiliary bus starts leaking 0.75A directly from the battery terminals — bypassing the traction current sensor. The sensor still reads 8A. The BMS, measuring cell voltage, reflects the true combined drain — the battery drops as if 8.75A were flowing. Innovation drifts negative: the BMS consistently reads a fraction of a percent lower than predicted. The shift is too small to see by eye on any single tick. But it is persistent, and persistence is what Page-Hinkley is built to find. About 70 ticks later (the **rose vertical**), Page-Hinkley has accumulated enough evidence to confirm the shift. At 60-second sampling, that is roughly 70 minutes from fault onset to automated detection — from a 0.75A leakage buried in sensor noise. **Toggle "Without Control."** The filter no longer uses current to predict — it just smooths the BMS readings. Innovation looks the same in both halves. No detection marker appears. The 0.75A leakage is indistinguishable from normal noise. The control input is what makes detection possible. --- ## Where This Pattern Fits | Domain | What leaks | Why it is invisible | |--------|-----------|---------------------| | EV fleets | Auxiliary loads downstream of the traction current sensor | Fleet telemetry reports "low battery" without identifying the cause | | HVAC compressors | Refrigerant leak forces longer run cycles at the same power draw | Energy meter reads higher but the compressor looks healthy | | Water distribution | Leak in a branch downstream of the main flow meter | Upstream meter cannot see downstream losses | | UPS systems | Parasitic load on a battery bank between inverter and load | UPS reports "battery weak" without pinpointing the drain | | Hydraulic presses | Internal seal leak wastes pressure the pump sensor cannot see | Pressure builds slower but the pump appears to run normally | --- ## How It Works The Kalman filter with a control input does coulomb counting: at each tick it predicts the SoC change from the known current. The `controlModel` parameter `G` converts amps to SoC% per tick. For a 100 Ah battery at 60-second intervals: > G = dt / (capacity × 3600) × 100 = 60 / (100 × 3600) × 100 ≈ 0.0167 At 8A of discharge, the filter predicts a 0.133% SoC drop per tick. Then it corrects with the BMS reading. When prediction and measurement agree, innovation hovers near zero. The `processVariance` (0.0005) is deliberately small — it tells the filter that SoC should change *only* because of the current, with very little unexplained drift. This makes the filter trust its coulomb-counting model. When the model is right (normal operation), innovation stays small. When the model is wrong (leakage), the unexplained SoC drop accumulates as a persistent innovation bias — 0.75A on a 100 Ah battery produces a −0.0125% bias per tick. That is one-eighth of the sensor noise on any single reading. No threshold alarm can catch it. But Page-Hinkley accumulates the evidence tick by tick until the shift is statistically confirmed. The `sensorVariance` (0.01) matches the BMS noise — σ² = 0.1² for a typical voltage-derived SoC estimate. Getting this right matters: too low and the filter over-trusts the measurement, passing noise through to the innovation. Too high and the filter ignores real BMS readings, delaying the diagnostic signal. ### Automated detection with Page-Hinkley The innovation is the diagnostic signal, but a human should not have to watch it. `pageHinkley` monitors the innovation stream and accumulates statistical evidence that its mean has shifted. The `delta` parameter (0.005) is the minimum shift worth detecting — below that, the change is within normal noise. `lambda` (3) controls how much evidence is needed before flagging. Higher lambda means fewer false alarms but slower detection. `detectDrop: true` tells Page-Hinkley to watch for a negative shift — the direction a leakage pushes the innovation. A positive shift (sensor reading *higher* than predicted) would indicate a different fault, like current sensor drift. ### Different faults, different signatures The innovation is a signed signal. Its direction and pattern identify the fault: - **Parasitic drain** (this recipe) — innovation drifts negative. The BMS consistently reads lower than the current-based prediction. The bias is proportional to the leakage: 0.75A on a 100 Ah battery creates a −0.0125% bias per tick — one-eighth of the sensor noise. - **Capacity fade** — the battery ages and its usable capacity drops, but the model still assumes the original 100 Ah rating. The filter under-predicts each tick's SoC change. Innovation drifts negative under load and returns to zero at rest — a load-dependent bias that grows over months. - **Current sensor drift** — the sensor develops a constant offset and reads high. The filter *overestimates* the drain each tick. Innovation flips *positive* — the BMS consistently reads higher than predicted. The opposite sign distinguishes a sensor problem from a battery problem. --- ## References - Kalman, R.E. (1960). *A new approach to linear filtering and prediction problems.* Journal of Basic Engineering, 82(1), 35–45. [doi:10.1115/1.3662552](https://doi.org/10.1115/1.3662552) - Page, E.S. (1954). *Continuous inspection schemes.* Biometrika, 41(1/2), 100–115. [doi:10.2307/2333009](https://doi.org/10.2307/2333009) - Plett, G.L. (2004). *Extended Kalman filtering for battery management systems of LiPB-based HEV battery packs.* Journal of Power Sources, 134(2), 252–261. [doi:10.1016/j.jpowsour.2004.02.031](https://doi.org/10.1016/j.jpowsour.2004.02.031) --- ## Next Steps - [Sudden Shifts](/docs/playground/recipes/sudden-shift) — the same node in a different mode: detect step changes with innovation gating instead of tracking gradual drift - [Kalman 1D Filter Explorer](/docs/playground/explore-nodes/kalman1d) — drag sliders to build intuition about each parameter - [Composition Patterns](/docs/concepts/composition-patterns) — how independent nodes compose into richer flows --- # PID Loop Hunting Source: https://composer.winkjs.org/docs/playground/recipes/loop-hunting This recipe counts how often the signal reverses direction inside a sliding window. The `winnow` node fires on every trajectory turning point, `swStats` averages those events into a continuous rate, and `categorize` bands the rate into a graded health state. > **Why frequency, not amplitude.** Two oscillations at the same > amplitude can produce opposite outcomes. A ten-minute period is a > tuning nuisance. A one-minute period at the same amplitude shortens > the modulating valve's life by years. A control chart (`mean ± 2σ`) > treats them as identical because it measures amplitude alone. This > recipe measures how often the signal turns around. ```javascript flow('loop-hunting-detector') .sanitize('sane', 'temperature', { failureReason: 'failReason' }, { ranges: { temperature: { min: 0, max: 200 } } }) .median3('med', 'temperature', { median3: 'medT' }) .esMean('slow', 'medT', { mean: 'slowT' }, { halfLife: 3 }) .esStats('noise', 'slowT', { stdev: 'tStdev' }, { halfLife: 30 }) .trend('dir', 'slowT', { trend: 'tDir', rocMean: 'tRoc' }, { rocStatsHalfLife: 30, rocThreshold: 0.015, warmupSamples: 30 }) .winnow('events', 'slowT', { significant: 'turningPoint' }, { K: 7, tightenBase: 5000, maxGap: 5000, noiseField: 'tStdev', // from esStats above dirField: 'tDir', // from trend above slopeField: 'tRoc' // from trend above }) .transform('encode', 'turningPoint', { result: 'eventBit' }, { using: ( v ) => v }) .swStats('rate', 'eventBit', { mean: 'reversalRate' }, { windowSize: 600 }) .categorize('level', 'reversalRate', { category: 'cyclingHealth' }, { thresholds: [ 0.001, 0.0115, 0.020 ], categories: [ 'stable', 'minor', 'major', 'severe' ] }) .run() ``` **Drag the slider** through the four phases and watch the recipe walk from `stable` through `minor` and `major` to `severe` — even though the three oscillating phases share the same amplitude. --- ## What You're Seeing The chart is divided into four 30-minute phases. Subtle background shading marks the three hunting phases; the unshaded first 30 minutes are a stable baseline. **The gray line** is raw temperature near 80°C. Look at the wave heights across all four phases — they are the same, about ±2°C. An amplitude alarm would see no difference between any of them. Now look at the wave speed. In the slow-hunt phase the signal completes one full cycle in about ten minutes. In the medium phase, five minutes. In the fast phase, two and a half. Same height, doubling speed — that is what a hunting control loop looks like as it worsens. **The amber curve** (right axis) is the reversal rate — how many direction changes the pipeline counted per sample over the last 600 samples. It sits near zero in the stable phase and steps up at each phase boundary as faster oscillations feed more turning points into the window. This single number is what the pipeline is built to produce. **The health card** below the chart shows the graded state at the slider position: `stable → minor → major → severe`. A standard control chart (mean ± 2σ) cannot make this distinction — all three hunting phases have the same amplitude and would land in the same bucket. This recipe separates them by frequency alone. Drag past minute 10 to see the first state appear — `trend` and `swStats` need their warmup windows to fill before they can report. --- ## Where This Pattern Fits | Domain | What hunts | Why it's hard to catch | |--------|-----------|------------------------| | Boiler temperature loop | Jacket or outlet temp under PID control | Oscillation amplitude stays within spec; only the period changes as tuning degrades | | Reactor jacket cooling | Inlet or outlet temperature | Slow oscillations look like normal batch behaviour on a dashboard | | Modulating control valve | Valve position or downstream flow | Mechanical wear produces frequent small reversals that no single reading flags | | Tank level under flow control | Level transmitter | Level oscillates around setpoint with no trend — every reading is in range | | Heat exchanger outlet | Downstream fluid temperature | Hunting appears only when the upstream loop adjusts to new operating conditions | --- ## How It Works Two design choices make this recipe work. Both are calibrated against the feasibility study. Remove either one and classification accuracy drops from 100% to 36% or worse. **Smooth twice before taking the slope.** `median3` absorbs single-sample spikes. `esMean` with `halfLife = 3` then drops the residual noise on the smoothed signal to about 0.013 — below `trend`'s `rocThreshold` of 0.015. Without `esMean`, `trend`'s rate-of-change estimate flickers on noise and `winnow`'s trend-reversal trigger fires on every flicker, flooding the rate with false events. The general rule: a slope estimator's noise floor scales as `2σ/√halfLife`; the signal slope you want to detect must sit above that floor. **Set winnow's `K` to 7, not the default of 2.** `winnow`'s deadband fires when the signal departs from its projected path by more than `K × stdev`. Think of K as a sensitivity dial — lower values trigger more fires, higher values suppress false alarms. At `K = 3`, about 3% of samples in the stable phase trigger a background fire from noise alone — enough to swamp the real hunting signal. At `K = 7` the false-fire probability drops below one per million samples. The general rule: when `winnow` feeds an event counter, `K` needs to be well above the compression-oriented defaults. **Keep the deadband width constant.** `winnow` progressively narrows its deadband as elapsed samples grow — a compression feature that inserts intermediate anchor points in long quiet segments. In a pure event-counting role that tightening would seed spurious fires inside the sliding window and inflate the rate. Setting `tightenBase: 5000` and `maxGap: 5000` — both well beyond the 600-sample `swStats` window — pins the deadband at its full `K × σ` width across the entire window and disables the gap-fill anchor. Lower values are correct for compression but wrong here. **Counting the events.** Each `winnow.significant = true` is a single event. The `transform` encoder converts the boolean to 0 or 1 so `swStats` can read it as a numeric field. `swStats(mean)` over a 600-sample window gives a continuous rate — fires per sample. Winnow fires at peaks, at troughs, and occasionally on large-enough noise excursions during the cycle, so the reported rate tracks the cycle frequency *monotonically* — faster hunting always produces a higher rate — but it is not literally `2 × cycles-per-sample`. That is why `categorize.thresholds` are set against observed rates on a clean baseline, not against a ground-truth formula. The operator gets a single state field — `stable`, `minor`, `major`, or `severe` — that they can log, alarm on, or display. **Wire winnow to the renamed fields.** Winnow reads four fields from the message on every tick: its primary input (`from.x`), plus a noise estimate, a trend direction, and a slope value for its deadband and reversal checks. By default these three supporting fields are named `stdev`, `trendDir`, and `roc`. The pipeline above renames them with a `t` prefix (`tStdev`, `tDir`, `tRoc`), so `winnow` must be pointed at the renamed fields via `noiseField`, `dirField`, and `slopeField`. Miss this wiring and `winnow` silently reads `undefined`, its warm-up guard fires on every sample, and the downstream rate saturates at 1.0. Three options — easy to overlook, load-bearing. The wider lesson is the composition pattern. A per-sample event detector, a sliding-window rate, and a graded state — the same shape applies wherever you need to turn discrete events into a continuous health indicator. ### Tuning to your loop The published thresholds are calibrated against the synthetic signal. On your own loop, calibrate from a clean baseline: 1. Run the pipeline for at least a week on a loop you know is healthy. Capture `reversalRate` to storage. 2. Note the stable-state value. Call it `r_normal`. On a healthy loop it is usually between 0 and ~0.002. 3. Set `categorize.thresholds` to `[ max(r_normal × 2, 0.001), r_normal × 5, r_normal × 10 ]`. 4. Leave the rest — `K`, the smoothing chain, `tightenBase`, `maxGap` — at the values in this recipe. They are calibrated against the noise floor of the smoothed signal, not against the loop. The table below is the reference for the other knobs. | Parameter | How to set it | |---|---| | `from.x` | Your loop variable | | `winnow.K` | 7 is the conservative floor. Drop to 6 only if your stable baseline is cleaner than the synthetic noise | | `categorize.thresholds` | Use the four-step procedure above | | `swStats.windowSize` | About three times the longest period of hunt you expect to detect. A ten-minute slowest hunt at one sample per second means `windowSize: 1800` | | `esMean.halfLife` | 3 is good for noise σ ≤ 0.1. Increase if your signal is noisier — but faster oscillations then get attenuated by the heavier smoothing | ### What this recipe is not **Period floor.** This recipe is calibrated for slow-to-medium control loops where one cycle contains at least about 150 samples. For HVAC compressor short-cycling at 30–60 second intervals, the `trend` node's slope estimator cannot resolve the oscillation — use a `kalman1d`-based detector instead. **Drift tolerance.** The slope-aware projection handles baseline drift up to roughly 0.005 units per sample. Faster drifts — strong setpoint ramps, large ambient shifts, slow production trends — walk `winnow`'s anchor out from under the signal and classification degrades. Place a `kalman1d`-based detrender upstream in that regime. **Sinusoidal vs real limit cycles.** The synthetic demo signal is sinusoidal. Real hunting — especially from valve stiction — is asymmetric: long dwells punctuated by sudden jumps at reversal. The reversal rate still rises with cycle frequency, but the `categorize` thresholds will need per-loop calibration against your own clean baseline (see the procedure above). **Other shapes of failure.** For a signal that drifts unidirectionally off its setpoint with no oscillation, use [Gradual Drift](/docs/playground/recipes/gradual-drift). For a signal that jumps to a new level, use [Sudden Shifts](/docs/playground/recipes/sudden-shift). --- ## References - Bristol, E.H. (1990). *Swinging Door Trending: Adaptive Trend Recording?* ISA National Conference Proceedings, pp. 749–756. (Direct heritage of winnow's deadband + trajectory model.) - Welford, B.P. (1962). *Note on a method for calculating corrected sums of squares and products.* Technometrics, 4(3), 419–420. [doi:10.1080/00401706.1962.10490022](https://doi.org/10.1080/00401706.1962.10490022) (The recurrence underneath esStats.) - Åström, K.J. & Hägglund, T. (2006). *Advanced PID Control.* ISA Publishing. (Loop-tuning and hunting diagnosis, chapter 7.) --- ## Next Steps - [Trajectory-Aware Adaptive Compression](/docs/playground/recipes/adaptive-compression) — the other winnow recipe; same per-sample event mechanism, a different downstream consumer - [Sudden Shifts](/docs/playground/recipes/sudden-shift) — step changes rather than oscillations - [Composition Patterns](/docs/concepts/composition-patterns) — understand the per-sample-event → rate → state pattern in detail --- # Frozen Sensors Source: https://composer.winkjs.org/docs/playground/recipes/sensor-freeze Sensor freeze is one of the most insidious data quality failures. It does not trigger range alarms, does not produce errors, does not look obviously wrong. Every downstream decision — mass balance, leak detection, process control — silently corrupts. This recipe builds a flow that continuously learns what "normal variability" looks like for a signal, and uses that knowledge to detect when variability disappears. **Learns, adapts, preserves.** A long-term esStats node *learns* the signal's normal variability. A dynamic threshold *adapts* from that learned baseline — no hardcoded values, works for any signal. A controller trigger *preserves* the learned state when a freeze is confirmed — anomalous data cannot corrupt what the system already learned. ```javascript flow( 'sensor-freeze-detector' ) .sanitize( 'sane', 'flowRate', { failureReason: 'failReason' }, { ranges: { flowRate: { min: 0, max: 500 } } } ) .median3( 'm3', 'flowRate', { median3: 'm3' } ) .esStats( 'stats', 'm3', { mean: 'mean', stdev: 'sigma' }, { halfLife: 3 } ) .esStats( 'baseline', 'm3', { stdev: 'baselineSigma' }, { halfLife: 60 } ) .threshold( 'thr', 'sigma', { active: 'sigmaLow' }, { mode: 'below', threshold: ( msg ) => msg.baselineSigma * 0.10, hysteresis: ( msg ) => msg.baselineSigma * 0.12 } ) .persistenceCheck( 'confirm', ( msg ) => msg.sigmaLow === true, { persistenceConfirmed: 'confirmedFreeze' }, { minVotes: 4, outOfTotal: 6, triggers: [ { control: 'pause', targets: [ 'baseline' ] } ] } ) .run() ``` **Drag the slider** and watch sigma collapse the moment the sensor stops changing — then watch the adaptive threshold catch it. --- ## What You're Seeing The gray line is the raw flow rate — noisy readings around 200 L/min. The **cyan line** is the exponentially smoothed mean, tracking the underlying signal. The **violet dashed curve** on the right axis is sigma — the running standard deviation. During normal operation, sigma fluctuates around 2–3. Between samples 150 and 200, a quiet period reduces the noise, and sigma dips — but stays well above the detection threshold. The detector does not fire. Two **amber dashed lines** on the right axis show the adaptive detection boundaries. These are not hardcoded — they are computed as fractions of the long-term baseline sigma (10% for the threshold, 22% for the reset point). Watch them converge during the first 50–60 samples as the baseline stabilises. At sample 200, the sensor freezes. Sigma decays exponentially toward zero and crosses the lower amber line. The **rose shaded region** marks where sigma is below threshold. The **rose vertical** marks the moment the persistence check confirms — 4 of 6 consecutive readings showed sigma below threshold. At confirmation, a controller trigger pauses the baseline to prevent the frozen data from dragging the reference down. After sample 320, the sensor recovers. Sigma climbs back above the upper amber line (the hysteresis reset point), and the detector clears. --- ## Where This Pattern Fits | Domain | What freezes | Why it is invisible | |--------|-------------|---------------------| | Water treatment | pH sensor stuck on last reading | Readings look normal — pH changes slowly | | HVAC | Temperature sensor on a frozen output | The building management system trusts the value and stops heating | | Oil and gas | Flow transmitter electronics lock up | Constant flow looks like stable operation | | Meteorology | Wind sensor bearings seize | Zero variation reads as calm wind, not a broken anemometer | | Manufacturing | Pressure transducer membrane stuck | Constant pressure looks like process stability | --- ## How It Works Two esStats nodes run in parallel on the same median-filtered input. The short-term node (halfLife=3) computes sigma — the fast-responding standard deviation that collapses during a freeze. The long-term baseline (halfLife=60) establishes what sigma normally looks like for this signal. The baseline converges slowly and is deliberately insensitive to short disruptions. The threshold node uses dynamic options — functions that receive the current message and return a value. The threshold is set to 10% of the baseline sigma; the hysteresis to 12%. With a typical baseline of 2.5, the threshold sits at about 0.25 and the reset point at about 0.55. These values adapt automatically to the signal — no manual calibration. Why 10%? Any estimate computed from a small number of samples fluctuates. The shorter the half-life, the fewer samples contribute, and the wider sigma swings during normal operation. Statistics gives us a precise floor: with halfLife=3, sigma is effectively computed from about 6 samples, and even in a 1-in-1000 unlucky stretch it will not dip below roughly 33% of its true value. Setting the threshold at 10% — well below that natural floor — makes a false alarm virtually impossible even before the persistence check adds its own safety margin. The multiplier is not a guess; it follows from the half-life. When the persistence check confirms a freeze (4 of 6), it fires a controller trigger that pauses the baseline esStats node. Pausing stops the update function while keeping the last-known baseline visible downstream. This prevents the frozen data — with its collapsing variance — from dragging the baseline down, which would lower the threshold and delay recovery detection. During the quiet period in this demo (samples 150–200), sigma drops but stays well above the adaptive threshold — the detector does not fire. This is the specificity test: reduced noise is not the same as no noise. --- ## References - Welford, B.P. (1962). *Note on a method for calculating corrected sums of squares and products.* Technometrics, 4(3), 419–420. [doi:10.1080/00401706.1962.10490022](https://doi.org/10.1080/00401706.1962.10490022) - Sharma, A.B., Golubchik, L. & Govindan, R. (2010). *Sensor faults: Detection methods and prevalence in real-world datasets.* ACM Transactions on Sensor Networks, 6(3), 1–39. [doi:10.1145/1754414.1754419](https://doi.org/10.1145/1754414.1754419) --- ## Next Steps - [Gradual Drift](/docs/playground/recipes/gradual-drift) — the complementary recipe for slow, invisible drift using fast/slow esMean crossover - [Sudden Shifts](/docs/playground/recipes/sudden-shift) — abrupt step changes detected by Kalman innovation gating - [Under the Hood](/docs/concepts/under-the-hood) — understand what happens inside the pipeline --- # Subtle Process Shifts Source: https://composer.winkjs.org/docs/playground/recipes/run-rules This recipe uses esStats to compute a z-score for each sample, a passIf gate to skip the warmup period, and four persistenceCheck nodes to evaluate the classic Western Electric rules against that z-score. No median3 — run rules must see isolated spikes, not smooth them away. ```javascript flow( 'run-rules' ) .sanitize( 'sane', 'measurement', { failureReason: 'failReason' }, { ranges: { measurement: { min: 0, max: 100 } } } ) .esStats( 'stats', 'measurement', { mean: 'mean', stdev: 'sigma', zScore: 'zScore' }, { halfLife: 30 } ) .passIf( 'warmup', ( msg, counter ) => counter > 60 ) .persistenceCheck( 'rule1', ( msg ) => msg.zScore > 3, { persistenceConfirmed: 'beyond3sigma' }, { minVotes: 1, outOfTotal: 1 } ) .persistenceCheck( 'rule2', ( msg ) => msg.zScore > 2, { persistenceConfirmed: 'twoOfThree' }, { minVotes: 2, outOfTotal: 3 } ) .persistenceCheck( 'rule3', ( msg ) => msg.zScore > 1, { persistenceConfirmed: 'fourOfFive' }, { minVotes: 4, outOfTotal: 5 } ) .persistenceCheck( 'rule4', ( msg ) => msg.zScore > 0, { persistenceConfirmed: 'eightSameSide' }, { minVotes: 8, outOfTotal: 8 } ) .run() ``` **Drag the slider** and watch the rules fire in sequence as the deviation grows. --- ## What You're Seeing The gray line is the raw process measurement — noisy readings around 50. The **cyan line** is the exponentially smoothed mean (halfLife=30), tracking the underlying process level. The **faded cyan band** marks the ±3σ control limits. The **violet dashed curve** on the right axis is the z-score — how many standard deviations each sample deviates from the baseline. At sample 80, an isolated spike pushes the measurement well above the upper control limit. The z-score jumps past 5, and **Rule 1** fires immediately — the first rose marker. Because there is no median3 in this pipeline, the spike reaches the rules unfiltered. After sample 100, the process mean begins creeping upward. As the deviation grows, the rules fire in sequence: **Rule 2** catches early noise peaks that push past 2σ. **Rule 4** fires when eight consecutive readings land above center — sustained bias that random noise cannot explain. **Rule 3** fires last, once the deviation has grown large enough that four of five readings consistently exceed 1σ. --- ## Where This Pattern Fits | Domain | What shifts | Why threshold alarms miss it | |--------|------------|------------------------------| | Pharmaceutical manufacturing | Tablet press fill weight after blade change | New blade shifts mean by 0.5% — within spec, but yield drops | | Semiconductor fabrication | Etch depth after chamber clean | 0.3 nm offset — no single wafer fails, batch yield declines | | Automotive assembly | Torque setting across operator shifts | New operator shifts mean by 2 Nm — within tolerance | | Dairy processing | Pasteurization temperature after CIP cycle | Thermocouple reseats 0.3°C lower — readings look normal | | Water treatment | pH after reagent batch change | New reagent lot shifts reading by 0.1 — compliance alarm is ±0.5 | --- ## How It Works ### No Median3 Unlike the drift and freeze recipes, this pipeline omits median3. The median filter smooths transient spikes — exactly the events Rule 1 is designed to catch. Including it would defeat the purpose. ### Z-Score as the Detection Signal The esStats node maintains exponentially smoothed estimates of the mean and standard deviation. For each incoming sample, it computes a z-score *before* updating the estimates — measuring how surprising the new value is relative to what the process has been producing, not what it is producing now. This preserves detection semantics: the baseline does not absorb the deviation before the rules notice it. A halfLife of 30 means that samples 30 readings ago carry half the weight of the current one. Short enough for a responsive baseline; long enough that the mean lags behind a growing deviation, keeping z-scores elevated. ### Warmup Gate A passIf node blocks the first 60 messages — giving esStats time to converge before the run rules begin evaluating. The gate sits between esStats and the rules: esStats receives every sample (building its baseline), but the rules only see messages after the baseline stabilizes. Without this gate, erratic early z-scores produce false detections. ### The Four Rules | Rule | What it detects | persistenceCheck | Sensitivity | |------|----------------|------------------|-------------| | **Rule 1** | Isolated spike | 1-of-1 beyond 3σ | Lowest — catches only large excursions | | **Rule 2** | Repeated exceedance | 2-of-3 beyond 2σ | Moderate — needs consistent deviation | | **Rule 3** | Sustained bias | 4-of-5 beyond 1σ | High — catches subtle shifts | | **Rule 4** | Process bias | 8-of-8 same side | High — catches any persistent asymmetry | Each persistenceCheck maintains a sliding vote window. It evaluates its predicate on each message, records a vote (true or false), and confirms when the minimum threshold is reached within the window. When confirmed — or when confirmation becomes mathematically impossible — the window auto-resets for the next detection. ### One-Sided Detection The predicates check the positive direction only (`msg.zScore > 1`, not `Math.abs( msg.zScore ) > 1`). This halves the false alarm rate compared to two-sided checking, and matches the classical Western Electric formulation: each rule is evaluated separately on each side of center. For below-center detection, add a parallel set of rules with `msg.zScore < 0`, `msg.zScore < -1`, `msg.zScore < -2`, and `msg.zScore < -3`. ### Progressive Sensitivity The rules form an early warning cascade. As a process deviation grows, the rules fire in order of what they can see: Rule 2 catches the first noise peaks that exceed 2σ. Rule 4 detects sustained bias. Rule 3 confirms once the deviation is large enough that most readings consistently exceed 1σ. Rule 1 catches isolated excursions that the pattern rules miss entirely. --- ## References - Western Electric Company (1956). *Statistical Quality Control Handbook.* AT&T Technologies, Indianapolis. - Montgomery, D.C. (2019). *Introduction to Statistical Quality Control,* 8th ed. Wiley. [ISBN 978-1-119-39930-8](https://www.wiley.com/en-us/Introduction+to+Statistical+Quality+Control%2C+8th+Edition-p-9781119399308) - Wheeler, D.J. (2000). *Understanding Variation: The Key to Managing Chaos,* 2nd ed. SPC Press. --- ## Next Steps - [Gradual Drift](/docs/playground/recipes/gradual-drift) — the complementary recipe for slow, continuous drift using fast/slow esMean crossover - [Sudden Shifts](/docs/playground/recipes/sudden-shift) — abrupt step changes detected by Kalman innovation gating - [Frozen Sensors](/docs/playground/recipes/sensor-freeze) — detect when a sensor stops changing using running standard deviation collapse - [Under the Hood](/docs/concepts/under-the-hood) — understand what happens inside the pipeline --- # Subsampling Noisy Signals with a Deadband Source: https://composer.winkjs.org/docs/playground/recipes/subsample-deadband A vibration sensor sampling at 20 kHz produces 1.7 billion readings a day. Most of those readings carry no new information — the signal is sitting on its noise floor. The trick is keeping the samples that depart from the noise and dropping the ones that don't, without having to pick a threshold by hand for every signal in the plant.} /> The first reach is usually fixed-rate subsampling: keep every Nth sample, drop the rest. One node, zero tuning, which is exactly why it's so common. Here's what that looks like on a test signal with a quiet baseline, three brief pulses, a slow ramp, and a noisier settling regime. ```javascript flow('fixed-rate-subsample') .passIf('every5', (msg, counter) => (msg.index % 5) === 0) .run() ``` **Drag the slider** and watch the reconstruction. Fixed-rate keeps the same 20% of samples regardless of what the signal is doing. Fixed-rate is blind to where the events actually are. It drops through the pulses — three events of five samples each are almost entirely thrown away — and it keeps samples in the quiet baseline just as densely as it does in the noisy regime. The reconstruction cannot recover what the gate discarded. The fix is an adaptive deadband. Run an `esStats` node to estimate the local noise, then use `passIf` with a predicate that keeps the sample only when its deviation from the running mean exceeds K times the local stdev. The threshold breathes with the signal — tight when quiet, wide when noisy — and the gate concentrates its budget where the information actually is. ```javascript flow('subsample-deadband') .sanitize('sane', 'value', { failureReason: 'failReason' }, { ranges: { value: { min: -50, max: 50 } } }) .median3('m3', 'value', { median3: 'smoothed' }) .esStats('stats', 'smoothed', { mean: 'mean', stdev: 'stdev' }, { halfLife: 50 }) .passIf('keep', (msg, counter) => ((msg.index % 5) === 0) || (Math.abs(msg.value - msg.mean) > 1.5 * msg.stdev)) .run() ``` **Drag the slider** again and compare: the kept-sample dots now cluster on the pulses, on the top of the ramp, and on the larger excursions in the noisy regime. Nothing is wasted on the quiet baseline. --- ## What You're Seeing Both demos run over the same four-region signal: a quiet baseline, three brief pulses, a slow upward ramp, and a noisier settling regime at a new level. The **slate line** is the raw signal; the **orange line** is the linear-interpolation reconstruction between consecutive kept samples — the signal a downstream consumer would see if it only stored the amber-dot points. In the **fixed-rate demo**, the kept dots sit at `msg.index % 5 === 0` — a regular lattice across the entire signal. Each of the three pulses catches at most one kept sample (and sometimes zero), so the reconstruction misses the pulse heights entirely. In the quiet baseline region the gate keeps 20% of samples even though almost none of them carry information worth storing. In the **adaptive demo**, the **cyan line** is the running mean from `esStats` and the **dashed violet lines** are the adaptive deadband at mean ± 1.5σ. The `passIf` gate combines two conditions — a heartbeat that keeps every fifth sample (the same base rate as fixed-rate), plus the adaptive deadband that keeps any sample whose deviation exceeds K times the local stdev. The deadband is a strict superset of fixed-rate — same lattice in quiet regions, plus event-concentrated extras. The pulses trip the adaptive gate for each excursion; the ramp produces a scattering as the lagging mean catches up; and in the noisy regime the stdev expands to match the new noise floor, so the dots scatter on the larger excursions. ### Reading the KPI strip Each chart shows three metrics, measured only over the *skipped* points — the ones the algorithm had to reconstruct, not the ones it stored exactly. - **Compression** — the percentage of samples dropped. Higher means less stored. - **Max Error** — the worst single-point reconstruction deviation. This occurs at event boundaries — the gap between the last quiet sample and the first event sample, where the reconstruction ramps linearly but the actual signal steps. The deadband improves on fixed-rate here because its adaptive extras provide closer interpolation anchors near events. - **RMS Error** — the average reconstruction quality across all skipped points. The deadband's extras pull the reconstruction closer to the signal in event regions, reducing the overall average. The deadband keeps more samples than fixed-rate (lower compression) but places them where they reduce reconstruction error. The trade-off is deliberate: spend storage budget on event fidelity, not quiet-region redundancy. For signals with sharp structural breaks (step changes, regime shifts), the [adaptive compression](/docs/playground/recipes/adaptive-compression) recipe adds a Kalman predictor and boundary anchoring to handle what the deadband cannot. --- ## Where This Pattern Fits | Domain | What you're keeping | Why a fixed threshold fails | |--------|---------------------|------------------------------| | Vibration monitoring | Excursions above the running noise floor | Bearing wear changes the noise floor over time | | Pressure logging | Pressure bumps from valve actions | The baseline drifts with temperature | | Power quality | Voltage sags below local nominal | Nominal voltage changes by region and season | | Process tags | Setpoint deviations | Recipes change the setpoint daily | | Battery monitoring | Step changes during load events | Resting voltage drifts with state of charge | --- ## How It Works The `esStats` node maintains an exponentially weighted running mean and stdev. With `halfLife: 50`, each sample's influence halves every 50 steps — the estimate tracks the most recent few hundred samples, long enough to be stable, short enough to follow real changes in the signal. The first ~50 samples are warmup; the deadband settles after that. The `passIf` predicate combines two conditions with OR. The first is a **heartbeat** — `msg.index % 5 === 0` — that keeps every fifth sample, matching the fixed-rate baseline. This makes the deadband a strict superset of fixed-rate: identical coverage in quiet regions, with adaptive extras layered on top during events. The second is the **adaptive deadband** — `|value - mean| > K * stdev` — the gate that keeps any sample whose deviation from the running mean exceeds K times the local stdev. Note the asymmetry: `esStats` reads from the median-smoothed stream so its stats are noise-protected, but the gate itself tests the raw `value` — the same signal the chart draws — so the reader's eye and the algorithm see the same thing. Because the predicate is multiplicative on stdev — not additive on the value itself — the threshold scales with whatever the local noise happens to be, with no absolute tolerance to pick per signal. K = 1.5 is the starting point used here. Under a Gaussian baseline, samples lying outside ±1.5σ from the mean account for roughly 13% of the population — so the adaptive gate fires on about one in eight samples during quiet operation. Combined with the every-fifth heartbeat, the total kept rate in quiet regions is around 28%. Tighter K (closer to 1.0) keeps more; wider K (closer to 2.0) keeps fewer. K itself is data-dependent: the value here was chosen for the four-region synthetic signal the demo runs on, and other datasets should sweep K on their own representative data before deploying. The `median3` node sits between sanitize and esStats to absorb single-sample spikes. Without it, one bad reading would inflate the running stdev and quietly widen the deadband for many samples afterwards. It's a standard hygiene step at the head of any pipeline that reads from a noisy sensor. --- ## References - Welford, B.P. (1962). *Note on a method for calculating corrected sums of squares and products.* Technometrics, 4(3), 419–420. [doi:10.1080/00401706.1962.10490022](https://doi.org/10.1080/00401706.1962.10490022) - Bristol, E.H. (1990). *Swinging Door Trending: Adaptive Trend Recording?* ISA National Conference Proceedings, pp. 749–756. --- ## Next Steps - [Trajectory-Aware Adaptive Compression](/docs/playground/recipes/adaptive-compression) — when reconstruction quality matters: adds a Kalman predictor and boundary anchoring to cut max error by 3–4× - [Sudden Shifts](/docs/playground/recipes/sudden-shift) — when the goal is detecting the events instead of keeping the samples that mark them - [Under the Hood](/docs/concepts/under-the-hood) — understand how messages flow through composed nodes --- # Trajectory-Aware Adaptive Compression Source: https://composer.winkjs.org/docs/playground/recipes/adaptive-compression This recipe wires seven nodes into a trajectory-aware subsampling flow. A `kalman1d` predictor with `followMode` runs alongside `esStats` and `trend`; the `winnow` node fuses the slope, the noise estimate, and the innovation gate into a single per-sample keep decision — its one-sample buffer anchors each spike to its baseline neighbor; `passIf` forwards the kept samples; a second `kalman1d` smooths measurement noise from the kept values before storage. ```javascript flow('adaptive-compression') .median3('med', 'value', { median3: 'smoothed' }) .kalman1d('kf', 'value', { filtered: 'filtered', innovation: 'innovation', innovationGate: 'gate' }, { sensorVariance: 0.005, processVariance: 2e-5, chi2Threshold: 6.63, followMode: true }) .esStats('stats', 'smoothed', { stdev: 'stdev', mean: 'mean' }, { halfLife: 50 }) .trend('slope', 'smoothed', { trend: 'trendDir', confidence: 'trendConf', rocMean: 'roc' }, { rocStatsHalfLife: 20, rocThreshold: 0.005, warmupSamples: 15 }) .winnow('compress', 'smoothed', { significant: 'store', deviation: 'dev', predicted: 'pred', xPrev: 'prevValue', tPrev: 'prevTimestamp' }, { K: 2, tightenBase: 100, maxGap: 50, chi2Threshold: 12, bufferPrev: true, timestampField: 'timestamp' }) .passIf('gate', (msg, counter) => msg.store === true) .kalman1d('kfSmooth', 'value', { filtered: 'storedValue' }, { sensorVariance: 0.005, processVariance: 0.0025, chi2Threshold: 6.63, followMode: true }) .run() ``` ### On the deadband's signal The [deadband recipe](/docs/playground/recipes/subsample-deadband) runs on a four-region test signal — quiet baseline, three pulses, a slow ramp, a noisier regime. **Drag the slider** to the end and compare the KPI numbers below with those on the deadband page. ### On a harder signal Now watch what happens on a signal with two sharp step changes — one up at sample 200, one down at sample 350 — plus high-amplitude vibration between them. **Drag the slider** through all five regions. --- ## What You're Seeing Both charts show the same three elements: the **cyan line** is the original signal, the **orange line** is the linear-interpolation reconstruction between consecutive kept samples, and the **amber dots** mark the samples the flow decided to store. **First chart (deadband's signal).** The same quiet-baseline, pulses, ramp, and noisy-regime signal the deadband recipe uses. Compare the KPI numbers directly: the adaptive flow achieves tighter reconstruction error because `winnow` fuses trajectory, noise estimate, and innovation into each keep decision — not just deviation from a running mean. **Second chart (harder signal).** A quiet sinusoid, a slow ramp, a sharp step UP at sample 200, high-amplitude vibration, then a sharp step DOWN at sample 350. Two structural breaks in opposite directions — this is where the trajectory awareness earns its keep. At each step, the Kalman innovation gate trips; KF1's `followMode` snaps the predictor to the new level in one tick; `winnow` forces a kept sample so the reconstruction tracks the step instead of trailing through it. A local-only gate like the [deadband](/docs/playground/recipes/subsample-deadband) would lag behind both transitions — the running mean cannot keep up with instantaneous structural breaks. The KPI strip under each chart shows three numbers: the compression ratio, the largest single-sample reconstruction deviation, and the RMS reconstruction error in the same units as the signal. --- ## Where This Pattern Fits | Domain | What you're keeping | |--------|---------------------| | Vibration monitoring | Excursions and inflections in 20 kHz accelerometer streams | | Current transducers | Step changes and ripple in motor drive feedback | | Acoustic emission | Bursts above the running noise in ultrasonic AE sensors | | Fleet telemetry | Speed, RPM, and torque excursions across vehicles | | Distributed motors | Bearing and stator currents across drives | --- ## How It Works The seven nodes split the work into single-purpose pieces. `median3` absorbs single-sample spikes. The first `kalman1d` predicts the signal's trajectory from a constant-velocity model and fires its innovation gate (a chi-squared test) when reality disagrees with the prediction — that gate is what catches step changes the deadband alone would miss. KF1's `followMode` snaps the filter to the new measurement when the gate fires. The predictor tracks the jump in a single tick instead of staying anchored to the old level. `esStats` maintains the running mean and stdev that set the local noise scale. `trend` classifies the signal's direction and its rate of change. `winnow` is the decision node — it fuses the slope, the noise estimate, the trend direction, and the innovation gate into a single per-sample `store` flag. Its one-sample buffer (`bufferPrev`) publishes the previous tick's value on gate-fire keeps, giving the reconstruction an anchor at the sample before each spike. `passIf` drops every sample whose `store` flag is false. The second `kalman1d` reads from the raw value (not the median3 output) and smooths measurement noise from the kept samples before storage. The single sensitivity parameter is `K` on `winnow`. K = 2 means "store when the deviation exceeds twice the current noise floor." Because the comparison is multiplicative on the running stdev, the threshold scales with whatever the local noise happens to be — there is no absolute tolerance to pick per signal. K itself is dataset-specific: the value here is tuned on the NASA IMS bearing data, and other datasets should sweep K on their own representative signals before deploying. The first `kalman1d` reads from the raw `value`, not from the median-smoothed stream. The Kalman filter's `sensorVariance` parameter is the right place to model measurement noise — pre-filtering with median3 would double-filter and leave the chi-squared innovation gate (6.63, a 1% false-alarm test) under-calibrated. `esStats` and `trend` still read from `smoothed` so their running statistics stay noise-protected. KF1 and `winnow` each have their own `chi2Threshold`. KF1's threshold (6.63) controls detection sensitivity — how surprising a sample must be before the innovation gate fires and `followMode` snaps the predictor. `winnow`'s threshold (12) controls storage sensitivity — how extreme a gate fire must be before it forces a kept sample. On vibration-only signals, set both to 6.63. On signals with step changes and high post-step noise, raise `winnow`'s threshold so it stores structural breaks without keeping every noisy post-step sample. `winnow`'s `maxGap = 50` guarantees at most 50 samples between kept points — a minimum sampling-rate fallback that rarely fires on vibrating data but prevents the reconstruction from flat-lining through a long quiet stretch. Like every recipe, these parameter values are a starting point. Adapt K, the chi-squared thresholds, and the Kalman variances to your own signal characteristics and storage budget before deploying. --- ## References - Bristol, E.H. (1990). *Swinging Door Trending: Adaptive Trend Recording?* ISA National Conference Proceedings, pp. 749–756. - Kalman, R.E. (1960). *A New Approach to Linear Filtering and Prediction Problems.* Journal of Basic Engineering, 82(1), 35–45. [doi:10.1115/1.3662552](https://doi.org/10.1115/1.3662552) - Welford, B.P. (1962). *Note on a method for calculating corrected sums of squares and products.* Technometrics, 4(3), 419–420. [doi:10.1080/00401706.1962.10490022](https://doi.org/10.1080/00401706.1962.10490022) --- ## Next Steps - [Subsampling Noisy Signals with a Deadband](/docs/playground/recipes/subsample-deadband) — the four-node entry point that this recipe builds on - [Frozen Sensors](/docs/playground/recipes/sensor-freeze) — the inverse problem, when a signal has stopped changing - [Composition Patterns](/docs/concepts/composition-patterns) — understand how independent nodes compose into richer flows --- # Explore Nodes Source: https://composer.winkjs.org/docs/playground/explore-nodes --- # Kalman 1D Filter — Interactive Explorer Source: https://composer.winkjs.org/docs/playground/explore-nodes/kalman1d A Kalman filter keeps a running estimate of a noisy signal. Each tick it predicts what it expects next, then corrects using the actual measurement. The difference — called the innovation — is the diagnostic signal: small means the model matches reality, large means something changed.} /> ## Smoothing and Outlier Detection The signal below is a noisy sensor reading with a few sudden spikes and a step change partway through. The filter smooths the noise while the innovation gate flags samples that are too far from what it predicted. **Parameters** — drag the sliders and watch the chart respond: - **`processVariance`** — the filter's assumption about how much the real signal drifts per tick. Raise it and the filter trusts each new measurement more (responsive). Lower it and the filter trusts its own prediction more (smooth). - **`sensorVariance`** — how noisy you believe the sensor is. Raise it and the filter discounts measurements — it leans on the model instead. Set it to your sensor's measured noise variance (σ²). - **`chi2Threshold`** — how large the prediction error must be before flagging an outlier. 3.84 = 5% false alarm rate, 6.63 = 1%, 10.83 = 0.1%. - **`followMode`** — Exclude rejects the outlier and continues from the prediction. Follow resets to the measurement — use this for real step changes, not sensor faults. **Try this:** - Set `chi2Threshold` low (~3) and watch false alarms multiply. - Switch `followMode` to Follow and see the filter track the step change at sample 200 instantly instead of slowly converging. - Drag `processVariance` to 2 — the filter follows every wiggle. Then drag it to 0.001 — it barely moves. --- ## Adding a Known Input The demo above treats the filter as a smoother — it has no idea *why* the signal changes. But when you tell the filter about a known influence on the signal, it becomes a diagnostic tool: the innovation stops reflecting noise and starts reflecting what the model cannot explain. You provide the filter with a second input — a known cause like discharge current, fuel flow, or motor speed — and a scaling factor (`controlModel`) that converts that input into the expected change per tick. Each tick the filter predicts: *given this much current, the charge should drop by this much.* Then it corrects with the actual measurement. When prediction and measurement agree, innovation hovers near zero. When they disagree persistently, something the sensor cannot see is affecting the system. Different faults produce different innovation signatures — parasitic drain drifts negative, capacity fade drifts negative under load, sensor drift flips positive. The direction and pattern identify the root cause. See [Hidden Parasitic Drain](/docs/playground/recipes/hidden-load) for an interactive recipe that uses this pattern to diagnose a parasitic battery drain — with a full pipeline you can grab and adapt. --- ## `kalman1d` Quick Reference Parameters exposed as sliders on this page: | Parameter | Default | What it controls | |-----------|---------|------------------| | **`processVariance`** | 0.01 | How much the signal can drift per tick beyond the model. Higher = responsive, lower = smooth. | | **`sensorVariance`** | 1 | Sensor noise variance. Set to your sensor's measured σ². | | **`chi2Threshold`** | 6.63 | Innovation gate. 3.84 = 5% false alarms, 6.63 = 1%, 10.83 = 0.1%. | | **`followMode`** | false | `false` = reject outliers. `true` = track step changes. | For the full parameter reference including `controlModel`, `stateTransition`, and `measurement`, see the [nodes reference](/docs/reference). --- ## References - R. E. Kalman, "A New Approach to Linear Filtering and Prediction Problems," *Journal of Basic Engineering*, 82(1), pp. 35--45, 1960. [doi:10.1115/1.3662552](https://doi.org/10.1115/1.3662552) --- ## Next Steps - [Hidden Parasitic Drain](/docs/playground/recipes/hidden-load) — a complete recipe that uses `kalman1d` control input to detect parasitic drain in a battery system - [Sudden Shifts](/docs/playground/recipes/sudden-shift) — detect step changes using `kalman1d` with innovation gating - [Under the Hood](/docs/concepts/under-the-hood) — how messages flow through a Composer pipeline at runtime --- # Kernel — Weighted Sliding Window Explorer Source: https://composer.winkjs.org/docs/playground/explore-nodes/kernel ## Presets **Preset** — selects which weights to apply. The weights panel shows the shape. **Signal** — switches the test input. **Try this:** - Pick `spike3` on **Impulses** — spikes become peaks, the flat baseline drops to zero. - Pick `rate3` on **Sawtooth** — each rising ramp becomes a flat line; the sharp resets become negative spikes. - Compare `sg5` and `smooth5` on **Step Changes** — Savitzky-Golay keeps the sharp edges. - Pick `jerk` on **Sine + Noise** — the output leads the input by a quarter cycle. --- ## One Signal, Two Questions The preset explorer shows what each kernel does to clean synthetic signals. With real data, the stakes are different — two presets of the same node, running simultaneously, answer two questions that neither can answer alone. ### The setup GPS speed from a heavy commercial vehicle, sampled once per minute. The 6-hour trip crosses steady highway stretches and congested urban zones where traffic slows to stop-and-go. | Assumption | Value | Why | |------------|-------|-----| | Vehicle | Heavy commercial truck | Standard freight vehicle | | Sampling | 60 s (GPS speed) | Typical telematics interval | | Highway cruise | 48–52 km/h | Indian national highway, loaded | | Sluggish traffic | 10–50 km/h oscillating | Urban stop-and-go | ### What smoothing alone misses `sg5` smooths the speed and answers "what regime is the truck in?" — the cyan line shows a clean trend. But it cannot distinguish steady cruising at 45 km/h from stop-and-go traffic that *averages* 45. Both look the same after smoothing. `momentum5` answers a different question: "how choppy is the driving?" Its amplitude is a seesaw — calm on the highway, violent in sluggish traffic. That amplitude is the missing signal. The top chart shows the speed regime. **Now look at the bottom chart** — compare the momentum trace in the first 100 minutes with minutes 180–260. ### What you're seeing **Top chart** — the smoothed line sits steady near 50 km/h for the first 120 minutes, then drops into a ragged 30–40 zone. The speed regime is clear: highway, then sluggish traffic. **Bottom chart** — in the highway zone, momentum hugs zero. In the sluggish zone, it erupts ±10–15 with every acceleration and braking cycle. The calm seesaw has become a violent one. The critical zone is the transition (minutes 120–180). The top chart still shows 45–48 km/h — it looks like highway. But the bottom chart is already swinging. The choppiness arrived before the speed dropped. That early signal is what `sg5` alone cannot see. ### Where the same pattern appears - **Fleet fuel efficiency** — choppy driving burns more fuel at the same average speed; momentum amplitude correlates with consumption - **Bus route punctuality** — a route averaging 30 km/h in free-flow vs stop-and-go has the same sg5 but different schedules - **Insurance telematics** — driving smoothness scoring beyond speed alone; momentum quantifies the seesaw - **Delivery ETA** — arrival time estimation in congested corridors needs choppiness, not just average speed --- ## Quick Reference Presets explored on this page. For the full catalog, see [Kernel in the Reference](/docs/reference/nodes/kernel). | Preset | What it reveals | |--------|----------------| | `smooth5` | Moderate noise reduction — good general-purpose smoother | | `sg5` | Smooths while preserving sharp edges — speed regime detection | | `rate3` | Rate of change with zero lag | | `jerk` | How sharply acceleration changes — leads the input signal | | `spike3` | Isolates sudden transients, removes slow trends | | `momentum5` | Weighted recent direction — driving choppiness indicator | --- ## References - A. Savitzky and M. J. E. Golay, "Smoothing and Differentiation of Data by Simplified Least Squares Procedures," *Analytical Chemistry*, 36(8), pp. 1627--1639, 1964. [doi:10.1021/ac60214a047](https://doi.org/10.1021/ac60214a047) - A. V. Oppenheim and R. W. Schafer, *Discrete-Time Signal Processing*, 3rd ed., Pearson, 2010. --- ## Next Steps - [Sudden Shifts](/docs/playground/recipes/sudden-shift) — `kalman1d` with innovation gating for step-change detection. - [Kalman 1D Filter](/docs/playground/explore-nodes/kalman1d) — adaptive smoothing that adjusts its own gain. - [Composition Patterns](/docs/concepts/composition-patterns) — chain a smoother with a derivative, or smooth before detecting. --- # Concepts Source: https://composer.winkjs.org/docs/concepts --- # Under the Hood Source: https://composer.winkjs.org/docs/concepts/under-the-hood In [Hello Flow](/docs/playground/hello-flow) you built a temperature monitor — four nodes that smooth, detect, confirm, and broadcast. This page explains what happens inside that pipeline and what the framework does to keep things running when data is messy or code throws errors. ## How messages *flow* A flow is a directed graph. A message enters, gets enriched or transformed at each node, and emerges with the accumulated work of the whole pipeline. Each node reads fields from the message, computes something, and adds new fields for downstream nodes to use: > [!NOTE] > > `emitIf` broadcasts a copy to an external system but does not stop the flow — if more nodes followed, the message would continue through them unchanged. ### Three Things That Can Happen | What happens | When | What downstream nodes see | |-------------|------|--------------------------| | **Process** | Node is active | Message gets new fields, continues forward | | **Disabled** | Node is turned off (via controller) | Message passes through unchanged — node does nothing | | **Filtered** | `passIf` condition returns false | Message **stops** — downstream nodes never see it | ### How a Disabled Node Differs from a Filter Disabled nodes pass messages through unchanged. Filters (`passIf`) drop messages entirely. See [disable vs passIf](/docs/concepts/composition-patterns#disable-vs-passif--the-key-difference) for when to use each. --- ## Timestamps > [!IMPORTANT] > > Every timestamp in winkComposer must be an integer representing **milliseconds since January 1, 1970 (UTC)**. Getting this wrong will silently produce incorrect durations, dwell times, and storage records. ### Why Milliseconds Since Epoch A millisecond epoch timestamp is a single number — like `1735500000000` — that pins down an exact moment. No timezone, no format ambiguity, no parsing. The same number means the same instant whether the pipeline runs in Tokyo, London, or New York. Timezones only matter when you *display* a timestamp to a person; the pipeline never does that. This is also what JavaScript's `Date.now()` returns, so it works naturally with the language. ### What You Need to Do If your messages include a timestamp field (for dwell time tracking, state change detection, time-based slope, or storage), make sure the value is **milliseconds since epoch**: ```javascript // Already correct — no conversion needed { ts: Date.now() } // 1735500000000 (13 digits) { ts: 1735500000000 } // From a system that already uses epoch ms // Needs conversion — seconds (10 digits) must be multiplied by 1000 { ts: 1735500000 } // ✗ This is seconds, not milliseconds { ts: 1735500000 * 1000 } // ✓ Now it's milliseconds // Needs conversion — ISO string must be parsed { ts: '2024-12-30T00:00:00Z' } // ✗ String, not a number { ts: Date.parse( '2024-12-30T00:00:00Z' ) } // ✓ Parsed to milliseconds ``` **Quick check:** if your timestamp has **13 digits**, it's milliseconds. If it has **10 digits**, it's seconds — multiply by 1000. ### What Happens If You Don't Provide a Timestamp Nodes that need timing — like `dwellTimeTracker`, `stateChangeDetector`, and `persistIf` — accept an optional `timestampField` option. When omitted, they use the system clock (`Date.now()`) automatically. This is fine when messages arrive in real time. But if you're replaying historical data or processing batched records, you **must** provide a timestamp field so that durations reflect the original event times, not the replay clock. ### Where Timestamps Are Used | Node / Component | What It Uses the Timestamp For | |------------------|-------------------------------| | `dwellTimeTracker` | How long a condition has been active (dwell time, duty cycle) | | `stateChangeDetector` | How long the current state has lasted (dwell time since last transition) | | `lag` | Time-normalized slope: (value change) ÷ (time change) | | `persistIf` | Row timestamp written to storage (QuestDB) | | Time-sliding windows | Evicting entries older than the configured duration | All of these compute **differences between timestamps** — subtracting one from another to get a duration. If timestamps are in the wrong unit, every duration will be off by a factor of 1000. --- ## What Happens When Things Go Wrong winkComposer handles bad inputs, throwing functions, and multi-asset isolation — no crashes, no manual resets. ### Bad Input Sensors go offline, PLCs emit sentinel values (-9999, 65535), division by zero produces meaningless results, and the first few readings after startup can't be trusted. winkComposer handles this at two levels: **First line of defense — the `sanitize` node.** Place it at the start of your pipeline to catch bad data before it reaches the rest of the chain. It validates against configured ranges, value lists, or custom checks — and marks failures as invalid for all downstream nodes: ```javascript .sanitize('check', 'temperature', { failureReason: 'tempErr' }, { ranges: { temperature: { min: -40, max: 150 } } // PLC sentinel -9999 → marked invalid }) ``` **Safety net — automatic invalid-value propagation.** Every computational node validates its input. When a node sees a bad value (missing, out of range, or the result of a failed upstream computation), it skips its own computation and marks all its outputs as invalid. Downstream nodes see the invalid marker, trigger their own validation, and propagate it further — a controlled cascade, not a crash. On the next valid message, nodes resume normal computation automatically. No reset needed. In **multi-field mode**, each field is processed independently. If one sensor produces bad data, only that sensor's outputs are marked invalid — the other sensors keep computing normally: ```text temperature: 92.5 → esMean computes → smoothTemp: 88.3 ✓ pressure: (invalid) → esMean skips → smoothPres: (invalid) ✗ vibration: 0.42 → esMean computes → smoothVib: 0.41 ✓ ``` ### User Functions That Throw You pass functions into the pipeline in two ways: as **conditions** that decide what happens (like the test in `emitIf` or `passIf`) and as **dynamic options** that adapt values at runtime (like a threshold that varies by operating mode). Both are guarded — every node catches the exception, continues processing, and logs once per error episode. Each node handles the error according to its role: a filter drops the message, an emitter skips the emission, a controller skips the failing condition and tries the next one. See [What Happens When a Predicate Throws](/docs/concepts/flow-language#what-happens-when-a-predicate-throws) for the per-node breakdown. When a dynamic option function throws, the node continues with the last value that worked. If there is no previous value (first message), it falls back to a safe default. See [What Happens When a Tunable Throws](/docs/concepts/flow-language#what-happens-when-a-tunable-throws) for details. When a controller resets a node, the error suppression clears too. If the same error happens again after the reset, a fresh log entry appears — confirming that the reset did not fix the problem. ### Each Asset Gets Its Own State When you configure `.assetId( 'machineId' )`, each unique asset gets its own complete pipeline instance — with independent buffers, counters, smoothing history, and threshold state. You write one flow definition; the framework creates isolated instances as new assets appear: ```text machineId: 'pump-A' → [esMean state A] → [threshold state A] → ... machineId: 'pump-B' → [esMean state B] → [threshold state B] → ... machineId: 'pump-C' → [esMean state C] → [threshold state C] → ... ``` Each asset lives in its own world — independent smoothing history, independent threshold state, independent controller decisions. A misbehaving sensor on one asset cannot corrupt another asset's results. A single message stream can carry data from hundreds of assets, and each one gets clean, dedicated processing as if it had its own pipeline. --- ## One Flow, One Process A flow owns its process. When you call `.run()`, it registers signal handlers for graceful shutdown. One running flow per process: - Use `.assetId()` for per-asset isolation within one flow - Use `.switch()`/`.case()` for specialized sub-pipelines within one flow - Use separate processes for truly independent flows --- ## Next Steps - [Flow Language](/docs/concepts/flow-language) — the full reference for building pipelines - [Composition Patterns](/docs/concepts/composition-patterns) — proven node combinations for common problems --- # Flow Language Source: https://composer.winkjs.org/docs/concepts/flow-language The flow language is how you build pipelines — a **fluent, chainable JavaScript DSL** where everything starts with `flow()` and chains naturally. It is executable code, not YAML or JSON configuration — options can be functions that adapt per message, control structures like `switch`/`case` compose pipelines, and `.build()` transpiles the flow to an optimized JavaScript module. ## Anatomy of a Flow Every pipeline starts with `flow()` and ends with a terminal method. In between, there are two phases — **configuration** (infrastructure setup) and **nodes** (processing logic): ```javascript flow( 'pipeline-name' ) // ── Configuration phase (optional, must come before nodes) ── .assetId( 'machineId' ) // each unique value gets its own pipeline instance .source( adapter, config ) // data source .emitter( adapter, config ) // output destination .storage( adapter, config ) // persistence .namingPolicy( '{param}_{stat}' ) // auto-naming for multi-field .assetClass( definition ) // semantics for storage schema // ── Node phase (processing, in order) ── .esMean( ... ) .threshold( ... ) .emitIf( ... ) // ── Terminal ── .run() // wire and start immediately ``` **Key rules:** - Configuration methods must appear **before** any node — the Flow Language enforces this at build time - Nodes execute in the order you write them — each reads fields added by earlier nodes - `.run()` wires sources, emitters, storage, and starts the pipeline The simplest possible flow has zero configuration and one node: ```javascript flow( 'minimal' ) .esMean( 'smooth', 'temperature', { mean: 'avg' } ) .run(); ``` --- ## Semantics As your pipeline grows, you add configuration. Semantics enter the flow through `.assetClass()` — they are the single source of truth for what your data means: column types, units, physical ranges, operational limits, and which columns belong to each storage table (`insightType`). The key design principle is that **facts live in semantics, decisions live in flows** — a column's physical range is a fact; the threshold that triggers an alert is a decision. Nodes don't consult semantics at runtime, but everything downstream does: the storage layer uses them to create schemas, the MCP server uses them to answer queries, and dashboards use them to render context-aware visualizations. --- ## Anatomy of a Node Call Most nodes follow this signature: ```javascript .nodeType( name, inputField, stats, options ) // ↑ ↑ ↑ ↑ // unique what to what to tuning // name read compute knobs ``` ```javascript .esMean('smooth', 'temperature', { mean: 'tempAvg' }, { halfLife: 5 }) ``` Variations exist — field-pair nodes take two input fields, condition nodes take a predicate instead of input/stats — but the pattern is consistent. The subsections below explain each part. --- ## Dynamic Options Any option that accepts a static value can also accept a **function**. The function receives the current message and returns the resolved value — evaluated fresh on every message: ```javascript // Static — same threshold for every message .threshold('check', 'temp', { active: 'hot' }, { threshold: 80 }) // Tunable — threshold adapts per message .threshold('check', 'temp', { active: 'hot' }, { threshold: ( msg ) => msg.baseline + 10 }) ``` This means pipelines adapt to context — operating mode, shift, sensor type, learned baselines — without restart. ### Built-in Helpers The winkComposer framework provides helpers for common patterns. Each helper returns a function that works anywhere a static value is accepted: | Helper | What It Does | Example | |--------|-------------|---------| | `lookupByField( field, map, default )` | Map a message field to a value | `lookupByField( 'shift', { day: 35, night: 30 }, 32 )` | | `scaleBy( field, factor, offset, step )` | Scale a message field | `scaleBy( 'stdev', 0.5 )` → `stdev * 0.5` | | `fromField( field, default )` | Read a field directly | `fromField( 'learnedBaseline', 50 )` | | `chooseWhen( predicate, ifTrue, ifFalse )` | Conditional value | `chooseWhen( msg => msg.isWarmup, 100, 78 )` | | `clampTo( field, min, max )` | Clamp a field to a range | `clampTo( 'requested', 10, 100 )` | | `offsetBy( field, offset )` | Add an offset to a field | `offsetBy( 'baseline', 10 )` | ### Real-World Example WiFi RSSI thresholds that vary by protocol — each message carries its own protocol type, and the threshold adapts: ```javascript .threshold('rssiAlert', 'rssi', { active: 'rssi_low' }, { threshold: lookupByField( 'protocolType', { '802.11ac': -70, '802.11n': -75 }, -75 ) }) ``` Individual node entries mark tunable-capable options with ⚡. ### What Happens When a Tunable Throws Dynamic option functions can throw at runtime — for example, if a message lacks an expected field. winkComposer handles this gracefully: - **Last good value:** The node continues with the last successfully resolved value. The throwing call is simply skipped — the node processes the message using whatever value worked last. - **First-message edge case:** If the function throws before any value has been resolved, the node falls back to a safe default — it either stays inactive until the function succeeds or marks its outputs as invalid, depending on the node. - **Log suppression:** The first error per episode is logged to console. Subsequent errors are suppressed until the function succeeds again (recovery), preventing console flooding at high message rates. - **Automatic recovery:** When the function succeeds again, the node resumes normal operation. A new error episode will be logged if the function fails again later. - **Reset:** A controller-triggered reset also clears the suppression. If the tunable still throws after the reset, a fresh error is logged. Static values (`threshold: 80`) cannot throw, so the guard only activates for user-supplied functions. --- ## Node Names The first argument to every node is its **name** — a unique identifier within the pipeline: ```javascript .esMean('smooth', 'temperature', { mean: 'tempAvg' }) // ↑ // node name — must be unique across the pipeline ``` Node names serve two purposes: - **Identity**: Each name must be unique — duplicates cause a build error - **Targeting**: Controllers reference nodes by name to send control signals (e.g., enable, disable, reset) ```javascript .esMean('smooth', 'vibration', { mean: 'smoothVib' }) // node named 'smooth' .esStats('stats', 'vibration', { stdev: 'vibStd' }) // node named 'stats' .controller('ctrl', [{ when: msg => msg.anomaly === true, triggers: [{ control: 'reset', targets: ['smooth', 'stats'] }] // ↑ references nodes by name }]) ``` --- ## Choosing What Each Node Computes and Adds to the Message Every node can compute one or more statistics — you choose which ones you want via the stats object. Each requested stat becomes a new field on the message, available to all downstream nodes. Keys choose **what to compute**, values choose **what to call it**: ```javascript .esMean('smooth', 'temperature', { mean: 'tempAvg' }) // ↑ ↑ // compute mean call it 'tempAvg' ``` Request multiple outputs from one node: ```javascript .esStats('monitor', 'temperature', { mean: 'avg', stdev: 'std', zScore: 'z' }) // Message gets three new fields: avg, std, z ``` Each node reads fields already on the message — including fields written by upstream nodes — and adds its own. The message grows as it flows: ```javascript .esMean('smooth', 'temperature', { mean: 'smoothTemp' }, { halfLife: 5 }) .esStats('monitor', 'smoothTemp', { stdev: 'tempStd', zScore: 'tempZ' }) .threshold('check', 'tempZ', { active: 'anomaly' }, { mode: 'above', threshold: 3 }) ``` ```text Incoming: { temperature: 92 } After esMean: { temperature: 92, smoothTemp: 88.5 } ↑ reads temperature After esStats: { ..., smoothTemp: 88.5, tempStd: 2.1, tempZ: 1.7 } ↑ reads smoothTemp After threshold: { ..., tempZ: 1.7, anomaly: false } ↑ reads tempZ ``` --- ## Single vs Multi-Field Processing Nodes can process fields in two modes: ### Single Field Mode (String Input) Pass a single field name as a **string**. You specify the exact output name: ```javascript // Single field - YOU control the output name directly .esMean('baseline', 'temperature', { mean: 'tempBaseline' }, { halfLife: 10 }) // Output field: tempBaseline (exactly what you specified) .threshold('check', 'pressure', { active: 'highPressure' }) // Output field: highPressure (exactly what you specified) ``` **Key Points:** - Input: String field name - Output: Exactly what you specify in the stats object - Naming policy: **NOT applied** - Use when: You want precise control over output names ### Multi-Field Mode (Array Input) Pass multiple field names as an **array**. The system auto-generates names using naming policy: ```javascript // Multi-field - SYSTEM generates names using naming policy .namingPolicy('{param}_{stat}') // Define the pattern .esMean('smooth', ['temperature', 'pressure'], { mean: 'smoothed' }, { halfLife: 10 }) // Output fields: temperature_smoothed, pressure_smoothed (auto-generated) .sanitize('validate', ['inlet_pressure', 'outlet_pressure'], // Array = multi-field { failureReason: 'error', failedValue: 'value' } ) // Output fields: inlet_pressure_error, inlet_pressure_value, // outlet_pressure_error, outlet_pressure_value ``` **Key Points:** - Input: Array of field names - Output: Auto-generated using naming policy pattern - Naming policy: **APPLIED** - Use when: Processing multiple fields with consistent naming --- ## Naming Policy (Multi-Field Only!) The naming policy **only applies to multi-field processing** (array inputs): ```javascript flow('my-pipeline') .namingPolicy('{param}_{stat}') // Template for multi-field auto-naming // Multi-field: naming policy APPLIES .esMean('smooth', ['temp', 'pressure'], { mean: 'avg' }) // Generates: temp_avg, pressure_avg // Single field: naming policy IGNORED .esMean('baseline', 'temp', { mean: 'myCustomBaseline' }) // Output: myCustomBaseline (your exact specification) ``` **Template Variables:** - `{param}` - The input field name (e.g., 'temperature') - `{stat}` - The name YOU provided in the stats object (e.g., 'avg') - `{name}` - The node instance name (e.g., 'smooth') - `|dv` - Drop vowels modifier (e.g., "temperature" → "tmprtr") **Important:** The `{stat}` variable uses the VALUE from your stats object, not the key: ```javascript .sanitize('check', ['temp'], { failureReason: 'invalid' }) // {stat} = 'invalid' (not 'failureReason') // Generates: temp_invalid ``` --- ## Per-Field Option Customization When using multi-field mode, options can be customized per field using object syntax: ```javascript // Same halfLife for all fields .esMean('smooth', ['temp', 'pressure'], { mean: 'avg' }, { halfLife: 10 }) // Different halfLife per field — temperature responds faster .esMean('smooth', ['temp', 'pressure'], { mean: 'avg' }, { halfLife: { temp: 5, pressure: 20 } }) ``` Each field gets independent state, namespaced control signals, and isolated failure handling. A processing error on one field does not affect others. --- ## Node Processing Types Beyond single/multi-field modes, nodes have different processing behaviors: | Type | Description | Example | |------|-------------|---------| | **Per-field** | Each field processed independently | `.esMean()`, `.threshold()` | | **Field-pair** | Operates between exactly two fields | `.diff()`, `.ratio()` | | **Field-Group** | Analyzes multiple fields together | `.esPairwiseCorrelation()` | | **Condition** | Evaluates boolean expressions | `.passIf()`, `.persistenceCheck()` | | **Control** | Orchestrates other nodes | `.controller()` | ### Per-Field Processing These nodes process each field independently. Supports both single and multi-field modes: ```javascript // Single field - direct naming .esMean('avg', 'temperature', { mean: 'avgTemp' }) // Output: avgTemp // Multi-field - auto-naming with policy .esMean('avg', ['temperature', 'pressure'], { mean: 'avg' }) // Outputs: temperature_avg, pressure_avg (with policy: '{param}_{stat}') ``` **Nodes supporting per-field processing:** - Signal Conditioning: `esMean`, `median3`, `sanitize`, `butterworthFilter`, `kernel` - Feature Extraction: `lag`, `momentsDigest`, `trend` - Detection: `threshold`, `pageHinkley` ### Field-Pair Operations These operate between exactly two fields. Always single mode (no arrays): ```javascript // Calculate difference between two fields .diff('delta', 'outlet_pressure', 'inlet_pressure', { diff: 'pressureDrop' }) // Output: pressureDrop // Calculate ratio .ratio('efficiency', 'output_power', 'input_power', { ratio: 'powerEfficiency' }) // Output: powerEfficiency // Correlation between two fields .esCorrelation('coupling', 'temp', 'pressure', { correlation: 'tempPressCorr' }) // Output: tempPressCorr ``` ### Field-Group Analysis These analyze multiple fields **as one group**: ```javascript // Compute all pairwise correlations .esPairwiseCorrelation('matrix', ['temperature', 'pressure', 'flow'], // Analyzes as a group { correlations: 'sensorCorr', // Vector of all correlations pairNames: 'corrLabels' // "temp-pressure", "temp-flow", etc. } ) // Outputs: sensorCorr (array), corrLabels (array) // Monitor multiple states together .stateChangeDetector('monitor', ['machineState', 'qualityLevel'], // Watches as a group { dwellTime: 'stateTime' }, { changeMode: 'any' } // Trigger on any change ) ``` > [!NOTE] > > Group analyzers emit shared outputs; naming policy does not apply. ### Condition-Based Processing These evaluate predicates on messages and compute stats from the result: ```javascript // Track binary conditions .dwellTimeTracker('uptime', msg => msg.running, { dwellTime: 'runDuration', dutyCycle: 'runPercentage' }) // Output: runDuration, runPercentage // Confirm persistence (2-of-3 voting) .persistenceCheck('stable', msg => msg.temperature > 80, { persistenceConfirmed: 'overheating' }, { minVotes: 2, outOfTotal: 3 } ) // Output: overheating ``` ### Control Flow Orchestrate other nodes without processing data: ```javascript .controller('adaptive', [{ when: msg => msg.vibration > 2.5, triggers: [ { control: 'enable', targets: ['esStats', 'correlation'] }, // targets by node name { control: 'reset', targets: ['baseline'] } // 'baseline' is an esMean node name ] }]) ``` ### What Happens When a Predicate Throws Predicate functions can throw at runtime — for example, if a message lacks a field the predicate reads. Every predicate-using node catches the exception and handles it according to its role: | Node | On exception | Effect | |------|-------------|--------| | **passIf** | Message dropped | Same as returning `false` — downstream nodes never see the message | | **persistenceCheck** | Treated as non-vote | The message doesn't count toward confirmation | | **dwellTimeTracker** | Outputs marked invalid | No state transition is recorded | | **controller** | Condition skipped | Remaining conditions still evaluate (first-match-wins) | | **emitIf** | Emission skipped | Emits a status signal to the target; recovers automatically | | **persistIf** | Write skipped | First error per episode logged; subsequent suppressed until recovery | | **sanitize** (custom predicate) | Value treated as invalid | Same as predicate returning `false` | All predicate exceptions are logged to `console.error` — once per error episode, not once per message. On recovery (successful predicate call), the suppression resets so the next episode is logged. A controller-triggered reset also clears the suppression. No predicate exception ever crashes the pipeline or affects other assets. --- ## Which Node Type to Choose | If you need to... | Use this node type | Example | |-------------------|-------------------|---------| | Process each field the same way | Per-field | `esMean`, `threshold` | | Compare two fields | Field-pair | `diff`, `ratio` | | Analyze relationships | Field-Group | `esPairwiseCorrelation` | | Filter or route messages | Condition | `passIf`, `emitIf` | | Track binary states | `dwellTimeTracker` | Running/stopped | | Track categorical changes | `stateChangeDetector` | State machines | | Coordinate nodes | Control | `controller` | --- ## Next Steps - [Composition Patterns](/docs/concepts/composition-patterns) — proven node combinations for common problems - [Semantics](/docs/concepts/semantics) — give your computed values meaning with metadata --- # Composition Patterns Source: https://composer.winkjs.org/docs/concepts/composition-patterns ## Pattern Catalog Proven node combinations for common problems: | Pattern | Pipeline | Use Case | |---------|----------|----------| | Bearing failure | `butterworthFilter → esMean → threshold` | Vibration anomaly | | Noise-tolerant alarm | `median3 → esMean → threshold` | Spike-resistant alerting | | Drift detection | `esMean(fast) → esMean(slow) → diff → pageHinkley` | Gradual shift detection | | Western Electric Rules | `esStats → persistenceCheck` | 2-of-3 beyond 2σ via zScore | | Correlation drift | `esPairwiseCorrelation → vectorDistance → emitIf` | Multi-sensor relationship change | | State-aware persistence | `dwellTimeTracker → invertFlag → persistIf` | Write on state exit, not entry | --- ## Layered Flows Flows can feed into other flows, building layers where each one produces higher-level insights. A **local flow** processes raw sensor data — smoothing, detecting, and emitting digests via MQTT. An **aggregate flow** subscribes to those digests, combines results from multiple assets, and persists to a database: Both are ordinary flows — same nodes, same configuration. Each layer can run anywhere (edge device, cloud, or both). The power is composition: each layer builds on what the previous one produced. --- ## Adaptive Diagnostics All nodes start active. To keep expensive nodes dormant until needed, pair them with a controller that disables them when the triggering condition is absent. On the very first message, the non-anomaly condition matches and the controller disables the expensive nodes — saving up to ~95% of compute: ```javascript .esMean('smooth', 'vibration', { mean: 'smoothVib' }, { halfLife: 5 }) .threshold('detect', 'smoothVib', { active: 'anomaly' }, { mode: 'above', threshold: 2.5 }) .controller('adaptive', [ { when: msg => msg.anomaly === true, triggers: [ { control: 'enable', targets: ['stats', 'correlation'] }, { control: 'reset', targets: ['stats'] } ] }, { when: msg => msg.anomaly === false, triggers: [{ control: 'disable', targets: ['stats', 'correlation'] }] } ]) .esStats('stats', 'vibration', { mean: 'vibMean', stdev: 'vibStd' }, { halfLife: 20 }) .esCorrelation('correlation', 'vibration', 'temperature', { correlation: 'vibTempCorr' }) ``` The two pipeline states (see [Pipeline Diagram Symbols](/docs/concepts/under-the-hood#pipeline-diagram-symbols) for the visual legend): --- ## Downsampling for Storage Collect raw samples into compact digests, then persist only when a window completes: ```javascript .momentsDigest('digest', 'temperature', { windowSize: 60 }) .digestMoments('stats', 'temperature', { mean: 'tempMean', stddev: 'tempStd', min: 'tempMin', max: 'tempMax' }) .persistIf('save', msg => msg.digest === true, { storageName: 'questdb', insightType: 'minuteStats' }) ``` For multi-level aggregation (seconds → minutes → hours), chain with `cascade: true`: ```javascript .momentsDigest('perSecond', 'temperature', { windowSize: 100 }) .momentsDigest('perMinute', 'temperature', { windowSize: 60, cascade: true }) ``` --- ## Choosing the Right Tool These four mechanisms control what happens in your pipeline. They look similar but serve different purposes: | What you're trying to do | Wrong tool | Right tool | Why | |--------------------------|-----------|------------|-----| | Suspend expensive nodes during normal operation | `passIf` — messages vanish, downstream sees nothing | `controller` + `disable` — messages pass through, nodes sleep | Stop *computation*, not *messages* | | Remove bad data before it propagates | `controller` + `disable` — bad data still flows through | `passIf` — message is dropped entirely | Stop *messages*, not *computation* | ### passIf — Drop messages that shouldn't continue Use when you want to **remove** messages from the pipeline entirely. Downstream nodes never see dropped messages. **Good for:** - Quality gates — only pass messages with valid data - Sampling — pass every Nth message to reduce volume - Startup warmup — skip the first N messages while sensors stabilize ```javascript .passIf('quality', msg => msg.confidence > 0.9) .passIf('downsample', ( msg, counter ) => counter % 100 === 0) ``` ### emitIf — Broadcast without disrupting the flow Use when you want to **send a copy** of the message to an external system (MQTT, terminal) while the original continues through the pipeline unchanged. **Good for:** - Alerts — broadcast when a condition is detected - Telemetry — send periodic status updates - Debugging — tap into the pipeline to see what's happening ```javascript .emitIf('alert', msg => msg.faultConfirmed, { target: 'mqtt', insightType: 'faultAlert' }) ``` ### controller — Change how the pipeline behaves Use when you want to **turn nodes on or off**, reset their state, or flush accumulated data. The controller reads the message but doesn't change it — it sends signals to other nodes. **Good for:** - Adaptive computation — enable expensive processing only during anomalies - State machines — switch between operational modes - Coordinated resets — clear state across multiple nodes at once ```javascript .controller('adaptive', [{ when: msg => msg.anomaly, triggers: [{ control: 'enable', targets: ['stats', 'corr'] }] }]) ``` ### disable vs passIf — The key difference | | `passIf` (filter) | `disable` (via controller) | |-|-------------------|---------------------------| | **Scope** | One message at a time | All messages while disabled | | **Message fate** | Dropped — gone forever | Passes through unchanged | | **Downstream impact** | Downstream sees nothing | Downstream sees the original message | | **Reversible?** | No — need a new message | Yes — `enable` signal resumes processing | | **When to use** | Bad data, sampling | Suspend expensive computation | **Rule of thumb:** If you want to stop *messages*, use `passIf`. If you want to stop *computation*, use a controller with `disable`. --- ## Error Handling Every user-supplied function — conditions and dynamic options — is guarded. A throwing function never crashes the pipeline or affects other assets. See [What Happens When a Predicate Throws](/docs/concepts/flow-language#what-happens-when-a-predicate-throws) and [What Happens When a Tunable Throws](/docs/concepts/flow-language#what-happens-when-a-tunable-throws) for how each node responds. --- ## Pipeline Ordering Guidelines The order you place nodes matters: 1. **Clean first** — `sanitize` at the start catches bad data before it propagates 2. **Smooth before detect** — `esMean` or `median3` before `threshold` reduces false alarms 3. **Confirm before act** — `persistenceCheck` before `emitIf` avoids alerting on transients 4. **Controller before targets** — controllers can only target nodes that appear **after** them 5. **Observe last** — `emitIf` and `persistIf` near the end see the fully enriched message --- ## Next Steps - [Semantics](/docs/concepts/semantics) — define what your data means so every downstream system speaks the same language - [Use Cases](/docs/use-cases) — see these patterns at production scale with real datasets --- # Semantics — Meaning Behind the Numbers Source: https://composer.winkjs.org/docs/concepts/semantics A pipeline computes numbers — smoothed values, trends, flags. But what do those numbers *mean*? Is `92.5` a temperature in Celsius or a pressure in bar? Is `0.42` vibration in mm/s or displacement in microns? Without metadata, a dashboard shows bare numbers and a query engine returns anonymous columns. Semantics answer this. They define what your data **is** — types, units, physical ranges, operational limits — so every system downstream speaks the same language. ## The Single Source of Truth Semantics are a shared metadata layer consumed by multiple parts of the ecosystem: Define a column once — its type, unit, and limits — and every consumer inherits the same definition. Change the unit from `bar` to `kPa` in one JSON file and the dashboard, the query engine, and the storage schema all update together. --- ## Facts vs. Decisions This is the core design principle: semantics describe **facts** about data, not **decisions** about what to do with it. | Semantics (facts) | Flow language (decisions) | |--------------------|--------------------------| | Outlet pressure is measured in bar | Alert when pressure exceeds 110 bar | | Temperature range is -40 to 150 °C | Switch to high-temp mode above 85 °C | | Vibration unit is mm/s RMS | Flag bearing as degraded when trend slope > 0.02 | | Pump status codes: 0=Idle, 1=Running | Only process messages where status is Running | A pump's outlet pressure has a **specification limit of 90 bar** (semantic fact). The flow uses a **threshold of 110 bar** to detect high-pressure events (policy decision). These are independent — the limit describes the equipment, the threshold drives the logic. --- ## What Semantics Define ### Columns Each column carries its full definition: - **Type** — `float64`, `int64`, `string`, `bool`, `timestamp` - **Unit** — engineering units (`bar`, `°C`, `mm/s`) - **Description** — human-readable explanation - **Resolution** — measurement precision - **Three-tier limits** — nested validation boundaries: Physical range catches impossible values (sensor faults). Operational limits define safe bounds with warning and critical thresholds. Specification limits track process capability for quality control. ### Asset Classes An asset class groups columns into an equipment type — like `industrialPump` or `bearingTestRig`. It defines which measurements belong together and how they map to storage: - **Columns** — all measurements for this equipment type - **Insight types** — column projections for storage tables (e.g., `operational` for continuous data, `faults` for event data) - **Enumerations** — coded-value lookups (e.g., `0` → `Idle`, `1` → `Running`) ### Semantic Digest A SHA-256 hash of the loaded semantics serves as a version fingerprint. Storage records carry this digest, so you can always verify which schema version produced a given dataset — essential for reproducibility and auditing. --- ## How Semantics Connect to Flows In the [flow language](/docs/concepts/flow-language), semantics wire in through configuration methods: ```javascript flow( 'pump-monitor' ) .assetClass( pumpSemantics ) // load the asset class .storage( questdb, config ) // columns, types, and units flow to storage .persistIf( 'operational', ... ) // "operational" is an insight type from the asset class .run() ``` The `persistIf` node references an insight type defined in the asset class. winkComposer validates at load time that the insight type exists and that the columns it references are defined — configuration errors surface immediately, not after hours of data collection. --- ## Next Steps - [Use Cases](/docs/use-cases) — see complete pipelines with semantics at production scale - [Node Index](/docs/reference/node-index) — find any building block by name or category --- # Streaming Intelligence in Action Source: https://composer.winkjs.org/docs/use-cases > [!NOTE] > > **These demos run real winkComposer pipelines on real datasets** — public research benchmarks and anonymised IIoT data. Parameters draw from domain knowledge and standard conventions; verdict thresholds and evidence-decay rates are tuned for narrative clarity at demo viewing speeds. As the framework matures and testing deepens, assessment behaviour and edge-case handling may be refined. --- # Diagnosing WiFi AP Health Source: https://composer.winkjs.org/docs/use-cases/wifi-ap-health A winkComposer flow turns a real four-day client telemetry export from a 5-client office network into one health verdict per AP, plus two kinds of signal worth acting on. Every number on the page is a direct read of a Composer node — the verdict, the drivers behind it, the roaming cost, the drop events. No AP-side telemetry is needed. **Drag the slider** and switch between access points to see how each one is doing. --- ## What You're Seeing **Below the verdict — what the flow is telling you.** Each card is a notice from the flow. **Watch** notices flag one driver highlighting on a single AP even though the overall verdict holds. **Fix** notices are sustained, site-wide observations worth acting on right now. **Per-AP verdict card.** Healthy / Monitor / Degraded / Critical, plus a key-signals table ranked by intensity and persistence. Each row is one driver watching a signal. The small amber pip on an AP tab means *"verdict still Healthy, but one driver is highlighting hard — worth a closer look."* **Three stacked charts.** Median RSSI · AP Health · drop events, all sharing one x-axis of active office hours. Overnight gaps are compressed out; per-AP silences inside a session are shown as grey bands labelled **Idle** at the bottom. The initial learning window is labelled **Calibrating**. **Flow matrix + roaming table.** Where clients actually go when they roam, and what the roam costs in dB. The matrix shows AP-3 is the hub; the table shows clients leave AP-3 at roughly −47 dBm — excellent signal — and arrive at AP-1 at around −57 dBm. Roughly 10 dB is lost per roam, and the policy is pushing clients off a strong AP onto a weaker one. One quirk worth naming: in this four-day sample, no AP's overall verdict ever leaves **Healthy**. That is the honest answer — the network is fine at the aggregate level. But one driver on AP-2 saturates at end-of-run because its per-client baseline becomes very tight — even a normal-quality reading registers as a significant deviation. The framework distinguishes this from a genuine AP failure: the verdict holds, and the driver's intensity combined with the Watch card raises the flag. --- ## How It Works The flow is layered — per-client locally, per-AP aggregated. The pattern is documented in [Composition Patterns](/docs/concepts/composition-patterns). The **per-client flow** cleans each RSSI sample, tracks a smoothed baseline, and ranks each signal dip by how large an amplitude would be needed to erase it — separating structural drops from routine jitter. A parallel step records the previous-tick RSSI per client, so each roam reports its own dB cost directly from flow output. An **aggregation step** groups all client records by access point and timestamp. For each group it takes the median RSSI, the median SNR, the mean retry rate, the client count, and the count of per-client drop events directed at that AP. That turns per-client streams into per-AP tick streams — without a single AP-side query. The **per-AP flow** runs six independent drivers in parallel — one each for signal strength, structural drops, interference, occupancy, retry, and event concentration — and fuses them into a verdict with the [appraise](/docs/playground/explore-nodes/appraise) node. Evidence for each driver builds while its signal is outside its normal range and fades as it returns; the verdict waits for combined evidence across multiple drivers before it escalates. That is why a single saturated driver does not flip the verdict — and why the Watch card exists to flag that case explicitly. The flow tracks **change against an adaptive baseline**, not absolute quality. A chronically poor but stable AP — one that has always sat at the edge of usable signal — does not register as a deviation, because the baseline learns its behaviour. Catching coverage holes that have always been holes needs a complementary fixed-threshold check sitting alongside this flow. No AP-side telemetry is needed. The universal client export — timestamp, client MAC, RSSI, retry rate, and AP name, plus SNR or the signal-plus-noise pair needed to derive it — is present in every major controller: UniFi, Omada, OpenWRT, and Cisco Catalyst Center. Catalyst Center reports a Client Health Coverage score based on fixed RSSI bins. The amplitude-ranked per-client drop-event stream this flow produces — with a per-client adaptive baseline — is not on its documented API surface. The same flow runs on a Raspberry-Pi-class board. --- ## References - Data: anonymised Cisco Catalyst Center client-health export from [Velocis Systems](https://velocis.in), used with permission. Five clients, six access points, approximately four days at 5-minute polling. PII removed: MAC addresses, usernames, IP addresses, and location identifiers replaced with anonymised labels. - Edelsbrunner, H., Letscher, D. & Zomorodian, A. (2002). *Topological Persistence and Simplification.* Discrete & Computational Geometry, 28, 511–533. [doi:10.1007/s00454-002-2885-2](https://doi.org/10.1007/s00454-002-2885-2) — the basis for ranking signal dips by the amplitude needed to erase them. --- ## Next Steps - [Detecting Bearing Failure](/docs/use-cases/bearing-health) — the same appraise-driven assessment on NASA vibration data - [Composition Patterns](/docs/concepts/composition-patterns) — the layered-flow pattern used here, in a more general setting --- # Detecting Bearing Failure Source: https://composer.winkjs.org/docs/use-cases/bearing-health This is a real bearing failure from the [NASA IMS Bearing Dataset](https://data.nasa.gov/dataset/ims-bearings) — one of the most cited run-to-failure experiments in predictive maintenance. A bearing degrades from healthy to catastrophic over 7 days while accelerometers sample every vibration at 20 kHz. Each 10-minute snapshot is condensed to window statistics — RMS, kurtosis, and more — using the same winkComposer building blocks that would run at the edge in production. A winkComposer flow then scores the bearing's health from these statistics in real time. **Drag the slider** and watch it detect the failure before it happens. ## What You're Seeing The chart shows vibration RMS (cyan) — the overall energy in the signal — tracked by an exponentially smoothed mean (lavender dashed). The filled **envelope** around the mean is the peak-tracking vibration envelope — tight when the bearing is healthy, widening as degradation introduces new vibration modes. For about the first day the system shows **Learning** — it is observing the equipment's baseline vibration automatically, with no manual threshold tuning required. The bearing holds steady for about 3.75 days. RMS and kurtosis stay normal, and the verdict reads Healthy. Around day 4, evidence builds across multiple signals at once — energy spikes, a rising trend, and falling signal clarity — and the verdict jumps straight to Critical. The change-point markers (vertical dashed lines) visually confirm the shift. The key signals table shows each signal's intensity and **persistence** — distinguishing sustained degradation (Continuous) from transient spikes (Intermittent). This is the first question maintenance engineers ask: is this real or a false alarm? What you're watching is a **P-F curve** — the trajectory from the point where a fault becomes detectable (P) to functional failure (F). The entire discipline of predictive maintenance exists to maximise the interval between P and F. --- ## How It Works One flow, six building blocks — each contributes a different perspective on bearing health: **ES Stats** smooths the raw RMS signal and tracks the vibration envelope — the lavender band on the chart. It also computes the z-score (how far the current reading deviates from normal) and signal-to-noise ratio (SNR), which drops as the bearing's behaviour becomes erratic. **RMS Trend** watches whether the smoothed RMS is climbing consistently — the degradation rate. Only a rising trend with sufficient confidence passes through to the assessment. **Kurtosis Trend** does the same for impulsiveness — often the first sign of a defect. **RMS Change-Point Detection** and **Kurtosis Change-Point Detection** run [Page-Hinkley](https://en.wikipedia.org/wiki/Page%27s_cumulative_sum_test) tests that fire when the signal shifts to a new level — the vertical markers on the chart. When both fire at roughly the same time, two independent physical signals confirm the same shift. The **health assessment** at the end of the pipeline fuses four signals into one verdict: - **Energy spikes** — z-score from ES Stats (current deviation from the running average) - **Impact events** — raw kurtosis (spiky vibration patterns) - **Energy trend** — trend confidence, gated to the rising direction only (zero when stable, regardless of confidence) - **Signal clarity** — SNR from ES Stats (drops as behaviour becomes erratic) During the first ~1.2 days, the node learns this bearing's normal vibration signature and calibrates a verdict scale from it — no manual threshold tuning per asset. Evidence for each signal then builds while the signal is outside its normal range and fades as it returns. Combined evidence determines the verdict — Healthy, Monitor, Degraded, or Critical — and the recommended action. Each signal's **intensity** and **persistence** appear alongside. --- ## References - Qiu, H., Lee, J., Lin, J. & Yu, G. (2006). *Wavelet filter-based weak signature detection method and its application on rolling element bearing prognostics.* Journal of Sound and Vibration, 289, 1066–1090. [doi:10.1016/j.jsv.2005.03.007](https://doi.org/10.1016/j.jsv.2005.03.007) - Page, E.S. (1954). *Continuous inspection schemes.* Biometrika, 41(1/2), 100–115. [doi:10.2307/2333009](https://doi.org/10.2307/2333009) - Dataset: [NASA IMS Bearing Data](https://data.nasa.gov/dataset/ims-bearings) — University of Cincinnati, 2004. Test 2, Bearing 1: inner race defect, ~164 hours run-to-failure. Four bearings under 6,000 lb radial load at 2,000 RPM. Accelerometers captured 20,480 samples at 20 kHz every 10 minutes (982 snapshots). --- ## Next Steps - [Catching Process Drift](/docs/use-cases/process-quality) — detect catalyst degradation in a chemical reactor - [Composition Patterns](/docs/concepts/composition-patterns) — the patterns behind this pipeline --- # Catching Process Drift Source: https://composer.winkjs.org/docs/use-cases/process-quality This is a real fault from the [Tennessee Eastman Process](https://doi.org/10.1016/0098-1354(93)80018-I) — the most widely used benchmark in process fault detection. A reactor converts gaseous feed into liquid product under high pressure (~2,800 kPa); unreacted gas recycles through a compressor. The pipeline streams raw sensor data at 1-minute resolution and monitors two signals: **reactor pressure**, where variance tells the story, and **compressor work**, where the correlation with pressure shifts. **Drag the slider** and watch the 2σ band widen as the process destabilises. ## What You're Seeing The chart shows reactor pressure (cyan) tracked by an exponentially smoothed mean (lavender dashed). The filled **2σ band** around the mean is the smoothed mean ± twice the smoothed standard deviation — tight during normal operation, widening as variance grows post-fault. Three detection events unfold in sequence: 1. **|z| > 3 exceedance fires first** (amber blocks). The adaptive z-score divides each pressure deviation by the smoothed standard deviation. Because the smoothed estimate lags behind the growing variance, early pressure swings punch through ±3σ — catching both positive and negative excursions while the 2σ band is still narrow. 2. **Correlation shift fires next** (orange vertical). The pressure–compressor correlation starts drifting as the physics decouple. A change-point detector accumulates evidence until the shift is confirmed. 3. **Compressor shift fires last** (red vertical). An independent change-point test on the smoothed compressor mean detects the energy level drop — a slower, confirmatory signal from a different physical variable. By the time the 2σ band visibly explodes, all three detectors have already fired. --- ## How It Works Three pipelines, three detectors — each answers a different question using the same composable building blocks: **Pressure Exceedance** catches individual swings. The exponentially smoothed standard deviation updates with a half-life of 90 samples (~1.5 hours), so it lags behind growing variance. Early pressure spikes produce large z-scores even while the mean barely moves — and the detector fires whenever |z| crosses ±3. The amber regions on the chart mark every excursion. **Correlation Drift** catches structural change. A [Page-Hinkley](https://en.wikipedia.org/wiki/Page%27s_cumulative_sum_test) change-point test accumulates evidence that the pressure–compressor correlation is shifting. A trend detector watches the rate of that accumulation — when evidence climbs consistently, it fires as an early warning before the change-point test confirms the structural shift. **Compressor Shift** provides independent confirmation from a different physical variable. The compressor's smoothed mean energy drops as slowing reactions reduce the gas volume to recycle. This is a subtle shift (~1–2%), so the change-point test fires late — serving as confirmation rather than early detection. --- ## References - Downs, J.J. & Vogel, E.F. (1993). *A plant-wide industrial process control problem.* Computers & Chemical Engineering, 17(3), 245–255. [doi:10.1016/0098-1354(93)80018-I](https://doi.org/10.1016/0098-1354(93)80018-I) - Page, E.S. (1954). *Continuous inspection schemes.* Biometrika, 41(1/2), 100–115. [doi:10.2307/2333009](https://doi.org/10.2307/2333009) - Dataset: [mv-per/tennessee-eastman-dataset](https://github.com/mv-per/tennessee-eastman-dataset) — 3,001 samples at 1-minute resolution (50 hours), fault injected at hour 8. Reactor pressure is XMEAS 7 (kPa), compressor work is XMEAS 20 (kW). Variable definitions from the [original FORTRAN simulation](https://github.com/camaramm/tennessee-eastman-profBraatz/blob/master/teprob.f). --- ## Next Steps - [Detecting Bearing Failure](/docs/use-cases/bearing-health) — predict bearing failure from vibration data - [Composition Patterns](/docs/concepts/composition-patterns) — the patterns behind this pipeline --- # Detecting Server Latency Degradation Source: https://composer.winkjs.org/docs/use-cases/server-health This is real data from the [Numenta Anomaly Benchmark](https://github.com/numenta/NAB) — a server in Amazon's East Coast datacenter that ended in complete system failure from a documented AWS API outage. A winkComposer flow extracts 4-hour statistical fingerprints from raw latency and fuses them into a health assessment that flags structural degradation 4 days before the first visible anomaly. **Drag the slider** and watch it detect what the raw chart hides. ## What You're Seeing The top chart shows raw request latency (cyan) tracked by an exponentially smoothed mean (lavender dashed). The filled **envelope** is the adaptive floor-to-ceiling range — tight when the server is stable, widening as latency behaviour becomes erratic. Below it, the **stddev sparkline** shows the standard deviation computed over non-overlapping 4-hour windows. This is where the hidden structure emerges: the mean barely moved (0.3ms over 14 days), but the window-to-window standard deviation climbed 21% from the first quarter to the third — invisible in the raw signal, clear in the sparkline. The vertical marker on the sparkline shows where a [Page-Hinkley](https://en.wikipedia.org/wiki/Page%27s_cumulative_sum_test) change-point detector fires — the noise floor has shifted to a new regime. With moderate sensitivity, this fires at Day 4.2, roughly 4 days before the first labeled anomaly at Day 8.2. Three labeled anomalies appear in the dataset: a transient glitch (Day 8.2), a burst overload (Day 12.8), and system failure (Day 15.0). The assessment card tracks the escalation from Normal through Monitor and Degraded as the statistical fingerprints deteriorate. The key signals table shows each signal's intensity and **persistence**. The leading signal is volatility (window standard deviation), not the mean — exactly the kind of structural change that threshold-based alerting on the mean would miss entirely. --- ## How It Works One flow, five building blocks — each extracts a different perspective on server health: **Tumbling window statistics** compute exact mean, standard deviation, range, kurtosis, and coefficient of variation over non-overlapping 4-hour windows of raw latency. No exponential decay, no tuning — pure batch statistics at streaming speed. This is where the hidden structure emerges: the standard deviation climbing from 1.63 to 2.02 while the mean barely moves. **Window range** computes the spread within each window — the difference between the highest and lowest latency reading. A widening range means tail latencies are growing, even if the average holds steady. **Exponential statistics** provide the visual layer — smoothed signal and adaptive envelope for the chart. The floor-to-ceiling envelope tracks recent extremes and makes the raw signal's behaviour legible. **Change-point detection** runs a [Page-Hinkley](https://en.wikipedia.org/wiki/Page%27s_cumulative_sum_test) test on the window standard deviation. When the noise floor shifts to a new level, it fires — the vertical marker on the sparkline. With moderate sensitivity (delta 0.008, lambda 0.8), it detects the first structural shift at Day 4.2 with zero false alarms in the first three days. The **health assessment** fuses four signals into one verdict: - **Volatility** — window standard deviation exceeding the 1.63ms baseline (the dominant signal, weighted 1.0) - **Tail latency** — window range exceeding the 7.6ms baseline (how wide the extremes swing) - **Distribution shape** — excess kurtosis rising above zero (outlier frequency increasing; weighted 0.5 because it is impulsive, not progressive) - **Consistency** — coefficient of variation exceeding the 3.6% baseline (reliability dropping) During the first few windows, the node calibrates a verdict scale from this server's own baseline. Consistently jittery servers and consistently quiet ones are both judged against their own normal, not an absolute bar. Evidence for each signal then builds while the signal is outside its normal range and fades as it returns. Combined evidence determines the verdict — Healthy, Monitor, Degraded, or Critical — and the recommended action. Each signal's **intensity** and **persistence** appear alongside. --- ## Why Tumbling Windows? Exponential smoothing adapts — it tracks the signal and decays old information. That is ideal for real-time alerting on individual samples. But for detecting structural change, you need statistics that do not adapt. A tumbling window computes the exact standard deviation of those 48 readings — no weighting, no decay. When that number changes from window to window, the change is real. The standard deviation of Window 20 is the true standard deviation of those 4 hours of latency. When it rises from 1.63 to 1.94, a genuine structural shift has occurred — not an artefact of exponential forgetting. --- ## References - Ahmad, S., Lavin, A., Purdy, S. & Agha, Z. (2017). *Unsupervised real-time anomaly detection for streaming data.* Neurocomputing, 262, 134–147. [doi:10.1016/j.neucom.2017.04.070](https://doi.org/10.1016/j.neucom.2017.04.070) - Page, E.S. (1954). *Continuous inspection schemes.* Biometrika, 41(1/2), 100–115. [doi:10.2307/2333009](https://doi.org/10.2307/2333009) - Dataset: [Numenta Anomaly Benchmark (NAB)](https://github.com/numenta/NAB) — `realKnownCause/ec2_request_latency_system_failure.csv`. 4,032 samples at 5-minute intervals, 14 days. Server in Amazon's East Coast datacenter; documented system failure from AWS API outage. --- ## Next Steps - [Detecting Bearing Failure](/docs/use-cases/bearing-health) — same approach, physical asset health - [Catching Process Drift](/docs/use-cases/process-quality) — catalyst degradation in a chemical reactor - [Composition Patterns](/docs/concepts/composition-patterns) — the patterns behind this pipeline --- # Classifying Driving Conditions Source: https://composer.winkjs.org/docs/use-cases/driving-modes A truck averaging 40{'\u00A0'}km/h could be cruising steadily or crawling through stop-and-go traffic. Speed alone cannot tell the difference — and the distinction drives fuel forecasts, arrival estimates, and driver coaching.} /> This is real GPS data from a heavy commercial vehicle on NH30 in Uttar Pradesh — a 4-hour run crossing highway stretches and congested zones approaching Budaun. A winkComposer flow extracts three perspectives from the raw speed signal and fuses them into a driving condition classification: Highway, City, or Jam. **Drag the slider** and watch the route paint itself green, amber, or red. **Hover over the labelled points** on the map to see Street View photos of what the truck actually encountered. ## What You're Seeing **The map** shows the truck heading northwest from east of Dataganj through Shahjahanpur toward Budaun. Each segment is coloured by classified driving condition — green for highway, amber for city traffic, red for jam. The white dot tracks the truck's position and coordinates update as you scrub. **The chart** shows the three signals the pipeline processes. Raw speed (grey) is noisy. The sg5-smoothed line (cyan) reveals the speed regime — steady near 50 km/h on the highway, dropping into a ragged 20–30 zone approaching Budaun. The choppiness trace (purple, right axis) is the missing dimension: calm on the highway, erupting in the congested zone. **The coloured bands** behind the chart mark the classified condition. Notice the transition — where smoothed speed still reads 40–45 km/h but choppiness is already rising. The classifier catches congestion before the speed drop makes it obvious. **The assessment card** shows which signals drive the current classification and how persistent they are — the same question fleet managers ask: is this real congestion or a momentary slowdown? Three labelled points on the map mark locations verified against Google Street View — hover over any label to see the actual road conditions. Watch for the brief amber/red flash around minute 57 on an otherwise green highway: that is a flyover construction zone where jersey barriers force the truck from 55 to 19 km/h. The classifier flags it and recovers within 2 minutes. Now scrub to minute 218 near Budaun — the truck crawls at 5–9 km/h through a narrow town road choked with parked vehicles. Same three signals, same appraise node, but the construction zone is a transient disruption while the town congestion is structural. Evidence builds across all three signals and the Jam verdict holds. --- ## How It Works One flow, four building blocks. Each kernel node applies a small window of weights to the speed stream — a lightweight **convolution**. Swap the weights and the same mechanism smooths, differentiates, or detects momentum. Appraise fuses the three outputs into a classification: **sg5** (Savitzky-Golay, 5-point) smooths the speed while preserving sharp transitions. Its output answers *"what regime is the truck in?"* — the trend line that separates highway cruising from sluggish urban driving. This is the dominant signal: evidence builds while smoothed speed drops below 42 km/h. **momentum5** computes a weighted recent direction — a seesaw that is calm when speed is steady and violent when the truck is accelerating and braking in quick succession. Steady cruising at 45 km/h and stop-and-go averaging 45 km/h produce the same smoothed speed but very different momentum amplitudes. Only oscillations outside a calm band of ±3 contribute — highway-level variation produces zero deviation. **sgRate5** (Savitzky-Golay first derivative) captures the rate of speed change. It supplements choppiness with directional information — is the truck accelerating or braking? Only changes exceeding ±2 register. The **condition assessment** fuses all three signals into one verdict. Evidence for each signal builds while the signal is outside its normal range, and fades as it returns: - **Speed regime** — smoothed speed below 42 km/h (highway floor); dominant signal, heavily weighted - **Driving choppiness** — momentum amplitude outside the calm band; catches stop-and-go that smoothing hides - **Speed changes** — acceleration magnitude beyond steady-state; supplements choppiness with direction The system learns the highway baseline automatically during the first ~28 samples. After calibration, combined evidence across signals determines the driving condition — Highway (low), City (moderate), or Jam (high). The half-life of 4 minutes keeps the classifier responsive: it detects a regime change within 2–3 samples and recovers just as fast when conditions improve. --- ## Where This Pattern Fits | Domain | What it classifies | Why speed alone fails | |--------|-------------------|---------------------| | Fleet fuel management | Driving regime per segment | Same average speed, vastly different consumption | | Insurance telematics | Driving smoothness scoring | Smooth highway and erratic city look identical after averaging | | Bus route punctuality | Free-flow vs stop-and-go segments | A bus averaging 30 km/h in free-flow keeps schedule; in stop-and-go it does not | | Delivery ETA | Congestion detection for corridor estimates | Arrival time in congested corridors needs choppiness, not just speed | | Driver coaching | Harsh driving vs traffic-forced braking | Frequent braking in a jam is unavoidable, not a coaching signal | --- ## References - A. Savitzky and M. J. E. Golay, "Smoothing and Differentiation of Data by Simplified Least Squares Procedures," *Analytical Chemistry*, 36(8), pp. 1627–1639, 1964. [doi:10.1021/ac60214a047](https://doi.org/10.1021/ac60214a047) - A. V. Oppenheim and R. W. Schafer, *Discrete-Time Signal Processing*, 3rd ed., Pearson, 2010. - Dataset: Anonymised GPS telemetry from a heavy commercial vehicle on NH30, Uttar Pradesh, India. 259 readings at 60-second intervals, ~4 hours. Speed and coordinates only — no vehicle or operator identifiers. - Street View imagery: Ground-truth photographs on the interactive map are from [Google Maps Street View](https://maps.google.com/), © 2026 Google. Used for editorial verification of classifier output at three locations along the route. --- ## Next Steps - [Kernel — Weighted Sliding Window](/docs/playground/explore-nodes/kernel) — explore sg5 and momentum5 interactively - [Detecting Bearing Failure](/docs/use-cases/bearing-health) — the appraise node in an industrial health monitoring context - [Composition Patterns](/docs/concepts/composition-patterns) — the patterns behind multi-kernel pipelines --- # Tracking Wash Cycle Quality Source: https://composer.winkjs.org/docs/use-cases/wash-cycle-quality A 90{'\u00A0'}bar pump drives a high-pressure cleaning cell. Precision-machined metal parts ride through it in cycles — each cycle either holds spec or does not. The raw outlet pressure is noisy with sensor dropouts and idle periods between washes, so the question is not whether the pump is running. It is whether each wash delivered consistent quality.} /> A winkComposer flow cleans the pressure signal, detects wash cycles, scores each cycle against specification limits (Cpk), and tracks sensor health as dropout frequency increases over the run. **Drag the slider** to move the reading point across the trace — clean wash cycles scored by Cpk, sensor dropouts caught and cleaned, and the health verdict escalating as the sensor degrades. ## What You're Seeing **The pressure trace** shows two lines. The faded grey line is raw outlet pressure — look for the sharp drops to near zero during wash cycles in the second half of the run. Those are sensor dropouts where the data acquisition system briefly loses the signal. The cyan line is the cleaned signal after spike rejection: each dropout is replaced with the median of its neighbours, restoring the true wash pressure. **The green bands** mark detected wash cycles. The first eight cycles (samples 0–830) arrive at high sampling rate — roughly 70 samples per cycle, each lasting about 3.7 minutes. After sample 830, the sampling rate drops to 60-second intervals. Cycles become thinner (3–5 samples each) and sensor dropouts appear. **The zoomed spec chart** below the main chart clips the Y-axis to 82–100 bar so the 85–98 bar spec band dominates the frame. Each plateau is one wash cycle; idle periods are dimmed to keep the temporal rhythm visible without competing for attention. The dashed orange lines are the upper and lower spec limits; the dotted line is the 90 bar target. A plateau that rides tight to target has high Cpk; one that skews toward a limit — or dips below the frame — is marginal or incapable. Cpk is computed once per cycle, at the moment the wash ends: hover a cycle-end sample for its Cpk, hover mid-cycle to see the previous cycle's. **The Cpk panel** shows per-cycle process capability scores as they appear. Cpk measures how tightly the wash pressure stays within specification limits (85–98 bar). A score above 1.33 means capable; below 1.0 means incapable. Notice Cycle 8 — its Cpk drops below 1.0 because a startup transient pulls pressure down to 79 bar. Scores are confidence-coded: full opacity for reliable cycles (30+ samples), faded for marginal (10–29), dim for unreliable (fewer than 10). **The assessment card** fuses four signals into one health verdict. The sensor-quality driver highlights with each dropout spike. The Cpk driver highlights while per-cycle Cpk drops below 1.33. The sample-confidence driver highlights while the cycle sample count drops below 30. The duration driver highlights on abnormally short washes (below 3 minutes). In the first half (clean signal, 70-sample capable cycles, normal duration) all four signals stay within their normal ranges and the verdict sits at Healthy. As dropouts accumulate, cycles thin out, and durations shorten in the second half, combined evidence climbs and the verdict escalates through Monitor toward Degraded. --- ## How It Works One winkComposer flow, eight building blocks plus a health assessment: **Range validation** catches out-of-gauge readings before any analysis begins. Pressures outside 0–120 bar are flagged — a first line of defence against sensor faults. **Spike rejection** detects single-sample sensor dropouts. The pressure drops from 90 bar to near zero for one reading, then recovers. A three-sample median filter identifies these events: if a sample differs from both its neighbours by more than 30 bar, it is a spike. The filter replaces it with the median, producing a clean pressure signal that downstream nodes can trust. **Wash detection** uses the cleaned signal with a threshold at 60 bar and 5 bar of hysteresis. Above 60 is washing; below 55 is idle. The hysteresis band prevents chatter at transitions — the pump does not flicker between states when pressure is near the boundary. **Dwell-time tracking** measures each cycle's duration in milliseconds and the pump's duty cycle. Duration appears at each falling edge (wash ends); duty cycle appears after two complete transitions. **Semantic correction** inverts the wash flag at cycle end. The dwell-time tracker reports duration when the state changes from washing to idle — but at that moment `isWashing` is false. The inversion produces `wasWashing = true`, the correct signal for "a wash cycle just ended." **Per-cycle flush** — a controller monitors the duration field and fires a flush command to the statistics accumulator at every wash end. This is the production recipe: the accumulator runs continuously, and the controller determines when to emit a complete cycle's statistics. **Per-cycle statistics** accumulate the cleaned pressure signal into mean, standard deviation, sample count, coefficient of variation, min, and max. The window is intentionally large (100,000) so it never auto-completes — only the controller's flush triggers emission. Wash samples dominate each accumulation window. **Per-cycle Cpk** is computed from the accumulated mean and standard deviation, scored against specification limits (USL 98 bar, LSL 85 bar). Cpk confidence depends on sample count — cycles with 30 or more samples produce reliable scores; fewer than 10 is statistically unreliable. The process capability index is described in Montgomery (2009). **Health assessment** fuses four signals into one verdict: - **Sensor quality** — spike magnitude; each dropout (~88 bar) builds evidence for the sensor-quality signal - **Process capability** — evidence builds while per-cycle Cpk drops below the capable threshold of 1.33 - **Sample confidence** — evidence builds while per-cycle sample count drops below 30; a cycle with only 3 samples cannot produce a reliable Cpk - **Cycle duration** — evidence builds on abnormally short washes (below 3 minutes); sensor-dropout fragments can trigger false cycle-ends lasting seconds Evidence fades between events with a half-life of 50 samples. The node self-calibrates against the first stretch of clean washes, so a pump that normally runs a little noisier than this one would hit the same verdict scale on its own baseline. Combined evidence across signals determines the verdict — Healthy, Monitor, Degraded, or Critical — and the recommended action. Each signal's **intensity** and **persistence** appear alongside. --- ## References - D. C. Montgomery, *Introduction to Statistical Quality Control*, 6th ed., Wiley, 2009. The standard reference for process capability indices (Cpk, Cp, Pp). - Dataset: 1,500 readings from an industrial washing pump, 3-second and 60-second sampling intervals, anonymised. Single outlet pressure channel. Real plant floor data from a production installation. --- ## Next Steps - [Detecting Bearing Failure](/docs/use-cases/bearing-health) — the appraise node tracking vibration-based degradation over days - [Catching Process Drift](/docs/use-cases/process-quality) — multi-signal fusion detecting catalyst degradation in a chemical reactor - [Composition Patterns](/docs/concepts/composition-patterns) — how building blocks combine into detection pipelines --- # Reference Source: https://composer.winkjs.org/docs/reference ## Nodes by Category Looking for a specific node? See the [Node Index](/docs/reference/node-index). ## Beyond Nodes ## Standards Alignment --- # Node Index Source: https://composer.winkjs.org/docs/reference/node-index

Arithmetic between fields

Running total of a field over a sliding or tumbling window. First-order difference between two fields, or against the previous message on the same key. Invert a boolean field — the standard adapter between detection and persistence. Divide one field by another with safe handling of zero and invalid inputs. Apply a pure function to each sample — custom arithmetic without writing a node.

Detection and thresholds

Page-Hinkley change detector; fires when a running mean drifts beyond threshold. Confirm a condition holds for N of M samples before firing — the standard noise filter. Compute Cpk/Ppk process capability against spec limits. Emits when a value crosses a configured bound in a given direction. Detect significant departure of a signal from a projected trajectory.

Feature extraction

Expand a compact digest into full statistics — mean, variance, skewness, kurtosis. Track how long a condition has been active — duty cycles, state lifetimes. Exponentially-smoothed Pearson correlation between two fields. All pairwise correlations across a field group — dense coupling in one pass. Rank local peaks and troughs by prominence over a sliding window; emit each as a completion event. Compare the current value with a lagged value or the same field one message back. Compact statistical digest over a window — pass downstream without materialising samples. Detect transitions between symbolic states with debounce against spurious flips. Classify a signal as stable, rising, or falling from its recent slope. Distance between two vector-valued fields — L1, L2, or cosine.

Flow control

Drops messages that fail a condition — the standard gate for downstream logic.

Intelligence and reasoning

Fuse multiple signals into one verdict — evidence builds per signal and fades between events. Scalar Kalman filter — model-based state estimation with outlier detection.

Observability and I/O

Broadcasts to MQTT when a condition is met. Writes to storage when a condition is met.

Orchestration and control

Enable, disable, reset, pause, and flush nodes on conditions — the runtime's conductor.

Signal conditioning

2nd-order low-pass or high-pass filter for frequency-domain shaping. Map values to named categories against a list of boundaries. Exponentially-smoothed mean; fast to respond, resistant to single-sample noise. Smoothed mean, stdev, envelope, and zScore in one pass. Convolutional windowed operator — smoothing, derivatives, and shaped detectors. Three-sample median filter — cheap, effective spike removal. Validate inputs against ranges, allow-lists, and custom checks. Detect and clean isolated spikes while preserving legitimate step changes. Sliding-window exact statistics over a configurable history. Tumbling-window statistics — one emission per complete window. --- # Mapping winkComposer to NIST AI RMF Source: https://composer.winkjs.org/docs/reference/nist-ai-rmf-mapping The NIST AI Risk Management Framework is widely referenced in AI governance procurement, often without a clear mapping behind the citation. This page provides that mapping for winkComposer — function by function, characteristic by characteristic, with the in-scope contributions and out-of-scope boundaries stated explicitly. **AI RMF 1.0** is the basis for this page; the page is updated when the framework is revised. > **In practice** — Composer is relevant when you need a transparent, streaming evidence layer around deployed AI or algorithmic systems. It helps monitor behaviour, detect drift, confirm sustained anomalies, emit events, and preserve evidence. It does not replace model validation, governance, security, privacy, fairness, or incident-response processes — those remain organisational responsibilities. ### Shared Responsibility Boundary Composer provides the streaming measurement, evidence, and eventing layer. The deploying organisation remains responsible for AI system purpose, risk categorisation, metric selection, human review, user feedback, incident response, model validation, security controls, privacy controls, and fairness decisions. This page maps Composer's contribution; it does not transfer organisational responsibilities onto the framework. ## What the AI RMF Is The Artificial Intelligence Risk Management Framework (AI RMF 1.0) is publication **NIST AI 100-1**, released by the National Institute of Standards and Technology, U.S. Department of Commerce, in **January 2023**, with DOI [10.6028/NIST.AI.100-1](https://doi.org/10.6028/NIST.AI.100-1). Two facts about the framework matter for any mapping exercise: 1. **It is voluntary.** The AI RMF is a guidance framework, not a regulation. There is no NIST certification scheme for AI RMF compliance. *Alignment with AI RMF* is an architectural statement, not an audit result. 2. **It is a living document.** NIST plans formal review with community input no later than 2028. Companion documents (the AI RMF Playbook and the Generative AI Profile, NIST AI 600-1) extend the core framework. This page is updated when the framework is revised. --- ## The Framework's Two Axes The AI RMF organises trustworthy AI along two complementary axes. **Axis 1 — Seven characteristics of trustworthy AI** (Section 3): | # | Characteristic | |---|---| | 3.1 | Valid and Reliable | | 3.2 | Safe | | 3.3 | Secure and Resilient | | 3.4 | Accountable and Transparent | | 3.5 | Explainable and Interpretable | | 3.6 | Privacy-Enhanced | | 3.7 | Fair – with Harmful Bias Managed | The framework notes that *Valid & Reliable* is the base condition for the others, and *Accountable & Transparent* applies across all of them. **Axis 2 — Four functions of the AI RMF Core** (Section 5): | Function | Role (paraphrased from NIST AI 100-1 §5) | |---|---| | GOVERN | Cross-cutting culture, accountability, policies, oversight; informs and infuses the other three functions | | MAP | Establishes context, categorises the system, identifies risks, characterises impacts | | MEASURE | Selects and applies metrics; evaluates trustworthiness; tracks risk over time; assesses measurement effectiveness | | MANAGE | Prioritises risks, allocates resources, plans response and recovery, handles residual risk, drives continual improvement | A complete AI risk management programme covers all four functions across the AI lifecycle. Composer is infrastructure, not a programme — it addresses some functions and characteristics, and is out of scope for others. --- ## Mapping Summary — by Function | Function | Composer's coverage | Why | |---|---|---| | GOVERN | Partial — infrastructure-side only | Composer flows are deterministic, versionable, and reviewable. The actual governance work — policies, accountability structures, training, oversight — is org-process, not infrastructure-feature. Composer makes governance *possible* but does not *implement* it. | | MAP | Out of scope | Context establishment, stakeholder identification, impact assessment, risk identification — these are pre-deployment activities that belong in MLOps governance tooling (model registries, model cards, feature stores). Composer is downstream of MAP. | | MEASURE | Strong fit | The MEASURE function explicitly covers continuous monitoring, drift tracking, and real-time evaluation of system behaviour in production. Several MEASURE subcategories map directly to Composer node families. | | MANAGE | Partial — trigger and escalation layer | Composer emits triggers and persists evidence. The treatment, prioritisation, and resource-allocation decisions sit in org-process. Composer is the *what changed* layer; MANAGE is the *what to do about it* layer. | The remaining sections work through each function and the seven characteristics with named subcategory references. --- ## Function-by-Function Detail ### GOVERN — Partial Contribution The GOVERN function is fundamentally about people, policies, and accountability — not about technology. NIST AI 100-1 §5.1 frames GOVERN as a cross-cutting function infused throughout AI risk management that enables the other three. Composer contributes to GOVERN only where infrastructure choices enable governance work: - **Flow definitions as a reviewable source of truth.** Every Composer flow is expressed as pipeline code — nodes, parameters, evidence sources, thresholds, emitters, and persistence choices live in one place. This supports GOVERN 1.5 (ongoing monitoring of the risk management process) by making behaviour reviewable. - **Deterministic, reproducible streaming.** Composer's hot path is allocation-free and the algorithms it runs are deterministic. Given the same flow definition and the same input stream, output is reproducible to within numerical precision. This supports GOVERN 4.3 (organisational practices to enable AI testing and identification of incidents) by keeping behaviour repeatable across environments. - **Configurable persistence for an evidence trail.** `persistIf` plus QuestDB can preserve raw inputs, derived signals, `appraise` scores, state, severity, timestamps, and emitted events. The audit value depends on what the deployment chooses to persist — Composer provides the mechanism, not an automatic audit trail. What Composer does *not* contribute to GOVERN: - Roles, responsibilities, and lines of communication (GOVERN 2.1) - Training (GOVERN 2.2) - Workforce diversity (GOVERN 3) - Decommissioning policies (GOVERN 1.7) These belong to the deploying organisation. ### MAP — Out of Scope The MAP function establishes the context that frames AI risk — intended use, stakeholders, impacts, system categorisation, risk identification (NIST AI 100-1 §5.2). Composer is downstream of every MAP activity. By the time a Composer flow exists, the deploying organisation has already established intended purpose, identified stakeholders, characterised impacts, and categorised the system. Composer does not implement context-establishment activities. MAP is a prerequisite to a useful Composer deployment, not a function Composer fulfils. A deployed flow can, however, preserve MAP decisions as implementation artefacts. Composer's [semantics](/docs/concepts/semantics) layer captures the *facts* — column types, units, physical ranges, operational limits, asset classes — as a single source of truth shared across dashboards, queries, and storage. The flow language captures the *decisions* — thresholds, evidence sources, severity labels, downstream event routes. Together they make context-of-use inspectable, even though Composer did not author it. ### MEASURE — Strong Fit The MEASURE function is where Composer's contribution is structural. NIST AI 100-1 §5.3 frames MEASURE as the application of quantitative, qualitative, and mixed-method tools to analyse, benchmark, and monitor AI risks and impacts — testing systems before deployment and continuously while in operation. Several MEASURE subcategories map onto Composer node families: | Subcategory | Composer contribution | |---|---| | **MEASURE 2.4** — production monitoring of AI system functionality and behaviour | `pageHinkley` for change-point detection; `esStats`/`twStats` for distributional baselines; `processIndex` for capability tracking (Cpk/Ppk) of deployed-model output streams; `threshold` and `emitIf` for departure flags; `stateChangeDetector` for mode transitions | | **MEASURE 2.6** — safety metrics covering reliability, robustness, real-time monitoring, and response times to failures | Composer's deterministic streaming pipeline; `persistenceCheck` for debounce against single-sample false positives; low-latency edge processing | | **MEASURE 3.1** — tracking of existing, unanticipated, and emergent AI risks based on actual performance in deployed contexts | QuestDB persistence of all evaluations; `momentsDigest` for distributional shape; `extremaRank` for amplitude-ranked deviations | | **MEASURE 2.9** — explanation, validation, and documentation of AI model output within its deployment context | `appraise`'s evidence decomposition exposes which evidence contributed how much to a decision | What Composer does *not* contribute to MEASURE: - The metrics themselves (MEASURE 1.1 — selection of approaches and metrics is a domain decision) - TEVV processes (MEASURE 2.1 — test sets, evaluation tools, formal documentation belong in MLOps tooling) - Feedback processes for end users (MEASURE 3.3) - Independent review processes (MEASURE 1.3) Composer is the engine; the metric definitions and the review processes wrap around it. ### MANAGE — Partial — Trigger and Escalation Layer NIST AI 100-1 §5.4 frames MANAGE as the allocation of risk resources to mapped and measured risks on a regular cadence set by GOVERN, with risk treatment plans for response, recovery, and communication around incidents or events. Composer contributes the trigger and escalation layer: | Subcategory | Composer's contribution | |---|---| | **MANAGE 1.3** — high-priority risk responses are developed, planned, and documented | Composer's `emitIf` raises events with explicit severity; MQTT/QuestDB egress routes events to consumers (alarm panels, ticket systems, on-call routers) | | **MANAGE 2.4** — mechanisms to supersede, disengage, or deactivate AI systems that show performance inconsistent with intended use | Composer detects performance regressions in real time; the *decision* to deactivate is upstream and out of scope | What Composer does *not* contribute to MANAGE: - Risk prioritisation policies (MANAGE 1.2) - Resource allocation (MANAGE 2.1) - Documentation of residual risk (MANAGE 1.4) - Third-party risk management (MANAGE 3) These are organisational decisions that consume Composer's emissions, not features Composer provides. --- ## Mapping by Trustworthy AI Characteristic The seven characteristics from NIST AI 100-1 §3 cut across the four functions. The mapping below names what Composer contributes to each characteristic and what falls outside its scope. | # | Characteristic | What Composer contributes | What is out of Composer's scope | |---|---|---|---| | 3.1 | Valid and Reliable | Deterministic streaming behaviour; reproducible outputs from the same spec and input | Validity of the upstream data sources or upstream models | | 3.2 | Safe | Fail-fast input validation via `sanitize`; per-field invalid-value cascade preventing silent corruption | Safety of the deployment context; controls in the OT layer | | 3.3 | Secure and Resilient | Edge-local processing can reduce unnecessary data movement; resilience contribution is runtime behaviour — `sanitize` at the boundary, per-field invalid-value cascade, guarded predicates and dynamic options, graceful recovery from bad inputs | Identity, authentication, access control, key management, encryption, and network segmentation are supplied by the surrounding deployment infrastructure | | 3.4 | Accountable and Transparent | Flow definitions are a reviewable source of truth for behaviour; configurable persistence supports an evidence trail | Org-process accountability — RACI, sign-off, governance committees | | 3.5 | Explainable and Interpretable | `appraise` is a small two-layer spiking neural network — one receptor neuron per evidence source (layer 1), one decision neuron (layer 2). It is natively decomposable: alongside the `combined` conviction score and `state` label, it publishes per-source `charge` (how much each evidence source contributed) and per-source `rate` (how fast each is accumulating); the configuration that produced those numbers — signed weights, half-lives, deviation type per source, and the monitor / degraded / critical thresholds — is visible in the flow definition. This explains Composer's assessment, not an upstream ML model's internal reasoning | Internals of an upstream ML model — those require SHAP, LIME, or similar techniques applied to that model | | 3.6 | Privacy-Enhanced | Composer's edge-deployable footprint enables edge-local processing as a deployment choice; the privacy property is realised by the deployment, not by Composer itself | Differential privacy primitives; k-anonymity; consent management | | 3.7 | Fair – with Harmful Bias Managed | Out of scope for typical industrial deployments. For AI applications where group-level monitoring is relevant, Composer can run an independent flow per `assetId` (or any other group key) — that supports observation, not fairness assessment | Bias detection algorithms; fairness metrics; the policy decisions themselves. Composer ships no fairness primitives | --- ## Worked Example — Monitoring a Deployed ML Model A common AI RMF deployment pattern: a deployed ML model produces predictions; Composer monitors its inputs and outputs in real time and surfaces a deployment-health assessment. | Function / Subcategory | What happens in the flow | |---|---| | MEASURE 2.4 — input distribution | `esStats` over input features establishes a streaming baseline; `vectorDistance` against the training-time baseline produces an input-drift score | | MEASURE 2.4 — output distribution | `pageHinkley` on the prediction stream detects mean shifts; `processIndex` tracks capability against expected output bounds | | MEASURE 2.6 — real-time safety | `persistenceCheck` debounces single-sample anomalies; `threshold` raises alarms when drift sustains | | MEASURE 2.9 — explainability | `appraise` fuses input-drift, output-drift, and persistence signals into a deployment-health score with visible evidence weights | | MEASURE 3.1 — tracking over time | QuestDB persists every assessment; the time series is queryable for trend review | | MANAGE 1.3 — escalation | `emitIf` raises an MQTT event when the deployment-health score crosses a threshold | This pattern assumes the deployment supplies the artefacts that any drift-monitoring layer needs: training-time or accepted-production baselines, feature schemas, expected output bounds, timestamp discipline, and escalation thresholds. Composer monitors against those artefacts; it does not infer their governance validity. --- ## Evidence an Adopter Can Request A standards-aware review benefits from naming the artefacts that can be inspected. For a Composer deployment those are: - The flow definition used for monitoring — every node in the pipeline, with parameters - Input and output schema, and the timestamp policy - Evidence fields persisted to QuestDB or downstream storage - An example emitted event payload, with topic structure and field naming - The `appraise` evidence decomposition — source weights, deviation types, half-lives, charge and rate fields, conviction score, and state thresholds - The semantics layer — column types, units, physical ranges, operational limits - The operational runbook describing who consumes emitted events and what action follows --- ## What This Mapping Is Not - **Not a certification claim.** AI RMF is voluntary. There is no NIST certification scheme for AI RMF compliance. This mapping is self-assessed by the Composer project; it does not imply review, approval, certification, or endorsement by NIST. - **Not exhaustive.** The mapped subcategories above are illustrative, not a complete enumeration. A full Composer-to-AI-RMF crosswalk would include every subcategory in Tables 1–4 of NIST AI 100-1. - **Not unchanging.** AI RMF is a living document; the framework is scheduled for review with community input no later than 2028. This page is updated when the framework is revised. - **Not a substitute for AI governance work.** Composer makes some governance possible; it does not replace policies, accountability structures, training, or oversight. - **Not coverage of NIST AI 600-1 (Generative AI Profile).** Composer's MCP layer surfaces structured pipeline state to LLMs for explanation but does not validate LLM outputs against the GenAI Profile's risks — confabulation, information integrity, prompt injection. LLM output validation is a deploying-organisation responsibility today, and a candidate node for future work. --- ## Cross-References - [Mapping winkComposer to ISO 13374](/docs/reference/iso-13374-mapping) — the parallel mapping for industrial condition monitoring --- ## References - NIST AI 100-1, *Artificial Intelligence Risk Management Framework (AI RMF 1.0)*, National Institute of Standards and Technology, U.S. Department of Commerce, January 2023. DOI: [10.6028/NIST.AI.100-1](https://doi.org/10.6028/NIST.AI.100-1) - NIST AI Risk Management Framework programme page: [nist.gov/itl/ai-risk-management-framework](https://www.nist.gov/itl/ai-risk-management-framework) - NIST AI Resource Center (AIRC): [airc.nist.gov](https://airc.nist.gov/airmf-resources/airmf/) - NIST AI 600-1, *Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile*, July 2024 — companion document for generative AI use cases --- # Mapping winkComposer to ISO 13374 Source: https://composer.winkjs.org/docs/reference/iso-13374-mapping ISO 13374 is widely cited in industrial monitoring procurement, often without a precise mapping behind the citation. This page provides that mapping for winkComposer — block by block, with the boundaries Composer respects and the gaps it leaves open named in plain language. ## What ISO 13374 Is ISO 13374 is published in four parts. The umbrella title varies between editions — Parts 1, 2, and 3 use *"Condition monitoring and diagnostics of machines"*; Part 4 (2015) broadened to *"Condition monitoring and diagnostics of machine systems"* — a renaming consistent with ISO/TC 108/SC 5's broader vocabulary shift around the same period. All four parts share the subtitle *Data processing, communication and presentation*. - **ISO 13374-1:2003** — Part 1: General guidelines (first edition, 2003-03-15) - **ISO 13374-2:2007** — Part 2: Data processing (corrected version 2008-01-15) - **ISO 13374-3:2012** — Part 3: Communication - **ISO 13374-4:2015** — Part 4: Presentation The standard was prepared by ISO/TC 108 (*Mechanical vibration, shock and condition monitoring*), Subcommittee SC 5 (*Condition monitoring and diagnostics of machines*). Part 1 establishes the six-block reference flow. Parts 2 through 4 detail data processing, communication, and presentation requirements that compliant specifications need to conform to — Part 2 §3.7 names the MIMOSA Open Systems Architecture for Enterprise Application Integration (OSA-EAI™) as one such compliant specification. There is no third-party ISO 13374 certification scheme. Specifications can be *compliant with* the standard's requirements (self-declared conformance), but no audit body issues an "ISO 13374 certified" mark. *Alignment with ISO 13374* in this page means the system's pipeline maps onto the six-block reference, not that it has passed a compliance test. A system that *implements* the reference architecture is not the same as a system that *claims compliance*. Composer is in the first category. --- ## How to Read This Mapping in a Compliance Context This page supports standards-aware architecture review. It helps system integrators identify which ISO 13374 functional blocks Composer can implement directly, which blocks require configuration, and which blocks remain the responsibility of upstream or downstream systems. It is not a certificate, conformance test report, or substitute for project-specific validation. What Composer does *not* replace — these belong outside Composer by design: - Sensors, ADCs, PLCs, or calibration systems - Operator workflow, CMMS, EAM, or SCADA alarm-management systems - Project-specific verification and validation What Composer does *not currently* replace — candidates for future work as the framework evolves: - Spectral preprocessing for vibration-heavy deployments (a future `fft` or `welch` node would close this gap) - A validated fault library (the standardised tables in ISO 17359 are a reasonable starting point; a packaged library is a candidate for future work) - Statistical or physics-based RUL models — `trend` already provides the smallest possible RUL primitive (rate-of-change with a confidence score, suitable for linear-extrapolation threshold-crossing estimates); richer RUL primitives (Weibull fits, survival analysis, physics-based degradation) are candidates for future work The first list reflects boundary by design. The second list reflects scope today. --- ## The Six Blocks ISO 13374-1:2003 §2.2.1 defines the six processing blocks as follows. The first three are usually technology-specific in configuration and signal interpretation — signal processing and data analysis targeted to a particular monitoring technology. The final three *combine monitoring technologies* to assess current health, predict future failures, and recommend action. | # | Block | Role (per ISO 13374-1:2003 §2.2.1) | |---|-------|------| | 1 | Data Acquisition (DA) | Converts transducer output into a digital parameter representing the physical quantity, plus context such as timestamp, calibration state, data quality, data collector identity, and sensor configuration. | | 2 | Data Manipulation (DM) | Runs signal analysis, computes meaningful descriptors, and derives virtual sensor readings from the raw measurements. | | 3 | State Detection (SD) | Establishes and maintains baseline profiles, screens new data for abnormalities, and assigns each observation to an abnormality zone (for example, alert or alarm). | | 4 | Health Assessment (HA) | Diagnoses faults and rates current health of the equipment or process, drawing on all state information. | | 5 | Prognostic Assessment (PA) | Predicts future health states, failure modes, and remaining useful life based on current health and projected usage loads. | | 6 | Advisory Generation (AG) | Generates actionable maintenance or operational recommendations to extend the useful life of the equipment or process. | Blocks consume the output of the block before them. A condition monitoring system covers some prefix of this sequence; a complete system covers all six. --- ## Where ISO 13374 Sits in the PHM Standards Landscape ISO 13374 is one part of a multi-standard ecosystem for condition monitoring and prognostics and health management (PHM). Vogl, Weiss, and Donmez of NIST surveyed the family in 2014 — the relevant siblings, with current editions: | Standard | Role | Current edition | |---|---|---| | ISO 17359 | Top-level general guidelines for setting up a condition monitoring programme; includes fault-symptom correlation tables across machine types | 2018 (third edition; cancels 2011) | | ISO 13374 (parts 1-4) | Open architecture for data processing, communication, and presentation — *the focus of this page* | 2003 to 2015 | | ISO 13379-1 | Data interpretation and diagnostics techniques; formalises failure mode symptoms analysis (FMSA) | 2025 (second edition; cancels 2012) | | ISO 13381-1 | Prognostics — formalises ETTF (Estimated Time to Failure), confidence levels, four prognostic phases | 2015 (cancels 2004) | | ISO 18435 (parts 1-3) | Integration with manufacturing automation applications — AIME and ADME structures | 2009 to 2015 | | MIMOSA OSA-EAI / OSA-CBM | Compliant implementation specifications, free for download | active | Composer maps to ISO 13374 specifically; this page does not duplicate the mapping for sister standards. Two adjacent standards are worth naming because they sharpen gaps acknowledged in the mapping that follows: - **ISO 17359:2018** contains standardised tables correlating possible faults with symptoms across common machine types — the standardised analogue of the application-specific fault library mentioned in Block 4 (Health Assessment). - **ISO 13381-1:2015** formalises prognostic vocabulary (ETTF, four phases: pre-processing, existing failure mode, future failure mode, post-action) — the standardised vocabulary for the RUL estimation Composer leaves to specialised libraries in Block 5 (Prognostic Assessment). --- ## Mapping Summary | Block | Coverage | Example implementations | |-------|----------|-------------------------| | 1 — DA | From the digitized-signal boundary onward | source-manager (OPC-UA, MQTT, custom adapters) | | 2 — DM | Time-domain: strong. Frequency-domain filtering: covered. Spectral analysis: gap | sanitize, kernel, butterworthFilter, esStats, twStats, momentsDigest, kalman1d | | 3 — SD | Strong — through configurable detection primitives | stateChangeDetector, dwellTimeTracker, pageHinkley, threshold, persistenceCheck | | 4 — HA | Fusion engine present; fault library is application-specific | appraise | | 5 — PA | Partial — trend-based. RUL estimation: gap | trend, lag | | 6 — AG | Trigger generation only; presentation is downstream | emitIf, persistIf, MQTT and QuestDB egress | The remaining sections name each boundary and gap. --- ## Block 1 — Data Acquisition The DA block converts the transducer output to a digital parameter with related information — time, calibration, data quality, data collector identity, sensor configuration (ISO 13374-1:2003 §2.2.1 a). Composer ingests *post-digitization* signals via its source-manager. Sensor-level acquisition (PLC, RTU, OPC-UA server) sits upstream of Composer. The standard's pre-digitization layer — transducer-to-digital conversion, calibration, sensor configuration capture — is satisfied by upstream automation infrastructure, not by Composer. What Composer covers from this block: - Time-stamping at ingestion, with the source timestamp preserved when the upstream system provides it - Field-level input validation via `sanitize` — range gates, type checks, invalid-value marking - Multi-source merge for assets with split telemetry streams What Composer does not cover: - Sensor signal conditioning — anti-aliasing filters, ADC range scaling, calibration curves - Sensor-side timestamping when the OPC-UA server's source timestamp is unavailable These responsibilities belong in the OT layer. For Composer's mapping to remain valid, the upstream layer should provide a minimum data contract: - A stable asset identifier, with an asset class where flows differ by equipment type - A timestamp in known units (Composer expects milliseconds since epoch; incorrect units silently corrupt dwell time, slopes, and stored records) - Engineering units for each measured field - Calibration status or calibration provenance, where available - A data quality flag or invalid-value convention - Source identity — the PLC, gateway, OPC-UA node, MQTT topic, or collector that produced the reading The asset identifier and asset class together drive Composer's per-asset isolation and its egress topic structure (`edgeDeviceId/assetId/assetClass/insightType`), which can carry an ISA-95 enterprise/site/area/cell hierarchy in the `edgeDeviceId` segment for Unified Namespace deployments. `sanitize` placed at the start of the pipeline catches many bad values when the upstream contract is incomplete; per-field invalid-value propagation prevents downstream nodes from silently corrupting derived signals. These are safety nets, not substitutes for the contract above. --- ## Block 2 — Data Manipulation The DM block runs signal analysis, computes descriptors, and derives virtual sensor readings from raw measurements (ISO 13374-1:2003 §2.2.1 b). In practice this covers signal filtering, feature extraction, and frequency-domain analysis — the standard names bearing vibration monitoring, infrared thermographic monitoring, acoustical monitoring, and motor current monitoring among the technology-specific applications (ISO 13374-1:2003 §2.2.1). Time-domain coverage is strong; spectral analysis is the gap: | DM capability | Composer node | Notes | |---|---|---| | Streaming statistical features (mean, variance, moments) | esStats, twStats, momentsDigest | Exponentially weighted or time-windowed | | Kernel-based smoothing (FIR) | kernel | Built-in presets plus custom weights | | State estimation | kalman1d | 1D Kalman with optional control input | | Frequency-domain filtering (IIR) | butterworthFilter | Lowpass, highpass, bandpass; cutoff in Hz | | Outlier rejection | spikeGuard, sanitize, median3 | Threshold-based and median-based | | Per-field invalid-value propagation | (built into every node) | Invalid inputs cascade so downstream consumers stay honest | | Spectral feature extraction (FFT, PSD) | not implemented | Vibration condition monitoring typically requires this | For applications dominated by time-domain features — process variables, electrical signals at the asset level, weighing, thermal monitoring — Composer's coverage is strong. For vibration-based bearing or rotor analysis, an FFT or PSD pre-processor is required upstream, or the spectral features are computed externally and fed into Composer as derived inputs. A future `fft` or `welch` node would close this gap. --- ## Block 3 — State Detection The SD block establishes and maintains baseline profiles, screens new data for abnormalities, and assigns each observation to an abnormality zone such as alert or alarm (ISO 13374-1:2003 §2.2.1 c). Composer covers this block through several primitives: - `threshold` — fixed and dynamic threshold tests against rolling baselines - `pageHinkley` — classical change-point detection (Page, 1954; Hinkley, 1971), suitable for streaming drift - `stateChangeDetector` — edge detection on derived flags - `dwellTimeTracker` — operating-mode classification by dwell time - `persistenceCheck` — confirms a condition holds for N of M samples before firing; the standard debounce against single-sample false positives The publishing of confirmed events — the egress of an SD result onto MQTT or into QuestDB — is handled by `emitIf` and `persistIf` and properly belongs to Block 6 (Advisory Generation), not to State Detection itself. The standard's emphasis on baseline comparison maps onto Composer's combination of streaming statistics — which establish the baseline — and detection nodes — which flag departures from it. --- ## Block 4 — Health Assessment The HA block diagnoses faults and rates current health of the equipment or process, drawing on all available state information (ISO 13374-1:2003 §2.2.1 d). Traditional condition monitoring systems implement this with a fault library — named modes such as imbalance, misalignment, bearing wear, each with a signature pattern. ISO 17359:2018 contains standardised fault-symptom correlation tables across common machine types, and ISO 13379-1:2025 formalises the diagnostic procedure (failure mode symptoms analysis, FMSA). Composer's `appraise` node provides the *fusion mechanism* — evidence-driven SNN accumulation that produces interpretable health scores. The fault library and the FMSA evidence patterns are application-specific: the system designer defines which evidence patterns map to which fault modes, with what weights and what decay constants. Composer does not ship with a built-in fault library; the standardised tables in ISO 17359 are a reasonable starting point for vibration-based applications. For applications where the fault modes are well understood — bearing wear, pump cavitation, valve sticking — the engine plus a small evidence specification is often sufficient. For applications without an established fault model, a domain study precedes the deployment. Composer runs the fault-evidence model. Checking that the model is right for your asset is the deploying organisation's job. Test it against historical or experimental data before you rely on it. Calibration is the bridge between an engine and a working assessment. With the right parameters a detection node catches genuine failures and stays quiet during normal operation. The approach is the same regardless of signal type: pick a window of known-healthy operation, measure each signal's mean and standard deviation during that window, then set every detection, trend, and assessment parameter relative to that baseline. Several Composer nodes carry a built-in warm-up phase that supports this work — `appraise` learns its firing threshold during warmup and publishes a `calibrating` flag while it does so; `pageHinkley` exposes `minWarmUpSamples`; `trend` reports a `learning` state before it commits to `stable`, `rising`, or `falling`; `lag` reports invalid output during its warmup window. Downstream consumers can branch on these signals to suppress alerts until calibration is complete. Dynamic options extend this further. A node parameter can be a function of the incoming message rather than a fixed value — a threshold that varies by asset class, a window size that scales with the operating regime, a setpoint that follows a control variable. The same flow then runs unchanged across heterogeneous fleets, with the calibration travelling with the data instead of being hard-coded into the pipeline. Three rules of thumb for evidence sources: - Quiet during normal operation. A source that drips signal even when the asset is healthy will accumulate into a false alarm. - Stronger when the fault is worse, not just different. - Driven by sustained behaviour. A single bad sample is not evidence. A property of this approach: every assessment is decomposable. The reader of an `appraise` output can see which evidence contributed to which decision and by how much. This matters for AI governance frameworks that require decision traceability — the companion [Mapping winkComposer to NIST AI RMF](/docs/reference/nist-ai-rmf-mapping) page covers that crosswalk. --- ## Block 5 — Prognostic Assessment The PA block determines future health states and failure modes based on the current health assessment, projected usage loads on the equipment, and remaining useful life predictions (ISO 13374-1:2003 §2.2.1 e). The standard's framing is broader than RUL alone — it includes failure-mode prediction conditioned on future operating loads. The companion standard ISO 13381-1:2015 formalises the prognostics vocabulary: *Estimated Time to Failure* (ETTF), confidence levels, root cause, and four prognostic phases — pre-processing, existing failure mode prognosis, future failure mode prognosis, post-action prognosis. Composer's coverage maps to the pre-processing phase plus light existing-failure-mode trending; the future-failure-mode and post-action phases sit outside Composer. Composer's coverage here is partial: - `trend` provides rate-of-change estimation with a confidence score derived from sample count, persistence, and noise. This supports linear-extrapolation-based threshold-crossing time estimates downstream. - `lag` provides historical comparison — *now* against *N samples ago*. Composer does not provide: - Statistical RUL estimation — Weibull fits, accelerated life models - Survival analysis primitives - Physics-based degradation models - Sequence-based forecasting — LSTM, transformer, ARIMA For applications that require RUL estimation, two integration patterns work: 1. **Stay narrow.** Cede this block to specialised libraries. Composer feeds features in, consumes RUL estimates back, and uses them as inputs to detection or advisory nodes. 2. **Linear extrapolation in-pipeline.** For degradation processes that are approximately linear in their late phase — battery capacity fade, filter pressure drop, brake-pad wear — `trend` plus a threshold-crossing computation is often sufficient. Many practical deployments can start with the second category. RUL estimation is hard at the streaming edge, and the failure mode of overconfident RUL is worse than no RUL at all. --- ## Block 6 — Advisory Generation The AG block generates actionable maintenance or operational recommendations aimed at extending the useful life of the equipment or process (ISO 13374-1:2003 §2.2.1 f). The standard's framing is broader than alerting — both maintenance and operational change recommendations fall within scope. Composer emits *advisory triggers* — events, alerts, and persistent records via MQTT and QuestDB. Composer is the trigger and routing layer. Presentation — the dashboard widget, the SCADA alarm panel, the mobile push notification, the MCP-mediated LLM that explains assessments in natural language — is downstream of Composer and is implemented by the consumer of Composer's emissions. For simple alerting cases, Composer's emit and persist path generates the advisory trigger and the supporting assessment record. The final advisory experience — acknowledgement, escalation, work-order creation, operator guidance, audit workflow — belongs to the consuming system. For applications where the advisory requires a recommendation engine — which corrective action, with what priority, escalating to whom — that recommendation engine consumes Composer's emissions. Communication of those emissions is configurable rather than conformant. Composer ships emitter and persistence adapters (MQTT, QuestDB), but ISO 13374-3-style interoperability depends on the chosen payload schema, topic structure, field naming, units, timestamps, and the contract with the downstream consumer. Existence of an MQTT egress is not, by itself, communication-layer conformance. --- ## Worked Example — A Bearing Condition Monitoring Flow A typical bearing condition monitoring deployment touches all six blocks: | Block | What happens in the flow | |---|---| | 1 — DA | Vibration RMS values arrive pre-computed; FFT and RMS are performed in the upstream signal processor | | 2 — DM | `kernel` smoothing, `esStats` for streaming mean and variance, `momentsDigest` for distributional shape | | 3 — SD | `threshold` against rolling mean ± kσ, `pageHinkley` for early drift detection, `persistenceCheck` for debounce | | 4 — HA | `appraise` fuses evidence from multiple detectors into a health score with explicit decay | | 5 — PA | Trend slope from `trend` extrapolated to a threshold-crossing estimate downstream of the pipeline | | 6 — AG | `emitIf` raises an alert, QuestDB persists the assessment record, MQTT pushes to the dashboard | This is one deployment-ready pattern. It is not the only pattern Composer supports for condition monitoring; it demonstrates how a Composer-centred deployment can participate in all six blocks, with upstream signal processing for spectral features and downstream presentation or recommendation handling where the deployment requires them. --- ## What Adopters Can Inspect A standards-aware architecture review benefits from naming the surfaces a reviewer can examine. For a Composer deployment, those surfaces are: - The flow definition — every node in the pipeline, its parameters, and the order of operations - Input fields and their units, and the timestamp source and unit - Invalid-value handling and any sanitize ranges - Threshold and baseline definitions, including dynamic options where used - For `appraise` — the evidence sources, weights, deviation types, and half-lives - Emitted MQTT payloads and persisted QuestDB records - The runbook describing who consumes those emissions and what action follows Each of these is a configuration artefact, not a black box. --- ## What This Mapping Is Not - **Not a certification claim.** ISO 13374 has no third-party certification scheme. The standard does support self-declared compliance with its requirements (Part 2 §3.7 names compliant specifications such as MIMOSA OSA-EAI), but the mapping presented here is an architectural statement, not a self-declared compliance claim. - **Not exhaustive.** Composer has more nodes than appear in the table above. `accumulate`, `categorize`, `extremaRank`, `vectorDistance`, `ratio`, `diff`, and others find use across blocks 2–6 depending on the application. - **Not unchanging.** A future `fft` node would close the spectral analysis gap in block 2. A future RUL primitive may close the prognostic gap in block 5. This page is updated when those land. --- ## References **The ISO 13374 series** (umbrella title varies; Parts 1-3 use *machines*, Part 4 uses *machine systems*; all share the subtitle *Data processing, communication and presentation*): - ISO 13374-1:2003 — *Part 1: General guidelines*. First edition 2003-03-15. ISO/TC 108/SC 5. Block role descriptions on this page draw from §2.2.1. [iso.org/standard/21832.html](https://www.iso.org/standard/21832.html) - ISO 13374-2:2007 — *Part 2: Data processing*. Corrected version 2008-01-15. The compliant-specifications discussion (MIMOSA OSA-EAI) is in §3.7. [iso.org/standard/36645.html](https://www.iso.org/standard/36645.html) - ISO 13374-3:2012 — *Part 3: Communication*. [iso.org/standard/37611.html](https://www.iso.org/standard/37611.html) - ISO 13374-4:2015 — *Part 4: Presentation*. [iso.org/standard/54933.html](https://www.iso.org/standard/54933.html) **Sister standards in the PHM landscape** cited on this page: - ISO 17359:2018 — *Condition monitoring and diagnostics of machines — General guidelines*. Third edition; cancels and replaces ISO 17359:2011. Contains fault-symptom correlation tables. [iso.org/standard/71194.html](https://www.iso.org/standard/71194.html) - ISO 13379-1:2025 — *Condition monitoring and diagnostics of machine systems — Data interpretation and diagnostics techniques — Part 1: General guidelines*. Second edition (October 2025); cancels and replaces ISO 13379-1:2012. [iso.org/standard/88027.html](https://www.iso.org/standard/88027.html) - ISO 13381-1:2015 — *Condition monitoring and diagnostics of machines — Prognostics — Part 1: General guidelines*. Cancels and replaces ISO 13381-1:2004. Defines ETTF and the four prognostic phases. [iso.org/standard/51436.html](https://www.iso.org/standard/51436.html) - ISO 18435-1:2009 — *Industrial automation systems and integration — Diagnostics, capability assessment and maintenance applications integration — Part 1: Overview and general requirements*. [iso.org/standard/38692.html](https://www.iso.org/standard/38692.html) - ISO 18435-2:2012 — *Part 2: Descriptions and definitions of application domain matrix elements*. [iso.org/standard/55419.html](https://www.iso.org/standard/55419.html) **Survey paper and implementation specifications:** - Vogl, G. W., Weiss, B. A., & Donmez, M. A. (2014). *Standards for Prognostics and Health Management (PHM) Techniques within Manufacturing Operations*. Annual Conference of the PHM Society, 6(1). Open access (CC BY 3.0). NIST Engineering Laboratory authors. [papers.phmsociety.org/index.php/phmconf/article/view/2503](https://papers.phmsociety.org/index.php/phmconf/article/view/2503) (NIST mirror: [tsapps.nist.gov/publication/get_pdf.cfm?pub_id=916376](https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=916376)) - MIMOSA Open Systems Architecture for Enterprise Application Integration (OSA-EAI) and Open Systems Architecture for Condition-Based Maintenance (OSA-CBM). Compliant implementation specifications named in ISO 13374-2 §3.7. [mimosa.org/mimosa-osa-eai/](https://www.mimosa.org/mimosa-osa-eai/) **Classical references for change-detection primitives** cited in Block 3: - Page, E. S. (1954). *Continuous Inspection Schemes*. Biometrika 41(1/2): 100–115. - Hinkley, D. V. (1971). *Inference about the change-point from cumulative sum tests*. Biometrika 58(3): 509–523. --- # Pipeline Throughput Benchmark Source: https://composer.winkjs.org/docs/benchmark The ~100K msg/sec on a Raspberry Pi and 1M+ on a modern server quoted on the home page come from this exact pipeline — 8 nodes, 900 temperature readings, end-to-end. The same flow now runs in your browser. The number below is yours, measured on your hardware.} /> --- ## What's Being Measured 900 temperature readings flow through an 8-node pipeline: noise filtering (sanitize, median3), dual exponential smoothing (fast and slow), signal differencing, Page-Hinkley cumulative sum test, persistence confirmation, and a controller that resets the smoothers when a change is detected. Each iteration creates fresh pipeline state, processes all 900 messages, and discards the output. The benchmark repeats until at least one second of wall-clock time has elapsed, then reports throughput. This is end-to-end — partition creation, message cloning, state updates, trigger dispatch, and result publishing. No shortcuts. --- ## Methodology - **Timer**: `performance.now()` — sub-millisecond, monotonic - **Warm-up**: one full pass before measurement (JIT priming) - **Duration**: repeats until ≥1 second elapsed - **Partitions**: single asset (one isolated state) - **State**: fresh partition per iteration - **Caveats**: browser results are typically 30–60% of native Node.js throughput due to JIT differences and sandbox overhead The reference hardware numbers use the same pipeline and dataset in Node.js v22. To reproduce natively: ```bash cd composer && node benchmark/compare.js 10 500 ``` winkComposer is transitioning to open source. The benchmark script will be available in the public repository once the transition is complete. --- ## Next Steps - [Hello Flow!](/docs/playground/hello-flow) — build this kind of pipeline from scratch, one node at a time - [Gradual Drift](/docs/playground/recipes/gradual-drift) — the same fast/slow crossover pattern applied to industrial drift - [Under the Hood](/docs/concepts/under-the-hood) — how messages flow through the closure chain --- # Engineering Notes Source: https://composer.winkjs.org/docs/notes --- # What 4KB of RAM Taught Me About Software Engineering Source: https://composer.winkjs.org/docs/notes/what-4kb-of-ram-taught-me
Sanjaya Kumar Saxena · April 8, 2026
TDC-12 — India's first indigenous digital computer. 12-bit architecture, 4KB RAM, 1978.
My first computer had 4KB of RAM. Technically, 4K Words — the TDC-12 was a 12-bit machine, and the byte wasn't its unit. But the constraint was the same. The TDC-12 — India's first indigenous digital computer. 12-bit architecture. I wrote self-modifying code on it. Not because it was clever. Because RAM was the constraint, and planning was the only way to survive. You couldn't waste a byte. Every abstraction had to earn its place. And floating-point arithmetic on a 12-bit machine? Wink at the wrong moment and your computation was silently corrupted. No exception. No warning. Just a wrong answer, quietly propagating forward. You learned a particular kind of paranoia — the kind that never quite leaves you. Four things I took away that I have never unlearned: 1. **Constraints are not the enemy of good engineering. They are the teacher.** Infinite resources stop you from thinking. A 12-bit machine with 4KB forces you to think very carefully indeed. 2. **Planning is not overhead. It is the work.** The code was almost a formality. The real engineering happened on paper, before the machine was ever touched. 3. **Abstractions must be earned.** Every layer has to justify itself. An abstraction that does not reduce complexity is just confusion with better naming. 4. **Numbers lie if you let them.** Floating-point errors are silent. They do not throw exceptions. On a 12-bit machine, the margin for error was so narrow that you had no choice but to think through every computation before trusting it. That instinct has informed every numerically stable algorithm I have written since. --- Forty-something years later — the tools have changed completely. There is something worth noting for anyone reading it now. Modern JavaScript runtimes like V8 do something conceptually similar — and far more sophisticated. On first execution, your code runs as interpreted bytecode. As functions get called repeatedly, V8 generates optimised machine code on the fly, based on observed runtime behaviour. If its assumptions are invalidated — a variable changes type mid-execution — it deoptimises, discards the compiled code, and revises its approach. The program's executable form changes during its own execution. This is the V8 warmup you may have noticed in benchmarks. It is, at its core, a highly engineered descendant of the same idea. The difference: in 1978, I did this by hand. In 4KB. With no safety net. The tools have changed. The underlying idea hasn't moved an inch. --- One acknowledgement I have carried for a long time: none of this would have happened without Prof. S. C. Gupta of IIT (BHU), Varanasi, India, who gave me unlimited access to the TDC-12. That kind of trust, given freely to a young engineer, shapes a career. It certainly shaped mine.