Methodology
Market Atlas never shows a value without saying where it came from, how fresh it is, and how many independent observers agree. This page explains each step from a raw payload to a canonical quote.
1. Observations, not prices
The atomic unit is an observation: one field (last price, bid, ask, volume, yield…) for one instrument, from one source, at one time. Every observation keeps its source timestamp when the source publishes one, the moment Market Atlas received it, its data-rights status, its real-time class and a reference to the raw payload it was extracted from (layers L0 raw → L1 parsed → L2 normalized → L3 canonical → L4 derived).
2. Instrument resolution
A ticker alone never identifies a security. Each source symbol (BTC-USD, BTC/USD, BTCUSD, BRK.B, BRK-B…) is resolved through a per-source alias table to one stable Market Atlas id such as eq_us_xnas_aapl, crypto_btc_usd, fx_eur_usd or index_us_spx. USD and USDT crypto pairs are different instruments. Listed US equities come from the Nasdaq Trader symbol directories and the SEC company directory (CIK), which is how filings link to instruments.
3. Consensus
For every instrument and field, the engine keeps the latest observation of each source, then computes:
- Freshness window. Real-time observations count for 15 seconds, delayed feeds for 30 minutes, indicative values for one hour, end-of-day values for three days. While a venue is closed, the last session value stays valid for up to four days and is reported as at close. Anything older is excluded and marked stale.
- Tiering. When a real-time or delayed observation exists, end-of-day and indicative values for the same instrument are superseded (kept visible in the provenance table, marked
superseded_by_live). - Weights. weight = source reliability × timestamp quality (exchange 1.0, source 0.9, connector 0.75) × real-time class (real time 1.0, delayed 0.6, indicative 0.5, end of day 0.4) × official-source bonus × freshness decay.
- Source families. Sources believed to share an upstream (for example several sites redistributing the same vendor) count as one vote; extra members of a family receive a quarter of their weight. The number of independent sources shown everywhere is the number of distinct families included.
- Outliers. With three or more candidates, values more than 2 % away from the weighted median are excluded (reason
outlier). - Canonical value. The weighted median of the remaining candidates. Dispersion is the spread between the highest and lowest included value, in basis points.
4. Confidence
Confidence is a 0–1 score of Market Atlas' own agreement, redundancy and freshness — never a prediction or a view on the instrument: 0.15 + 0.35 × agreement + 0.25 × redundancy + 0.10 × freshness + 0.15 × mean reliability, capped at 0.995 so the interface never claims false precision. Agreement falls to zero at 1 % dispersion; redundancy is 1 − e^(−families/2) (one family ≈ 0.39, three ≈ 0.78, five ≈ 0.92); freshness decays with the age of the newest observation (fixed at 1 while a venue is closed). A single stale observation yields confidence 0.
5. Data status labels
| Label | Meaning |
|---|---|
| Live | At least one real-time source within the freshness window (crypto venue feeds). |
| Delayed | Value from a feed that is contractually delayed (Cboe quotes are 15 minutes behind). |
| At close | The venue is closed; the value is the last session's, shown with the closing time. |
| End of day | Official or licensed daily value (ECB/Bank of Canada fixings, Treasury par yields, daily bars). |
| Stale | No source within its window; the last known value is displayed with its age and confidence 0. |
| Withheld | The value exists but its data rights do not allow public redistribution; provenance metadata is still shown. |
6. Source reliability
Each connector carries an operational score from 0 to 100: 40 % availability (share of minutes healthy), 20 % parse success, 15 % latency (p95 under 1 s = full marks), 15 % connection stability, 10 % error rate. It measures how well the integration behaves, not the quality of the venue. The best score across a source's connectors feeds the consensus weight.
7. Events
Ticks are noisy; events are meaningful. Derived events use per-instrument baselines: a price move fires when the change since the last event exceeds max(0.5 %, 8 × the typical tick move); session highs/lows require 200 observations and a 0.1 % improvement; a volatility spike compares realized volatility over the last 30 ticks with an exponentially weighted baseline (≥ 4×); source divergence fires when included sources disagree by more than 50 bps. Venue and regulator events (halts, filings) are deduplicated on their native identifiers; additional sources confirming the same event increase its source count instead of creating a duplicate.
8. Bars and derived metrics
One-minute bars are aggregated from canonical prices and rolled up to 5m/15m/1h/1d; daily history for equities, indices, FX, crypto and continuous futures comes from licensed end-of-day bars. Market Atlas never overwrites an official daily bar with a derived one. Returns, volatility, drawdown, correlation, breadth and “what changed” counts are computed from these bars and labelled derived.
9. Market hours
Session states (pre, open, post, closed, holiday, early close) come from a calendar engine using IANA time zones, per-venue sessions and holiday tables; US holidays are refreshed from the Nasdaq Trader schedule page by an HTML change-detection connector. Nothing is hard-coded to 09:30–16:00.
10. Limitations
- Equity and index quotes are delayed 15 minutes; Market Atlas does not hold real-time US equity redistribution rights.
- FX and rates are official reference values, published once a day — indicative, not tradable.
- Intraday history starts when Market Atlas first observed an instrument live; gaps are never interpolated.
- Coverage measures Market Atlas observation, not market size or quality.
See also: data rights, sources, developer documentation.