--- title: "Feed Quality Validator" description: "Paste OHLCV bar data and run automated feed quality checks: OHLCV consistency, outlier detection, gap flagging, and large return anomaly detection." canonical: https://www.getswoopr.com/trading-technology/market-data-engineering/feed-quality-validator/ source: "Swoopr Investment: https://www.getswoopr.com" --- ## Direct Answer > A feed quality validator runs automated data-quality checks on OHLCV bar data, internal consistency, outlier price detection, bar gap flagging, and large-return anomaly alerts, and flags each issue with a severity level and a plain-language explanation. Paste in CSV-format bar data and set your QA thresholds to run the checks. Catching bad market data before it reaches a backtest or live strategy prevents corrupted signals and false performance results. # Feed Quality Validator Paste CSV-format OHLCV bar data, set your QA thresholds, and run automated data quality checks: OHLCV internal consistency, outlier price detection, bar gap flagging, and large return anomaly alerts. Each issue is flagged with severity and a plain-language explanation. ![A person using a tablet for stock market analysis in an office setting.](https://www.getswoopr.com/images/hero/trading-technology-market-data-engineering-feed-quality-validator.webp) *Photo by [Yan Krukau](https://www.pexels.com/@yankrukov/) via [Pexels](https://www.pexels.com/photo/a-person-holding-tablet-with-graph-on-screen-7698884/)* ## Input Data OHLCV Data (CSV, one bar per line: date,open,high,low,close,volume) No data? Load sample data with planted errors Outlier Detection Z-Score Threshold 2.5 (sensitive) 3.0 (standard) 3.5 (conservative) 4.0 (very conservative) Large Return Alert Threshold (%) 10% 15% 20% 25% Run Quality Checks Reset ## How to Use This Tool 1. **Paste data:** Enter OHLCV bars in CSV format, date (any format), open, high, low, close, volume, one bar per line. No header row needed. Commas or tabs as delimiters both work. 2. **Choose thresholds:** The Z-score threshold controls outlier price detection sensitivity (lower = more sensitive, more false positives). The large-return threshold flags close-to-close changes that may indicate an unaccounted corporate action or bad tick. 3. **Run checks:** Click "Run Quality Checks." The tool runs four check categories and reports each issue with severity (PASS / WARN / FAIL) and the specific bars involved. ### Check Categories - **OHLCV Consistency:** Verifies High ≥ max(Open, Close), Low ≤ min(Open, Close), and all prices positive. A consistency failure is a data pipeline error, not a market event. - **Outlier Detection:** Computes a z-score for each close price against the full sample's mean and standard deviation. Any close with |z| above the threshold is flagged as a potential bad tick or encoding error. - **Bar Gaps:** Checks for missing bars using the date sequence. A gap of more than 4 calendar days between consecutive bars, larger than a typical weekend or single holiday, is flagged as a possible missing bar. - **Large Returns:** Flags any bar where the close-to-close return exceeds the configured threshold. These should be cross-referenced against corporate action calendars. ## Related Guides - [Market Data QA and Data Lineage](https://www.getswoopr.com/trading-technology/market-data-engineering/market-data-qa-and-data-lineage/) - [Missing Data, Gaps, and Bad Tick Detection](https://www.getswoopr.com/trading-technology/market-data-engineering/missing-data-gaps-and-bad-tick-detection/) - [Corporate Actions and Historical Price Adjustments](https://www.getswoopr.com/trading-technology/market-data-engineering/corporate-actions-and-historical-price-adjustments/) - [Market Data Engineering hub](https://www.getswoopr.com/trading-technology/market-data-engineering/) ## FAQ ### Is the pasted data sent anywhere? No. The checks run in the browser on the page itself, so the pasted content is not transmitted to a server and nothing is stored between visits. That design keeps proprietary or licensed data from leaving the machine, which matters because market data is usually subject to redistribution terms. It also means the tool works without an account and that reloading the page discards whatever was entered. ### What does a consistency failure indicate compared with an outlier flag? A consistency failure means the bar is internally impossible: a high below the open or close, a low above them, or a non-positive price. No market condition produces that, so it is always a pipeline defect, typically a column mapping error, a bad parse, or a merge that combined fields from different bars. An outlier flag is statistical and can be either an error or a genuine large move, so it needs a human judgement that a consistency failure does not. ### Why is the outlier score computed against the whole sample rather than a rolling window? A full-sample score is simple, deterministic, and needs no warm-up period, which suits a tool run on a pasted extract. The tradeoff is that it assumes a roughly stable level: a series with a strong trend will flag its own recent values as extreme simply because they sit far from the overall mean. For a trending series, running the check on returns rather than on price levels, or on shorter segments, gives a more meaningful result. ### How should the outlier threshold be chosen? A lower threshold catches more genuine errors and produces more false positives, which costs review time. The sensitive setting suits a first pass over unfamiliar data where missing an error is worse than reviewing extra bars. The conservative settings suit a series known to be volatile, where flagging every large move buries the real problems. Running the same data at two thresholds and looking at what appears between them is often more informative than choosing one. ### Why does the gap check use calendar days rather than a trading calendar? A calendar-day rule needs no exchange calendar, so it works on any market including instruments that trade continuously, and it does not go wrong when a calendar is out of date. The cost is precision: a multi-day holiday can exceed the threshold without any bar being missing, and a single missing bar in the middle of a normal week will not trigger it. Treat a gap flag as a prompt to check against the venue calendar rather than as a confirmed missing bar. ### What does a large return alert usually turn out to be? Most often an unadjusted corporate action, where a split or a large distribution produces a step change in an otherwise continuous series. Other frequent causes are a decimal or currency unit error, a stale bar repeated from the previous day, and a genuine market event. The distinguishing feature of an unadjusted split is that the ratio between the two closes is close to a simple fraction, which is worth checking before assuming the move was real. ### Does a clean result mean the data is safe to backtest on? No. These checks cover internal consistency, statistical outliers, obvious gaps, and large jumps. They cannot detect survivorship bias in the universe, silently restated values, a subtly wrong timestamp convention, prices adjusted with the wrong factor but still continuous, or volume reported on a different basis than expected. A clean result removes a class of obvious defects, which is a starting point for trusting a dataset rather than a conclusion. ### Can the validator be used on intraday bars? The consistency, outlier, and large-return checks apply to any bar frequency, since none of them assumes a daily interval. The gap check is the exception: it reasons in calendar days, so it will not identify a missing five-minute bar. For intraday data, the practical approach is using the tool for the other three categories and checking interval continuity separately against the expected number of bars in a session. ### How would these checks fit into an automated pipeline? The same four categories translate directly into a validation stage that runs before data is written to a store, with each check producing a severity rather than a pass or fail for the whole file. Consistency violations usually justify rejecting a file outright, while outliers and gaps are better routed to review with the affected rows quarantined. Recording the thresholds used alongside the result keeps a later reader from having to guess why a bar was accepted. ## References - [SEC: Regulation Systems Compliance and Integrity (Reg SCI)](https://www.sec.gov/rules-regulations/2015/12/regulation-systems-compliance-integrity) - [SEC: Market Data Infrastructure (Release No. 34-90610)](https://www.sec.gov/rules-regulations/2020/12/market-data-infrastructure) ## Related Reading - [Market Data Engineering & Feed Quality](https://www.getswoopr.com/trading-technology/market-data-engineering/) the section this guide belongs to - [Corporate Action Adjustment Checker](https://www.getswoopr.com/trading-technology/market-data-engineering/corporate-action-adjustment-checker/) a companion guide in this cluster - [Corporate Actions and Historical Price Adjustments](https://www.getswoopr.com/trading-technology/market-data-engineering/corporate-actions-and-historical-price-adjustments/) a companion guide in this cluster - [Exchange vs. Consolidated Market Data](https://www.getswoopr.com/trading-technology/market-data-engineering/exchange-vs-consolidated-market-data/) a companion guide in this cluster - [Late, Corrected, and Out-of-Order Market Data](https://www.getswoopr.com/trading-technology/market-data-engineering/late-corrected-and-out-of-order-market-data/) a companion guide in this cluster - [Market Data QA and Data Lineage](https://www.getswoopr.com/trading-technology/market-data-engineering/market-data-qa-and-data-lineage/) a companion guide in this cluster - [Market Data Timestamps and Clock Synchronization](https://www.getswoopr.com/trading-technology/market-data-engineering/market-data-timestamps-and-clock-synchronization/) a companion guide in this cluster - [Missing Data, Gaps, and Bad Tick Detection](https://www.getswoopr.com/trading-technology/market-data-engineering/missing-data-gaps-and-bad-tick-detection/) a companion guide in this cluster - [Point-in-Time Storage and Historical Replay](https://www.getswoopr.com/trading-technology/market-data-engineering/point-in-time-storage-and-historical-replay/) a companion guide in this cluster - [Real-Time vs. Cleaned Historical Data Differences](https://www.getswoopr.com/trading-technology/market-data-engineering/real-time-vs-cleaned-historical-data-differences/) a companion guide in this cluster