Info on Data Quality

Wall Street is the only place that people ride to in a Rolls Royce, to get advice from those who take the subway.

More of Warren Buffet, and you'll realize this when you start to analyse why your trade decisions do not yield expected results that it starts to become clear the importance of data quality in the process.

nude A
Having a motorbike and helmet is not the whole picture, biker clothing is part of the total solution. Just as you would need biker clothing, so you need the most accurate data available from your data feed.

You see, the software is applying various technical indicators to the stream of price data, and variances in that data will affect the outcome of those indicators, meaning your conclusions may be incorrect and so you trade when you shouldn't or you don't trade when you should. You now begin to appreciate there may very well be problems with your data feed.

Warning signs of data feed problems
InvestorData founder, Mike Jenkins, supported corporate and private investors in the mid-nineties, and established that they were not entirely happy with what was being distributed by their data vendors, who took almost no corrective action to reported data issues and that those with multiple packages had to buy the same data from different data vendors as formats were not interchangeable. The biggest frustration was not having an alternative source of data.

Just as your cherished child can cause pain that is not easy to deal with, so your data vendor can lock you into a contract where you know something is not right, but cannot prove anything to anyone - you have been warned.

So, to minimize frustrations you need to be aware of these aspects
  • After-hours support responsiveness - if you cannot resolve an issue 20h00 then you will not be able to trade the next morning
  • Data issue resolution - if someone highlights a data issue, then you would want it fixed for you as well
  • Multiple charting systems invariably requiring multiple data subscriptions
  • Since you are investing real after-tax money, you simply have to use the most accurate data available, to get the most accurate signals possible. Do not assume this to be the case with all data vendors
  • Quality of the data you receive - when is a close not a close and an open not an open?
When is a close not a close and an open not an open

couple joke A
Amazing how words can have such different meanings.

We suspect that data feeds used to rely on their customers being too trusting.

In the beginning, InvestorData received many calls due to differences in CLOSE / OPEN prices between InvestorData and some comparative source, which suggested data being published by the source was not correct.

CLOSE - The JSE used to publish the RULING PRICE as the CLOSE, with the factually correct last traded price buried in their published data somewhere. It took some footwork to identify the actual last trade for each item, but InvestorData took the trouble to do this, so when comparing our published CLOSE with another, they would not always match.

To explain the difference between the two by way of example ... if you purchase a house for R1m and a drunk ambles by the day you moved in and offered you R1.2m (obviously the offer is irrelevant) . . . InvestorData would publish the last trade price of R1m while other feeds would publish the ruling price (last trade price unless a subsequent bid / offer) of R1.2m. Now when you apply technical analysis to this data, it stands to reason that a buy or sell signal from data provided by InvestorData is more relevant, generating buy / sell signals based on factually correct data, not someone's testing of the market to see if they can get a higher / lower price. The fact that the bid / offer was not accepted proves the markets unwillingness to move in that direction . . . so publishing that price was not the smartest thing to do. The JSE have since changed their system to provide the correct close price, but historical data from these data feeds carries the problem into the future, especially when you try to backtest any form of trading algorithms.

OPEN - The opening price also required some footwork but other data feeds took the easy way out. The data provided by InvestorData has always contained a clean OPEN (being the first trade for the day) price, which means that Japanese candlesticks and proper bar charts can be correctly drawn. Many other data feeds don't provide an OPEN price or they provide a dirty OPEN, which is the previous day's CLOSE amended to fit within today's HI-LO range. The charts drawn with a dirty OPEN cannot be compared to the ones our data will generate. Many (not all apparently) data feeds have corrected their systems, but historical data from these data feeds carries the problem into the future, especially when you try to backtest any form of trading algorithms.

You begin to appreciate how
quality data from InvestorData can impact your trading performance !

Decisions made using data that is not factually correct will not perform as well as decisions made on data that is factually correct !

Now that you are aware of the type of issues you can encounter when signing up for a data feed, or you are smiling at recalling the above when you signed up for your data feed, proceed to
JSE endorsing improved data quality.

Perhaps you are a returning subscriber, or a friend has given you the heads-up on dealing with InvestorData so wish to
request a subform.