MasterDATA Composite Breadth Data Methodology

MasterDATA compiles its historical composite/breadth datafiles using the current component list only.  There are at least two points of view regarding historical data on composites (indexes and ETFs) as follows:

• Historical  There is logic to the argument that historical composite datafiles should reflect every component list change over the history of an index or ETF.  In other words, Unocal (previously symbol UCL), a component in many major indexes and ETFs, should be included in the composite/breadth data of any index or ETF for which it was a component.  The fact that Unocal was bought out by Chevron in August, 2005, makes no difference.  The price movements of Unocal "should" (according to this point of view) be included in the S&P 500 Index, its related ETF (SPY), the Russell 1000 Index, its related ETF (IWB), the Energy SPDR and many, many more indexes and ETFs permanently and for all time even though it no longer exists.

• Survivor:  There is nothing sacrosanct about any index or ETF.  Indexes and ETFs are simply a list of stocks, often with some commonality, perhaps, but a list nevertheless.  The Russell 2000 Index is a list of (approximately) 2000 mostly unrelated stocks and the fact that the list was different last year, from this point of view, is irrelevant.  In fact, the Russell 2000 Index seldom has exactly 2000 components except on its reconstitution date once each year.  From this point of view, focus should be on the historical composite data only for the current list of components.  After all, you can't trade the historical list.
  
  Another way to look at it is that if you, personally, constructed a "basket" of stocks to analyze for your own purposes and, over time, made changes to the list, would you analyze the list including all the changes you made or would you analyze your current "basket" of stocks?  Most likely, the latter.  What many investors call a "basket" is a composite just like any other composite (indexes and ETFs included) and should be, from this point of view, treated the same way.

Only one of these two points of view is even feasible in the real world.  While it might be interesting to apply the first point of view, historical changes to the component lists of most indexes and ETFs is simply not available.  Although I am certain, records exist somewhere, the index and ETF sponsors are either unwilling or unable to produce such historical changes.  At best, sponsors provide a readily accessible list of current components (some not even that).  The very best in our experience is Standard and Poors which not only provides a current list but a list of changes to the component list over the last few years.  But even the best sponsor does not provide component list changes for the life of its indexes.

We are therefore left with the Survivor approach as our only realistic alternative.  MasterDATA's historical composite data is therefore based only on the current list of component stocks making up any index or ETF.  This means that when one component is deleted and a new one added to replace it, we recalculate the entire database of historical composite breadth data using the current list only (just as if Unocal never existed).

I will add, however, that I leaned in the Survivor direction in the first place.  In scientific studies it is common practice to discard the best and worst results of any study and calculating on those results remaining.  Eliminating merger and buy out activity as well as bankruptcies and stocks dropping to Pink Sheets, being moved from one index or ETF to a different index or ETF and only using the current list of components approximates a somewhat similar practice.  Whether or not this is a valid argument or simply a rationalization, is really not important.  The bottom line is this.  With the information available, its either this approach or nothing.  I strongly believe it makes more sense, from an investors point of view, to only work with a list of stocks currently trading.  Frankly, it would not be particularly difficult to generate the composite breadth datafiles the other way (if the information was available), but I am convinced the results could not be any more productive.

top

The S&P 500 Index is capital weighted.  Component weighting utilizing this method is based upon market capitalization, so one component basically counts more or less (carries more or less weight) than the next when figuring the current value of the total composite.  A one point price move in a component with a larger market capitalization affects the parent index or ETF total value more than a component with a smaller market capitalization.

The Dow Jones Industrial Average is price weighted.  In this method, component weighting is based upon the price of the component issue.  In other words, if you started a new 30 component index today, this method would add the current price of the 30 components and divide by 30.  Similar to capital weighting, one component thus carries more or less weight than the next, but in this method the higher priced components carry more weight than lower priced components (as simple as this method sounds, it gets very complicated, very quickly when the divisor is changed - in other words, in our example, you would no longer divide by 30, but instead, another number calculated to compensate for various component changes such as spin-offs, etc. - the Dow's divisor is currently less than one).

composite breadth data is inherently equal weighted.  Each component of an index or ETF carries exactly the same weight as the next.  If 316 components of the S&P 500 Index increase in price for the day, the composite breadth statistic, advancing issues (or simply "advances"), is 316.  Because most indexes and ETFs are not similarly equal weighted, a possible conflict presents itself.  For example, using a capital weighted index like the S&P 500 Index with equal weighted composite breadth data could potentially be akin to comparing "apples and oranges".

To address this issue, in addition to providing historical market values, MasterDATA recalculates all followed indexes and ETFs as equal weight.  Both sets of historical data are included in downloads from this site, both actual values and recalculated values. 

The process of recalculating each index and ETF as equal weight developed into a much larger project than one might anticipate from the idea's initial conception.  For one thing, each of as many as 3400 component issues had to be filtered and manually corrected for historic price errors (our data vendor is one of the biggest and "best", but an error here and there can quickly result in a major impact on total values).  Additionally, numerous methods were implemented before arriving at one that displayed absolutely no "drift", but instead provides meaningful values comparable to the indexes' and ETFs' actual market valuation.

The result of this work is very intriguing.  While the overall chart patterns remain basically the same, price moves are generally smoother.  A large price move in a heavily weighted component does not overly impact the index or ETF value unless other components experience similar movement.  From a technical analysis point of view, equal weighting might be considered the ideal weighting methodology for composites.  In any event, the data is provided at no additional charge.  It is your decision and your decision alone to use it or not.

For a chart comparison between index and ETF market values and MasterDATA's recalculated equal weight values, click here.  For more about the primary weighting methods, click here for S&P's brief presentation.

top

MasterDATA will only generate historical composite datafiles on indexes and ETFs with a readily accessible and accurate list of current component issues.  Some ETFs in particular, are hesitant or unable to provide an up to date list.  Indexes and ETFs with components not trading on a US exchange are also not currently included in MasterDATA's followed list (this will change in time).

top

We make every effort to make changes to our component list on the day of the change or, at worst, shortly thereafter.  Buy outs and mergers often cause a component to be dropped from a component list.  Many indexes and ETFs replace such components systematically (for instance, the S&P 500 virtually always has 500 components).  Others refill their component lists at given time intervals (i.e. the Russell indexes).  Still others, such as those with a unit investment trust structure, never replace component issues (i.e. HOLDRS).  Deletions due to bankruptcy, delisting and dropping to the Pink Sheets, are handled similarly.

MasterDATA's practice is to immediately delete any security that is no longer trading or that moves to the Pink Sheets.  If the index or ETF lists a replacement component, that component issue is added simultaneous to the deletion or as soon thereafter as the sponsor makes the information available.

top

The entire MasterDATA database of historical composite data is rebuilt from scratch nightly and reflects the currently published component list of each index or ETF followed.  During the trading session, updates to the full historical database are generated hourly (prices as of the "top" of each hour) and published to the web site by about 20 minutes after the hour.  These "snapshots" are less for the purpose of intraday trading and more specifically to facilitate entry or exit of positions on the close of the trading session.  After the close of trading, numerous updates are generated to reflect adjustments by the exchanges and data vendors to price and volume statistics for the day.  Such adjustments often occur days and even weeks later.  All adjustments are reflected in MasterDATA's full nightly rebuild of all historical composite datafiles.

top

MasterDATA applies it utmost best efforts to maintain its historical composite datafiles to the highest standards of accuracy and timeliness.  Occasionally, uncontrollable events temporarily preclude these efforts including inaccurate information from our data vendors, late information provided by the sponsor of a particular index or ETF, computer malfunction, Internet disruptions, human error, or some other unforeseen event.  In any case, all errors, for any reason will be corrected as soon as possible.  Subscribers will be notified on the web site when an issue arises.

top