The ValueLine DowJones Model: Does It Have Predictive Content? A USEFUL RULE OF THUMB  “Data Mining” by Michael Lovell (1983, RES) The Lovell Pretesting Rule for Coefficient Significance
 Start MLR with c candidate variables.
 Use A Best Subset Method to obtain
 A MLR with k final variables
 PValue(actual) = (c/k)*PValue(stated)
A MLR PREDICTION RULE OF THUMB  “On the Usefulness of Macroeconomic Forecasts as Inputs to Forecasting Models” Richard Ashley Jo. of Forecasting. 1983
 Var(x(hat))/Var(x) versus 1
 Ratio greater than one, x generally not useful
 Ratio less than one, x possibly useful
 It seems a lot of practitioners ignore this rule at their peril
The ValueLine Dow Jones Stock Evaluation Model  Regression model used by the ValueLine Corporation in its endofyear report (Value Line Investment Survey) to provide its readers a forecast range for the DowJones Index in the coming years. (Model builder: Samuel Eisenstadt)
 DJ — Dow Jones Industrial Average,
 EP —Earnings Per Share on the Dow Jones,
 DP — Dividends Per Share on Dow Jones, and
 BY — Moody’s AAA Corporate Bond Yield
 logarithm transformation linear form:

Motivation  No evaluation of the model in existing literature, although the model is in use for over twenty years and possibly by millions of readers who may have made decisions upon forecasting results from the model. It would be interesting and useful to see how precise and reliable these forecasts are.
 Arguments in the literature about the forecasting competence of regression model vs. univariate models, eg. Ashley (1983). Accuracy of the model depends on the accuracy of the forecasts of the independent variables. Are the independent variables making the forecast better or worse?

Outline of Presentation  Data
 Stability Analysis
 OutofSample Forecast Evaluation
 (Predictive Content of Input Variables)
 Conclusions
Data  Annual observations (19202002) on
 DJ: Dow Jones Industrial Average, annual averages
 EP: Earnings Per Share on the Dow (data point 1932 adjusted
 for convenience of log transformation)
 DP: Dividends Per Share on the Dow
 BY: Moody’s AAA Corporate Bond Yield
 Data source: “Long Term Perspective” chart of the Dow Jones Industrial Average, 19202002, published by the Value Line Publishing, Inc. in Value Line Investment Survey
 Logarithm transformation used to obtain linear regression
 Comparisons are made among forecasts of DJ
Stability Analysis for VLDJ Model: Recursive Coefficients Diagrams  As reported in end of year ValueLine Investment Survey, coefficients are estimated as follows:
 2002: (1.030, 0.210, 0.350, 0.413); 1999: (1.034, 0.217, 0.332, 0.468);
 2001: (1.032, 0.218, 0.336, 0.463); 1998: (1.032, 0.216, 0.335, 0.473), and so on.
 2000: (1.033, 0.214, 0.340, 0.480);
Stability Analysis for VLDJ Model: CUSUM and CUSUMSQ Test Results  The CUSUM test is based on the statistic:
 The CUSUMSQ test is based on the statistic:
 Where is recursive residual defined as
 S is the standard error of the regression fitted to all T sample points.
Test for Structural Change of Unknown Timing: Wald Test Sequence as a Function of Break Date  Andrews (1993, 2003) critical values
The Models for “DLDJ” (specified using insample data only)  Transfer function model (in same form as the ValueLine Model): DLDJ at time t is a function of DLEP, DLDP and DLBY at time t ; where
 DLEP ~ MA(2) , DLDP ~ MA(1) and DLBY ~ AR (1)
 BoxJenkins univariate model: DLDJ ~ MA(1).
 Note: Transform Predictions for DLDJ to DJ in two steps:
 Step 1:
 Step 2:
ExAnte Forecast Accuracy— Transfer Function vs. BoxJenkins (Imperfect Foresight) Usefulness of Explanatory Variables in the Transfer Function Model— Forecast AccuracyRMSFE: Assuming Perfect Foresight for Leading Indicators in Transfer Function Model  *Disadvantage: Loss of forecast accuracy relative to TFPerfect
Value Line Forecasts vs. TF and BJ Forecasts  *The MAFE and RMSFE are computed based on years 19832002 except 19931995
Combination Forecasts of TF and BJ  Simple Average (CF1)
 Nelson Combination (CF2)
 GrangerRamanathan Combination (CF3)
 FairShiller Combination (CF4)
 Note: We apply dynamic weights
Forecast Accuracy (RMSFE)— BoxJenkins vs. Combinations Ways of Combating Weak Input Variables  Drop input variables that don’t satisfy Ashley’s Criterion (Forecast could have bias but less variance)
 Use improved input variables: Combination of sample mean and forecasts of input variable
  Simple average
  Ashley (1985) combination
Forecast Accuracy— Dropping Inadequate Input Variables Forecast Accuracy— Input Variables From Combination Forecasts Conclusions  In the absence of perfect foresight, TF (Value Line) forecasts are less accurate than the BJ benchmark forecasts for any forecast horizons.
 Ashley (1983) criterion shows that the leading indicators are very noisy and inhibit ex ante forecasting accuracy of TF model.
 If future values of leading indicator variables are assumed known, (perfect foresight), TF forecasts improve considerablybeat the BJ forecast for 26 stepahead forecast horizons, but do not for the 1stepahead forecast horizon.
Conclusion (cont.)  With respect to Ex Ante combination forecasting, BJ forecasts perform better for short horizons and combinations of the TF and BJ are best for longer horizons.
 For Ex Ante forecasts, differences in accuracy between TF forecasts and the most accurate forecasts are not statistically significant. Ashley (2003)
 Dynamic Combination forecasts perform better than combinations with fixed weights.
 Dropping inadequate input variables did not improve forecast accuracy. Using combination forecasts for the input variables only improved the forecast accuracy of some horizons.
Conclusion (cont.)  Evidently, the Value Line personnel have been pretty astute with respect to choosing future values of the independent variables of their model. Their published 1stepahead forecasts have smaller MAFE than the ex ante TF model and the BJ model. With respect to the RMSFE, however, the BJ model provides a more accurate 1stepaheadforecast.
 Remember forecasting accuracy is only one way to evaluate the VLDJ model. Irrespective of its forecasting powers, it should be recognized that the VLDJ model is potentially quite useful for examining “what if” scenarios and understanding historical causal factors in the stock market.
 It would be interesting to compare competing models based on interval forecast accuracy and density forecast accuracy.
Thank you! References  Andrews, D. W. K. (1993): “Tests for Parameter Instability and Structural Change with Unknown Change Point,” Econometrica, 61, 821856.
 Andrews, D. W. K. (2003): “Tests for Parameter Instability and Structural Change with Unknown Change Point: A Corrigendum,” Econometrica, 71 (1), 395397.
 Ashley, R. (1983): “On the Usefulness of Macroeconomic Forecasts as Inputs to Forecasting Models,” Journal of Forecasting, 2, 211223.
 Ashley, R. (2003): “Statistically Significant Forecasting Improvements: How Much OutofSample Data Is Likely Necessary?” International Journal of Forecasting, 19(2), 229239.
 Bai, J. (1997): “Estimation of A Change Point in Multiple Regression Models,” Review of Economics and Statistics, 79 (4), 551563.
References (cont.)  Brown, R. L., J. Durbin, and J. M. Evans (1975): "Techniques for Testing the Constancy of Regression Relationships Over Time," Journal of the Royal Statistical Society, Series B, 37, 149192.
 Diebold, F. X. and R. S. Mariano (1995): “Comparing Predictive Accuracy,” Journal of Business and Economic Statistics, 13 (3), 253263.
 Fair, R. C. and R. J. Shiller (1990): “Comparing Information in Forecasts from Econometric Models,” American Economic Review, 80 (3), 375389.
 Nelson, C. R. (1972): “The Prediction Performance of the FRBMITPENN Model of the U.S. Economy,” American Economic Review, 62 (5), 902917.
Combinations of the TF and BJ models  Naïve combination: simple average (weight=0.5)
 Outofsample
 (obs. 5483)
Combinations of the TF and BJ models (dynamic weights applied)  Dynamic Nelson combination (weights sum to 1)

 where weight is obtained from LS regression
 Test Data
 (Outofsample)
Combinations of the TF and BJ models (dynamic weights applied)  Dynamic GrangerRamanathan combination (weights obtained from unrestricted regression)
 where weights are obtained from regression
 Dynamic Fair and Shiller Combination
 where weights are obtained from regression
Data LDJ Data LEP Data LDP Data LBY
Share with your friends: 