#DataScience

Data is based on 2007 Statistical Abstract of the United States

I’ve thoroughly analyzed my “best” cross validated model and further pruned the model using backwards stepwise regression on the final dataset from the cross validated term algorithm and did model diagnostics and highlighted in green variables that had positive residuals marked out and red for negative residuals (for further analysis)

This is the type of work I was thinking about publishing

This model is the “Income” model.  I’ve included quadratic terms and got an amazing MAPE of 2.65% and an Adjusted R^2 of .97

I’ve noticed the cross validation pruned the hierarchical dependency (crime).  I’m not sure what to make of that atm but I trust it knowing the MAPE was cross validated.  I know I can exclude INTERACTION hierarchy dependencies (unsure about quadratic), but I know Crime is also captured in the interactions, so technically maybe that’s why it’s showing significant.

I’ve included studentized residuals and mapped it to cook’s distance which gives a great view of outlier’s.

I would give my stamp of approval on this model and say it passes the 4 model assumptions

State’s of Note

* Alaska

* Arizona

* Connecticut

* Illinois

* Lousiana

* California

* Maryland

Some best formula inferences

• Income,Poverty,White,Unemployed,Doctors*Infant.Mort,Doctors*Traf.Deaths,Doctors*Unemployed,Doctors*University,Infant.Mort*Unemployed,Infant.Mort*White
  • mape of 4.37%
• Poverty,Crime,Traf.Deaths,Unemployed,Crime*Infant.Mort,Crime*Unemployed,Crime*White,Doctors*University,Income*Infant.Mort
  • MAPE of 6.57%
• University,Poverty,Infant.Mort,White,Crime*Income,Doctors*Traf.Deaths,Doctors*Unemployed,Doctors*White
  • MAPE of 7.26%

I used concepts from Backward stepwise regression to find the best set of factors for inclusion in regression. I got the idea from p values, but I wanted to use cross validation. There is a post that does exactly this, but I realized the results might not be the best CV scores. So I coded a loop, a few actually. This has been an idea I’ve been working on on my linkedin for a while, going back and forth trying to derive millions of combinations of factors (interactions, quadratic terms, etc) to test.

I came up with the most ingenious solution for the best cross validated formula using interactions.

1. Derive all interactions.

2. Develop Cross Validation train/test splits

3. Build full model

3a. For each full model, subtract 1 variable. Find model that performs best over all folds.

3b. Repeat 3 until model does not improve anymore. That is the best formula.

It’s kind of like genetic algorithm, except no mutations.

I got the idea from backwards selection. It’s also a lot easier to code than trying to build a manual backwards selection.

I realized overfitting all terms wouldn’t be an issue as the validation partition wasn’t used for training. So variables that were not meant to be in the equation will stand out the most.

I imagine something similar could be done for forward regression. Start with the single most powerful variable and work your way forward.

Github (private): https://github.com/thistleknot/multipleCorrelation/blob/master/interactionsBackwardsCrossValidationBestSubset.R

Code: https://hastebin.com/sapixopete.bash

Outputted scores: https://hastebin.com/polamefugo.css

data (states.csv): https://hastebin.com/apuvusevir.css

Note: the source call to mape.r can be commented out

I learn and then apply

I dropped the self filtering method of deriving multiple R manually using matrix equations to using cross validation and then when the best model (set of parameters with the lowest RMSE) is chosen. I will then derive the final model against the full dataset.

I use combn to iterate over every combination of factors hashtag#dataScience

I redid my Matrix Multiple Regression Spreadsheet so it’s easier to read and has less needless matrix multiplication outputs and less cluttered formula’s.

Includes

* Inverted Transposed Predictor Matrix * Predictor Matrix
* Covariance Matrix
* Inverted Correlation Matrix

I’ve

* reproduced all Data Analysis toolpack data
* didn’t have to use linest
* nor results from the toolpack
* nor any 3rd party plugins

file: https://drive.google.com/…/1eQcyp9hoqeJDkw8QfP-cjtV1i…/view…

I’m quite proud of myself.  I spent the good portion of the day trying to figure out how to calculate p-scores by hand (by hand I mean in excel as opposed to using R or data analysis toolpack).

Granted.  I did use the analysis toolpack to derive the residuals, but that was to shortcut rather than using linest.  I DID derive the coefficients manually using matrix algebra, so I could have derived the y’s and then the residuals.  But since I wasn’t focused on that, but the t scores and the subsequent p values.  I went straight to that, because that’s what I want to model in R.

Now I can derive the P Scores of traincontrol in R 🙂

Next I’m going to add correlation matrix to this sheet, which is pretty easy since I have the covariance matrix

This file can be found in my uploads

and I got it!

https://drive.google.com/file/d/1mpgHHFwPEoU68Wy9LhYxJQrW88UR08Tb/view?usp=sharing