Lecture 1: Mod-01 Lec-01 Lecture-01-Simple Linear Regression
Lecture Details :
Regression Analysis by Prof.Soumen Maity, Department of Mathematics ,IIT Kharagpur. For more details on NPTEL visit http://nptel.iitm.ac.in
Course Description :
Introduction: the need for statistical analysis, Straight line relationship between two variables. SIMPLE LINEAR REGRESSION: fitting a straight line by least squares - Useful properties of Least squares fit, Statistical properties of least squares
estimators, Analysis of Variance (ANOVA) - Confidence intervals and tests for 0 and 1. F-test for significance of regression. The correlation between X and Y - Interval estimation of the mean response, Prediction of new observation - Coefficient of determination - MULTIPLE LINEAR REGRESSION, Estimation of model parameters. Properties of least squares estimators.
Hypothesis testing in multiple linear regression, Analysis of variance, Test
for significance of regression, Tests on individual regression coefficient - Extra sum of squares method and tests for several parameters being zero,
The extra sum of squares principle, Two alternative forms of extra SS, Two
predictor variables: Example - Multiple regression-Special topics: Testing a general linear hypothesis - Confidence intervals in multiple regression: Confidence intervals on the
regression coefficients, Confidence interval estimation of mean response,
Prediction of new observations - EVALUATING THE PERFORMANCE OF A REGRESSION MODEL, Residual Analysis: Method for scaling residuals, Standardized residuals, Studentized residuals, PRESS residuals.
Residual plots: Normal probability plot, Plot of predicted response (^y)
against observed response (y), Plot of residuals (ei) against fitted values
(^y). Partial residuals plot - Serial correlation in residuals, The Durbin-Watson test for a certain type of
serial correlation - Examining Runs in the time sequence plot of residuals: Runs test - More on checking fitted models, The hat matrix H and the various types
of residuals. Variance-covariance matrix of e, Extra sum of squares attributable to ei - DIAGNOSTICS FOR LEVERAGE AND INFLUENCE, Detection of influential observations: Cooks D,DFFITS and DFBETAS.
POLYNOMIALREGRESSIONMODELS,Polynomial models in one variable: Example - Picewise Polynomial Fitting (splines), Example: picewise linear regression - Orthogonal polynomials regression - Models containing functions of the predictors,including polynomialmod-els,Worked examples of second- order surface fitting for k=3 and k=2 predictor variables - TRANSFORMATIONS AND WEIGHTING TO CORRECT MODEL INADEQUA-CIES.Variance-stabilizing transformations,Transformations to linearize the model.
Analytical methods for selecting a transformation.Transformations on y: TheBox-CoxMethod, Transformations on the regress or variables - Generalized least squares and weighted least squares.An example of weighted least squares,A numerical example of weighted least squares - DUMMY VARIABLES: Dummy variables to separate blocks of data with different intercepts, same model - Interaction terms involving dummy variables, Dummy variables for segmented models - SELECTING THE BEST REGRESSION EQUATION. All possible regressions and Best subset regression
Forward Selection, Stepwise Selection, Backward elimination, Significanc levels for selection procedures - MULTICOLLINEARITY: Sources of multicollinearity, Effects of multicollinearity.
Multicollinearity diagnostics: Examination of the correlation matrix, Variance Inflation Factors, Eigen system Analysis of X'X - Methods for dealing with multicollinearity: Collecting additional data,Re-move variables from the model,Collapse variables - Ridge regression: Basic form of Ridge Regression, In what circumstances is ridge regression absolutely the correct way to proceed?
THE GENERALIZED LINEAR MODELS (GLIM): The exponential family of distributions: examples - Logistic regression models: models with binary response variable.Estimating the parameters in alogistic regression model,Interpretation of the parameters in logistic regression model,Hypothesis tests on model parameters - The Generalized Linear Models (GLIM): Link functions and linear predictors, Parameter estimation and inference in the GLM - AN INTRODUCTION TO NON LINEAR ESTIMATION,Linear regression models,Non linear regression models,Least squares for non linear models
Estimating the Parameters of a non linear systems,An example.
Robust Regression:Least absolute deviations regression(L1regression),M-estimators,Steelemploymentexample - Least median ofsquares regression,Robust regression with ranked residuals - EFFECT OF MEASUREMENT ERRORS INREGRESSORS:Simple linear regression,The Berkson Model - INVERSE ESTIMATION-The calibration problem - Resampling procedures(BOOTSTRAPPING):Resampling procedures for regression models,Example:Straight line fit