Regression reveals relationships between variables to predict outcomes. Models train algorithms to identify patterns in data distribution. Two main types: linear regression fits data points along line, logistic regression determines class membership
Loss functions evaluate model performance and guide optimization. Loss measures how well predicted outputs match true labels. Loss minimization helps model make fewer mistakes on training data
Gradient boosting combines weak prediction models into a single strong learner. It optimizes pseudo-residuals instead of traditional residuals. Typically uses decision trees as weak learners
Developed by Fix and Hodges in 1951, later expanded by Cover. Classifies objects by majority vote among k nearest neighbors. Can be used for both classification and regression. Input consists of k closest training examples
AI enables machines to think and make decisions like humans. Machine Learning is a subfield of AI that helps machines learn from data. AI algorithms combine inputs and outputs simultaneously for learning
Multicollinearity occurs when independent variables are highly correlated in regression. It leads to skewed results and wider confidence intervals in statistical analysis. Perfect multicollinearity shows exact correlation between variables (+1.0 or -1.0)