Probability (Data Distributions review), Bayesian statistics

•        Matrix calculus(Take derivative from matrix)
•        Matrix decomposotion techniques(SVD, LR)
•        Math operations with numpy, efficient matrix operations, broadcasting, vectorization
•        KDE
•        Q-Q Plots

General Stages in ML project

• Problem Definition
• Research
• Data Aggregation/Mining/Scraping
• Data Preparation/Preprocessing
• Modelling
• Evaluation
• Deployment
• Data Preprocessing

Data Import (pandas, numpy)

• Handling missing values
• Categorical data encoding
• Outlier detection
• Date data processing (time series)
• Dataset splitting
• Cross Validation (Overfitting, Underfitting)
• Feature Scaling: standardization, normalization
• PCA + Theory

Confusion Matrix and Model Performance metrics

• Recall
• Precision
• Accuracy
• Root Mean Square Error
• F1 score
• ROC AUC
• Confusion Matrix
• Gini Coefficient

Regression Algorithms (with Matrix notation):

• Linear regression + Theory
• Logistic regression + Theory
• Regularization techniques (Lasso, Ridge)
• Knn

Decision Tree Algorithms:

• CART + Theory
• Bagging & Boosting + theory
• Random Forest + theory

Boosting Algorithms:

• AdaBoost
• XGBoost + Theory
• LightGBM
• CatBoost

Neural Networks

• Neural Networks, backpropagation and forward propagation (single layer perceptron)
• Multilayer perceptron
• Cost functions
• Activation functions

Optimization Methods (Hyper-parameters, Gradient Descent etc.)

• Gradient Descent
• Stochastic Gradient Descent

Hyperparameter optimization

• Grid search
• Random search
• Bayesian optimization

     Imbalanced classification

• ROC curve and threshold choice
• SMOTE (Oversampling)
• Near-Miss (Undersampling)

Model explainability

• Permutation importance
• Partial dependency plots
• SHAP values

Unsupervised learning:

• K_Means clustering + Theory
• Clustering evaluation metrics
• Dimensionality reduction (e.g. t-SNE, UMAP)

DL Frameworks (Tensorflow2 , Keras)

Introduction to Tensors, variables

Introduction to Gradients and Automatic Differentiation (tf.GradientTape())

Introduction graphs and functions

Introduction to modules, layers, and models

Basic training loops

Advanced Automatic Differentiation

Keras Sequential model

Training & evaluation with the built-in methods

Save and load Keras models

Working with preprocessing layers

Working with the model’s layers

Dropout, Weight initialization

Optimization algorithms (Mini batch gradient descent, Momentum, RMSProp, Adam)

Batch normalization

CNN (Convolutional Neural Network)

Convolutions

Pooling

Fully connected

RNN (Recurrent Neural Networks)

RNN

LSTM

GRU

NLP

Classical methods overview(language model, tf-idf vectorization)

Word embeddings, CBOW and skip-gram methods, word2vec