Welcome

Welcome to the Optimization Techniques for Big Data Analysis (OTBD) course, which is part of the Signal Processing and machine learning for big data track of the Master of Science in Signal Theory and Communications.

This course delves into the fundamental concepts and theoretical underpinnings of convex optimization, offering a comprehensive set of algorithms designed to address a wide range of standard optimization challenges in the field of Machine Learning. Additionally, the course explores algorithm variations tailored for local processing within distributed environments, particularly suitable for handling large-scale data scenarios.

---

Course content and slides

Part1: Fundamentals of Optimization and First Order Methods

• Introduction

• Machine Learning Contextualisation

• Review of Fundamentals of Convex Optimisation

• First Order Methods Applied to Neural Networks

Part2: Second Order Methods and Large Scale Optimisation

• Second Order Methods

• Coordinate Descent Methods

• Augmented Lagrangian Methods

Part3: Automatic differentiation and hyperparameter selection

• Symbolic differentiation and computational graphs

• Introduction to Bayesian Optimisation

---

Interactive course material

Fundamentals of Optimization and First Order Methods

• Machine Learning Contextualisation
• Review of Fundamentals of Convex Optimisation
• First Order Methods Applied to Neural Networks

Second Order Methods and Large Scale Optimisation

• Second Order Methods
• Coordinate Descent Methods
• Augmented Lagrangian Methods

Automatic differentiation and hyperparameter selection

• Symbolic differentiation and computational graphs
• Introduction to Bayesian Optimisation

---

Course’s project

• Instructions

---

Download a ZIP file with the course’s notebooks