I have been reading a book recently about algorithms in the wider sense
(40 algorithms every programmer should know -book and github link below)
I spend a lot of time with algorithms considering my teaching (AI at University of Oxford).
For Machine Learning and Deep Learning, we need to study a suite of algorithms – but its nice to go back to first principles to study algorithms more broadly. Typically, we don’t have that luxury when we apply them – but many of us have studied these algorithms in Computer Science bachelors or masters.
Hence, this was a good read.
I was also interested more broadly on How to approach the study of algorithms – which is the theme of this post based on the book
Below I list considerations to approach the study of algorithms (from the book)
- Understanding the logic and flow of algorithms
- Algorithm design techniques (data dimensions and compute dimensions)
- Performance analysis (Space complexity, Time complexity, best case, worst case, average case)
- Validating an algorithm (exact, approximate, randomised, explainablity)
- Data Structures Used in Algorithms (List, Tuples, Dictionary, Sets, DataFrames, Matrix)
- Exploring abstract data types (Vector, Stacks, Queues, Trees)
- Basic concepts of designing an algorithm (categorizing complexity, dataset size, analysis of results)
- Understanding algorithmic strategies(divide and conquer, dynamic programming, greedy, brute force)
- Data algorithms(data storage algorithms, streaming data algorithms, data compression algorithms)
- Large-Scale Algorithms (latency, throughput, bandwidth, elasticity)
- design of parallel algorithms
- The explainability of an algorithm
- ethics and algorithms
- Reducing bias in models
- Tackling NP-hard problems
Following is a list of algorithms
Sorting algorithms (Bubble sort, Insertion sort, merge sort, shell sort, selection sort)
Search algorithms(Linear, Binary, Interpolation)
Graph Algorithms(Representation of Graphs, network analysis, graph traversals)
Unsupervised Machine Learning Algorithms(Clustering, Dimensionality reduction, Association rules mining)
Traditional Supervised Learning Algorithms(Classification and Regression)
Neural Network Algorithms
Recommendation Engines
Cryptography algorithms
Missing in this book: (maybe for future editions) time series, Markov decision processes, Bayesian
Book link
40 Algorithms Every Programmer Should Know
Github link – 40 algorithms every programmer should know
Image – Euclid’s elements – one of the oldest and best known textbooks on Algorithms – including Euclid’s algorithm
Image source: http://readingeuclid.org/category/guest-blog-post/
