Summary: Python’s open-source and high-level nature, as well as its comprehensive libraries, make it the perfect fit to solve the numerous real-life ML challenges.
The increasing popularity and accessibility of Artificial Intelligence solutions is rapidly reshaping many industries, from healthcare through finance to aviation. Although the application of the latest technologies has always been an essential consideration for companies striving to get ahead of the curve, the ubiquity of AI means that it’s becoming the core of many operations.
But the increased use of ML, understood as a technical implementation of AI concepts, across the board has generated a wide range of problems that require both a creative approach and efficient tools to solve.
Python, a highly versatile programming language, aids both; its significant flexibility and comprehensiveness, as well as availability of libraries and a supportive community, make it the perfect choice for developers who create ML solutions.
Let’s take a closer look at some of Python’s qualities below.
a) Flexibility
Python is a highly flexible, scalable, and portable platform that allows developers to choose between OOPs and scripting, as well as combine the language with others.
b) Popularity
According to the 2018 Python Developers Survey, conducted by the Python Software Foundation, the language’s adoption is already at 84% and it’s growing rapidly worldwide. Stack Overflow has found that Python is the fastest-growing major programming language.
As a result of its spread, Python is becoming more and more popular among data scientists, too.
c) Community support
Being a wildly popular and open-source language, Python enjoys a strong and responsive community of developers, constantly striving to improve it and make it more accessible to beginners.
d) Libraries
The availability of a large number of open-source libraries compatible with Python is one of the language’s greatest assets. Although they can be used for a variety of purposes, there are many libraries that provide solutions to ML challenges.
But what are some of the ML challenges that Python can help solve?
Supervised machine learning
a) Problems
Supervised machine learning, where the algorithm learns form a labeled dataset and its output is already known, is by far the most common across a variety of use cases. Here are the two main areas where supervised learning is applied, with examples of their real-life usage.
Classification: this technique is used to categorize data into desired and distinct classes. It predicts a discrete value.
For instance, classification can be used to assess creditworthiness, or to make more accurate readings of X-ray scans to detect early signs of cancer.
Regression: the predicted outcome is an estimation of a numeric value, the so-called “dependent variable,” which depends on other variables (known as independent variables). Regression is one of the most common algorithms, used in problems that involve continuous numbers. Some of its uses include demand forecasting, property price estimation, and financial forecasting.
b) Python solutions
There are many Python libraries that can be used to solve both classification and regression problems. Some of the most popular ones with substantial community support include:
- scikit-learn (for instance, the SVM algorithms, Linear and Quadratic Discriminant Analysis, Nearest Neighbors, Naive Bayes, decision trees, ensemble methods, and others)
- TensorFlow
- Keras
- PyTorch
- Caffe2 (deep learning)
- XGBoost
- CatBoost
- LightGBM (gradient boosting)
The most popular data science frameworks and libraries according to the Python Developers Survey 2018:
Source: https://www.jetbrains.com/research/python-developers-survey-2018/
38% respondents of the Python Developers Survey 2018 said they use Python for Machine Learning, up from 31% a year before:
Unsupervised machine learning
a) Problems
In unsupervised learning, ML is handed an unlabeled dataset without any training instructions or known outcome. It relies on its own ability to solve complex problems based on the input data.
Clustering and matrix factorization are two examples of unsupervised learning methods. They are used to group elements based on the similarity between object properties. Clustering has been used successfully across many industries in the form of user segmentation, and matrix factorization is the backbone behind recommender systems used by Netflix, YouTube, and many online shopping platforms.
b) Solutions
Although it hasn’t been used as often as supervised learning, the unsupervised method can also count on a wide range of Python-compatible libraries. The most popular one is scikit-learn (with algorithms such as k-means, hierarchical clustering, DBSCAN, OPTICS, BIRCH). Some of the most popular libraries used in recommender systems are:
- Surprise (neighborhood-based methods, SVD, PMF, SVD++, NMF)
- LightFM (hybrid latent representation recommender and matrix factorization)
- Spotlight (which uses PyTorch to build recommender models)
Reinforcement learning
a) Problems
Reinforcement learning refers to algorithms that learn to achieve particular objectives and make the right decisions by taking a number of actions and receiving feedback on them. They are able to modify their behavior to attain the best possible result and avoid mistakes.
The reinforcement learning algorithms have been used in video games to achieve above-human performance, for instance in the case of AlphaGo.
Reinforcement learning was also tested in a traffic light control system and reported to show superior results to traditional methods.
b) Solutions
The problems found in reinforcement learning tend to be much more specific than the ones related to supervised and unsupervised machine learning, with solutions more difficult to find. However, there are some libraries that come in handy when dealing with these questions, including:
- keras-rl (Deep Reinforcement Learning for Keras)
- TensorForce (TensorFlow library for applied reinforcement learning)
- Coach (NAF, DQN, DFP, and more)
Given its ability to tackle a wide variety of ML challenges, Python has been used across a number of industries, from fintech to healthcare. The language’s high-level and open-source nature means its popularity is growing among programmers and data scientists alike.
In addition, Python is considered the top choice for ML-related problems thanks to its:
- flexibility,
- simple syntax,
- ability to allow rapid testing of complex algorithms,
- access to comprehensive libraries.
—
About the author: Łukasz Grzybowski is a Machine Learning Consultant at STX Next with over 8 years of data science and development experience in research, insurance, banking, telecommunications, and classified sectors. He’s an entrepreneur and a big fan of startups, new technologies, and innovations, especially in Fintech, IoT, and AI.
