Gartner and Forrester Begin to Weigh in on Automated Machine Learning (AML)

Summary:  AML has been around since at least 2016 but only in the last year have Gartner and Forrester begun to offer their opinions.  Here’s where we stand.

This has been a big year for AML (automated machine learning).  A number of new players have emerged and pretty much everyone acknowledges that some level of automation is appropriate to enhance the productivity of your data science team.  And no, data scientists shouldn’t be alarmed by this trend.  None of this approaches letting an untrained person push the button and have a useful model pop out.

And yet something keeps bugging me.  I’ve been following this trend since its emergence in about 2016 and there’s still no good single source to turn to for comprehensive reviews and comparisons.  Another way of saying this is that if Forrester and Gartner haven’t reviewed AML, does this really represent something to pay attention to.

The Forrester New Wave™: Automation-Focused Machine Learning Solutions, Q2 2019

We were excited earlier in the year when Forrester published its first ever AML review.  They picked some winners and losers and gave some good guidance for comparison.

While the analysis was certainly up to Forrester’s high professional standards it was disturbing that there were many high ranked players I had never even heard of.  Turns out participants were not selected based on some broad industry survey but from among those Forrester thought were leaders and who had submitted an RFP-like response.  Research was limited to a 1 ½ hr. interview / demo and calls to some of their installed customers.  Inclusion criteria required the vendor to have at least five installed customers and have a full spectrum AML offering. 

So the vendors who participated were to an extent self-selected and there’s no indication of who might have applied and been rejected.  I suspect there are a number of other competitors who could have passed these hurdles.

Well, some information is better than none so long as readers see that the field of competitors reviewed is not altogether objective.

What about Gartner?

So I was excited to see a recent report from Gartner “How Augmented Machine Learning is Democratizing Data Science” hoping this might be the first in a much awaited review of AML competitors.  Unfortunately it is not.

The report covers the generalities of AML platforms that can be used by professionals, proceeding on by category until arriving at the most completely automated like DataRobot and H2O.ai. 

Gartner also seems insistent on applying its own naming convention referring to the category as ‘augmented’ machine learning which really isn’t going to help the reader gain clarity since ‘automated machine learning’ is already the much more common phrase.

There are a couple of observations here worth pointing out.

• Gartner believes that fewer than 2% of models are currently being produced using AML (2019) but offers no data to support this.
• They project 40% by 2023 (four years) and that 50% of DS activities (a different measure) will be automated by 2025. Wish they would provide some support for these forecasts.

Perhaps most interesting, and I wonder if this is an oversight in the report, is that their interpretation of AML includes the automation of deep learning models (ADL) for image and text.  There are a handful of current AML competitors able to do this but it’s unclear how much customers want this bulked up capability. 

See our recent analysis of Booz Allen’s Modzy launch as perhaps a more efficient path to providing DNN models to those who need them.  Let’s be honest, probably 90% of the value being created by models comes from our well understood ML techniques and not the unusual problems that rely on text, voice, and image classification.

Who Do Forrester and Gartner Think Should be Considered

Neither of these reports is based on a comprehensive market penetration survey but it’s interesting to see the overlap.

Who Got Left Out

The reader needs to be aware that your choices are broader than these two reports.  First, we’re missing the cloud based AML offerings from the likes of Amazon, Microsoft, Google and others. 

Second, there’s a long list of stand-alones that might be added to this list.  There’s no way to know if they were invited and didn’t make the cut, or simply missed the invite.  Based on my reading over the last few years my guess is that there are at least 30 AML platforms in some early stage of release.

This is not an endorsement, but I was surprised for example not to see some of the following included:

• Compellon
• PurePredictive
• x.ai
• Xpanse Analytics
• BigML

Some even offer automated deep learning as part of their platform if you’re ready to move in that direction.

So these were not the comprehensive studies we’ve been waiting for.  But given the fact that, as Gartner puts it, 40% of our ML tasks will be automated over the next four years, we are all anticipating the full on AML studies from both these excellent sources.

Additional articles on Automated Machine Learning, Automated Deep Learning, and Other No-Code Solutions

Complete Hands-Off Automated Machine Learning (October 2019)

Automated Machine Learning for Professionals – Updated (August 2019)

Thinking about Moving Up to Automated Machine Learning (AML) (July 2019)

Automated Machine Learning (AML) Comes of Age – Almost (July 2019)

Practicing ‘No Code’ Data Science  (October 2018)

What’s New in Data Prep  (September 2018)

Democratizing Deep Learning – The Stanford Dawn Project  (September 2018)

Transfer Learning –Deep Learning for Everyone  (April 2018)

Automated Deep Learning – So Simple Anyone Can Do It  (April 2018)

Next Generation Automated Machine Learning (AML) (April 2018)

More on Fully Automated Machine Learning  (August 2017)

Automated Machine Learning for Professionals  (July 2017)

Data Scientists Automated and Unemployed by 2025 – Update!  (July 2017)

Data Scientists Automated and Unemployed by 2025!  (April 2016)

Other articles by Bill Vorhies

About the author:  Bill is Contributing Editor for Data Science Central.  Bill is also President & Chief Data Scientist at Data-Magnum and has practiced as a data scientist since 2001.  His articles have been read more than 2 million times.

He can be reached at:

[email protected] or [email protected]