Gartner and Forrester Begin to Weigh in on Automated Machine Learning (AML)
Updated: Dec 10, 2019
Summary: AML has been around since at least 2016 but only in the last year have Gartner and Forrester begun to offer their opinions. Here’s where we stand.
This has been a big year for AML (automated machine learning).
A number of new players have emerged and pretty much everyone acknowledges that some level of automation is appropriate to enhance the productivity of your data science team. And no, data scientists shouldn’t be alarmed by this trend. None of this approaches letting an untrained person push the button and have a useful model pop out.
And yet something keeps bugging me. I’ve been following this trend since its emergence in about 2016 and there’s still no good single source to turn to for comprehensive reviews and comparisons. Another way of saying this is that if Forrester and Gartner haven’t reviewed AML, does this really represent something to pay attention to.
The Forrester New Wave™: Automation-Focused Machine Learning Solutions, Q2 2019
We were excited earlier in the year when Forrester published its first ever AML review. They picked some winners and losers and gave some good guidance for comparison.
While the analysis was certainly up to Forrester’s high professional standards it was disturbing that there were many high ranked players I had never even heard of. Turns out participants were not selected based on some broad industry survey but from among those Forrester thought were leaders and who had submitted an RFP-like response. Research was limited to a 1 ½ hr. interview / demo and calls to some of their installed customers. Inclusion criteria required the vendor to have at least five installed customers and have a full spectrum AML offering.
So the vendors who participated were to an extent self-selected and there’s no indication of who might have applied and been rejected. I suspect there are a number of other competitors who could have passed these hurdles.
Well, some information is better than none so long as readers see that the field of competitors reviewed is not altogether objective.
What about Gartner?
So I was excited to see a recent report from Gartner “How Augmented Machine Learning is Democratizing Data Science” hoping this might be the first in a much awaited review of AML competitors. Unfortunately it is not.
The report covers the generalities of AML platforms that can be used by professionals, proceeding on by category until arriving at the most completely automated like DataRobot and H2O.ai.
Gartner also seems insistent on applying its own naming convention referring to the category as ‘augmented’ machine learning which really isn’t going to help the reader gain clarity since ‘automated machine learning’ is already the much more common phrase.
There are a couple of observations here worth pointing out.
Gartner believes that fewer than 2% of models are currently being produced using AML (2019) but offers no data to support this.
They project 40% by 2023 (four years) and that 50% of DS activities (a different measure) will be automated by 2025. Wish they would provide some support for these forecasts.
Perhaps most interesting, and I wonder if this is an oversight in the report, is that their interpretation of AML includes the automation of deep learning models (ADL) for image and text. There are a handful of current AML competitors able to do this but it’s unclear how much customers want this bulked up capability.
Who Got Left Out
The reader needs to be aware that your choices are broader than these two reports. First, we’re missing the cloud based AML offerings from the likes of Amazon, Microsoft, Google and others.
Second, there’s a long list of stand-alones that might be added to this list. There’s no way to know if they were invited and didn’t make the cut, or simply missed the invite. Based on my reading over the last few years my guess is that there are at least 30 AML platforms in some early stage of release.
This is not an endorsement, but I was surprised for example not to see some of the following included:
Some even offer automated deep learning as part of their platform if you’re ready to move in that direction.
So these were not the comprehensive studies we’ve been waiting for. But given the fact that, as Gartner puts it, 40% of our ML tasks will be automated over the next four years, we are all anticipating the full on AML studies from both these excellent sources.
Some opinions expressed in this article may be those of a guest author and not necessarily Analytikus.