2018 will be the year of the three As: AI, Automation, and Advancing analytics skills
In 2017, advanced analytics maintained its momentum in the enterprise. Open source technologies such R and Python gained ground. Other technologies such as machine learning continued to pique interest. Use of the cloud become more mainstream. TDWI expects these technologies to continue to grow in importance. We also anticipate other advanced analytics hotspots in 201
2017 Trends
Open source. Open source has become quite popular, especially for big data and data science, because it is a low-cost source community for innovation, which appeals to many data scientists and analytics application developers- especially those who like to code. During 2017, R maintained its popularity. R has been in use for several decades but exploded on the scene anew, a number of years back, with the advent of data science. Python, an interpreted, interactive, easy to read object-oriented scripting language really gained steam. According to Stack Overflow, a site that helps developers solve coding questions, visits to its site for Python-tagged questions was higher than Java in 2017.[1] We saw increased interest in Python, as well.
Commercial software vendors also saw the writing on the wall and fully embraced open source, integrating Python and R into their environment.
Machine learning. Machine learning — the science of getting computers to act without being explicitly programmed — was, and will continue to be, an extremely hot topic in 2018. Although this technology too has been around for decades, it continues to receive renewed interest as data volumes increase. Deep learning- which uses algorithms to learn functions that can classify complex patterns– in particular continues to gain steam as organizations are interested in using it to classify images and sound. Vendors released “data science workbenches” which provide open source environments for data scientists to develop models and build analytics applications.
Machine learning is an important component of AI (artificial intelligence) that we see as a hot spot in 2018 (see below).
The cloud as part of a data strategy. The cloud has been hyped as the go-to platform for analytics for years. Yet, at TDWI we saw resistance to it from a number of quarters, mostly based on security concerns. In 2017, however, organizations seemed to realize that the cloud could be especially useful for analyzing big data. Cloud data warehouses gained popularity, for instance. Organizations began to understand that it made sense to analyze data where it lands and more often, with ‘new’ data sources such as machine data, this is in the cloud. As organizations look to integrate disparate data for analytics, the cloud is more often becoming part of their data strategy
2018 Anticipated Hot Spots
The trends above will continue to be hot trends in 2018. However, there are other emerging trends that are also worth noting.
The Growth of AI and AI Applications Will Continue
The idea that machines could act “intelligently” has been around since the ancient Greeks, but there has been no real consensus about what artificial intelligence actually means. Back in the 1950s, when John McCarthy of Dartmouth College coined the term, he described it as, “Making a machine behave in ways that would be called intelligent if a human were so behaving.” There has been debate about its meaning ever since. One thing is clear though – “AI” has become the buzzword du jour and it will continue to be in 2018. Many vendors are hyping AI capabilities whether they have them as not.
AI and its subcomponents (machine learning, deep learning, and natural language processing are some) are being woven into the analytics arsenal in marketing, sales, and operations, across industries to increase insights and take action. This trend will continue in 2018.
Additionally, research in AI will continue with new algorithm development. We will see new companies forming around AI and vertical applications built that have machine learning or other AI components woven into them. We will also continue to see the rise of personal assistants and (probably much to our dismay) chatbots becoming ubiquitous. The bar on what is considered AI will continue to rise.
Automating the analytics life cycle becomes more widespread
The analytics life cycle starts with data ingestion and continues onto deployment. In 2017, we heard a lot more from vendors about embedding advanced analytics across the entire analytics life cycle. In other words, vendors are using advanced technology to perform tasks that normally required human intervention. For instance, machine learning is being used in data integration to automatically identify schemas or metadata. It is being used in data preparation for data cleansing and profiling or to suggest transformations. It is being used in visualization to suggest what to plot. It is being used in predictive analytics to automatically build a model. The idea is to make analytics “smarter” so that it is easier for everyone in the business to make use of analytics, continuing the democratization trend we’ve seen over the past few years. This trend made market noise in 2017 and will make bigger noise in 2018.
Advancing Analytics Skills/Organizational processes
As the “smart”/automation/ intelligent app phenomenon mentioned above continues to march ahead, smart organizations will realize that the tools can’t be smarter than they are. It is one thing to automate the building of a predictive model when you’re exploring your data. It is another thing entirely to put that model into production if you don’t know what it actually means.
The automation phenomenon is in the early adoption stage, and as organizations continue to adopt it, they will realize that it can’t stand alone. Perhaps this realization will occur when a model that was built automatically has issues and costs the company money. Or, the marketing director can’t explain his model to the head of the organization and it costs the company money. That means that organizations will wake up to the fact that either the models that have been automatically built need to explain their output better and/or people need the skills to interpret model output – or both. In either case, 2018 should a year of reckoning.
The Bottom Line
Much of what is old is new again, thanks to some new algorithm development and the processing power to make advanced analytics buzz. Advanced analytics in 2018 will build upon what we’ve seen hyped in 2017. TDWI will be following these trends as well as a number of others. For more information on other trends see: TDWI Upside
[1] http://www.zdnet.com/article/java-javascript-c-no-python-is-the-fastest-growing-programming-language/
Fern Halper's data makes the world go 'round 