AI is maturing

Yes. AI (artificial intelligence) is maturing. AI is still limited to its narrow definition. It is not yet the general AI you see taking over Earth in those crazy science fiction movies. So, in this context, what does it mean that AI is maturing?

I just came back from IBM’s Think conference. Think is to IBM fans what Burning Man is to Californians and hippies, even if there is no obvious direct correlation between the two. It is a major conference, where over 30,000 people gather to learn and celebrate technology and its usage. I wanted to share with you some takeaways, specifically in the field of AI.

Maturing means that processes are coming into place. Maturing means that tools are addressing key issues like bias or usability. Maturing means that people have learned from previous errors. Those errors are analyzed and built open.

In a short interview with theCUBE, my friend Rob Thomas, summarizes some his vision and explains some key concepts:

  • There is no AI without IA (information architecture).
  • Methodology is definitely part of the process.

Let’s drill down into both of these concepts.

AI without IA

Information architecture is a requirement to artificial intelligence. At first, it almost seems tautologic, right?

However, when you look at the enterprise, this is not true anymore – if it has been at anytime. A customer of ours had 83 databases mixed in both on premise and in the cloud. Another started

Figure 1 An illustrated way to do machine learning, from xkcd.

development on Azure before settling to AWS, forcing the company to manage both. The latter also combined on-premise data, AWS, and using various Watson services.

IT is mostly about processing data. In French, IT is called “informatique”, which translates to informatics. It means automated processing of information. That’s what humans have been trying to do since the beginning of this science. Since the beginning of those times, we learned the idiom: garbage in, garbage out.

Machine learning and now AI is an evolution of data processing. In an oversimplified definition, humans create a model from an existing dataset and then the computer reapplies the model, a bit like a function. However, in AI, you do that over and over.

You now start to understand that the quality of the data coming into an AI system needs to be even higher than for standard data processing. This is where the need for an information architecture becomes important. Starting an AI practice or project in your organization, without having the proper architecture in place may not bring the quality of the data (or metadata) expected by your AI system.

Not having the architecture will also harm the result and maintainability of your AI system after the initial implementation. Models built by humans are naturally biased, as we all are. Bias is amplified by the system, like this case, a few months ago, of an AI-based recruiting system at Amazon who discriminated against women.

Part of their suite of AI-oriented building tools, IBM offers a solution for detecting and helping you correct bias. Correcting and adapting bias is doable, but it means you have to manage different versions of the same model. Tracking the various versions of models and their parameters (like hyperparameters) is a challenge that traditional source control tools like GitHub may not be equipped to do.

The AI ladder

To get outcomes from data science (see this post) and AI, you will need a methodology. Thomas and his teams call this process the AI ladder. The foundation remains data, but this data needs to be organized, rationalized, governed and architected: in two words, you need information architecture. Once more, the baseline becomes obvious: there is no AI without IA.

Figure 2 Thomas’ AI ladder

Figure 3 The virtuous circle

BM has decided to split IT data-oriented offerings in three groups: collect, organize, and analyze. Those are building blocks. You do not really have to start at a particular spot. Nevertheless, my experience showed me that companies are pretty good at collecting. Those companies directly want to analyze (and this is the second step of the ladder). Although they can see some outcomes, they quickly realize that without organizing the data, they fail or hit a glass ceiling.

One of the key elements remains: the talented people you will put in your data science team and how you can manage them. At Think, I talked with peers about how to organize data science teams. It helped me realize

that, twelve years ago, I built the perfect data science team (without knowing it). It’s not rocket science, I share some takeaways on my blog.

If you want to know more, you can watch Rob Thomas’ full interview on YouTube.

Veracity can definitely help you in building the information architecture you will need to get to AI and help you climb your ladder, whether you are an IBM shop or not.