Taipy or How to Remove Major Hurdles with Your AI/Data Projects

Over the years, I have been involved in implementing many “smart software” projects that demonstrated high benefits to major organizations. At the heart of these different software projects were algorithms based on Mathematical Programming, Simulation, and Heuristics, as well as AI models based on ML and generative AI. Most of these projects led to substantial ROI for these organizations; some have even shaped their company’s future.

Despite all the hype around AI and Data, many organizations (outside of the software industry)  struggle to implement a successful AI strategy. Most CIOs/CDOs involved have mostly produced “standard” data projects (data lakes/warehouses/data management/Dashboarding), some implemented several AI pilots, and very few have generated deployed projects showing substantial ROI for their company. 

One could consider the distribution of companies in terms of AI penetration as a highly left-skewed fat-tail distribution.

The purpose of this article is not to list all the obstacles preventing the wider penetration of AI projects inside companies. For this purpose, I would recommend these two enlightening articles:

Why businesses fail at Machine Learning 

How AI can help leaders make better decisions under pressure 

Instead, we focus on two gaping holes in the current software implementation approach.

Gaping hole 1: A very siloed Environment

Visualizing the various groups involved in a typical AI project is interesting.

Of course, there are valid reasons for having these different roles, let alone the need for specialization. However, it is worth noting that:

On a real project, the gap between the data scientists and end-users is substantial.

Each silo uses different technology stacks. It is not uncommon for data scientists to develop mainly in Python, while IT developers use JavaScript, Java, Scala, etc.

There has never been a wider variety of programming skills between and within each siloes.

Gaping hole 2: Getting acceptance from the end-users / business-users

As highlighted in a previous article, end-users seem to have disappeared from the AI landscape. It is all about data, technologies, algorithms, testing, deployment, etc. As if all AI projects will necessarily replace completely human experts. I am convinced that the future of AI in the industry lies in the hybrid collaboration between business users and AI software. 

However, end-users are an integral part of AI software development. Not getting them fully involved during the development process puts you at risk of not having your software used when the system goes live. 

Our strategy is to ensure that these two steps get implemented:

A smooth end-user Interaction with the algorithm(s)

And an easy tracking of business-user satisfaction

How to fill Gap 1? 

Some obvious directions are:

To standardize as much as possible on a single programming language.

Provide an easy-to-learn/use programming experience to cater to all programming levels. 

Python is the ideal candidate for this. It is at the heart of the AI stack and ideal for integrating with other environments.

Many Python libraries are available and provide an easy learning curve (including low code); unfortunately, they often suffer from performance issues and lack of customization.

Let’s consider, for instance, the development of graphical Interfaces: One has the choice of using full-code libraries like Plotly Dash (or even development in Java Script) or easy-to-develop libraries like Streamlit or Gradio. However, these libraries do not scale performance-wise and will set you into a strict framework forbidding most customization. 

A Python developer shouldn’t have to arbitrage so much between programming productivity and performance/customization.

We spent a lot of time on the design/implementation of our product, Taipy, to go one step further by guaranteeing ease of development while providing a huge leap in performance and customization. Here are two examples of performance issues (amongst many others) solved with Taipy:

How to fill Gap 2?

 Addressing the two salient points mentioned above is crucial:

A smooth end-user Interaction with the back-end algorithm(s)

And an easy tracking of the business-user satisfaction

Addressing Point 1: the end-user needs to interact with the algorithm/back-end. 

For this purpose, it is essential to:

Provide variables/parameters that the end-user can control through the GUI.

Allow the end-user to execute backend algorithms using these different parameter values, leading to different results.

Provide the possibility to compare these different runs and track KPI performance over time.

In Taipy, we have introduced the ‘scenario’ concept that addresses all of the above requirements.

A scenario consists of the execution of the algorithm/pipeline where Taipy stores all the data elements (data sources, data outputs)

Taipy’s scenario registry enables the end-user to:

keep track of all of its runs, 

revisit a past scenario, understand its results, scan its input data, etc.

Addressing Point 2: easy tracking of the business-user satisfaction

Another great benefit of Taipy’s Scenario function is that it reduces the gap between the end-user and the data scientists. The Taipy scenario registry is a gold mine for data scientists since they can access all end-user’s runs. In addition, the end-user can tag any of these scenarios and share them with the data scientists for examination.

This scenario feature can dramatically increase the software’s acceptance by the end-user. Unfortunately, in practice, testing AI algorithms is generally limited to a few test cases and the usage of drift detection. More is needed to guarantee a high acceptance of the software. And Taipy’s scenarios will help a lot here.

Here are some examples of Taipy AI applications enabling the business user to explore previously generated scenarios.

Conclusion

To conclude with, Taipy has proven instrumental in the success of AI projects for leading corporations, offering an efficient and user-friendly Python framework. With the launch of Taipy Designer, we continue to democratize AI development, focusing on accessibility for Data Analysts and ensuring the seamless integration of AI into business processes.

This article was originally published on Taipy.

Thanks to Taipy team for the thought leadership/ Educational article. Taipy team has supported us in this content/article.