How AI is redefining data-based roles

Today’s AI capabilities rely on enormous quantities of data — and as a result, data professional are reimagining their roles in the enterprise.

The AI revolution in which we’re embroiled today has progressed at such breakneck speed, 77% of business leaders already worry they’re missing out on the benefits, according to a November 2023 Salesforce survey.

But with AI’s almost limitless applicability, where should an organization focus first? On the most valuable commodity possessed by an enterprise — its data — and the positions most closely associated with maintaining, manipulating and consuming it. After all, today’s celebrated generative AI models produce results only as good as the huge quantities of data on which they are trained. Capable stewards of that data estate are essential.

AI will replace few if any data-related roles. Instead, AI-powered software will enhance their capabilities — and encourage ambitious data professionals to jump on acquiring whatever new AI-related skills may be demanded. Here’s a quick rundown of the impact AI will have on data roles across the organization.

Chief data officers (CDOs)

The CDO role is one of the toughest C-level jobs in IT, with CDOs retaining their positions for just 2-and-a-half years on average, according to the Harvard Business Review. AI is a potential CDO game-changer, as it offers new opportunities to deliver value to the enterprise.

Until recently, the office of the CDO was considered a cost center that ensures data governance, integrity and security. AI elevates the CDO’s standing in key ways. First, it adds an abundance of automation to improve data quality, database performance and data analytics, yielding better outcomes across the board. Second, AI applications, from chatbots to pricing optimizers to predictive analytics, depend on giant repositories of quality data — and many of those apps are already driving new revenue.

But AI also adds an important new obligation for CDOs: They must ensure that AI training data does not yield biased outcomes. The classic example is inadvertent association of risk with minority borrowers, job candidates, business partners and so on. Avoiding AI bias is also the responsibility of AI app developers, so collaborative testing must be ongoing.

Data architects

Data architects bring the vision, policies and initiatives of the CDO to life through effective planning and design. That begins with data modeling: Collecting and analyzing data requirements and developing the logical and physical models to accommodate them. AI-powered data modeling is at an early phase, but as the technology matures, it will help architects produce more sophisticated and accurate models.

Data architects can use AI-enabled tools to identify trends in data usage, with the goal of establishing optimal data location, storage performance and data security to serve applications across an organization. Such analysis can extend to predictive capacity planning so that architects can determine which data to store on which platforms, now and in the future, on premises or in the cloud.

Data engineers and integration specialists

Data engineers generally manage data at the system rather than the organizational level, with an emphasis on infrastructure, whereas data integration specialists address the age-old problem of blending and reconciling data from multiple repositories for any number of business applications. These two overlapping roles are already benefiting from AI.

The key issue in this area is metadata management — that is, organizing all salient information that describes data useful to the enterprise, regardless of origin or platform. AI tools already exist that can help surface and regularize metadata schema for data mapping and integration. Some also automate the creation of data pipelines, which form the fabric of data integration. Newer AI offerings can continuously monitor data quality as it flows through pipelines, flagging inconsistencies in real time.

Database administrators (DBAs)

Managing an enterprise database is a job with many facets, from performance tuning to intensive SQL querying to ensuring availability and security. DBAs typically need to balance the requirements of different sets of users while minimizing disruption as data stores scale and new database software versions arrive. Here again, AI can reduce the time spent on menial tasks, enabling DBAs to spend more time capturing and fulfilling stakeholder needs.

But the big win is in optimization. Using AI-powered tools to analyze performance characteristics enables DBAs to flag bottlenecks and anticipate upcoming infrastructure limitations — or actually add capacity without human intervention. AI tools that plumb the database itself can suggest indexing tweaks and recommend changes to queries that deliver better results faster. 

Data scientists

AI arguably provides the greatest benefit of all to the data scientist, a job that demands advanced skills in programming, machine learning (ML), mathematics and data analysis tools. For example, automated ML (AutoML) greatly eases the task of model development, including choosing the right machine learning algorithm for the job. Plus, as with any programming, data scientists writing Python or R code can benefit from the increased productivity offered by AI coding assistants.

Data scientists enjoy a broad purview, tapping huge quantities of data to identify long-term enterprise trends, risks and opportunities — a process enriched by a new crop of AI-infused analytics software. But the job comes with a dirty little secret: Data scientists spend most of their time sourcing, cleaning and preprocessing data. AI-powered data cataloging accelerates sourcing, while AI tools are emerging to help fulfill the six elements of data quality: Accuracy, completeness, consistency, uniqueness, timeliness and validity. That groundwork adds value to data analytics across the enterprise.

Data analysts

Like data scientists, data analysts are capitalizing on new AI capabilities baked into the latest analytics tools, although data analysts typically focus on domain-specific decision support rather than big-picture insights. For years, AI has powered predictive analytics, but new, iterative ML capabilities are improving pattern (and anomaly) recognition to yield much more accurate predictions. AI can also serve up the best visualization for the task at hand and even automatically generate dashboards.

All this automation has the effect of widening access to data analytics. Natural language interfaces are enabling those lacking query language skills to perform their own analysis, while the guidance offered by AI helps prevent the unwashed from making rookie mistakes. AI is changing analytics forever at an astounding clip, vastly expanding capabilities and equipping a broader swath of business analysts with more powerful self-service tools.

Software developers

Strictly speaking, software developers are not data professionals, but obviously they deal with huge quantities of data in the form of millions of lines of code. At the same time, many developers are integrating ML capabilities into applications that process all sorts of enterprise data. In both cases, AI-based coding assistants are having a double-digit impact on developer productivity.

Coding assistants go way beyond simply completing repetitive lines of code. Using natural language queries of vast open source code repositories, plus their own company’s proprietary code base, developers no longer need to heroically track down obscure syntax details. Coding assistants can serve them up well-formed — and in accordance with the coding rules established by a developer’s organization. In some cases, coding assistants also recommend the right machine learning algorithms for specific application tasks.

AI’s conquest of the enterprise

It’s safe to say that no emerging technology has had a broader impact as quickly as AI. Although data wranglers and developers are seeing the greatest impact, professionals in marketing, product development, service operations, risk analysis and more are riding a hockey stick of AI adoption. Improvements in data quality and analysis are already being felt across the enterprise. Perhaps the most astonishing fact is we’re just getting started.

Jozef de Vries is chief product engineering officer for EnterpriseDB.