OpenAI’s ChatGPT and other generative AI technologies have certainly opened a Pandora’s box for businesses to streamline daily operations leveraging the huge array of new AI-enabled tools. Indeed, Microsoft has very quickly integrated Copilot into a range of their business tools including the Microsoft 365 suite. Google has begun rolling out Gemini across their Google Workspace products providing immediate access to AI prompt interfaces in almost every application. 

For most people, the integration of LLM’s into their daily workflow is a means to rapidly research concepts, punch out email responses, gain summaries of Teams meetings, translate documents or better organise their calendars. Power users will get more creative and build out business strategies, write code or even develop financial models. Conversely, IT staff are being asked how to prompt their generative AI tools to get the best results. In short, we’re all learning a lot about prompt engineering and the many creative ways to use generative AI. 

Besides the governance and security concerns that AI introduces, CIO’s are becoming increasingly interested in finding significant productivity gains from AI – especially in Business Intelligence and data reporting. BI and data management can often pose significant technical and financial challenges to IT departments with many businesses driving their IT chiefs to democratise data sets such as marketing, sales, financials, inventory, logistics, production, and operations. 

Indeed, as executives become more data-savvy, they are demanding a broader array of reports, insights and predictions – all delivered via secure self-service BI tools. 

To meet this demand, forward-thinking CIOs may consider building a ‘data-driven organisation’ strategy. Often this strategy will also encapsulate the concept of data democratisation: the process of making data accessible and usable by a wider range of employees, beyond just data analysts and IT professionals with the goal to empower business users with the ability to access, understand, and use data to make better decisions and improve their work.

The value of this often complex initiative aims to deliver improved decision-making, increased efficiency, enhanced customer experiences, competitive advantage, innovation, risk mitigation, compliance and sustainability. Such initiatives can take months or years due to the diversity, complexity, quality and volume of data. Add the tens of spreadsheets used by staff across the business and one quickly forms a nightmarish picture. 

With these challenges in mind, CIOs are asking ‘how can AI impact Business Intelligence practices?’ and ‘what data security measures need to be taken to leverage AI while not exposing the data sources to a public LLM?’ 

AI-powered analytics offers an excellent opportunity to deliver quick wins to business units seeking intelligence from their diverse structured and unstructured datasets. 

The value of introducing AI to your data stores

While LLMs are now presenting us with the ability to interrogate our file and website content repositories using natural language prompts, vendors such as Microsoft are now turning their attention to exposing these tools to data sets within spreadsheets and even relational databases. By giving users the ability to ask questions of their data, they can immediately view the results via the generative AI interface, visualise the results as charts, and even drill down into the results within the query thread. 

Think…

What were the total sales for Q1 2024?
Of those sales, how many included ‘product x’?
Which salesperson sold the most ‘product x’?
Chart ‘salesperson y’s’ sales performance in FY2024

Where Business Intelligence Analysts would normally be required to tailor datasets for business units, AI-driven BI tools take us one step closer to offering democratised data to all permitted users throughout the business. Once users develop prompting skills, they also tend to increase their data literacy enabling them to gain deeper insights and solve problems faster. From an IT cost centre perspective the reliance on analysts diminishes, theoretically reducing labour costs. 

AI analytics and business intelligence tools available today

While there are a range of cost effective tools available on the market, it is hard to go past Microsoft’s Copilot as an excellent place to start your AI-BI journey. Copilot sits within the Microsoft 365 product suite, including Edge, allowing users to interact with the GPT functions within each application. With the addition of Copilot Studio, IT teams may quickly build custom Copilots (GPT agents) so that users can type conversational prompts to gain AI-driven insights during their BAU tasks. These custom Copilots can be made available where users need them most. 

This is where things get really interesting. A custom Copilot can be added to a Dynamics 365 instance (many other prebuilt data connectors are available) allowing authorised users to interact with the Copilot within the CRM, Marketing, Finance or Operations applications or even added as plugins within Power Apps mobile applications or Power Pages web portals. Better still, the underlying user roles and permissions are enforced for any queries performed by users making security relatively straightforward. 

Imagine providing field service and operational staff with an AI chat tool allowing them to ask conversational questions such as: 

HEALTH CARE: What are my client’s care needs today?

CONSTRUCTION: What was the last incident for ‘site x’?

IT: How many urgent support tickets are due today?

Copilot embedded into Dynamics 365 offers an endless array of opportunities for users to gain immediate access to data insights that pertain to their job role. A number of predefined prompts will also help users learn how to wield this new analytics power. Notably, Microsoft have also recently released Copilot for Dynamics 365 Sales which provides a range of simple AI tools to boost sales team productivity. 

For businesses seeking alternatives to Microsoft’s offering may select from a variety of low-cost options including datalang.io, SQLAI, SQL AI chat, and Insightbase. 

Microstrategy offers an Enterprise-grade SaaS platform with many bells and whistles including consultancy services to help organisations build the underlying data models, train the connected LLM and deploy to user groups. 

AI for BI: how does it work?

Many of the low cost tools mentioned above work on the basis of offering middleware, including a user agent, dashboard interface and a web service. The web service securely connects to your database(s) on one end and an LLM such as OpenAI GPT engine on the other end. Once your database is connected, users may interact with the chat agent by using conversational prompts on data. In a relatively unsophisticated manner, these prompts are converted into SQL queries that run on your target database. The results are presented to the user via the agent or dashboard. Importantly, OpenAI never actually stores or understands the data – it is simply producing the results and assisting in the interpretation of those results. Your database data is not exposed or used to train public AI services GPT4.0. 

In the case of Copilot, the technology is much more sophisticated. Delivered by Microsoft’s Azure OpenAI service, Copilot will draw from a range of user interactions, Microsoft 365 application data sources (Outlooks, Word, etc), Dynamics 365 dataverse, and potentially external databases (via authenticated data connectors). The Copilot “agent” is available through many Microsoft applications allowing users to constantly converse with the agent. 

Microsoft repeatedly underscores that data privacy is paramount to offering a secure and trusted service. Rest assured, Public ChatGPT models are not trained directly on your data. All data remains within your tenant and access is restricted by the roles and permissions set within the Azure tenancy. Instead, Copilot leverages concepts such as grounding, Retrieval Augmented Generation (RAG), and fine-tuning to help bring real-world context to the answers derived from the underlying data models. 

Data security and governance concerns

We recommend that technology leaders be aware of data security and privacy concerns associated with AI technologies. There are some excellent online resources to help CIOs formulate AI usage policies and reshape existing data security and privacy policies. Ironically, generative AI can deliver the bones of a set of guidelines, but we recommend a more robust approach is taken. The Australian Institute of Company Directors offers guidance on matters of IT governance including AI. 

Again, Microsoft also offers comprehensive documentation on how tools such as Copilot inherit the overarching security policies of your Azure tenant. 

“The permissions model within your Microsoft 365 tenant can help ensure that data won’t unintentionally leak between users, groups, and tenants. Microsoft Copilot for Microsoft 365 presents only data that each individual can access using the same underlying controls for data access used in other Microsoft 365 services. Prompts, responses, and data accessed through Microsoft Graph aren’t used to train foundation LLMs, including those used by Microsoft Copilot for Microsoft 365.” (source: Microsoft.com).

Importantly, statements like this should not be taken out-of-hand. IT teams building chat agents should review data security policies when connecting to internal or external databases – especially if existing roles and permissions are not enforced by API connectors. 

Building your AI BI business case

With GPTs (generative pre-trained transformer) being an assistant tool, creating a solid business case may be relatively straightforward. The direct word from Microsoft (per a partner conference in Melbourne, March 2024) is that many of their customers are taking the approach of testing the waters by deploying AI solutions such as Copilot to small user groups. Out-of-the-box functionality is readily available within the Microsoft 365 suite making it easy for IT departments to roll out Copilot to small control groups for testing. Similarly, custom Copilots can be built and deployed to these groups for building the underlying data models, LLM model training, testing the accuracy of AI-BI conversational queries, and verifying enforcement of data security protocols – all prior to exposing to larger user groups. As of April 2024, Microsoft offers the most accessible and easy to implement AI solutions in the market, hands-down. 

Like many tech initiatives, we recommend commencing your AI-BI journey by identifying a clear hypothesis of where value is expected to be derived for each user and a business unit as a whole. Once success markers have been identified, IT teams may move to building the first prototype project, simultaneously aiming to develop internal capability and a workflow that suits your organisation in the process. Start with simple, non-sensitive data sets and begin generating queries that align with common business reporting requests for data queries. If the underlying data models and GPT are performing well – ie returning accurate results, abiding by user permissions – then you may wish to broaden your scope. Importantly, AI will not resolve your underlying data inconsistencies. Some work may need to be done to cleanse data prior to exposing it to GPTs. AI-BI platform vendor, Microstrategy, stresses the importance of data preparation in their “semantic graph” which effectively offers a unified library of data sources and the ability to create relationships between data sets. 

Once your value hypothesis has been proven, and you have pushed the solution into the production environment, be sure to re-run your data validation process alongside good old-fashioned SQL queries to demonstrate the accuracy of results building confidence in users and stakeholders.