Prologue

The Intelligence of Humanity

LLMs are ML Models

Business Intelligence

You Got AI in My BI! You Got BI in My AI!

Putting the ‘I’ Back into BI

Chapter 1: Introduction

The Puppeteers of an Enterprise

Fragmented Knowledge

Can a Single Entity Know Most Things About Most Things?

Beyond Traditional BI Analysts and Managers

What Do Human BI Analysts See?

The Big Gap Between Human and Machine Intelligence

AI Gone Viral

My Research Assistant, ChatGPT

My Feeling About AI

David and Goliath

We Think Because We Are

The Book’s TL;DR

Could a Private LLM have Written this Book?

ChatGPT’s Visual Representation for this Book

The Book Structure

Topics Punted to Future Books in the Series

Query Languages other than SQL

Non-Time Series Conditional Probabilities

Bayesian Probabilities

Functions

Linear and Non-linear Regression

The Many Ways to Talk to an LLM

Digital Twins

Frightening Things: Code and Math

Striving for Vendor Neutrality

Chapter 2: BI in the Era of ML, DS, and AI

Even an R&D-Focused Enterprise is still an Enterprise

My BI Misunderstanding

Encoding Knowledge

It’s Like Books but Looks Like Tinker Toys

Expert Systems and the Semantic Web

[Diverse Human Expertise] + ((KG + LLM) + BI) = AEI

Intuition for this Book

Version 1—SQL Server Performance Tuning Web (ca. 2004)

Version 2—Map Rock (ca. 2011)

Version 3—Augmented Enterprise Intelligence (Present)

KG Development with LLM Assistance

The Puppet Looks More Lifelike

Chapter 3: The Intelligence of a Business

A Business is as an Organism

The Brain: A Centralized Repository of Knowledge

EKG versus Conventional Databases

Humans as Components of the Business Organism

Survival of the Smartest?

Business Intelligence, Performance Management,  Process Management

Too-puhls (Tuples) and Dataframes

Embracing Complexity

Novel Solutions to Novel Problems

Tell Me Something of Value that I Don’t Already Know

We Don’t Know What We Don’t Know

Tribal Knowledge: Most Understanding is Trapped in the Heads of Individuals

But Things Somehow Work

Sampling of the Incalculably Large Knowledge Space of an Enterprise

A Rockhound Analogy

Widening Breadth of Data

LLMs Opened Floodgates for Inclusion of Unstructured Data

The Unstructured Data Majority Unlocked with AI

AI Quality and Acceptance Hit a Critical Mass

Very Book Smart, Although a bit Lacking in Common Sense … For Now

The Ubiquitous Time Dimension

Conclusion to The Intelligence of a Business

Chapter 4: Knowledge Graphs

Symbiotic Relationship between KGs and LLMs

The Elusive Knowledge Graph

The KG and LLM Symbiotic Relationship

Mitigating Friction Between Human and Machine Intelligence

Knowledge Graph Foundations

The Effort to Create KGs

Nature of Representation

Probabilistic versus Assertive Knowledge

Interactivity and Dynamic Responses

Complementing Each Other

LLMs Queries can be slow compared to more traditional database queries

Trustworthiness and Verification

Regulation

Cost

The Sweet Spot of LLM Ability for KGs

LLMs are Very Far from Six Sigma!

Ontologies and Taxonomies

Relational Models are Ontologies

Ontology Example

Knowledge Graph Artifact Sources

Feasibility of Developing and Maintaining KGs

Semantic Web and KGs

Semantic Web, RDF, SPARQL, and OWL

Semantic Layer versus Knowledge Graph

Enterprise Knowledge Graph (EKG) versus Knowledge Graph

Node Identifiers

KG Users Ontology

Conclusion to Knowledge Graphs

Chapter 5: Data Catalog

The Ontology of the Databases

Data Catalog Foundations

Semantic Layer and Data Fabric

RDF-encoded Data Catalog

Retrieving Metadata

URI for Tables and Columns

OLAP Cubes in the Enterprise-Scoped Data Catalog

Source to Target Mapping

Database Views as Abstractions

“Lowly-Curated” Data

Data from Partners, Vendors, Customers

The Value of Questionable Data

Ad-hoc Data

Non-BI Data Left on the Cutting Room Floor

Chapter 6: Business Intelligence

Benefits of Traditional BI

Highly Curated

Highly Performant

But My BI Seems Fast Enough

Federated Databases

Kyvos Insights Cloud-based OLAP

User-Friendly

BI Users

Traditional BI Users

Knowledge Workers – Massive Expansion of BI Consumers

AI Agent

Append-Only Data Strategy in Business Intelligence

Visualization Tools

Chapter 7: Data Mesh

Expanding the Reach for BI

The Centralized BI Team Bottleneck

Decentralization of BI Effort

Data Governance Team

Data Products

Data Product Service Level Agreements (SLA)

Data Products Per Domain

But Wait, Aren’t LLMs Monolithic Models?

Data Mesh, Data Fabric, Data Catalog, and Semantic Layer

Optimized Cubes as the Consumer-Facing Semantic Layer

The KG Use for Data Mesh

Knowledge Bog

KG “Data Products”

OLAP Cubes as Data Products with Coherent Bounded Contexts

Distributed Construction of the KG in the Data Mesh Paradigm

Data Products for Mappings

Master Data Management

RDF Mapping

Automated Similarity Mapping with LLMs

Database-Derived Ontologies

KG Update Tracking

KG Update Exceptions

Art Gallery Ontology Creation Example Using Protégé and GitHub

Chapter 8: Architecture of the AEI

Architecture Overview

Development Tools

GitHub

Visual Code

Ontology Composition Tools

Databases

Data Warehouses, Data Marts

Graph Databases

Relationships Only—But Know Where to Find the Information

Graph Database Options

Neo4j

Optimized cubes

IT Manager’s Nightmare

Optimized Cubes Make a Comeback

SQL Server Analysis Services and Kyvos

Pre-Aggregation for Consistently High Query Performance

The Unique Cache of Pre-Aggregations

Navigating the Vast Cubespace

Big Data Systems and the Expanding User Base

Kyvos: Beyond Aggregation to Aggregation Management

The Balanced Approach of the Dimensional Model

The Critical Nature of BI Data

End-User Perspectives: The Persistence of Cubes

Enhancing Analytical Speed with Optimized OLAP

High Query Concurrency

The Two Extremes of Cubes and Graphs

Large Language Model

Private Vector Database

Relational Database

Cloud File Storage

Chapter 9: BI-Charged EKG Components

Why Go Through the Trouble of Storing BI Insights?

Guided Exploration with the ISG and TCW

Navigating with Intuition versus Guided Systems

The Value of Collective Exploration

Finding Indirect Pathways versus Direct Correlation

Big Brother?

Events

Sources with a Plethora of Events

Derived Events

The Event Ensemble

Generalized Schema for Events

Event Ensemble in a Mosaic of Cubes

Event Ensemble Integration

Avoid Copying Massive Numbers of Facts

Event Storming as a KG Kickoff

Event Storming as a Periodic Sanity Check

A Simple Event Storming Example

Ask ChatGPT to Organize the Event Storming Session

Rare Events and Risk Management

The Insight Space Graph

The QueryDef Object

A KG of Dataframes

Dataframe Individuals

Spaces

Physical Space

Data Space

Cube Space (Semantic Layer)

Insight Space—the Final Frontier

Business Intelligence Insights

Line Graphs

Bar Charts

Pie Charts

Stacked Bar Chart

Scatter Plots

ML Model Visualizations

Models

ML Models Built from a Dataframe

Business Rules

ML Models are Business Rules

BI Analyst AI Use Cases

Contextual Information Delivery

Guided Exploration

Enhanced Data Analysis

The Tuple Correlation Web

Time

Time Events

The “Social Network” of Tuples

Putting Old Myths and Legends to the Test

Correlation Algorithms

Bayesian Probabilities

Conditional Probabilities

Pearson Correlation

Spearman Correlation

Pearson Correlation Example

Transforming to Stationary Values

Plotting the Rolling Average Correlation

Correlation Parameters

Determining Lag for Cause and Effect

Time Series Intervals

Casting a Wide Net

Intelligent Query Patterns

Actionable Insights

Detective Query Pattern

The Coach Query Pattern

The MacGyver Query Pattern

Stressing the Correlations and Probabilities

Confounding Correlations

Obvious Correlations

Ask the Experts (the KG)

Why not just ask ChatGPT?

Coefficient of Variation and Slope

EKG Analysis

Object Schema

Objects Dimension Table

ObjectLog Fact Table

Object Archive

Users Schema

User Validation Table

Chapter 10: Special Structures

KPI Strategy Map

Compact Smarts

The Theory of our Corporate Strategies

Gaming KPIs

KPI Correlation Score

Bayesian Belief Networks and Causal Diagrams

Business Process Knowledge

Simple AI-Assisted Strategy Map Example

Refining the Workflow

Limitations of Encoding Knowledge

Trophic Cascades in the TCW

Time Series ISG Models

Time Series Inflection Points

Frequency Domain Analysis

Minimum Data Points

Spectral Components

Component Similarity

Chapter 11: Implementation

Environment

Set Up the KG and DC Environment

Create a Sample Ontology

Load the Data Catalog

Retrieve Data Catalog from SQL Server

Upload Data Catalog into the EKG

Processing BI Queries into the ISG

Query Parsing and Saving

Reusable Components

Filter Components

Set Components

Tuples Components

Query Functions

Uploading to the ISG

Tuple Correlation Example

Web of Pearson Correlations and Conditional Probabilities

Conditional Probabilities

Tuple-Level as Opposed to Conditional Probability Table for Conditional Probabilities

Conditional Probabilities versus Pearson Correlations

Conditional Probability Example

Casting a Wide Net―Pearson Example

Casting a Wide Net with Conditional Probabilities

Retrieval Augmented Generation

RAG Intuition

Self-Reflective RAG

Vector Database

Leveraging Neo4j GDS for Embedding Management and Similarity Analysis

Offloading Vector Tasks to a Vector Database

Prompt Enrichment

Query the TCW like a Social Network

Query the ISG like any Other Ontology

Graph Query Intent Templates

LLM API

Maintenance

Updating the KG

Updating of the TCW and ISG

Batch Updates for KG Data Products

Pruning the ISG and TCW

Re-query with Fast Optimized OLAP

QueryDef Count Settles Down on its Own

BI Consumers with Widely Diverse Focuses

Compelling Statistics Model

Time to Live

AI Pruning

Minimal Relationship Options

Turning off Features

QueryDef Identifiers

Conditional Probability Table

Offloading to a Relational Database

Off-Loading to a LLM

Security

Securing BI OLAP Cubes

Securing the Graph Database

Securing the Data Catalog

Securing the ISG/TCW

Securing the Vector Database

Securing Ourselves from the LLM

Chapter 12: Future Steps

Fine-Tuning LLMs with BI Data

Inference

Inference on the Semantic Web

Subgraphs and Paths

Metrics at Scale

Where Does Data Science and Machine Learning Fit In?

Investment and Sacrifice

No Pain, No Gain

Levels of Pain

Running the Process

Conclusion to Investment and Sacrifice

Conclusion

Appendix: Ontologies and Taxonomies  in OOP Terms

Glossary

References