The Elephant in the Fridge: Guided Steps to Data Vault Success through Building Business-Centered Models, by John Giles
You want the rigor of good data architecture at the speed of agile? Then this is the missing link – your step-by-step guide to Data Vault success.
An “owner’s manual” is a good start, but …
Did I hear a warning?
A vital context
Seeding the Data Vault project
Task #1: Define how the business sees their data
Task #2: Design the Data Vault, based on the business view
Task #3: Bottom-up Source-to-Data Vault mapping
Task #4: Define business rules
A bit of data warehouse history
Data Vault made (too) easy
One position on Data Vault standards
A case study
Satellites (on a Hub)
A drill-down into cardinality on Links
Satellites (on a Link)
Untangling “raw” and “business” Data Vault
Setting our compass bearing
Is the enterprise view really that important?
Tool vendors, and “source system Data Vault”
Consultants, and “source system Data Vault”
A case study in conflicting designs
Before we go any further, what is an ‘enterprise data model’?
Sample snippets from enterprise data models
One model, many views?
Where can we find an enterprise data model?
Commercially available industry models
Existing, home-grown models
Is “agile evolution” an option?
Patterns to the rescue?
How to build a “sufficient” enterprise data model yourself
Step #0: Challenge anyone who says “Follow Step #1, Step #2 …”!
Step #1: Pick the team
Step #2: Define the “enterprise”
Step #3: Get a feel for the enterprise
Step #4: Facilitator learns the pattern basics
Introducing data model patterns
Could I see an example, please?
A palette of patterns
No pattern is an island
Sharing the understanding
Step #5: Assemble a lightweight Enterprise Framework (in days)
Meet and greet
Introduction to patterns
Starting the model
A consolidated enterprise model
Step #6: “Pain Point” & “Process” drill-down (in weeks)
Using “Pain Points” to drive the drill-down
Can “Process” help, too?
Step #7: Cast the net wider
The important bit: Getting business data out
Well-formed Hubs: Getting the right foundations
Confusion over “Semantics & Grain”?
Finding the sweet spot
Not only Semantics & Grain – now there’s Key Values
Passive Integration: Same “semantics-&-grain”, same key value for the same real-world object
Different, non-overlapping business key values
Different, overlapping key values
Hubs with more than one business key
Business-sourced Hub and Satellite instances
Using business rules to populate instances in Satellites
Using business rules to populate instances in Hubs
Well-formed Links versus “ugly” Links
When the business and systems views happen to align
When life is a bit more complex
Business-sourced Data Vault Link and Satellite instances
What if we get it wrong?
Understanding Data Vault by understanding data loading
The case study expanded
Populating a Hub’s Satellites
Populating a Link’s Satellites
Some other data load considerations
Extracts versus Change Data Capture (CDC)
Restarting the load
Doing the source-to-Data Vault mapping
Business rules, and the “Transformation” bit
Applying business rules to “close the gap”
A quick glance at some more advanced topics
Point-In-Time (PIT) and Bridge tables
Using a Data Vault to Shine Some Light on the “Process” World
Gap analysis using a Data Vault
To understand “Process”, we need more than a Data Vault
A bit on projects
Platform & tooling projects versus the Data Vault project
A few hints
A few controversies
Modeling a Link or a Hub?
Modeling a Satellite on a Link?
Normalization for Links, and Links-on-Links
Closing comments on controversies
More uses for top-down models
Pattern: Party & Party Role
Pattern: Product (tangible goods, intangible services)
Success with a Data Vault starts with the business and ends with the business. Sure, there’s some technical stuff in the middle, and it is absolutely essential – but it’s not sufficient on its own. This book will help you shape the business perspective, and weave it into the more technical aspects of Data Vault modeling.
You can read the foundational books and go on courses, but one massive risk still remains. Dan Linstedt, the founder of the Data Vault, very clearly directs those building a Data Vault to base its design on an “enterprise ontology”. And Hans Hultgren similarly stresses the importance of the business concepts model. So it’s important. We get that. But:
What on earth is an enterprise ontology/business concept model, ‘cause I won’t know if I’ve got one if I don’t know what I’m looking for?
If I can’t find one, how do I get my hands on such a thing?
Even if I have one of these wonderful things, how do I apply it to get the sort of Data Vault that’s recommended?
It’s actually not as hard as some would fear to answer all of these questions, and it’s certainly worth the effort. This book just might save you a world of pain. It’s a supplement to other material on Data Vault modeling, but it’s the vital missing link to finding simplicity for Data Vault success.
“Data Warehousing in the context of large healthcare organizations is notoriously difficult – largely in part due to challenges around Health Information Modeling. The Elephant in the Fridge provides clear and rational perspectives that have allowed my team to breakthrough some of our circular debates and ‘too hard basket’ challenges. Success in this space will require technical and health professionals to co-design solutions around a shared information model. This book is written in a way that they can both consume. John’s experience and emphasis on art of Information Modelling is a welcome complement to other works on Data Vault. I wish we had this book a year ago!”
Benson Choy, Enterprise Information Architect, eHealth Queensland.
“John has a wonderful way of explaining complicated topics in an uncomplicated way. If you’ve heard about taxonomies and are wondering how to apply these to your Data Warehouse, then this is the book for you. Data modelling is about the business, and John explains how this can be achieved by using proven business templates which help to quickly and efficiently define a solid data model. Don’t re-invent the wheel, but start with the model patterns that are already available! This is a highly relevant book, because it helps to match the information delivery to business expectations – by correctly applying taxonomies and (business) model archetypes in a clear and simple way.”
Roelant Vos, General Manager – Enterprise Data Management
“It has been a delight and privilege working with John on a Data Vault project. I have enjoyed his practical and pragmatic approach and readiness to share his wealth of knowledge. This is an excellent book written based on real life experience. It provides valuable and timely insights into Data Vault modeling and delivery, and can be easily understood by those who are new to the Data Vault. The book contains practical advice and sample patterns to help people get started on a Data Vault project and avoid costly mistakes.”
Natalia Bulashenko, Information Solution Architect
“This book covers off one of the key aspects of Data Vault – the Modelling. Far too many DV practitioners come from a technical database background and lack the fundamental Data Modelling skills necessary to identify the “Key Business Concepts” that are core to successful DV projects. Far too many of us start with the low hanging fruit that we have available to us – the source systems. John explains why this is an extremely bad approach with examples, identifying the signs that the project is heading into trouble and then describes what needs to be done to get back on track. This book is NOT about the implementation phase, it’s about setting the corner stone for the whole project. Get this right and you are on the road to a successful project.”
Peter Dudley, Consultant, Interactive Innovations
John Giles is an independent consultant, with a passion for seeing ideas taken to fruition. For two decades his focus has been on enterprise information modeling, enterprise information integration, and enterprise information architecture. Over the last few years he has also gained international recognition in Data Vault modeling. John is primarily a practitioner, having been responsible for leading teams to successful delivery of IT solutions across a wide diversity of industries. However, his pragmatic focus is backed up by a solid appreciation of the underlying theory, having presented internationally, and published widely, including in his book, The Nimble Elephant: Agile delivery of data models using a pattern-based approach.
Please complete all fields.