The core components of traditional databases, such as tables and columns, obfuscate the natural relationships between data points. Users of these databases struggle with creating a meaningful entity relationship diagram – even though that’s how the data originated.
The Need for a NoSQL Database
When I was a developer in the 90’s working on IBM Lotus Notes, we faced a similar problem. Lotus Notes was a product that allowed thousands of collaborators to send email and share ideas and information, and we needed to store all of that data in a meaningful way. This problem led to the creation of a new type of database, which we called NSF.
NSF wasn’t your typical RDBMS database. It stored data in documents instead of tables and let us put an arbitrary number of fields in each document. It was even possible to create views on top of those documents to pick which fields to show and where. That NSF technology later inspired a new type of NoSQL database, starting with CouchDB.
The Right Technology for the Right Time
A lot of the technologies that we now use on a day-to-day basis have ancestors rooted in IBM’s history.
For example, back in the 60’s, IBM founded the notion of utility computing, or time-sharing, where companies could rent computing power and apps that ran on IBM data centers. A few years later, IBM embarked on an unusual project: the VM operating system, which allowed the creation of customized virtual machines for individual users on top of IBM’s mainframes.
Some of those technologies were too far ahead of their time and didn’t catch on until decades later. I believe there are two reasons for this:
The first is that the infrastructure at the time didn’t provide a decent user experience, and the on-prem alternative was much more effective. The second reason is that there was no impetus to change; users were able to make do using tools they had in-house, without incurring new expenses, learning new skills, or altering the way they worked.
Graph Databases Help Explore Complex Data
In recent years we have seen a major shift, particularly as smaller and smaller companies are collecting larger and larger quantities of data that needs to be analyzed and understood in as near to real time as possible.
Enter graph databases: a new, old technology that can help users both maintain the natural way data exists and easily uncover the insights hidden within it. The current technological landscape and today’s changing business needs are paving the road for graph databases to become the natural pick for anyone who wants to get value from their complex data.
Graph databases store data as entities (vertexes) and relationships (edges). They can be traversed using graph queries, which simplify the process of getting insights from data and eliminate the need to write complex joins. They are able to scale to accommodate the largest of graphs and serve complex queries in real-time. Currently, graph databases are in production at many Fortune 100 companies in various sectors and are continuing to see a steady increase in popularity as they become better understood.