Data and Databases ,Read and make your choice SQL or NoSQL
Hi everyone My Name is ALOUANI YOUSSEF and this is my first article. I am a junior software engineer ,when I started working on my dream project, many issues showed in my face ,one of the big problems was the database choice, it was a big problem for me ,as an independent developer working on large web application “cloud native application” ,to make the best choice ,because it will store a business critical Data , and I need to make my app Secure , Scalable , easy to maintain and give me the opportunity to use the data to make a complex decisions using data analysis ,in the third step in information system . In this phase I understand a big thing that software engineering is not about coding it is about making sensitives decisions and for that we need a deep understanding of the process of development of the tools and technologies used in the world of IT .and because of the importance of metaphors in building software so lets walk on the history and watch the development of databases .
Data and Humanity
From the Beginning of Consciousness , the Human being tried to describe and transform what he saw or what he heard to something meaningful for him as the fees on the rocks or the caverns , with development of humanity we start writing language around 3200 BCE .
century after century , and with the development of humanity , also with agriculture revolution the needs to store data about crops and the needs of accounting they caused the born of tools for storing data as the paper records, those tools expanded with the industrial revolution .
From paper to Data management software
The concept of a database “ database refers to a set of related data and the way it is organized , and the access to this data is usually provided by a DBMS” was made possible by the emergence of direct access storage media such as magnetic disks, which became widely available in the mid 1960 , in this year the first DMBS was born when charles W. Bachman designed the integrated database System the first DBMS ‘DATABASE Management System’ . after that IBM launched The IBM IMS their own database systems , the IMS and the first DBMS are described as the forerunners of navigational databases .
DataBases Models and categories
Computer scientists classify DBMS according to the database model that they support . for example navigational data model use the hierarchical model “ NETWORK model ”. Also we can categorized databases according to their model , the computer it run on , its internal engineering
The Beginning of Relational model and SQL
this model relational model first proposed in 1970 by Edgar F. Codd , this model born to make that application able to search based on the content rather than following links ‘ using pointers in Navigation model’ the model is based on storing data as rows and columns in series of tables by associating a special key to each row id as PK primary key and we can establish relationships between tables that why we call it relationel there is a relation between tables by add what we call a Foreign key ‘put the ids of table as properties ‘columns’ of other table’ and to retrieve the full data we need to make a join operation , when computers become more powerful and are able to run this model and its spread like wildfire. the SQL was initialy in the early 70s Donald Chamberlin to mutate and retrieve data from IBM R system and by 1986 it was standardized by ISO/IEC 9075 into the syntax that remains extremely popular . the relational databases are so secure and that is the point.
When to use relational databases SQL databases and when not
the relational databases are designed for all purposes also are useful and really good when we have structured data in this case we can normalize our data also is a good choice if we need to deal with some quicker more complex queries ,they have a strong consistency , concurrency , recovery the data is following a specific format , it is also great for dealing with not a lot of data , when the data became large the join operation became more expensive ,they are so hard to scale horizontally with relational database ‘use multiple computers to store the data’ impedance mismatch occurs , bad performance and weak in high availability , some of SQL DBMS they have some NOSQL components inside them like PostgresSQL where we can have a column with JSON type that is mean that we can store a document as field in SQL DBMS , also with DataWarhouse we can make this data useful to help us for make decision based on our stored data.
The birth of NoSQL
in the databases context it is better to use high level programming language for data manipulation instead of using MapReduce (MapReduce is a programming model and an associated implementation for processing and generating big data sets with a parallel, distributed algorithm on a cluster) pattern , but a really high level language as SQL make data manipulation not easy at all .
The World heard the NoSQL “Not Only SQL” term for the first time at 1998 by Carlo Strozzi while naming his lightweight, open-source “relational” database that did not use SQL , but the relation between NoSQL and Non-relational databases start in 2009 Eric Evans and Johan Oskarsson used it to describe non-relational databases . the most popular event about NoSQL is in 2007 when a company called DoubleClick serving 4000per second have issues with scalability and flexibility using existing DBMS its faced the 3V (Volume, Velocity, Variety)monster ,that inspired for create an other type of database called MongoDB which based on different Database model its the document model where data is stored in binary json which are organized into collection each one have an id which unique in the collection . for better storage and manipulation of data we have 4 models of databses in the NoSQL
The Gang of four Models in NoSQL
1Key-Value :efficiency and simplicity a key/value system like JavaScript object or python dictionary or hash table ,a set of keys for every keys is unique and point to some value. They are great in using the cache at of some other persisting data layer,like Redis
2 wide column : reverse the traditional model of rows and columns here we store data in columns the most popular case to use it is by netflix for scaling tine series data storge , like Casandera
3Document : in this model we have document each document is a container for key-value pair they are unstructured and the document are group together in collections and collections can be organized into a logical hierarchy they don’t require schema and they don’t support joins it is push us to demoralized data format the best choice for mobile app. like MongoDB
4 Graph : the stored data are nodes and the relation between nodes as edges , often used when we have a lot of disconnect but related data ,like FLOCKDB
When to use NoSQL when Not :
before start discussion a would say don’t came with a knife in gun fire and also don’t ride a donkey for cars race , the NoSQL DBMS are a good choice for fast development they built to support large data volumes , they allows to store complex forms of data, they are fast in writing and they can scale and handle a lot of connections at the same times which impossible with SQL DBMS the NoSQL are build for the distributed systems they allows us to scale horizontally , we can also use NoSQL databases for Big Data and real-time applications , they allows us to build data Lake . but it is required to choice a secure solution if you have a sensitive data , or use SQL DBMS if your data should flow a required strict ACID compliance.
The Solution In My Case :
for me in this situation with a sensitive data and requirement of cloud native application also with a distributed architecture , I chose to work with the NOSQL , with an SQL DBMS in a different server for duplicate my sensitive data , Using the NoSQL and exactly MongoDB make my development process much ease and allows me to use the data science to provide the best user experience to my users .
thanks for reading this .
