Grady Booch, the much respected software architect asserts that “at some level of abstraction all complex systems are message passing systems". He means all systems, from cell biology to plate tectonics, not just software systems.
Software in many ways are inherently state management systems where functions act upon data. State in turn has to be passed between functions and larger modules. We built the concept of Object Oriented Design and Programming over this simple idea.
If you carefully observe, all complex systems both organic (eg: protein synthesis) and inorganic (eg: payment systems), are essentially abstractions of message passing systems. …
Database space estimation is a relic of the past. Space estimation is usually done after Physical Data Model has been designed. Architects had to fill these estimations on Excel sheets and convince infrastructure managers (and sometimes finance) that they need the requested GBs to run their applications. Then they wait.
Today we start from a guesstimate and let the serverless database auto-increment storage based on demand. The most we do is point and click on our cloud provider’s console on the browser.
But it’s good to learn it due to the off chance that an old school architect asks this…
DDD is a widely discussed and implemented pattern for microservices. Microservices interact with domain models to save state of business. Domain models are data models that are domain specific (specific to each microservice). For example,
All three are instances of customer business object, but applied in different context. This essentially duplicates data across microservices, and that’s ok per DDD. What…
Microservices is a way of building applications as smaller chunks of services each doing independent self-contained work (as opposed to a single large monolith that does many things).
Microservices are usually implemented following REST principles. It means that a software module to be called microservice, it should follow some patterns — patterns like service not having to keep track of the state of application (state is just a nerdy term for data) or multiple services not having to exchange state changes to each other creating inter-dependencies.
Database character set is the combination of symbols (alphabets, numerals, punctuation etc.) and their encoded numeric values (as defined by an encoding scheme like ASCII or Unicode).
Default character set for user data in MySQL is ‘latin1’ which includes English and some European languages.
Collation implies the rules to compare and manipulate symbols in character set of data (alphabets, numerals etc.). Collation doesn’t apply to symbols in identifiers (table name, column name etc.) or keywords (select, from, where etc.)
The two predominant modeling techniques for applications are
(1) fully normalized models (based on normalization principles evolved in the 70’s) usually following 2NF for batch-based databases and 3NF for DML-intensive hot OLTP databases
(2) read-optimized data warehouse models known as Star Schema or Dimensional model (based on Kimball’s methodology)
All other techniques (for instance, top-down data warehouse design known as Inmon methodology, snowflake models etc.) are a combination or slight variation of the above two.
Data Vault modeling solves a problem that these two techniques doesn’t solve. And that problem is rigidity. It is better explained with an example.
This write-up is about Oracle on RDS but some of these apply to any RDS databases.
Before highly autonomous cloud databases like Snowflake and Serverless Aurora, RDS was the easiest no-DBA cloud database. It’s a PaaS solution, which means it doesn’t give you any control of the host machine or operating system. At the same time it lets you control all (or most) of the configurable properties of the product hosted on it (the database) in order to “administer” it. That configurability is what makes it a non-SaaS solution. …
OLAP cube or multi-dimensional cube is equivalent to implementing aggregated tables on DB. Data is pre-calculated and stored in the cube. Users running analysis don’t have to join and group by.
Cubes are not necessarily a feature of the database engine, it is commonly built by the BI tool. Cubes can be built very easily without having to do any data modeling. If you know what measures and dimensions are the users interested in, you can drag-and-drop facts and dimensions to build a cube using a GUI tool like Microsoft SSAS.
Basic objects are: cubes, measures and dimensions. Measures are…
I think of modeling as an art, something that requires intuition, albeit scientific methods behind it. There is more than one way to model a business. It’s easy to criticize a data model after the fact. Part of our technical maturity is in knowing that the specific nature of universe at the time of modeling— the budget, the pressure, the people you interact with, the timelines, the blindspots and so on — probably favored that particular design.
OLTP vs OLAP
If you have only modeled data for read intensive workloads, as in dimensionally modeled data warehouses, you haven’t had any fun…
Abstraction is a way of building software that generalizes a business process or a functionality without having to write separate logic for specific use cases and without giving away details of implementation. By doing this a module of software can do many things with one, can be reused by many, and can hide complexity.
A data modeler for instance is abstracting away business rules inside a schema. Similarly, the implementation of an SQL engine is an abstraction of datasets that are warped in normalization principles . Regardless of how a database is designed a user can query it with SQL…