![[data-analytics-platform.drawio.svg]] | INGESTION LAYER | Collecting/extracting data from sources<br><br>**Application**: Fivetran/Airbyte/Custom… | | ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **TRANSFORMATION LAYER** | Converting extracted data in a consistent format suitable for analysis<br><br>**Application**: DBT + Snowflake OR DBT + Bigquery OR Databricks + Azure data lake storage Gen2 | | **SEMANTIC LAYER** | By centrally defining critical business metrics we can ensure consistent self-service access to metrics in downstream tools:<br>- BI dashboards<br>- LLM agents/AI chatbots <br>- Jupyter notebooks for data scientists<br>- Reverse ETL<br><br>**Application:** Cube.dev | | **CONSUMPTION LAYER** | **Consumption patterns:**<br>- BI dashboards<br>- LLM agents/AI chatbots <br>- Jupyter notebooks for data scientists<br>- Reverse ETL (=sending analytic data products back to operational systems)<br><br>**Application**: MSTR/Power BI/Tableau/Looker.... | | **ORCHESTRATION LAYER** | Manages execution of workflows in the ingestion & transformation layer (loading, cleaning, aggregation, …)<br><br>**Application:** Airflow,Dagster or Azure Data factory | | **DATA DOMAINS** | Full data products (datamarts/cubes), created in transformation layer & consumed through a semantic layer <br><br>e.g. stores, services, supply chain, … | | **DATAOPS** | Automation, CI/CD pipelines, Monitoring , Alerting, Version Control<br>IAM management<br> <br>**Application:** <br>CI/CD - Jenkins, Bitbucket pipelines, Github actions<br>Monitoring data sources - Grafana, AWS Cloudwatch, GCP Stackdriver<br>Alerting - Microsoft Teams, Slack channel<br>Version Control - Github, Bitbucket, Gitlab | | **PLATFORM GOVERNANCE LAYER** | Governance process for optimizations: naming conventions, change request flow, … | | **FINOPS** | A data product that collects costs from all layers of the platform (ingestion,transformation & consumption) into 1 report to achieve<br>financial follow-up of data analytics platform. |