README.md

Design a data platform solution (15-20%)

Storage account type	Supported services	Supported performance tiers	Supported access tiers	Replication options	Deployment model1	Encryption2
General-purpose V2	Blob, File, Queue, Table, Disk, and Data Lake Gen26	Standard, Premium5	Hot, Cool, Archive3	LRS, GRS, RA-GRS, ZRS, GZRS (preview), RA-GZRS (preview)4	Resource Manager	Encrypted
General-purpose V1	Blob, File, Queue, Table, and Disk	Standard, Premium5	N/A	LRS, GRS, RA-GRS	Resource Manager, Classic	Encrypted
BlockBlobStorage	Blob (block blobs and append blobs only)	Premium	N/A	LRS, ZRS4	Resource Manager	Encrypted
FileStorage	File only	Premium	N/A	LRS, ZRS4	Resource Manager	Encrypted
BlobStorage	Blob (block blobs and append blobs only)	Standard	Hot, Cool, Archive3	LRS, GRS, RA-GRS	Resource Manager	Encrypted

Design a data management strategy

• choose between managed and unmanaged data store

  • Unmanaged Data Stores (choose when need high performance tuning or hight degree of control over hosting environment and offering)

    • Table
    • Blob
    • File
    • Queue
    • SQL Server in VM
    • Orable DB in VM

  • Managed Data Stores (highly available, auto-scaling, threat detection, auto tuning)

    • VM Data Disk
    • Azure SQL Database
    • Cosmos DB
    • Azure Database for MySQL, PostgreSQL, MariaDB
    • Managed SQL Server
    • Redis Cache

• choose between relational and non-relational databases

  • Criteria for choosing a data store

  • Lab - classify your data

  • Choose relational database for strong consistency and integrity

  • Choose non-relational database for scalability and flexibility

• design a data auditing strategy

  • Take advendate of SQL audit log

  • Use Azure Monitor Logs to connect to SQL Databases and other data sources

  • With Azure Monitor workbooks you can create interactive reports including data auditing

• design a data caching strategy

  • Architect Caching strategy

  • Plularsight - Data Caching

• identify data attributes

  • Plularsight - understanding data attributes

• recommend database service tier sizing

  • Understand DTU-based pricing model - DTU

  • Understand Reuqest Units pricing model - RU/s

• design a data retention policy

  • Regular retention up to 35 days

  • Long Term Retention policy up to 10 years

  • Automated geo-redundant SQL Database backups

• design for data availability

  • Plularsight - Data Availability

• design for data consistency

  • 

  • How to choose right consistency level for Cosmos DB

• design for data durability

  • Understand what is RPO and RTO

    • RPO stands for recovery POINT objective, i.e., how much data is one potentially prepared and willing to lose, worse case
    • RTO stands for recovery TIME objective, i.e., if/when the ‘bad thing’ happens, how much time does it take to be back up and running again

• design a data warehouse strategy

  • Use when need to do reports and analytics

  • Read only data

  • Use Azure Analysis Services to combine data from different sources and perform ad hoc analysis

  • Big Data Architecture

Design a data protection strategy

• recommend geographic data storage

	LRS	ZRS	GRS	RA-GRS
Overview	Replicates data in a single datacenter	Stores copies of data across multiple datacenters	Stores copies in a local datacenter, like LRS, but then stores three more copies in a datacenter in another region	Same as GRS, but offers read access in the secondary datacenter in the other region
Data copies	3	3	6	6
Use case	Ensures that your data is highly available but, for compliance reasons, must be kept local	A higher durability option for block storage, where data can stay in only one region	Where you need to ensure that your data and systems are always available despite datacenter or region outages	Where your applications (especially those with many read requests) can read data from other regions, but also to ensure that read operations are always available even if the primary region is down

• design an encryption strategy for data at rest

  • Azure Storage encryption for data at rest

  • Azure Data Encryption-at-Rest

  • TDE - Transparent data encryption for SQL Database and Data Warehouse

• design an encryption strategy for data in transmission

  • Force SSL connections to data storage

  • Use VPN (site-to-site or point-to-site) if connecting from outside Azure network

  • Plularsight - data encryption in transit

  • Dynamic data masking for Azure SQL Database and Data Warehouse

• design an encryption strategy for data in use

  • Plularsight - data encryption in use

• design a scalability strategy for data

  • Plularsight - data scalability

  • Scaling out with Azure SQL Database

• design secure access to data

  • Secure access to data on network level using Virtual Network service endpoints

  • Whitelist or blacklist IP ranges using Azure SQL Database and Azure SQL Data Warehouse IP firewall rules

  • Use Azure Active Directory to access SQL database

  • Plularsight - secure data access

• design a data loss prevention (DLP) policy

  • Plularsight - data loss prevention

Design and document data flows

• This section is covered by Plularsight - Design and Document Data Flows with Microsoft Azure

• identify data flow requirements

• create a data flow diagram

• design a data flow to meet business requirements

• design data flow solutions

• design a data import and export strategy

Design a monitoring strategy for the data platform

• Azure SQL Monitoring and tuning

• Query Performance Insight for Azure SQL Database

• General advice: For any monitoring solution, check Azure Advisor and Azure Monitor, from there you can often connect to resources to setup additional monitoring

• Enable automatic tuning to monitor queries and improve workload performance

• Troubleshoot Azure SQL Database performance issues with Intelligent Insights

• design for alert notifications

• design an alert and metrics strategy

• monitor Azure Data Factory pipelines