【代理投稿】The Journey from MySQL to TiDB

146 Views

July 09, 24

スライド概要

2024-06-27 TiUGでのPingCAP CTO Ed Huangの発表スライドです。

profile-image

PingCAP株式会社 Developer Advocate & Solution Architect

シェア

またはPlayer版

埋め込む »CMSなどでJSが使えない場合

関連スライド

各ページのテキスト
1.

The Journey from MySQL to TiDB The Story about PingCAP Ed Huang Co-founder & CTO, PingCAP [email protected]

3.

The Beginning -> From OLTP to HTAP -> From Database to Database Cloud

4.

The beginning... ● The story begins in 2015… ○ Max, Dylan and I were working for a fast-growing startup (Wandoujia.com) ■ The AppStore for Android Phone ○ Max and I were working in Storage/Database Infrastructure Team ■ We all love Golang…We build Codis (https://github.com/CodisLabs/codis, I believe u-next is a big user of Codis, also TiDB ) ■ Fun facts: ● We started using Go even before its 1.0 ● We have zero background about SQL database ● No one use Rust in production before 😂 ○ We ran/maintained a huge amount of MySQL instances & HBase! (since 2013) ■ MySQL Sharding Cluster ● ~50 nodes ■ HBase ● >200 nodes

5.

Why not MySQL? The different requirements: ● ● ● ● ● Cost-based model Distributed Computing Online DDL Different storage API abstraction … MySQL codebase is not easy to hack… We love Golang! We want to use the programing language we like :)

6.

The Beginning ● The Beginning: We just want to build a better solution to Sharded MySQL ○ 100% Scalable OLTP ○ MySQL binlog listener (syncer, now is DM), helps user reduce migration cost Binlog Binlog dm Binlog dm Binlog dm dm Binlog https://github.com/go-mysql-org/go-mysql dm

7.

The Journey

8.

The Journey OLTP Workload insert into github_events values(...) insert into github_events values(...) insert into github_events values(...) select … from github_events where actor='xxx' … Binlog Binlog dm Binlog dm dm Binlog dm OLAP Workload

9.

From OLTP to HTAP ● We introduce TiSpark to enhance TiDB's analytical capabilities (TiDB 3.0) TiSpark Binlog Binlog dm Binlog dm Binlog dm dm Binlog dm

10.

From OLTP to HTAP ● ● Is Spark enough? ○ Spark only solve the computing problem We introduce Columnar Storage (TiFlash) into the storage layer of TiDB 4.0 AdHoc Reporting TiKV (Row-based Storage) TiSpark TiFlash (Columnar Storage)

11.

From OLTP to HTAP ● ● TiFlash solves the Storage problem for high speed OLAP And then, we introduce MPP mode in TiDB 5.0 AdHoc Reporting TiKV (Row-based Storage) TiDB SQL TiFlash MPP Mode (Massively Parallel Processing)

12.

What's next? Database Cloud! TiDB Dedicated TiDB Serverless

13.

Is TiDB-Operator a cloud? No, not even close. It's just only a deployment tool for TiDB in k8s

14.

Building a DBaaS is hard, the architecture overview of the first version of TiDB Cloud

15.

Two different approaches to build DBaaS

16.

Two different approaches to build DBaaS Shared Gateway Gateway Gateway Gateway On Demand Pool TiDB Virtual Cluster - Tenant 1 Isolated SQL Layer Shared Storage Layer TiDB Row Engine MPP Row Engine Virtual Cluster - Tenant 2 TiDB MPP MPP MPP MPP Columnar Engine Row Engine Virtual Cluster - Tenant n TiDB TiDB TiDB TiDB Columnar Engine Columnar Engine MPP Worker MPP Worker MPP Worker Shared Storage Pool S3 Service Shared Services Compaction Analyze DDL Remote Copr Service Other Region

17.

Building a DBaaS actually gives us more 1. 2. 3. 4. Multi-tenancy a. Resource Control b. Online/Concurrent DDL c. 1,000,000+ Table Support d. … Cloud native architecture a. Compute/Storage separation TiFlash b. TiDB Serverless Auto-Scaler c. … Performance improvement New workload support a. Vector Search on TiDB Serverless Tier

18.

The philosophy of TiDB ● ● ● ● ● Cloud is the Future ○ Embrace Kubernetes & Cloud Infrastructure early Keep Neutral ○ Not rely on specific hardware ○ Not depend on a specific cloud vendor Be Open ○ Open-Minded & Open-Source Database is not only Database ○ Eco-system also matters! Simplicity is the best weapon to defeat complexity