Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MaterializedMySQL can't start with MySQL 8.3.0 #60479

Closed
Slach opened this issue Feb 28, 2024 · 6 comments · Fixed by #63101
Closed

MaterializedMySQL can't start with MySQL 8.3.0 #60479

Slach opened this issue Feb 28, 2024 · 6 comments · Fixed by #63101
Labels
experimental feature Bug in the feature that should not be used in production st-wontfix Known issue, no plans to fix it currenlty

Comments

@Slach
Copy link
Contributor

Slach commented Feb 28, 2024

MySQL 8.3.0

log_bin_use_v1_row_events: Whether server is using version 1 binary log row events. Removed in MySQL 8.3.0.

so adding --log-bin-use-v1-row-events=OFF will not work

i created reproduced environments
https://gist.github.com/Slach/1db8f5ec1df586f5f6427b0188977fdd

materialized_mysql.xml

<clickhouse>
    <profiles><default><allow_experimental_database_materialized_mysql>1</allow_experimental_database_materialized_mysql></default></profiles>
</clickhouse>

docker-compose.yaml

services:
  mysql:
    image: docker.io/mysql:${MYSQL_VERSION:-latest}
    command: --default-authentication-plugin=mysql_native_password --gtid_mode=on --enforce_gtid_consistency=ON
    hostname: mysql
    container_name: mysql
    environment:
      MYSQL_ROOT_PASSWORD: "root"
    healthcheck:
      test: mysqladmin -p=root ping -h localhost
      timeout: 20s
      retries: 10
  clickhouse:
    image: docker.io/${CLICKHOUSE_IMAGE:-clickhouse/clickhouse-server}:${CLICKHOUSE_VERSION:-latest}
    links:
      - mysql
    healthcheck:
      test: clickhouse client -q "SELECT 1"
      interval: 10s
      timeout: 2s
      retries: 30
      start_period: 5s
    depends_on:
      mysql:
        condition: service_healthy

reproduce.sh

docker-compose pull
docker-compose down
docker-compose up -d clickhouse
docker-compose exec mysql mysql --user=root --password=qwerty -e "CREATE DATABASE mydb"
docker-compose exec clickhouse clickhouse-client -q "CREATE DATABASE mysql ENGINE = MaterializedMySQL('mysql:3306', 'mydb', 'root', 'qwerty') SETTINGS allows_query_when_mysql_lost=true, max_wait_time_when_mysql_unavailable=10000;"

Return error

Code: 695. DB::Exception: Received from localhost:9000. DB::Exception: Load job 'startup MaterializedMySQL database mysql' failed: Code: 537. DB::Exception: Illegal MySQL variables, the MaterializedMySQL engine requires log_bin_use_v1_row_events='OFF'. (ILLEGAL_MYSQL_VARIABLE),. (ASYNC_LOAD_FAILED)
(query: CREATE DATABASE mysql ENGINE = MaterializedMySQL('mysql:3306', 'mydb', 'root', 'qwerty') SETTINGS allows_query_when_mysql_lost=true, max_wait_time_when_mysql_unavailable=10000;)

reproduced on 23.3, 23.8, 24.1

@Slach Slach added the potential bug To be reviewed by developers and confirmed/rejected. label Feb 28, 2024
@alexey-milovidov alexey-milovidov added st-wontfix Known issue, no plans to fix it currenlty experimental feature Bug in the feature that should not be used in production and removed potential bug To be reviewed by developers and confirmed/rejected. labels Mar 15, 2024
@alexey-milovidov
Copy link
Member

We don't support MaterializedMySQL.

@Slach
Copy link
Contributor Author

Slach commented Mar 16, 2024

Why you don't remove it in this case with the same manner as clickhouse-copier?

@alexey-milovidov
Copy link
Member

@stigsb promised to contribute a new implementation.

@Slach
Copy link
Contributor Author

Slach commented Mar 17, 2024

@stigsb hope you will improve MaterializedMySQL

@kolya7k
Copy link
Contributor

kolya7k commented Apr 19, 2024

#62793

@kolya7k
Copy link
Contributor

kolya7k commented Apr 19, 2024

@Slach you can use my mysql-8.3.0 patches for compatible with ClickHouse
https://github.com/kolya7k/mysql-patches/blob/main/0009-clickhouse-compat.patch
It just adds absolete readonly log_bin_use_v1_row_events variable and removes mysql_native_password warning

Slach added a commit to Slach/ClickHouse that referenced this issue Apr 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
experimental feature Bug in the feature that should not be used in production st-wontfix Known issue, no plans to fix it currenlty
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants