-
Notifications
You must be signed in to change notification settings - Fork 590
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor(connector): use config file to initiate a debezium source connector #8539
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
java/connector-node/risingwave-connector-service/src/main/resources/postgres.properties
Outdated
Show resolved
Hide resolved
|
||
// if offset is specified, we will continue reading changes from the specified offset | ||
if (null != startOffset && !startOffset.isBlank()) { | ||
postgresProps.setProperty("snapshot.mode", "never"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not also put this in config file?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is a default config for snapshot.mode
in config file, but we need to disable snapshotting when startOffset
is set (e.g. recovery scenario). This logic cannot be achieved by the config file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if property
is a widely accepted config file format, I would agree with the change
It's quite popular. |
02a996e
to
afdb30a
Compare
commit 96ff27f Author: August <[email protected]> Date: Mon Mar 27 13:24:29 2023 +0800 feat: support alter rename relations including table/mview/view/sink/index (risingwavelabs#7745) commit 0069678 Author: Runji Wang <[email protected]> Date: Mon Mar 27 11:29:49 2023 +0800 feat: add more string functions (risingwavelabs#8767) Signed-off-by: Runji Wang <[email protected]> commit f4e2bdc Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Sat Mar 25 12:12:53 2023 +0800 chore(deps): bump openssl from 0.10.47 to 0.10.48 (risingwavelabs#8769) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 5992017 Author: Richard Chien <[email protected]> Date: Fri Mar 24 20:22:10 2023 +0800 refactor(hash agg): directly use `AggGroup` instead of `Box<AggGroup>` (risingwavelabs#8745) Signed-off-by: Richard Chien <[email protected]> commit 5f41727 Author: Renjie Liu <[email protected]> Date: Fri Mar 24 20:07:17 2023 +0800 fix: Use debug log to record simulation time (risingwavelabs#8768) commit 7302424 Author: Dylan <[email protected]> Date: Fri Mar 24 18:34:47 2023 +0800 feat(meta): support group notification (risingwavelabs#8741) commit be9723e Author: Liang <[email protected]> Date: Fri Mar 24 18:25:00 2023 +0800 fix(state clean): state clean should not delete NULL (risingwavelabs#8737) commit 4b03a93 Author: Yuhao Su <[email protected]> Date: Fri Mar 24 17:11:15 2023 +0800 feat: introduce generated columns (risingwavelabs#8700) commit 98ea76a Author: Noel Kwan <[email protected]> Date: Fri Mar 24 16:57:19 2023 +0800 feat(optimizer): Add `StreamProjectMergeRule` (risingwavelabs#8753) Co-authored-by: lmatz <[email protected]> commit 380e104 Author: stonepage <[email protected]> Date: Fri Mar 24 16:39:31 2023 +0800 perf(stream): add simple strategy for if the stream project compact the chunk (risingwavelabs#8758) commit 8bedb3c Author: Eric Fu <[email protected]> Date: Fri Mar 24 16:21:12 2023 +0800 feat: generate example.toml from default config (risingwavelabs#8735) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> commit 2493fc1 Author: stonepage <[email protected]> Date: Fri Mar 24 15:15:48 2023 +0800 fix(bench): shorten the benchmark id (risingwavelabs#8754) commit 44191a7 Author: Runji Wang <[email protected]> Date: Fri Mar 24 15:01:58 2023 +0800 refactor(expr): generate build-from-prost with procedural macros (risingwavelabs#8499) Signed-off-by: Runji Wang <[email protected]> commit 58f29a6 Author: Shanicky Chen <[email protected]> Date: Fri Mar 24 14:47:37 2023 +0800 fix: correctly handle graceful shutdown after losing leader (risingwavelabs#8734) commit 1c6b246 Author: zwang28 <[email protected]> Date: Fri Mar 24 11:45:29 2023 +0800 chore(test): fix config for compaction test (risingwavelabs#8752) commit e951cde Author: Kamalesh Palanisamy <[email protected]> Date: Thu Mar 23 15:40:56 2023 -0500 feat: Add support for array_length function in psql (risingwavelabs#8636) Co-authored-by: xxchan <[email protected]> commit aeddef3 Author: Bohan Zhang <[email protected]> Date: Thu Mar 23 20:11:44 2023 +0800 fix: avro timestamp_mills and timestamp_micros parse to timestamptz (risingwavelabs#8730) Signed-off-by: tabVersion <[email protected]> commit 24ecd4d Author: Bohan Zhang <[email protected]> Date: Thu Mar 23 18:19:17 2023 +0800 chore: fix unstable unit test (risingwavelabs#8674) Signed-off-by: tabVersion <[email protected]> commit d967fcc Author: Shanicky Chen <[email protected]> Date: Thu Mar 23 17:53:45 2023 +0800 chore: use a separate option for etcd election client (risingwavelabs#8726) Signed-off-by: Shanicky Chen <[email protected]> commit 92584b6 Author: Wallace <[email protected]> Date: Thu Mar 23 17:01:14 2023 +0800 fix(meta): fix trivial move failed because of no member table ids (risingwavelabs#8725) Signed-off-by: Little-Wallace <[email protected]> commit 1df800a Author: stonepage <[email protected]> Date: Thu Mar 23 16:49:55 2023 +0800 perf(hashkey): add benchmark for hash key ser/deser (risingwavelabs#8733) commit 96aa23d Author: Noel Kwan <[email protected]> Date: Thu Mar 23 16:16:49 2023 +0800 docs(common): add more docs for `DataChunk` (risingwavelabs#8736) commit f6ccfd5 Author: lmatz <[email protected]> Date: Thu Mar 23 16:13:15 2023 +0800 perf(connector): use Vec<u8> instead of Bytes and &[u8] (risingwavelabs#8732) commit 0e8d81f Author: Yuhao Su <[email protected]> Date: Thu Mar 23 16:02:52 2023 +0800 fix: evict hash join cache every n messages. (risingwavelabs#8731) commit 8261a30 Author: Zhidong Guo <[email protected]> Date: Thu Mar 23 16:02:45 2023 +0800 fix(risedev): disallow in memory state store when there is a compactor (risingwavelabs#8729) commit 9d64e50 Author: ZENOTME <[email protected]> Date: Thu Mar 23 14:27:19 2023 +0800 test: add e2e extended mode test (risingwavelabs#8710) commit 7bac239 Author: Bugen Zhao <[email protected]> Date: Thu Mar 23 13:57:21 2023 +0800 feat(meta): support scaling delta join (risingwavelabs#8694) Signed-off-by: Bugen Zhao <[email protected]> commit 837d43a Author: odysa <[email protected]> Date: Wed Mar 22 22:45:57 2023 -0700 fix: integration tests telemetry env (risingwavelabs#8705) commit 3e8ce31 Author: Renjie Liu <[email protected]> Date: Thu Mar 23 13:36:03 2023 +0800 chore: Add time for each slt record in simulation test (risingwavelabs#8724) commit ea72ed7 Author: Liang <[email protected]> Date: Thu Mar 23 12:11:42 2023 +0800 fix(watermark): fix watermark derivation in stream dynamic filter in … (risingwavelabs#8719) commit 1b008f4 Author: Renjie Liu <[email protected]> Date: Thu Mar 23 11:59:01 2023 +0800 chore: Remove unnecessary spawn (risingwavelabs#8722) commit a1d084d Author: Bugen Zhao <[email protected]> Date: Thu Mar 23 11:21:33 2023 +0800 fix(streaming): also enable schema check in release profile (risingwavelabs#8711) Signed-off-by: Bugen Zhao <[email protected]> commit ad61a71 Author: Eric Fu <[email protected]> Date: Thu Mar 23 11:16:45 2023 +0800 test: add integration test of tidb/ticdc (risingwavelabs#8708) commit 3a0ee3d Author: lmatz <[email protected]> Date: Thu Mar 23 08:50:15 2023 +0800 perf(parser): do to_ascii_lowercase only once (risingwavelabs#8718) commit 669087e Author: Renjie Liu <[email protected]> Date: Wed Mar 22 20:15:36 2023 +0800 fix: Dashboard main should also setup protoc (risingwavelabs#8717) commit c34ffc1 Author: Renjie Liu <[email protected]> Date: Wed Mar 22 19:36:28 2023 +0800 chore: Remove proto gen in dashboard (risingwavelabs#8716) commit 815a51a Author: Renjie Liu <[email protected]> Date: Wed Mar 22 17:52:39 2023 +0800 fix(batch): Cancel task should not propagate error. (risingwavelabs#8675) commit c784683 Author: idx0-dev <[email protected]> Date: Wed Mar 22 16:32:47 2023 +0800 fix: csv empty cell should be parsed to null (risingwavelabs#8709) commit 359fbf5 Author: Wallace <[email protected]> Date: Wed Mar 22 15:43:50 2023 +0800 feat(meta): support move state-table between compaction-group (risingwavelabs#8390) Signed-off-by: Little-Wallace <[email protected]> commit 52caa65 Author: lmatz <[email protected]> Date: Wed Mar 22 15:32:32 2023 +0800 revert: feat(streaming): call may_exist when insert cache miss in join executor (risingwavelabs#7957) (risingwavelabs#8655) commit 86ffe99 Author: Zhidong Guo <[email protected]> Date: Wed Mar 22 14:45:24 2023 +0800 deprecate(config): deprecate state store url on worker nodes (risingwavelabs#8704) commit 9e774bb Author: Richard Chien <[email protected]> Date: Wed Mar 22 14:33:00 2023 +0800 refactor(hash agg): split building change and applying change (risingwavelabs#8706) Signed-off-by: Richard Chien <[email protected]> commit b474059 Author: WillyKidd <[email protected]> Date: Wed Mar 22 12:29:22 2023 +0800 test(connector): add test cases for postgres validation permission checks (risingwavelabs#8662) commit 4738ee9 Author: Eric Fu <[email protected]> Date: Wed Mar 22 12:07:54 2023 +0800 fix(ci): fix github workflow syntax error (risingwavelabs#8702) commit 655d2bd Author: Zhanxiang (Patrick) Huang <[email protected]> Date: Wed Mar 22 11:55:38 2023 +0800 feat(sst-dump): support more features (risingwavelabs#8580) commit 8648ff8 Author: Bugen Zhao <[email protected]> Date: Wed Mar 22 11:35:40 2023 +0800 refactor(simulation): interface of running multiple queries in the same session (risingwavelabs#8697) Signed-off-by: Bugen Zhao <[email protected]> commit 6313c42 Author: TennyZhuang <[email protected]> Date: Wed Mar 22 11:26:49 2023 +0800 refactor(dashboard): ignore `dashboard/proto/gen` and gen them during `npm i` (risingwavelabs#8695) Signed-off-by: TennyZhuang <[email protected]> commit 3a06226 Author: xxchan <[email protected]> Date: Tue Mar 21 12:02:55 2023 +0100 fix(stream): careful cache invadiation for TopN (risingwavelabs#8659) Co-authored-by: st1page <[email protected]> commit 9abe5dc Author: Richard Chien <[email protected]> Date: Tue Mar 21 18:18:39 2023 +0800 perf(hash agg): use `get_mut` instead of `pop`+`put` pattern (risingwavelabs#8691) Signed-off-by: Richard Chien <[email protected]> commit bf95473 Author: Eric Fu <[email protected]> Date: Tue Mar 21 18:14:20 2023 +0800 fix(connector): support more data types in JDBC sink (risingwavelabs#8678) Signed-off-by: tabVersion <[email protected]> Co-authored-by: tabVersion <[email protected]> commit 694c446 Author: Liang <[email protected]> Date: Tue Mar 21 17:50:40 2023 +0800 fix(watermark): avoid panic in watermark derivation (close risingwavelabs#8689) (risingwavelabs#8690) commit 870ba34 Author: TennyZhuang <[email protected]> Date: Tue Mar 21 17:20:06 2023 +0800 chore(deps): bump minitrace,toml,auto_enums,bit_flags,... (risingwavelabs#8682) Signed-off-by: TennyZhuang <[email protected]> commit 0e8f518 Author: TennyZhuang <[email protected]> Date: Tue Mar 21 16:59:02 2023 +0800 perf(array): avoid double bounds check in ArrayIterator (risingwavelabs#8685) Signed-off-by: TennyZhuang <[email protected]> commit 16c9708 Author: Bugen Zhao <[email protected]> Date: Tue Mar 21 16:22:57 2023 +0800 fix(streaming): drop subtask on another blocking thread (risingwavelabs#8672) Signed-off-by: Bugen Zhao <[email protected]> commit 83057e5 Author: congyi wang <[email protected]> Date: Tue Mar 21 15:46:31 2023 +0800 feat(storage): support azblob (risingwavelabs#8257) commit dd34989 Author: Bohan Zhang <[email protected]> Date: Tue Mar 21 14:40:15 2023 +0800 chore: fix docker compose test (risingwavelabs#8666) Signed-off-by: tabVersion <[email protected]> commit 02b3ea2 Author: odysa <[email protected]> Date: Mon Mar 20 22:44:59 2023 -0700 feat: telemetry (risingwavelabs#7384) commit 3d4bca7 Author: Bugen Zhao <[email protected]> Date: Tue Mar 21 13:24:36 2023 +0800 fix: also enable jemalloc on macOS (risingwavelabs#8665) Signed-off-by: Bugen Zhao <[email protected]> commit 3079441 Author: xiangjinwu <[email protected]> Date: Tue Mar 21 13:16:49 2023 +0800 fix(optimizer): `PlanCorrelatedIdFinder` should be aware of agg filter (risingwavelabs#8667) commit d557a6c Author: Dylan <[email protected]> Date: Tue Mar 21 13:05:15 2023 +0800 fix(meta): fix alter table add/drop column with indexes (risingwavelabs#8664) commit 59a0947 Author: Yuanxin Cao <[email protected]> Date: Tue Mar 21 11:17:54 2023 +0800 fix(sink): pass downstream pk when starting a sink (risingwavelabs#8660) commit 8c5489e Author: Bugen Zhao <[email protected]> Date: Tue Mar 21 11:06:04 2023 +0800 fix(meta): correctly resolve update of vnode mapping after scaling (risingwavelabs#8652) Signed-off-by: Bugen Zhao <[email protected]> commit cc6e687 Author: Clearlove <[email protected]> Date: Mon Mar 20 12:51:14 2023 -0400 feat(optimizer): optimize always-false filter for batch (risingwavelabs#8629) Signed-off-by: Clearlove <[email protected]> commit ae99e55 Author: Li0k <[email protected]> Date: Mon Mar 20 20:38:58 2023 +0800 feat(storage): basic compactor scheduler policy (risingwavelabs#6986) Co-authored-by: Yuhao Su <[email protected]> Co-authored-by: Dylan <[email protected]> Co-authored-by: Yuanxin Cao <[email protected]> Co-authored-by: ZENOTME <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: congyi wang <[email protected]> commit eddb2fc Author: xiangjinwu <[email protected]> Date: Mon Mar 20 19:41:21 2023 +0800 fix(optimizer): `ApplyAggTransposeRule` should handle `CorrelatedInputRef` in agg filter (risingwavelabs#8650) commit 32f4925 Author: WillyKidd <[email protected]> Date: Mon Mar 20 18:08:45 2023 +0800 fix(connector): add postgres permission checks in validation phase (risingwavelabs#8525) commit ce9e519 Author: Dylan <[email protected]> Date: Mon Mar 20 18:01:00 2023 +0800 feat(frontend): Enhance parsing error reporting by providing location information. (risingwavelabs#8646) commit 7fbab01 Author: Bugen Zhao <[email protected]> Date: Mon Mar 20 16:23:14 2023 +0800 chore: remove accidentally pushed files (risingwavelabs#8645) Signed-off-by: Bugen Zhao <[email protected]> commit 44943ec Author: Bohan Zhang <[email protected]> Date: Mon Mar 20 16:06:31 2023 +0800 fix: display error message on the frontend (risingwavelabs#8638) Signed-off-by: tabVersion <[email protected]> Co-authored-by: lmatz <[email protected]> commit 1dee824 Author: xxchan <[email protected]> Date: Mon Mar 20 08:27:14 2023 +0100 doc: add notes for `.cargo/config.toml` (risingwavelabs#8632) commit 13641c8 Author: congyi wang <[email protected]> Date: Mon Mar 20 15:16:30 2023 +0800 feat(test): test opendal fs engine on main cron with large dataset (risingwavelabs#8637) commit 014f6db Author: Shanicky Chen <[email protected]> Date: Mon Mar 20 13:55:14 2023 +0800 feat: remove the exchange after append_only source (risingwavelabs#8532) commit 19b1be3 Author: Eric Fu <[email protected]> Date: Mon Mar 20 13:25:06 2023 +0800 fix(risedev): connector-node start-up script (risingwavelabs#8590) Co-authored-by: lmatz <[email protected]> commit da57717 Author: Bugen Zhao <[email protected]> Date: Mon Mar 20 13:08:44 2023 +0800 fix(streaming): drop actor on another blocking thread (risingwavelabs#8624) Signed-off-by: Bugen Zhao <[email protected]> commit f111bfb Author: Zhidong Guo <[email protected]> Date: Mon Mar 20 11:39:15 2023 +0800 fix(config): bring back system params to config file (risingwavelabs#8623) commit 3789023 Author: xxchan <[email protected]> Date: Mon Mar 20 03:56:27 2023 +0100 fix: fix connector start in risedev (risingwavelabs#8587) commit 41323a9 Author: Richard Chien <[email protected]> Date: Mon Mar 20 01:04:51 2023 +0800 feat(common): support `NULLS {FIRST | LAST}` (risingwavelabs#8485) Signed-off-by: Richard Chien <[email protected]> commit 6940ae4 Author: Tesla Zhang <[email protected]> Date: Sat Mar 18 16:57:04 2023 -0400 feat(pretty): introduce the proposed pretty printer in stream_graph_formatter (risingwavelabs#8576) commit 99f4604 Author: Bohan Zhang <[email protected]> Date: Sat Mar 18 16:09:40 2023 +0800 chore: add timeout to fix main ci (risingwavelabs#8627) Signed-off-by: tabVersion <[email protected]> commit 9a818b8 Author: xiangjinwu <[email protected]> Date: Sat Mar 18 14:53:22 2023 +0800 fix(common): interval multiplication and division (risingwavelabs#8620) commit 53261c5 Author: Kevin Axel <[email protected]> Date: Sat Mar 18 11:47:22 2023 +0800 feat: Bushy tree join ordering (risingwavelabs#8316) Signed-off-by: Kevin Axel <[email protected]> commit a3dc882 Author: StrikeW <[email protected]> Date: Fri Mar 17 23:52:09 2023 +0800 fix(compaction-test): add --advertise-addr to meta's opts (risingwavelabs#8611) commit 99501d1 Author: Yufan Song <[email protected]> Date: Fri Mar 17 10:39:28 2023 -0400 feat(connector-node): support stream chunk payload in connector node (risingwavelabs#8548) Co-authored-by: William Wen <[email protected]> commit 8c95702 Author: Eric Fu <[email protected]> Date: Fri Mar 17 21:06:36 2023 +0800 fix(sink): remove `unimplemented` (risingwavelabs#8622) Signed-off-by: tabVersion <[email protected]> Co-authored-by: tabVersion <[email protected]> commit 7cd7c9d Author: William Wen <[email protected]> Date: Fri Mar 17 20:47:53 2023 +0800 fix(conector-node): do not store sink row inside upsert iceberg sink (risingwavelabs#8625) commit 6fd8821 Author: Runji Wang <[email protected]> Date: Fri Mar 17 20:19:22 2023 +0800 chore: replace all `ProstXxx` with `PbXxx` (risingwavelabs#8621) Signed-off-by: Runji Wang <[email protected]> commit d90165a Author: StrikeW <[email protected]> Date: Fri Mar 17 18:41:37 2023 +0800 refactor(connector): use config file to initiate a debezium source connector (risingwavelabs#8539) commit bba43ba Author: Bohan Zhang <[email protected]> Date: Fri Mar 17 16:49:00 2023 +0800 feat: make gen timestamp deterministic (risingwavelabs#8619) Signed-off-by: tabVersion <[email protected]> commit 18863e0 Author: congyi wang <[email protected]> Date: Fri Mar 17 16:25:10 2023 +0800 refactor(frontend): replace dist_key_indices with dist_key_in_pk_indices in frontend (risingwavelabs#8617) commit 961e342 Author: Yuanxin Cao <[email protected]> Date: Fri Mar 17 16:10:39 2023 +0800 feat(sink): enable user-defined primary key for upsert sink (risingwavelabs#8610) commit 88aa6a4 Author: Bugen Zhao <[email protected]> Date: Fri Mar 17 16:00:45 2023 +0800 fix(test): use correct type for nexmark source planner test (risingwavelabs#8618) Signed-off-by: Bugen Zhao <[email protected]> commit 9be17af Author: xiangjinwu <[email protected]> Date: Fri Mar 17 15:07:54 2023 +0800 fix(common): interval input/output overflow and negative handling (risingwavelabs#8613) commit 1b208bb Author: Bugen Zhao <[email protected]> Date: Fri Mar 17 14:57:38 2023 +0800 feat(streaming): separate `BarrierRecv` executor (risingwavelabs#8595) Signed-off-by: Bugen Zhao <[email protected]> commit 78ddbce Author: William Wen <[email protected]> Date: Fri Mar 17 14:12:06 2023 +0800 feat(connector-node): specify sink payload format in start sink and call close for iterator and sink row (risingwavelabs#8585) commit f4a2f8d Author: congyi wang <[email protected]> Date: Fri Mar 17 13:49:55 2023 +0800 refactor(storage): sstable iter compare fullkey struct instead of encoded key to avoid memory allocation (risingwavelabs#8607) commit ab701c7 Author: Wallace <[email protected]> Date: Fri Mar 17 13:48:25 2023 +0800 fix(meta): limit file count and l0 compact score (risingwavelabs#8563) Signed-off-by: Little-Wallace <[email protected]> commit 4f430ac Author: StrikeW <[email protected]> Date: Fri Mar 17 13:34:25 2023 +0800 feat(source): support private link for kafka connector (risingwavelabs#8247) Signed-off-by: Runji Wang <[email protected]> Co-authored-by: Runji Wang <[email protected]> commit 7b5ffb4 Author: Yuanxin Cao <[email protected]> Date: Fri Mar 17 12:33:19 2023 +0800 feat(connector): validate sink primary key and sink type on connector node (risingwavelabs#8599) commit 08fc246 Author: Zhanxiang (Patrick) Huang <[email protected]> Date: Thu Mar 16 22:33:56 2023 +0800 feat(source): store source split state as jsonb (risingwavelabs#8602) commit cfc0349 Author: Liang <[email protected]> Date: Thu Mar 16 22:19:45 2023 +0800 perf(compaction): avoid duplicate data in LSM (risingwavelabs#8489) commit ad7e21b Author: Clearlove <[email protected]> Date: Thu Mar 16 10:08:18 2023 -0400 refractor(optimizer): replace `StreamIndexScan` by `StreamTableScan` on logical index scan (risingwavelabs#8567) Signed-off-by: Clearlove <[email protected]> commit 5efe089 Author: Clearlove <[email protected]> Date: Thu Mar 16 09:42:52 2023 -0400 feat(frontend): ban `update` statements modifying pk columns (risingwavelabs#8569) Signed-off-by: Clearlove <[email protected]> commit 25a9127 Author: ZENOTME <[email protected]> Date: Thu Mar 16 21:04:50 2023 +0800 feat(frontend): support extended query protocol handle (risingwavelabs#8565) commit 018fe9e Author: Runji Wang <[email protected]> Date: Thu Mar 16 20:09:27 2023 +0800 doc(udf): improve UDF documentation (risingwavelabs#8597) Signed-off-by: Runji Wang <[email protected]> commit 2eaea50 Author: TennyZhuang <[email protected]> Date: Thu Mar 16 19:59:45 2023 +0800 test(stream): add join and temporal_filter state-cleaning test (risingwavelabs#8596) Signed-off-by: TennyZhuang <[email protected]> Co-authored-by: Yuhao Su <[email protected]> commit a4bd877 Author: congyi wang <[email protected]> Date: Thu Mar 16 19:57:17 2023 +0800 chore: remove dist_key_indices in state table and storage table (risingwavelabs#8601) commit c683098 Author: zwang28 <[email protected]> Date: Thu Mar 16 19:09:16 2023 +0800 fix(test): fix test config (risingwavelabs#8586) commit bcb324b Author: Dylan <[email protected]> Date: Thu Mar 16 18:38:46 2023 +0800 feat(meta): add internal table to pg_class (risingwavelabs#8594) commit 632423a Author: Dylan <[email protected]> Date: Thu Mar 16 18:08:13 2023 +0800 fix(optimizer): fix hash join distribution (risingwavelabs#8598) commit 582307d Author: Bugen Zhao <[email protected]> Date: Thu Mar 16 18:05:16 2023 +0800 refactor(playground): refinements on connector node (risingwavelabs#8582) Signed-off-by: Bugen Zhao <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> commit 9b89bb0 Author: stonepage <[email protected]> Date: Thu Mar 16 17:47:55 2023 +0800 fix(optimizer): projectSet && overAgg should call input's predicate push down && prune col (risingwavelabs#8588) commit 14bfc62 Author: Shanicky Chen <[email protected]> Date: Thu Mar 16 17:42:50 2023 +0800 chore: add is_visible in column for connector (risingwavelabs#8592) commit 075d50a Author: ZENOTME <[email protected]> Date: Thu Mar 16 17:29:52 2023 +0800 feat(frontend): support BATCH_PARALLELISM (risingwavelabs#8552) commit bd9d156 Author: Bohan Zhang <[email protected]> Date: Thu Mar 16 17:28:09 2023 +0800 fix: load pk from the downstream instead of Risingwave (risingwavelabs#8457) Signed-off-by: tabVersion <[email protected]> commit 25a4809 Author: Bugen Zhao <[email protected]> Date: Thu Mar 16 16:16:25 2023 +0800 feat(streaming): only output required columns for MV-on-MV (risingwavelabs#8555) Signed-off-by: Bugen Zhao <[email protected]> commit 64a5f88 Author: congyi wang <[email protected]> Date: Thu Mar 16 15:13:09 2023 +0800 feat(storage): do not compress table_id (risingwavelabs#8512) commit 7641b15 Author: congyi wang <[email protected]> Date: Thu Mar 16 15:12:01 2023 +0800 feat(test): add e2e test for OpenDAL fs backend (risingwavelabs#8528) commit 65a641d Author: TennyZhuang <[email protected]> Date: Thu Mar 16 15:00:39 2023 +0800 feat(frontend): support SET TIME ZONE (risingwavelabs#8572) Signed-off-by: TennyZhuang <[email protected]> commit c19fc72 Author: Eridanus <[email protected]> Date: Thu Mar 16 14:29:43 2023 +0800 feat: Support optional parameter `offset` in `tumble` and `hop` (risingwavelabs#8490) commit 777e836 Author: stonepage <[email protected]> Date: Thu Mar 16 14:09:19 2023 +0800 perf(streaming): add the missed read prefix hint on state table (risingwavelabs#8545) commit e61af5a Author: xiangjinwu <[email protected]> Date: Thu Mar 16 14:08:58 2023 +0800 fix(common): interval overflow panic / wrap during comparison and justify (risingwavelabs#8556) commit 04a2885 Author: lmatz <[email protected]> Date: Thu Mar 16 14:05:30 2023 +0800 fix: add --advertise-addr to playground meta's opts (risingwavelabs#8581) commit 6060672 Author: lmatz <[email protected]> Date: Thu Mar 16 13:51:41 2023 +0800 deprecate: remove --host and --client-address args (risingwavelabs#8575) commit 30a8946 Author: odysa <[email protected]> Date: Wed Mar 15 22:07:44 2023 -0700 fix(common): decimal round digits test (risingwavelabs#8568) Co-authored-by: Noel Kwan <[email protected]> commit 88550e4 Author: Liang <[email protected]> Date: Thu Mar 16 13:07:42 2023 +0800 fix(hash join): avoid emitting chunks that violate `UpdateDelete` ass… (risingwavelabs#8579) commit dc76ad7 Author: lmatz <[email protected]> Date: Thu Mar 16 12:39:41 2023 +0800 deprecate: remove host cmd line argument when starting meta node (risingwavelabs#8574) commit b0f276b Author: Zhidong Guo <[email protected]> Date: Thu Mar 16 01:41:51 2023 +0800 fix(config): remove system params from config file (risingwavelabs#8366) commit 24fe1e8 Author: jon-chuang <[email protected]> Date: Wed Mar 15 23:35:00 2023 +0800 fix(batch, source): Propagate user errors (risingwavelabs#8493) Signed-off-by: Runji Wang <[email protected]> Co-authored-by: jon-chuang <[email protected]> Co-authored-by: Runji Wang <[email protected]> commit 81b4d59 Author: Liang <[email protected]> Date: Wed Mar 15 21:11:24 2023 +0800 perf(prefetch): enable prefetch for hash join with degree table (risingwavelabs#8566) commit 7bd208d Author: stonepage <[email protected]> Date: Wed Mar 15 20:41:28 2023 +0800 fix(explain): add missed fields in explain streaming dist plan (risingwavelabs#8544) commit 03cc2ae Author: Runji Wang <[email protected]> Date: Wed Mar 15 16:52:47 2023 +0800 refactor(expr): make evaluation async (risingwavelabs#8229) Signed-off-by: Runji Wang <[email protected]> commit 1a11c3f Author: ZENOTME <[email protected]> Date: Wed Mar 15 15:58:03 2023 +0800 feat(frontend): support bind paramater (risingwavelabs#8543) commit f92d7f6 Author: Runji Wang <[email protected]> Date: Wed Mar 15 14:39:07 2023 +0800 feat(udf): support user-defined table function (UDTF) (risingwavelabs#8255) Signed-off-by: Runji Wang <[email protected]> Co-authored-by: xxchan <[email protected]> commit 61191c2 Author: Shanicky Chen <[email protected]> Date: Wed Mar 15 14:36:56 2023 +0800 feat(common): Add support for DataType::Serial (risingwavelabs#8392) commit f907452 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed Mar 15 13:56:34 2023 +0800 chore(deps): bump golang.org/x/net from 0.0.0-20220927171203-f486391704dc to 0.7.0 in /integration_tests/datagen (risingwavelabs#8521) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Yuhao Su <[email protected]> commit 1449439 Author: Bugen Zhao <[email protected]> Date: Wed Mar 15 13:47:22 2023 +0800 refactor(fragmenter): remove `is_singleton` workarounds on `Chain` (risingwavelabs#8536) Signed-off-by: Bugen Zhao <[email protected]> commit 9f68cef Author: broccoliSpicy <[email protected]> Date: Wed Mar 15 12:13:52 2023 +0800 feat(expr): support builtin function pi. (risingwavelabs#8509) commit 8be4734 Author: William Wen <[email protected]> Date: Wed Mar 15 12:12:44 2023 +0800 feat(java-binding): support java binding on stream chunk (risingwavelabs#8517) commit 8be23f4 Author: Noel Kwan <[email protected]> Date: Wed Mar 15 12:03:41 2023 +0800 feat(sqlsmith): add statement-level reducer (risingwavelabs#8507) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> commit 320d755 Author: TennyZhuang <[email protected]> Date: Wed Mar 15 12:01:24 2023 +0800 test(stream): add state cleaning test (risingwavelabs#8546) Signed-off-by: TennyZhuang <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> commit 8b09f5e Author: Liang <[email protected]> Date: Tue Mar 14 22:14:47 2023 +0800 feat(sst id): avoid SST ID gap when generating (risingwavelabs#8542) commit 53c6a4c Author: Bugen Zhao <[email protected]> Date: Tue Mar 14 20:55:17 2023 +0800 fix(expr): do not construct error for extracting time subfield (risingwavelabs#8538) Signed-off-by: Bugen Zhao <[email protected]> commit 53eaa24 Author: Liang <[email protected]> Date: Tue Mar 14 20:47:57 2023 +0800 refactor(compactor): remove useless code (risingwavelabs#8514) commit 18b04a4 Author: Li0k <[email protected]> Date: Tue Mar 14 20:34:52 2023 +0800 feat(storage): flexible KeyPrefix encoding for Block (risingwavelabs#8379) commit 0c53349 Author: Zhanxiang (Patrick) Huang <[email protected]> Date: Tue Mar 14 19:38:52 2023 +0800 fix: move sst meta offset to the end of the object (risingwavelabs#8524) commit d49a4c5 Author: stonepage <[email protected]> Date: Tue Mar 14 19:35:05 2023 +0800 refactor(optimizer): move fd dervie into core (risingwavelabs#8540) commit 305c864 Author: ZENOTME <[email protected]> Date: Tue Mar 14 19:31:56 2023 +0800 feat(frontend): support infer param in binder (risingwavelabs#8453) commit 9d5ff78 Author: Liang <[email protected]> Date: Tue Mar 14 19:23:36 2023 +0800 refactor(storage): distinguish SST id and object id (close risingwavelabs#8434) (risingwavelabs#8436) commit 6d28cf6 Author: Bugen Zhao <[email protected]> Date: Tue Mar 14 18:24:16 2023 +0800 refactor(streaming): only scan necessary columns in backfill (risingwavelabs#8533) Signed-off-by: Bugen Zhao <[email protected]> commit 42f17a6 Author: ZENOTME <[email protected]> Date: Tue Mar 14 17:21:22 2023 +0800 fix(pgwire): process empty query correctly (risingwavelabs#8535) commit 84a9831 Author: William Wen <[email protected]> Date: Tue Mar 14 17:20:04 2023 +0800 feat(connector): unify and simplify path config of minio and s3 (risingwavelabs#8508) commit 0e93998 Author: August <[email protected]> Date: Tue Mar 14 16:38:53 2023 +0800 fix: avoid panic when upstream input is closed for lookup (risingwavelabs#8529) commit 338ace9 Author: Dylan <[email protected]> Date: Tue Mar 14 15:41:25 2023 +0800 chore(batch): reuse common logical for index selection (risingwavelabs#8531) commit eb56b9f Author: Noel Kwan <[email protected]> Date: Tue Mar 14 15:23:36 2023 +0800 fix(ci): run entire snapshot in `main`, `pr` and `main-cron` workflows (risingwavelabs#8523) commit d06674b Author: Shanicky Chen <[email protected]> Date: Tue Mar 14 14:53:40 2023 +0800 refactor: move row_id to common (risingwavelabs#8513) commit 9bf2e8c Author: Bugen Zhao <[email protected]> Date: Tue Mar 14 14:47:15 2023 +0800 fix(ci): increase scaling test timeout for nexmark recovery test (risingwavelabs#8526) Signed-off-by: Bugen Zhao <[email protected]> commit a4afac3 Author: Yuanxin Cao <[email protected]> Date: Tue Mar 14 14:21:29 2023 +0800 refactor: refine conditional compilation for mem control on different OSs (risingwavelabs#8504) Co-authored-by: Bugen Zhao <[email protected]> commit 9564db0 Author: Dylan <[email protected]> Date: Tue Mar 14 14:12:55 2023 +0800 feat(batch): Support index selection for sort aggregation with a descending ordering (risingwavelabs#8515) commit 97b021d Author: stonepage <[email protected]> Date: Tue Mar 14 14:00:59 2023 +0800 fix(streaming): ignore null stream key from full outer join to workaround (risingwavelabs#8520) commit 53da2e3 Author: Yuanxin Cao <[email protected]> Date: Tue Mar 14 13:58:49 2023 +0800 fix(ci): exclude `go.mod` and `go.sum` files for typos (risingwavelabs#8527) commit 8183b41 Author: Yuanxin Cao <[email protected]> Date: Tue Mar 14 12:49:45 2023 +0800 feat(meta): introduce sink validation in meta (risingwavelabs#8417) commit fbcd407 Author: Bugen Zhao <[email protected]> Date: Tue Mar 14 12:05:02 2023 +0800 refactor: use u16 representation for virtual node (risingwavelabs#8385) Signed-off-by: Bugen Zhao <[email protected]> Co-authored-by: Renjie Liu <[email protected]> commit 4f34ade Author: Eric Fu <[email protected]> Date: Tue Mar 14 11:13:27 2023 +0800 chore: migrate demo projects here as intergration tests (risingwavelabs#8502) Co-authored-by: TennyZhuang <[email protected]> commit c11fb64 Author: Dylan <[email protected]> Date: Tue Mar 14 10:41:24 2023 +0800 fix(optimizer): ban scalar subquery for project set (risingwavelabs#8519) commit 428354d Author: zwang28 <[email protected]> Date: Mon Mar 13 19:08:46 2023 +0800 feat(backup): support mutating backup config (risingwavelabs#8505) Co-authored-by: Zhidong Guo <[email protected]> commit 85e450d Author: Bugen Zhao <[email protected]> Date: Mon Mar 13 18:42:27 2023 +0800 fix(streaming): map watermark in dispatcher with output indices (risingwavelabs#8506) Signed-off-by: Bugen Zhao <[email protected]> commit cdaa8cf Author: xiangjinwu <[email protected]> Date: Mon Mar 13 16:59:20 2023 +0800 fix(common): interval should have microsecond precision (risingwavelabs#8501) commit b235f68 Author: Renjie Liu <[email protected]> Date: Mon Mar 13 16:42:34 2023 +0800 refactor: ExchangeWriter should report error throught status (risingwavelabs#8478) commit 2db01f9 Author: Dylan <[email protected]> Date: Mon Mar 13 16:01:35 2023 +0800 feat(streaming): support temporal join part 3 (risingwavelabs#8480) Co-authored-by: Bugen Zhao <[email protected]> Co-authored-by: st1page <[email protected]> commit f36bf0b Author: William Wen <[email protected]> Date: Mon Mar 13 14:37:17 2023 +0800 feat(frontend): support SET SESSION CHARACTERISTICS AS TRANSACTION (risingwavelabs#8497) commit 79a3786 Author: Bugen Zhao <[email protected]> Date: Mon Mar 13 14:36:33 2023 +0800 fix(common): only warn on dropping non-empty builder (risingwavelabs#8494) Signed-off-by: Bugen Zhao <[email protected]> commit e1ae04e Author: stonepage <[email protected]> Date: Mon Mar 13 14:34:24 2023 +0800 fix(streaming): hop executor handle watermark (risingwavelabs#8498) commit c0aa78b Author: congyi wang <[email protected]> Date: Mon Mar 13 13:59:17 2023 +0800 refactor(storage): sstable and block level interfaces use fullkey struct (risingwavelabs#8419) commit 32100f3 Author: William Wen <[email protected]> Date: Mon Mar 13 13:05:23 2023 +0800 feat(sink): set parallelism of iceberg sink to 1 (risingwavelabs#8476) commit a6c8c86 Author: Yuanxin Cao <[email protected]> Date: Mon Mar 13 11:59:26 2023 +0800 feat: user configurable memory control policy (risingwavelabs#8475) commit 28c539c Author: Bugen Zhao <[email protected]> Date: Sun Mar 12 15:52:17 2023 +0800 refactor: use a macro for logical vs physical type matching (risingwavelabs#8479) Signed-off-by: Bugen Zhao <[email protected]> commit e51f639 Author: Eric Fu <[email protected]> Date: Sun Mar 12 14:22:16 2023 +0800 fix: improve CDC connector param check (risingwavelabs#8450) commit b7c46d4 Author: zwang28 <[email protected]> Date: Sun Mar 12 11:50:53 2023 +0800 feat(storage): limit hummock write based on LSM tree stats (risingwavelabs#8383) commit 4b49428 Author: TennyZhuang <[email protected]> Date: Sat Mar 11 23:44:56 2023 +0800 feat(parser): report nearby tokens when parse failed (risingwavelabs#8465) Signed-off-by: TennyZhuang <[email protected]> commit 00ea62a Author: TennyZhuang <[email protected]> Date: Sat Mar 11 23:13:11 2023 +0800 chore(sqlparser): add do-apply-parser-test (risingwavelabs#8486) Signed-off-by: TennyZhuang <[email protected]> commit a77f6cc Author: StrikeW <[email protected]> Date: Sat Mar 11 21:27:45 2023 +0800 fix(connector): fix logging after bump to log4j2 (risingwavelabs#8487) commit d07f2bb Author: Liang <[email protected]> Date: Sat Mar 11 13:13:12 2023 +0800 fix(compaction group): trivial adjust should also increase divide ver… (risingwavelabs#8484) commit 6797904 Author: Xuanwo <[email protected]> Date: Fri Mar 10 22:09:00 2023 +0800 chore: Bump OpenDAL to latest version (risingwavelabs#8481) Signed-off-by: Xuanwo <[email protected]> commit 38edae8 Author: Dylan <[email protected]> Date: Fri Mar 10 21:10:22 2023 +0800 feat(streaming): Temporal join part2 (risingwavelabs#8466) commit e18f243 Author: idx0-dev <[email protected]> Date: Fri Mar 10 20:33:44 2023 +0800 feat(connector): unified csv parser (risingwavelabs#8463) commit 2ae019d Author: Runji Wang <[email protected]> Date: Fri Mar 10 18:19:30 2023 +0800 test(recovery): add recovery test for nexmark stream (risingwavelabs#7623) Signed-off-by: Runji Wang <[email protected]> Co-authored-by: Liang <[email protected]> commit 25499e3 Author: Liang <[email protected]> Date: Fri Mar 10 17:54:11 2023 +0800 fix(recovery): wait_epoch should be called in recovery (close risingwavelabs#8467) (risingwavelabs#8468) commit 70f46f1 Author: Renjie Liu <[email protected]> Date: Fri Mar 10 16:59:25 2023 +0800 refactor: Remove state reporter for local mode (risingwavelabs#8477) commit eb24cda Author: Bugen Zhao <[email protected]> Date: Fri Mar 10 16:43:59 2023 +0800 fix: revert extension of scaling test timeout (risingwavelabs#8462) Signed-off-by: Bugen Zhao <[email protected]> commit 8d0e869 Author: Richard Chien <[email protected]> Date: Fri Mar 10 16:00:53 2023 +0800 refactor(common): unify order-related types (risingwavelabs#8449) Signed-off-by: Richard Chien <[email protected]> Co-authored-by: Bugen Zhao <[email protected]> commit 4b008ac Author: WillyKidd <[email protected]> Date: Fri Mar 10 15:21:52 2023 +0800 test(connector): add test cases for debezium json test (risingwavelabs#8334) Co-authored-by: StrikeW <[email protected]> commit 79b499c Author: stonepage <[email protected]> Date: Fri Mar 10 15:02:19 2023 +0800 refactor(optimizer): move some methods into core struct && refactor the join's predicate push down (risingwavelabs#8455) commit 64d80d2 Author: August <[email protected]> Date: Fri Mar 10 14:22:46 2023 +0800 feat(test): optimize set stmts in simulation to avoid duplicate replay (risingwavelabs#8420) commit b6244d7 Author: Eric Fu <[email protected]> Date: Fri Mar 10 13:43:09 2023 +0800 chore: update and add connector-node to docker-compose.yml (risingwavelabs#8427) commit 0a7a47e Author: Bohan Zhang <[email protected]> Date: Fri Mar 10 11:21:18 2023 +0700 chore: add license check path (risingwavelabs#8459) Signed-off-by: tabVersion <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> commit 614b6c5 Author: William Wen <[email protected]> Date: Fri Mar 10 11:41:18 2023 +0800 fix(sink): fix connector node sink json payload serialization (risingwavelabs#8461) commit 2f626d9 Author: xiangjinwu <[email protected]> Date: Fri Mar 10 07:40:09 2023 +0800 refactor(common): make certain `IntervalUnit` constructors test-only (risingwavelabs#8464) commit 86188ef Author: xxchan <[email protected]> Date: Thu Mar 9 23:20:59 2023 +0100 chore: change storage log level to WARN for playground (risingwavelabs#8406) commit 6f3eb54 Author: Shuxian Wang <[email protected]> Date: Thu Mar 9 06:38:47 2023 -0800 feat(temporal-join): Temporal join executor (risingwavelabs#8412) Co-authored-by: Dylan Chen <[email protected]> Co-authored-by: Dylan <[email protected]> commit 5294f43 Author: xiangjinwu <[email protected]> Date: Thu Mar 9 20:50:54 2023 +0800 refactor(common): cleanup unused methods on `IntervalUnit` (risingwavelabs#8456) commit f12b263 Author: Li0k <[email protected]> Date: Thu Mar 9 20:06:49 2023 +0800 feat(storage): monitor avg_key_size and avg_epoch_count (risingwavelabs#8297) commit 953e4b2 Author: ZENOTME <[email protected]> Date: Thu Mar 9 17:38:57 2023 +0800 feat(common): support from_binary and from_text in ScalarImpl (risingwavelabs#8421)
I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
Refactor the connector node to load a
.properties
file to initiate a Debzium source connector.debezium.properties
: common configs shared by all connectorsmysql.properties
: config for the MySQL connectorpostgres.properties
: config for the PG connectorWe can easily implement a new connector by adding a new config file and the required maven dependencies to the project.
Checklist For Contributors
./risedev check
(or alias,./risedev c
)Checklist For Reviewers
Documentation
fix #8471