Refactor: Clean up compaction config classes #16810
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
...rc/test/java/org/apache/druid/tests/coordinator/duty/ITAutoCompactionLockContentionTest.java
Dismissed
Show dismissed
Hide dismissed
| .forDataSource(UPGRADE_DATASOURCE_NAME) | ||
| .withSkipOffsetFromLatest(newSkipOffset) | ||
| .withTuningConfig( | ||
| new UserCompactionTaskQueryTuningConfig( |
There was a problem hiding this comment.
A builder can be introduced for this one as well in future.
There was a problem hiding this comment.
Yeah, it's already in the works. 😄
| private final DataSourceCompactionConfig compactionConfig; | ||
| private final AuditInfo auditInfo; | ||
| private final DateTime auditTime; | ||
|
|
||
| @JsonCreator | ||
| public DataSourceCompactionConfigAuditEntry( | ||
| @JsonProperty("globalConfig") GlobalCompactionConfig globalConfig, | ||
| @JsonProperty("globalConfig") ClusterCompactionConfig globalConfig, |
There was a problem hiding this comment.
| @JsonProperty("globalConfig") ClusterCompactionConfig globalConfig, | |
| @JsonProperty("globalConfig") ClusterCompactionConfig clusterCompactionConfig, |
| @@ -50,7 +51,7 @@ public DataSourceCompactionConfigAuditEntry( | |||
| } | |||
|
|
|||
| @JsonProperty | |||
| public GlobalCompactionConfig getGlobalConfig() | |||
| public ClusterCompactionConfig getGlobalConfig() | |||
There was a problem hiding this comment.
| public ClusterCompactionConfig getGlobalConfig() | |
| public ClusterCompactionConfig getClusterCompactionConfig() |
There was a problem hiding this comment.
globalConfig is also a term used by the web-console.
Renaming this getter would require us to use @JsonProperty("globalConfig") on the getter for backwards compatibility. Leaving the names as is seemed cleaner.
Let me know if you feel strongly about this.
| /** | ||
| * A DTO containing audit information for compaction config for a datasource. | ||
| */ | ||
| public class DataSourceCompactionConfigAuditEntry | ||
| { | ||
| private final GlobalCompactionConfig globalConfig; | ||
| private final ClusterCompactionConfig globalConfig; |
There was a problem hiding this comment.
Consistent naming may be better. Can use older annotation for getter property name.
| private final ClusterCompactionConfig globalConfig; | |
| private final ClusterCompactionConfig clusterCompactionConfig; |
| ); | ||
| } | ||
|
|
||
| public Optional<DataSourceCompactionConfig> findConfigForDatasource(String dataSource) |
There was a problem hiding this comment.
Thanks for this handy function!
server/src/main/java/org/apache/druid/server/coordinator/DruidCompactionConfig.java
Outdated
Show resolved
Hide resolved
| @JsonProperty("compactionTaskSlotRatio") @Nullable Double compactionTaskSlotRatio, | ||
| @JsonProperty("maxCompactionTaskSlots") @Nullable Integer maxCompactionTaskSlots, | ||
| @JsonProperty("useAutoScaleSlots") @Nullable Boolean useAutoScaleSlots, | ||
| @JsonProperty("compactionEngine") @Nullable CompactionEngine compactionEngine | ||
| @JsonProperty("engine") @Nullable CompactionEngine engine | ||
| ) | ||
| { |
There was a problem hiding this comment.
I think the constructor should take in null values but create a compaction config with the non-null default fields.
Otherwise the comparison in the Audit history can be incorrect when there are null values but the default values change
There was a problem hiding this comment.
I considered that. But that becomes problematic since this payload is used to update cluster configs.
If we assign non-null values in the constructor, it would seem that the values are being updated even if we don't want them to be updated.
The audit entry thing is not an issue as there we always use non-null values.
See DruidCompactionConfig.clusterConfig() method which always returns non-null values.
There was a problem hiding this comment.
Thanks for the clarification!
|
Thanks a lot for the prompt reviews, @AmatyaAvadhanula , @gargvishesh !! |
Changes: - Rename `CoordinatorCompactionConfig` to `DruidCompactionConfig` - Rename `CompactionConfigUpdateRequest` to `ClusterCompactionConfig` - Refactor methods in `DruidCompactionConfig` - Clean up `DataSourceCompactionConfigHistory` and its tests - Clean up tests and add new tests - Change API path `/druid/coordinator/v1/config/global` to `/druid/coordinator/v1/config/cluster`
Changes
CoordinatorCompactionConfigtoDruidCompactionConfigCompactionConfigUpdateRequesttoClusterCompactionConfigDruidCompactionConfigDataSourceCompactionConfigHistoryand its tests/druid/coordinator/v1/config/globalto/druid/coordinator/v1/config/clusterThis PR has: