Skip to content

docs(dm): explain printing source config as YAML - #24686

Open
onlyxiaohuang wants to merge 1 commit into
pingcap:masterfrom
onlyxiaohuang:docs/dm-source-config-jq
Open

onlyxiaohuang wants to merge 1 commit into
pingcap:masterfrom
onlyxiaohuang:docs/dm-source-config-jq

Conversation

@onlyxiaohuang

@onlyxiaohuang onlyxiaohuang commented Nov 12, 2026 •

Copy link
Copy Markdown

First-time contributors' checklist

What is changed, added, or deleted? (Required)

Document how to extract the YAML in the cfg field with jq -r '.cfg', including an example that saves the result to source.yaml. Also clarify the get-config syntax for DM versions earlier than v6.0. This addresses the documentation direction discussed in pingcap/tiflow#3937; command output behavior is unchanged.

Which TiDB version(s) do your changes apply to? (Required)

  • master (the latest development version)
  • v9.0 (TiDB 9.0 versions)
  • v8.5 (TiDB 8.5 versions)
  • v8.1 (TiDB 8.1 versions)
  • v7.5 (TiDB 7.5 versions)
  • v7.1 (TiDB 7.1 versions)
  • v6.5 (TiDB 6.5 versions)

What is the related PR or file link(s)?

AI agent involvement

  • The changes in this PR were primarily made by an AI agent on behalf of the PR author.

Do your changes match any of the following descriptions?

  • Delete files
  • Change aliases
  • Need modification after applied to another branch
  • Might cause conflicts after applied to another branch

Summary by CodeRabbit

  • Documentation
    • Clarified how to extract and save YAML configuration from data-source output.
    • Added guidance to use get-config source before DM v6.0.

Document how to extract the YAML config with jq and note the legacy command for DM versions before v6.0.
@ti-chi-bot

ti-chi-bot Bot commented Nov 12, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign kissmydb for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added contribution This PR is from a community contributor. first-time-contributor Indicates that the PR was contributed by an external member and is a first-time contributor. labels Nov 12, 2026
@ti-chi-bot

ti-chi-bot Bot commented Nov 12, 2026

Copy link
Copy Markdown

Hi @onlyxiaohuang. Thanks for your PR.

I'm waiting for a pingcap member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@ti-chi-bot ti-chi-bot Bot added the needs-ok-to-test Indicates a PR created by contributors and need ORG member send '/ok-to-test' to start testing. label Nov 12, 2026
@ti-chi-bot

ti-chi-bot Bot commented Nov 12, 2026

Copy link
Copy Markdown

Welcome @onlyxiaohuang!

It looks like this is your first PR to pingcap/docs 🎉.

I'm the bot to help you request reviewers, add labels and more, See available commands.

We want to make sure your contribution gets all the attention it needs!



Thank you, and welcome to pingcap/docs. 😃

@ti-chi-bot ti-chi-bot Bot added missing-translation-status This PR does not have translation status info. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Nov 12, 2026
@pingcap-cla-assistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@coderabbitai

coderabbitai Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Explain Change →

Walkthrough

The DM source inspection instructions now show how to extract the YAML configuration from command output and specify the command to use before DM v6.0.

Changes: DM source documentation

Layer / File(s) Summary
Source configuration extraction
dm/dm-manage-source.md
The instructions add a jq -r '.cfg' pipeline for formatting or saving the configuration. They specify get-config source <source-id> for DM versions before v6.0.

Estimated code review effort: 1 (Trivial) | ~3 minutes

Change: Other

Possibly related issues

Suggested reviewers: qiancai

Merge Risk: Low · up to a75f2

The new documentation example writes extracted configuration to source.yaml, which is also the file name used earlier as the user's source configuration input. Following it literally can overwrite that file with masked output. Change the example to a different file name before merging; no runtime behavior is affected.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title "docs(dm): explain printing source config as YAML" matches the main change. The diff documents extracting YAML from the cfg field of config source output with jq -r '.cfg', and it documents …
Description check Passed The description follows the repository template. It fills in the required 'What is changed' section, the version checkboxes (master only), the related reference link, and the AI agent involvement sect…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title "docs(dm): explain printing source config as YAML" matches the main change. The diff documents extracting YAML from the cfg field of config source output with jq -r '.cfg', and it documents get-config syntax for DM versions before v6.0. The title covers the primary change and omits the get-config clarification, which is acceptable.

Full details: Description check

Explanation

The description follows the repository template. It fills in the required 'What is changed' section, the version checkboxes (master only), the related reference link, and the AI agent involvement section. It removes the first-time contributor checklist and the related-PR section headings are reduced, but all required information is present. The contributor license checkbox is left unchecked, which is a minor gap.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Additional comments: 1

dm/dm-manage-source.md: 1

113-120: Minor · Functional Correctness · Quick win

Unverified finding
Verification ran but could not confirm this finding. It is shown for review, not as a verified issue.

Correct the JSON output example.

The cfg string contains unescaped line breaks and has no closing double quote. jq cannot parse this example. Update it to show the actual JSON output, including escaped line breaks.

As per coding guidelines, “Commands, code, configuration names, API fields, UI strings, and literal values remain correct.”

Source: Coding guidelines

Review info

Run configuration

  • Source mode: Auto
  • Repository: pingcap/docs
  • Branch: master
  • File: .coderabbit.yaml
  • Commit: 0696463
  • Layers: Repository YAML
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 54809a24-d329-4f24-895e-00965dd82990
Commits

Reviewing files that changed from the base of the PR and between 0696463 and a75f269.

Files selected for processing: 1

  • dm/dm-manage-source.md

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.

Comment thread dm/dm-manage-source.md
> The `config` command is only supported in DM v6.0 and later versions. For earlier versions, you must use the `get-config` command.

If you know the `source-id`, you can run `dmctl --master-addr <master-addr> config source <source-id>` to get the data source configuration.
If you know the `source-id`, you can run `dmctl --master-addr <master-addr> config source <source-id>` to get the data source configuration. To print the YAML with line breaks or save it to a file, pipe the output to `jq -r '.cfg'`, for example, `dmctl --master-addr <master-addr> config source <source-id> | jq -r '.cfg' > source.yaml`. Before DM v6.0, use `get-config source <source-id>` instead of `config source <source-id>`.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major · Data Integrity & Integration · Quick win

Avoid overwriting source.yaml with the extracted configuration.

The earlier operate-source create ./source.yaml example uses source.yaml as the source configuration file. The > source.yaml redirection in this paragraph truncates that file before dmctl runs, even if the command or jq fails. The printed cfg also masks passwords (password: '******'), so the overwritten file can lose its credentials.

Write the extracted configuration to a different file name, and use the <source-id> placeholder already in the example:

Suggested fix

--- "a/dm/dm-manage-source.md"
+++ "b/dm/dm-manage-source.md"
@@ -98,7 +98,7 @@
 >
 > The `config` command is only supported in DM v6.0 and later versions. For earlier versions, you must use the `get-config` command.
 
-If you know the `source-id`, you can run `dmctl --master-addr <master-addr> config source <source-id>` to get the data source configuration. To print the YAML with line breaks or save it to a file, pipe the output to `jq -r '.cfg'`, for example, `dmctl --master-addr <master-addr> config source <source-id> | jq -r '.cfg' > source.yaml`. Before DM v6.0, use `get-config source <source-id>` instead of `config source <source-id>`.
+If you know the `source-id`, you can run `dmctl --master-addr <master-addr> config source <source-id>` to get the data source configuration. To print the YAML with line breaks or save it to a file, pipe the output to `jq -r '.cfg'`, for example, `dmctl --master-addr <master-addr> config source <source-id> | jq -r '.cfg' > <source-id>.yaml`. Before DM v6.0, use `get-config source <source-id>` instead of `config source <source-id>`.
 
 {{< copyable "" >}}
 
Committable suggestion

IMPORTANT
Review the replacement and indentation, then test before committing.

Suggested change
If you know the `source-id`, you can run `dmctl --master-addr <master-addr> config source <source-id>` to get the data source configuration. To print the YAML with line breaks or save it to a file, pipe the output to `jq -r '.cfg'`, for example, `dmctl --master-addr <master-addr> config source <source-id> | jq -r '.cfg' > source.yaml`. Before DM v6.0, use `get-config source <source-id>` instead of `config source <source-id>`.
If you know the `source-id`, you can run `dmctl --master-addr <master-addr> config source <source-id>` to get the data source configuration. To print the YAML with line breaks or save it to a file, pipe the output to `jq -r '.cfg'`, for example, `dmctl --master-addr <master-addr> config source <source-id> | jq -r '.cfg' > <source-id>.yaml`. Before DM v6.0, use `get-config source <source-id>` instead of `config source <source-id>`.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contribution This PR is from a community contributor. first-time-contributor Indicates that the PR was contributed by an external member and is a first-time contributor. missing-translation-status This PR does not have translation status info. needs-ok-to-test Indicates a PR created by contributors and need ORG member send '/ok-to-test' to start testing. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant