Configuration

Before running the command:

• Enable DynamoDB Streams on the source table with NEW_IMAGE or NEW_AND_OLD_IMAGES.
• Create the target table with the same partition key and optional sort key names and data types as the source table.

This requirement applies only to key attributes. DynamoDB non-key attributes are not declared when the table is created, and ddb-migrate preserves their data types and values.

Configuration file

ddb-migrate.toml:

# ddb-migrate 先全量扫描 AWS DynamoDB 源表，再持续读取 DynamoDB Streams。

[source]
access_key_id = ""                        # AK/SK 都为空时使用 AWS 默认凭证链
secret_access_key = ""
region = "us-east-1"                     # 必填，不读取 AWS_REGION
table = "source_table"
checkpoint_enabled = true                 # 启用断点续传
checkpoint_file_path = "checkpoint.json"
scan_segments = 4                         # 全量 Scan 并行分段数

[target]
access_key_id = "username:password"        # Tair 使用“账号:密码”格式
secret_access_key = "dummy"               # 固定占位值
region = "us-east-1"                     # 自定义 endpoint 可省略，默认 us-east-1
endpoint_url = "http://your-endpoint:80"  # 自定义 endpoint 默认跳过 TLS 验证
table = "target_table"
workers = 4                                # 目标端并发写入 worker 数

The example lists every field. An empty string and an omitted string field have the same effect; omit Boolean and integer fields directly. The tables below describe the result. The [source] and [target] credentials are independent.

source

The migration source always uses official AWS DynamoDB and DynamoDB Streams endpoints and has no endpoint_url field.

Option	Purpose	Omission rules and result
access_key_id, secret_access_key	Source AWS credentials	Omit both or leave both empty to use the AWS default credential chain. Set both to use static AK/SK. Setting only one fails at startup.
region	AWS Region containing the source table	Required. It must be set in TOML even when AWS_REGION exists in the runtime environment.
table	Source table name	Required.
checkpoint_enabled	Persist full-scan and incremental progress	Optional; the default is false. When disabled, an interrupted run cannot resume and the next run performs the full Scan again.
checkpoint_file_path	Checkpoint file path	Optional; the default is checkpoint.json. The file is unused when checkpoint_enabled = false.
scan_segments	Number of parallel full-Scan segments	Optional; the default is 4, and values less than or equal to 0 also use 4. Changing it prevents an incomplete old Scan checkpoint from being resumed, so the source table is scanned again.

target

Option	Purpose	Omission rules and result
access_key_id, secret_access_key	Target credentials	Omit both for an AWS target using the default credential chain. For Tair, set both: use account:password for access_key_id and the fixed value dummy for secret_access_key. Setting only one fails at startup.
region	Region of an official AWS endpoint, or signing Region for a custom endpoint	Required when endpoint_url is empty. Optional when endpoint_url is non-empty; the default is us-east-1. It can be omitted for Tair.
endpoint_url	Target DynamoDB API address	Optional for an official AWS DynamoDB target, where region selects the endpoint. Required for Tair.
table	Target table name	Required.
workers	Number of concurrent target write workers	Optional; the default is 4, and values less than or equal to 0 also use 4.

For a Tair target, the minimum required fields are access_key_id, secret_access_key, endpoint_url, and table. Both region and workers can be omitted.

AWS credential sources

When AK and SK are both non-empty in TOML, the program uses only those static credentials. When both fields are omitted or empty, the program uses the AWS SDK for Go v2 default credential chain.

Runtime	AK/SK in TOML	Runtime configuration
Environment variables	Omit both or leave both empty	Set AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY; also set AWS_SESSION_TOKEN for temporary credentials.
Web Identity	Omit both or leave both empty	Let the runtime provide Web Identity settings, such as an IAM role for an EKS Pod.
AWS profile or IAM Identity Center	Omit both or leave both empty	Use ~/.aws/credentials and ~/.aws/config; select a named profile with AWS_PROFILE and complete the login required by that profile.
ECS task role	Omit both or leave both empty	Assign an IAM role to the ECS task.
EC2 instance role	Omit both or leave both empty	Assign an IAM role to the EC2 instance; the SDK obtains temporary credentials from the instance metadata service.

The default chain checks environment variables, Web Identity, shared credentials/config files, an ECS task role, and an EC2 instance role in order, and uses the first credentials it obtains. AK/SK environment variables therefore override an EC2 instance role. They also override a profile selected with AWS_PROFILE.

For example, on an EC2 instance with an IAM role, remove access_key_id and secret_access_key from [source] and keep region and table. If the target is Tair, its Tair account and password and the dummy value are still required in [target].

Example using a named profile:

AWS_PROFILE=production ./ddb-migrate ddb-migrate.toml

If the default chain finds no credentials, the first AWS API request fails. See Configure the SDK - AWS SDK for Go v2 for the complete loading order.

Start

./ddb-migrate ddb-migrate.toml