WRKLDS-1191: oc rsync: Add --last flag

[tchap]

The flag can be used to only sync N most recently modified files.

The flag is mutually exclusive to:

• --include
• --exclude
• --delete
• --watch

It is also ignored when the source is a local directory when using tar, because the implementation doesn't allow to select particular files. We would need to exclude all other files, which can explode easily. Simply use a different strategy in that case.

Generally when there is any problem when using --last, the error is ignored and sync happens as if the flag was not specified.

Regarding implementation details, oc rsync performs an extras step when --last is specified, and that is discovering relevant files to select. This is done using manual directory walking when local, for remote the remote executor is used to invoke a shell using find+sort+head.

The resulting filenames are then passed to --files-from for rsync, for tar they are simply passed to the command as arguments.

Tests were added for testing the discovery mechanism, the rest has been tested manually. oc rsync is rather poorly unit-tested in general.

[openshift-ci-robot]

@tchap: This pull request references WRKLDS-1191 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set.

Details

In response to this:

The flag can be used to only sync N most recently modified files.

The flag is mutually exclusive to:

• --include
• --exclude
• --delete
• --watch

It is also ignored when the source is a local directory when using tar, because the implementation doesn't allow to select particular files. We would need to exclude all other files, which can explode easily. Simply use a different strategy in that case.

Generally when there is any problem when using --last, the error is ignored and sync happens as if the flag was not specified.

Regarding implementation details, oc rsync performs an extras step when --last is specified, and that is discovering relevant files to select. This is done using manual directory walking when local, for remote the remote executor is used to invoke a shell using find+sort+head.

The resulting filenames are then passed to --files-from for rsync, for tar they are simply passed to the command as arguments.

Tests were added for testing the discovery mechanism, the rest has been tested manually. oc rsync is rather poorly unit-tested in general.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@tchap: This pull request references WRKLDS-1191 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set.

Details

In response to this:

The flag can be used to only sync N most recently modified files.

The flag is mutually exclusive to:

• --include
• --exclude
• --delete
• --watch

It is also ignored when the source is a local directory when using tar, because the implementation doesn't allow to select particular files. We would need to exclude all other files, which can explode easily. Simply use a different strategy in that case.

Generally when there is any problem when using --last, the error is ignored and sync happens as if the flag was not specified.

Regarding implementation details, oc rsync performs an extras step when --last is specified, and that is discovering relevant files to select. This is done using manual directory walking when local, for remote the remote executor is used to invoke a shell using find+sort+head.

The resulting filenames are then passed to --files-from for rsync, for tar they are simply passed to the command as arguments.

Tests were added for testing the discovery mechanism, the rest has been tested manually. oc rsync is rather poorly unit-tested in general.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: tchap
Once this PR has been reviewed and has the lgtm label, please assign ardaguclu for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[tchap]

I guess this can also wait for 4.21 as nobody is going to use it in 4.20 anyway...

[tchap]

/test okd-scos-e2e-aws-ovn
/test e2e-metal-ipi-ovn-ipv6
/test e2e-aws-ovn

[tchap]

/retest

[openshift-ci]

@tchap: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[coderabbitai]

Important

Review skipped

Auto reviews are limited based on label configuration.

🚫 Review skipped — only excluded labels are configured. (1)

• do-not-merge/work-in-progress

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}