added step callback for tracking generation #144
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
4 issues found across 4 files
Prompt for AI agents (all 4 issues)
Understand the root cause of the following 4 issues and fix them.
<file name="workflows/README.md">
<violation number="1" location="workflows/README.md:84">
The new docs/PROGRESS_TRACKING.md link points to a file that doesn’t exist, so following the README link will 404.</violation>
<violation number="2" location="workflows/README.md:121">
The structure diagram advertises progress_tracking_example.py, but that file doesn’t exist anywhere in the repo, which misleads readers.</violation>
</file>
<file name="workflows/workflow_use/healing/service.py">
<violation number="1" location="workflows/workflow_use/healing/service.py:831">
The final status update always reports 0 steps when no step callback is registered because the counter only increments inside the callback block. Increment the counter regardless of callback availability (or compute the count from history) before emitting the status update so progress is accurate.</violation>
</file>
<file name="tests/test_progress_tracking.py">
<violation number="1" location="tests/test_progress_tracking.py:79">
This unit test instantiates a real ChatOpenAI client, so running the test suite now requires an OpenAI API key and network access just to inspect method signatures. Replace the real LLM with a mock or stub so the test remains deterministic and credential-free.</violation>
</file>
Reply to cubic to teach it or ask questions. Re-run a review with @cubic-dev-ai review this PR
| │ │ ├── variables/ # Variable feature examples | ||
| │ │ ├── demos/ # Advanced demos | ||
| │ │ └── runner.py # Generic workflow runner | ||
| │ ├── progress_tracking_example.py # ⭐ NEW: Real-time progress tracking |
Contributor
There was a problem hiding this comment.
The structure diagram advertises progress_tracking_example.py, but that file doesn’t exist anywhere in the repo, which misleads readers.
Prompt for AI agents
Address the following comment on workflows/README.md at line 121:
<comment>The structure diagram advertises progress_tracking_example.py, but that file doesn’t exist anywhere in the repo, which misleads readers.</comment>
<file context>
@@ -98,6 +118,7 @@ workflows/
│ │ ├── variables/ # Variable feature examples
│ │ ├── demos/ # Advanced demos
│ │ └── runner.py # Generic workflow runner
+│ ├── progress_tracking_example.py # ⭐ NEW: Real-time progress tracking
│ └── workflows/ # Example workflow JSON files
│ ├── basic/ # Basic workflow examples
</file context>
|
|
||
| - **[docs/DETERMINISTIC.md](docs/DETERMINISTIC.md)** - Deterministic workflow generation | ||
| - **[docs/VARIABLES.md](docs/VARIABLES.md)** - Variables guide | ||
| - **[docs/PROGRESS_TRACKING.md](docs/PROGRESS_TRACKING.md)** - Real-time progress tracking ⭐ NEW |
Contributor
There was a problem hiding this comment.
The new docs/PROGRESS_TRACKING.md link points to a file that doesn’t exist, so following the README link will 404.
Prompt for AI agents
Address the following comment on workflows/README.md at line 84:
<comment>The new docs/PROGRESS_TRACKING.md link points to a file that doesn’t exist, so following the README link will 404.</comment>
<file context>
@@ -57,12 +57,32 @@ python cli.py run-workflow-no-ai my_workflow.json
- **[docs/DETERMINISTIC.md](docs/DETERMINISTIC.md)** - Deterministic workflow generation
- **[docs/VARIABLES.md](docs/VARIABLES.md)** - Variables guide
+- **[docs/PROGRESS_TRACKING.md](docs/PROGRESS_TRACKING.md)** - Real-time progress tracking ⭐ NEW
+- **[QUICK_START_PROGRESS_TRACKING.md](QUICK_START_PROGRESS_TRACKING.md)** - 5-minute integration guide
- **[examples/README.md](examples/README.md)** - Example scripts
</file context>
| print(f'✅ Agent completed. Captured {len(element_text_map)} element mappings total.') | ||
|
|
||
| if on_status_update: | ||
| on_status_update(f'Completed recording {step_counter["count"]} steps') |
Contributor
There was a problem hiding this comment.
The final status update always reports 0 steps when no step callback is registered because the counter only increments inside the callback block. Increment the counter regardless of callback availability (or compute the count from history) before emitting the status update so progress is accurate.
Prompt for AI agents
Address the following comment on workflows/workflow_use/healing/service.py at line 831:
<comment>The final status update always reports 0 steps when no step callback is registered because the counter only increments inside the callback block. Increment the counter regardless of callback availability (or compute the count from history) before emitting the status update so progress is accurate.</comment>
<file context>
@@ -665,15 +820,27 @@ async def act(self, action, browser_session, *args, **kwargs):
print(f'✅ Agent completed. Captured {len(element_text_map)} element mappings total.')
+ if on_status_update:
+ on_status_update(f'Completed recording {step_counter["count"]} steps')
+
# Store the history so it can be accessed externally (for result caching)
</file context>
✅ Addressed in 3d44586
tests/test_progress_tracking.py
Outdated
| from langchain_openai import ChatOpenAI | ||
|
|
||
| # This should work without callbacks | ||
| llm = ChatOpenAI(model="gpt-4o") |
Contributor
There was a problem hiding this comment.
This unit test instantiates a real ChatOpenAI client, so running the test suite now requires an OpenAI API key and network access just to inspect method signatures. Replace the real LLM with a mock or stub so the test remains deterministic and credential-free.
Prompt for AI agents
Address the following comment on tests/test_progress_tracking.py at line 79:
<comment>This unit test instantiates a real ChatOpenAI client, so running the test suite now requires an OpenAI API key and network access just to inspect method signatures. Replace the real LLM with a mock or stub so the test remains deterministic and credential-free.</comment>
<file context>
@@ -0,0 +1,324 @@
+ from langchain_openai import ChatOpenAI
+
+ # This should work without callbacks
+ llm = ChatOpenAI(model="gpt-4o")
+ service = HealingService(llm=llm)
+
</file context>
Contributor
There was a problem hiding this comment.
1 issue found across 14 files (reviewed changes from recent commits).
Prompt for AI agents (all 1 issues)
Understand the root cause of the following 1 issues and fix them.
<file name="workflows/examples/progress_tracking_example.py">
<violation number="1" location="workflows/examples/progress_tracking_example.py:8">
The new usage instructions reference `examples/progress_tracking_example.py`, but the script actually lives under `workflows/examples/`, so the documented command fails when run from the repo root. Update the path to the real file so the example can be executed as written.</violation>
</file>
Reply to cubic to teach it or ask questions. Re-run a review with @cubic-dev-ai review this PR
| callbacks to track workflow generation progress in real-time. | ||
| Usage: | ||
| python examples/progress_tracking_example.py |
Contributor
There was a problem hiding this comment.
The new usage instructions reference examples/progress_tracking_example.py, but the script actually lives under workflows/examples/, so the documented command fails when run from the repo root. Update the path to the real file so the example can be executed as written.
Prompt for AI agents
Address the following comment on workflows/examples/progress_tracking_example.py at line 8:
<comment>The new usage instructions reference `examples/progress_tracking_example.py`, but the script actually lives under `workflows/examples/`, so the documented command fails when run from the repo root. Update the path to the real file so the example can be executed as written.</comment>
<file context>
@@ -3,12 +3,15 @@
callbacks to track workflow generation progress in real-time.
+
+Usage:
+ python examples/progress_tracking_example.py
"""
</file context>
Suggested change
| python examples/progress_tracking_example.py | |
| python workflows/examples/progress_tracking_example.py |
…/browser-use/workflow-use into add-step-callback-for-generation
Contributor
There was a problem hiding this comment.
1 issue found across 16 files (reviewed changes from recent commits).
Prompt for AI agents (all 1 issues)
Understand the root cause of the following 1 issues and fix them.
<file name="workflows/tests/test_step_counter_without_callback.py">
<violation number="1" location="workflows/tests/test_step_counter_without_callback.py:17">
The test opens workflow_use/healing/service.py but that path does not exist in the repo (file lives under workflows/workflow_use/), so the test will always crash before asserting anything. Update the path to point to the actual file.</violation>
</file>
Reply to cubic to teach it or ask questions. Re-run a review with @cubic-dev-ai review this PR
| callback conditional block. | ||
| """ | ||
| # Read the service.py file and verify the counter increment is outside the callback check | ||
| with open('workflow_use/healing/service.py', 'r') as f: |
Contributor
There was a problem hiding this comment.
The test opens workflow_use/healing/service.py but that path does not exist in the repo (file lives under workflows/workflow_use/), so the test will always crash before asserting anything. Update the path to point to the actual file.
Prompt for AI agents
Address the following comment on workflows/tests/test_step_counter_without_callback.py at line 17:
<comment>The test opens workflow_use/healing/service.py but that path does not exist in the repo (file lives under workflows/workflow_use/), so the test will always crash before asserting anything. Update the path to point to the actual file.</comment>
<file context>
@@ -0,0 +1,108 @@
+ callback conditional block.
+ """
+ # Read the service.py file and verify the counter increment is outside the callback check
+ with open('workflow_use/healing/service.py', 'r') as f:
+ content = f.read()
+
</file context>
Suggested change
| with open('workflow_use/healing/service.py', 'r') as f: | |
| with open('workflows/workflow_use/healing/service.py', 'r') as f: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary by cubic
Added real-time progress tracking to workflow generation with optional step and status callbacks, enabling live UIs and better debugging. Also fixed the step counter so status messages report accurate step counts even when no callback is provided.
New Features
Bug Fixes
Written for commit 53fe4b2. Summary will update automatically on new commits.