Summary
When the safe_outputs job fails to submit a PR review (e.g. GitHub API returns 422), the conclusion job still updates the PR comment with "✅ completed successfully!" because it only checks Agent Conclusion and Detection Conclusion, not whether safe outputs were actually delivered.
This makes it look like the reviewer ran and found nothing, when in reality it generated comments that failed to post.
Repro
- Trigger an agentic workflow that uses
safe-outputs with submit_pull_request_review
- If the GitHub API returns a transient 422 error during PR review submission, the
safe_outputs job logs ##[error] but the conclusion job still reports success
Example run: https://github.com/dotnet/android/actions/runs/25446317518
Logs from safe_outputs job
Submitting PR review on dotnet/android#11254: event=COMMENT, comments=3, bodyLength=1962
POST /repos/dotnet/android/pulls/11254/reviews - 422 with id 11C0:1C2012:7B8F57:1D76664:69FB6710 in 626ms
##[error]Failed to submit PR review: Unprocessable Entity: "An internal error occurred, please try again."
Logs from conclusion job
Agent Conclusion: success
Detection Conclusion: success
Successfully updated comment
The conclusion job never checks the safe_outputs job result.
Expected behavior
The conclusion comment should show an error (e.g. ❌ or ⚠️) when safe_outputs fails to deliver its outputs, so the user knows to re-trigger the workflow.
Actual behavior
The comment shows "✅ completed successfully!" even though no review comments were posted to the PR.
Summary
When the
safe_outputsjob fails to submit a PR review (e.g. GitHub API returns 422), theconclusionjob still updates the PR comment with "✅ completed successfully!" because it only checksAgent ConclusionandDetection Conclusion, not whether safe outputs were actually delivered.This makes it look like the reviewer ran and found nothing, when in reality it generated comments that failed to post.
Repro
safe-outputswithsubmit_pull_request_reviewsafe_outputsjob logs##[error]but theconclusionjob still reports successExample run: https://github.com/dotnet/android/actions/runs/25446317518
Logs from
safe_outputsjobLogs from
conclusionjobThe
conclusionjob never checks thesafe_outputsjob result.Expected behavior
The conclusion comment should show an error (e.g. ❌ or⚠️ ) when
safe_outputsfails to deliver its outputs, so the user knows to re-trigger the workflow.Actual behavior
The comment shows "✅ completed successfully!" even though no review comments were posted to the PR.