Configuring Pull Request Draft Review Execution A Comprehensive Guide

In this comprehensive guide, we'll dive into the details of configuring pull request (PR) draft review execution within the Kodus platform. This feature allows teams to control whether automatic code reviews should run while a PR is in draft state, offering greater flexibility and control over the review process. So, let's get started, guys, and explore how to make the most of this powerful feature!

Overview of Pull Request Draft Review Configuration

Configuring pull request draft review execution is a key aspect of managing your team's code review workflow. Let's delve into the goal and key components of this feature. Our primary goal is to provide teams with the ability to configure whether automatic code reviews should run while a Pull Request (PR) is in a draft state, both globally and on a per-repository basis. By default, to maintain existing functionality, these reviews will continue to run. However, with this configuration, teams can choose to postpone reviews until a PR is out of the draft stage. The key to this configuration is the runOnDraft boolean parameter. When set to true (the default), reviews run as usual, even in the draft state. Setting it to false, however, will delay reviews until the PR is ready for formal review. This functionality will be available across multiple platforms, including GitHub, GitLab, Bitbucket, and Azure Repos, ensuring a consistent experience regardless of your chosen platform. This flexibility ensures that code reviews are conducted at the most appropriate time, reducing noise and improving the efficiency of the review process.

Scope of the Configuration

To effectively manage pull request draft reviews, understanding the scope of the configuration is crucial. The scope of this feature encompasses several key areas within the Kodus platform. Firstly, a configuration surface (runOnDraft) will be added in both web parameters and the kodus-config.yml file. This provides flexibility in how teams choose to manage their settings, whether through a user interface or via configuration files. Secondly, the system will be able to detect the draft state from each platform (GitHub, GitLab, Bitbucket, Azure Repos) and carry this information through the internal mapped PR object. This ensures that the system accurately understands the current state of the PR. Thirdly, review execution will be gated when a PR is in draft and runOnDraft is disabled. This means that automatic reviews will be skipped, but manual triggers via command will still be preserved, offering a balance between automation and manual control. Finally, a review will be triggered automatically when the PR transitions out of the draft state (e.g., GitHub's ready_for_review event). This ensures that no PR is missed once it’s ready for review, streamlining the workflow and ensuring timely feedback. Understanding these aspects of the scope helps in tailoring the configuration to fit specific team needs and workflows.

Configuration Model Details

The configuration model is central to how pull request draft reviews are managed within Kodus. Let's explore the types, schema, default YAML, and overrides and precedence that define this model. In terms of types, a runOnDraft?: boolean parameter is added to the CodeReviewConfig and KodusConfigFile (located in src/config/types/general/codeReview.type.ts), as well as to CodeReviewConfigWithoutLLMProviderDto (in src/core/infrastructure/http/dtos/create-or-update-code-review-parameter.dto.ts). This boolean allows you to specify whether reviews should run on draft PRs. For the schema, the codereview.json file will be updated to include runOnDraft with type: "boolean" and default: true (file path: src/shared/schemas/codereview.json). This ensures that the configuration is properly validated and that the default behavior is maintained. The config versioning in the validator (file: src/shared/utils/validateCodeReviewConfigFile.ts) will be aligned with the YAML default to prevent unnecessary deprecation warnings. In the default YAML configuration (default-kodus-config.yml), runOnDraft: true is added, ensuring that, by default, reviews will run on draft PRs. Understanding the overrides and precedence is crucial for managing configurations effectively. Per-repository values will override global (team) values, allowing for specific configurations for different projects. Additionally, if kodusConfigFileOverridesWebPreferences is set to true, the settings in kodus-config.yml will take precedence over UI-stored preferences, providing a clear hierarchy for configuration management. This model ensures flexibility and control, enabling teams to tailor their review processes to their specific needs.

runOnDraft: true

Draft Detection in the Mapped PR Model

Effective draft detection is vital for the proper execution of pull request draft reviews. The system needs to accurately identify when a PR is in a draft state to apply the configured settings. To achieve this, the IMappedPullRequest interface is extended with isDraft?: boolean (file location: src/core/domain/platformIntegrations/types/webhooks/webhooks-common.type.ts). This boolean flag indicates whether the PR is currently in draft. The next step is to populate this isDraft flag in the platform mappers, which are responsible for translating platform-specific events into a common format. For GitHub, this is done in src/shared/utils/webhooks/github.ts by setting isDraft = !!payload.pull_request.draft. Similarly, for Azure Repos, the draft status is determined in src/shared/utils/webhooks/azureRepos.ts with isDraft = !!pullRequest.isDraft. It's important to note that draft state detection is platform-dependent. Currently, GitLab and Bitbucket do not offer native draft functionality, meaning the isDraft flag will not be applicable for these platforms. This platform-specific handling ensures that the system accurately reflects the state of the PR, allowing for correct application of the runOnDraft configuration. Accurate draft detection is a foundational element for the successful implementation of this feature, ensuring that reviews are skipped or triggered as intended.

Execution Gating Skipping Reviews in Draft

To effectively skip reviews in the draft state when configured, a robust execution gating mechanism is essential. The primary gate for this functionality is implemented in the RunCodeReviewAutomationUseCase.execute method (file path: src/ee/automation/runCodeReview.use-case.ts). The process begins by locating the team and mapping the PR. Once this is done, the configuration is fetched via CodeBaseConfigService.getConfig(...). The core logic then checks if the review origin is not a manual command (origin !== 'command') and if runOnDraft is disabled (!config.runOnDraft) and the PR is in draft (pullRequest.isDraft). If all these conditions are met, the execution is skipped, and the reason for skipping is logged. This ensures that automatic reviews are effectively prevented for draft PRs when the configuration dictates. An optional defense-in-depth measure is also implemented in the pipeline within the validate-config.stage.ts file (located in src/core/infrastructure/adapters/services/codeBase/codeReviewPipeline/stages/validate-config.stage.ts). In the shouldProcessPR(...) method, the same conditions are checked, and if met, the method returns false, further ensuring that the review is skipped. This dual-layer approach provides a robust mechanism for preventing reviews on draft PRs, aligning with the configured preferences and improving workflow efficiency.

Triggering Reviews Upon Leaving Draft

Ensuring reviews are triggered automatically when a pull request leaves the draft state is a crucial aspect of this feature. This functionality guarantees that no PR is missed once it's ready for review. The implementation varies slightly depending on the platform due to differences in how each platform handles draft transitions. For GitHub, the system accepts the pull_request action ready_for_review. This involves whitelisting this action in the controller (file: src/core/infrastructure/http/controllers/github.controller.ts), the handler's allowedActions (file: src/core/infrastructure/adapters/webhooks/github/githubPullRequest.handler.ts), and the automation's shouldRunAutomation allowedActions (file: src/ee/automation/runCodeReview.use-case.ts). By accepting this action, the system is notified when a PR is marked as ready for review, triggering the review process. For Azure Repos, the approach is a bit different. On the git.pullrequest.updated event, the system detects the transition from draft to ready for review by checking if isDraft flips from true to false. When this transition is detected, the automation is triggered. This method ensures that even without a specific ready_for_review event, the system can identify when a PR is ready for review. These platform-specific mechanisms ensure that reviews are promptly triggered when a PR leaves the draft state, streamlining the review process and enhancing team collaboration.

Manual Command Behavior for Code Reviews

To provide flexibility and control, the @kody start-review command is designed to bypass the draft gating mechanism. Regardless of the runOnDraft configuration, when a user manually triggers a code review using this command, the review will always be executed. This behavior is intentional, allowing users to initiate reviews even when a PR is in draft if they deem it necessary. This is particularly useful in situations where a preliminary review is needed, or when specific aspects of the code need to be examined before the PR is fully ready. The manual command serves as an override, ensuring that users retain the ability to initiate reviews at any stage of the pull request lifecycle. This approach balances the automation provided by the runOnDraft configuration with the need for manual intervention and control, providing a comprehensive and adaptable code review workflow.

Logging and Telemetry for Monitoring and Debugging

Effective logging and telemetry are critical for monitoring the behavior of the pull request draft review feature and for debugging any potential issues. When a review is skipped due to the draft state, detailed logs are generated to provide insights into the reasons for the skip. These logs include the following information: { reason: 'skipped-draft', runOnDraft, isDraft, platformType, prNumber, repositoryId }. This data allows developers to quickly understand why a review was skipped, whether it was due to the runOnDraft configuration, the draft status of the PR, or other factors. Similarly, when a review is triggered upon a PR leaving the draft state, logs are generated with the following structure: { reason: 'ready-for-review', platformType, prNumber, repositoryId }. These logs confirm that the review was triggered as expected and provide context about the platform, PR number, and repository. The combination of these logs provides a comprehensive view of the feature's operation, enabling proactive monitoring and efficient troubleshooting. By capturing these events, teams can ensure that the feature is functioning correctly and can quickly address any discrepancies or unexpected behavior.

Backwards Compatibility and Versioning for Seamless Transitions

Maintaining backwards compatibility and managing versioning are essential for ensuring a smooth transition and preventing disruptions for existing users. The default setting of runOnDraft = true is crucial for preserving the current behavior for teams already using the Kodus platform. This means that, out of the box, there will be no change in how code reviews are triggered, minimizing any potential impact on existing workflows. Aligning the validator's currentVersion with the version in default-kodus-config.yml (e.g., 1.3) is also important. This alignment prevents false deprecation warnings, which can be confusing and unnecessary for users. Older configuration files (e.g., 1.2) will continue to be accepted by the system. In cases where the runOnDraft setting is not explicitly present in these older files, the system will fall back to the default behavior of true. This ensures that older configurations remain valid and functional, preventing any unexpected changes in behavior. This careful approach to backwards compatibility and versioning ensures a seamless experience for all users, regardless of when they adopted the Kodus platform or how their configurations are set up. This thoughtful approach minimizes disruptions and ensures a consistent experience during and after the update.

Acceptance Criteria for Feature Validation

To ensure the pull request draft review configuration feature works as expected, specific acceptance criteria have been defined. These criteria serve as a checklist to validate that the feature meets the intended functionality and behaves correctly across various scenarios. Firstly, with runOnDraft = false, no automatic review should run while a PR is in the draft state across all supported platforms (GitHub, Azure Repos, GitLab, and Bitbucket). However, the @kody start-review command should still function, allowing for manual review initiation. Secondly, a review must run automatically when a PR transitions out of the draft state, such as the ready_for_review event on GitHub or equivalent events on other platforms. This ensures that reviews are triggered promptly when a PR is ready. Thirdly, repository-level configurations should override global configurations, providing flexibility for teams to customize settings for individual projects. Additionally, when enabled, settings in the kodus-config.yml file should override UI-stored preferences, offering a clear hierarchy for configuration management. Finally, there should be no change in behavior when runOnDraft = true, maintaining the existing functionality for teams that prefer to review draft PRs. Meeting these acceptance criteria ensures that the feature is robust, flexible, and aligns with the needs of different teams and workflows.

Test Plan for Ensuring Quality and Reliability

A comprehensive test plan is crucial for ensuring the quality and reliability of the pull request draft review configuration feature. The test plan encompasses both unit tests and integration tests to validate different aspects of the feature. Unit tests are designed to verify the correctness of individual components and functionalities. These tests include: testing the config merge logic to ensure that global, repository, and YAML configurations are correctly merged with the appropriate precedence; verifying that platform mappers set the isDraft flag correctly for GitHub and Azure Repos; confirming that the gating mechanism in RunCodeReviewAutomationUseCase respects the runOnDraft configuration and the origin of the review (manual vs. automatic); and optionally, testing the gating mechanism in ValidateConfigStage.shouldProcessPR. Integration tests, on the other hand, focus on testing the interaction between different components and the overall behavior of the feature. These tests include: verifying that a pull_request with ready_for_review on GitHub triggers a review when runOnDraft = false; ensuring that a leaving-draft transition on Azure Repos triggers a review without requiring new commits; and confirming that the manual @kody start-review command bypasses the gating mechanism. This combination of unit and integration tests provides a thorough validation of the feature, ensuring that it functions correctly in various scenarios and meets the defined acceptance criteria. Thorough testing is essential for delivering a robust and reliable feature that enhances the code review workflow.

Implementation Tasks Checklist

To ensure the successful implementation of the pull request draft review configuration feature, a detailed checklist of tasks has been created. This checklist provides a structured approach to the development process, ensuring that all necessary steps are completed. The tasks include: adding runOnDraft to the CodeReviewConfig, KodusConfigFile, and the DTO (CreateOrUpdateCodeReviewParameterDto); updating codereview.json and ensuring validator version alignment; adding runOnDraft: true to default-kodus-config.yml; extending IMappedPullRequest with isDraft and populating it in all platform mappers; adding gating logic in RunCodeReviewAutomationUseCase.execute and optionally in ValidateConfigStage; accepting ready_for_review events from GitHub in the controller, handler, and automation; implementing logic to detect leaving-draft events on Azure Repos and trigger reviews; and adding both unit and integration tests as per the test plan. This checklist serves as a roadmap for the development team, ensuring that each aspect of the feature is addressed and implemented correctly. By following this structured approach, the implementation process is streamlined, and the likelihood of overlooking important details is minimized.

Out of Scope Considerations

To maintain focus and prevent scope creep, certain items have been explicitly defined as out of scope for this feature. Changes to approval rules or merge strategies that are unrelated to draft handling are considered out of scope. The primary focus is on controlling the execution of code reviews based on the draft status of a pull request, and modifications to broader approval or merging processes are not part of this effort. Additionally, UI changes beyond exposing the new runOnDraft toggle, if needed, are also out of scope. The intention is to minimize the user interface impact and concentrate on the core functionality of draft review configuration. This clear delineation of scope helps the development team stay on track and ensures that the feature is delivered efficiently and effectively, addressing the specific goals without unnecessary complexity. This focused approach ensures that the core functionality is robust and well-implemented.

Conclusion

In conclusion, this comprehensive guide has walked you through the intricacies of configuring pull request draft review execution within the Kodus platform. By understanding the goals, scope, configuration model, draft detection, execution gating, trigger mechanisms, manual command behavior, logging, backwards compatibility, acceptance criteria, test plan, implementation tasks, and out-of-scope considerations, you're well-equipped to leverage this powerful feature effectively. So go ahead, guys, and tailor your code review workflow to perfection!