Uploaded image for project: 'Yetus'
  1. Yetus
  2. YETUS-681

Add robots plugin type

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.9.0
    • Precommit
    • None
    • Hide
      <!-- markdown -->

      This change makes significant changes to precommit to support a variety of automation systems for continuous integration (aka robots or the `--robot` flag) as well as rework the existing Jenkins support (`--jenkins`).

      End User Changes
      ================

      * `--jenkins` flag has been removed. `test-patch`/`qbt` should now autodetect not only if it is running Jenkins, but also Circle CI, Gitlab CI, and Travis CI. Please see the robots document for specifics as to how that works, limitations, etc.
      * In the specific case of Jenkins, much better support for general Jenkins settings but also Github Branch Source Plugin using Pipelines and Github Pull Request Plugin. In many cases, options that were previously set specifically on the command line may be unnecessary/duplicate information and should be removed.
      * Configuration for how to run Apache Yetus against Apache Yetus for various CI systems is now bundled with the project source.
      * When running under a robot, `test-patch`/`qbt` may update the git remote repository information for branches. This is done because in some configurations, the information is missing and is required for proper functionality. In cases where this is not possible, the programs will treat the remote repositories as offline or the local repository as shallow. As a result, some functionality may not be available. The limitations may include `test-patch` functioning as though it was `qbt`. In other words, performing a full build rather than a patch-based build.
      * `--git-shallow` may be used to pre-warn that the local checkout is shallow/other branches are not available.
      * `--git-offline` may be used to pre-warn that remote repositories are not available/pulls will fail.
      * `--offline` sets `--git-offline` as well.
      * As a result of that work, git repositories that require authentication, are non-HTTP-based, and various other configurations should be much better understood and handled much more appropriately.
      * precommit will now attempt to guess what build tool a repository may be using. This may be overridden by specifying the build tool with `--build-tool` set to the appropriate plugin, including `nobuild` to disable build tools altogether.
      * If Apache Yetus is launched in a Docker container under a Jenkins pipeline as the agent definition, `${HOME}` should now be set to somewhere valid if it isn't valid. This will enable Apache Maven and other utilities that would otherwise crash due to broken homes.


      Developer Changes
      =================

      * Personalities setting `${GITHUB_REPO}` information will likely cause problems with autodetection of Github information. See YETUS-721 for more information.
      * Developers that check for `${JENKINS}` should specifically start checking for `${ROBOTTYPE}=jenkins` for equivalent functionality. But be aware that coding specifically for Jenkins may be incorrect.
      * Environment variable handling for Docker has been given a rewrite with the introduction of `add_docker_env`. For environment variables, this should be used instead of directly setting `${DOCKER_EXTRAARGS}`. Environment variables set via `add_docker_env` will be resolved just prior to Docker run thus enabling the value to change.
      * It is now possible set `${BUILDMODE}` during `locate_patch` resolution. This change was made due to limitations with some bug system/CI integrations. See `set_buildmode` for more information.
      * `github_breakup_url` can now be used for Github repo URLs and not just pull requests.
      * `gitlab_breakup_url` can now be used for Gitlab repo URLs and not just merge requests.
      * A lot of defaults are now defined much later. This should only impact plug-ins and overrides that are triggered extremely early in the boot up process.
      Show
      <!-- markdown --> This change makes significant changes to precommit to support a variety of automation systems for continuous integration (aka robots or the `--robot` flag) as well as rework the existing Jenkins support (`--jenkins`). End User Changes ================ * `--jenkins` flag has been removed. `test-patch`/`qbt` should now autodetect not only if it is running Jenkins, but also Circle CI, Gitlab CI, and Travis CI. Please see the robots document for specifics as to how that works, limitations, etc. * In the specific case of Jenkins, much better support for general Jenkins settings but also Github Branch Source Plugin using Pipelines and Github Pull Request Plugin. In many cases, options that were previously set specifically on the command line may be unnecessary/duplicate information and should be removed. * Configuration for how to run Apache Yetus against Apache Yetus for various CI systems is now bundled with the project source. * When running under a robot, `test-patch`/`qbt` may update the git remote repository information for branches. This is done because in some configurations, the information is missing and is required for proper functionality. In cases where this is not possible, the programs will treat the remote repositories as offline or the local repository as shallow. As a result, some functionality may not be available. The limitations may include `test-patch` functioning as though it was `qbt`. In other words, performing a full build rather than a patch-based build. * `--git-shallow` may be used to pre-warn that the local checkout is shallow/other branches are not available. * `--git-offline` may be used to pre-warn that remote repositories are not available/pulls will fail. * `--offline` sets `--git-offline` as well. * As a result of that work, git repositories that require authentication, are non-HTTP-based, and various other configurations should be much better understood and handled much more appropriately. * precommit will now attempt to guess what build tool a repository may be using. This may be overridden by specifying the build tool with `--build-tool` set to the appropriate plugin, including `nobuild` to disable build tools altogether. * If Apache Yetus is launched in a Docker container under a Jenkins pipeline as the agent definition, `${HOME}` should now be set to somewhere valid if it isn't valid. This will enable Apache Maven and other utilities that would otherwise crash due to broken homes. Developer Changes ================= * Personalities setting `${GITHUB_REPO}` information will likely cause problems with autodetection of Github information. See YETUS-721 for more information. * Developers that check for `${JENKINS}` should specifically start checking for `${ROBOTTYPE}=jenkins` for equivalent functionality. But be aware that coding specifically for Jenkins may be incorrect. * Environment variable handling for Docker has been given a rewrite with the introduction of `add_docker_env`. For environment variables, this should be used instead of directly setting `${DOCKER_EXTRAARGS}`. Environment variables set via `add_docker_env` will be resolved just prior to Docker run thus enabling the value to change. * It is now possible set `${BUILDMODE}` during `locate_patch` resolution. This change was made due to limitations with some bug system/CI integrations. See `set_buildmode` for more information. * `github_breakup_url` can now be used for Github repo URLs and not just pull requests. * `gitlab_breakup_url` can now be used for Gitlab repo URLs and not just merge requests. * A lot of defaults are now defined much later. This should only impact plug-ins and overrides that are triggered extremely early in the boot up process.
    • Important

    Description

      Umbrella JIRA to rework how robots are used in the system

      Attachments

        1. YETUS-681.00.patch
          15 kB
          Allen Wittenauer
        2. YETUS-681.YETUS-15.001.patch
          44 kB
          Allen Wittenauer
        3. YETUS-681.YETUS-15.002.patch
          47 kB
          Allen Wittenauer
        4. YETUS-681.YETUS-15.003.patch
          48 kB
          Allen Wittenauer
        5. YETUS-681.004.patch
          56 kB
          Allen Wittenauer
        6. YETUS-681.005.patch
          68 kB
          Allen Wittenauer
        7. YETUS-681.006.patch
          56 kB
          Allen Wittenauer
        8. YETUS-681.007.patch
          80 kB
          Allen Wittenauer
        9. YETUS-681.008.patch
          80 kB
          Allen Wittenauer

        Issue Links

          Activity

            People

              aw Allen Wittenauer
              aw Allen Wittenauer
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: