Internal documentation about Devuan infrastructure.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
 
 

137 lines
5.7 KiB

  1. This pad is at: https://pad.dyne.org/code/#/1/edit/usQoFADFDsJ9UAFbVm0H7A/FPGyscaoxWTAqqgscT1bw7VA
  2. Previous pad discussing a part of this: https://pad.dyne.org/code/#/1/edit/nXiySd0FPHvG8pBgGgImxA/aJ87TdwIEWiIk96GZcFwypnb
  3. This should help us document the *big picture* of Devuan's current infrastructure.
  4. ToC
  5. ===
  6. - Overview
  7. - gdo (GitLab/GL)
  8. - Releasebot
  9. - Jenkins
  10. - Amprolla (Moved to parazyd's email August 20th 2017)
  11. - Mirrors (Moved to parazyd's email August 20th 2017)
  12. - TODO / Wishlist
  13. Overview
  14. ========
  15. http://devuan.evilham.com/ci.png
  16. (from slide 7 https://www.slideshare.net/opennebula/opennebulaconf2015-112-the-status-of-devuan-project-alberto-zuin)
  17. ## Ralph:
  18. I think it would help if
  19. a) the arrows in the image were labelled
  20. b) it separated the people/user roles from the packages and logs
  21. c) identified the roles for all transitions
  22. ## end Ralph
  23. Different components:
  24. * gdo
  25. * Jenkins
  26. * Amprolla
  27. * The communication between the components itself:
  28. * Releasebot (gdo --> Jenkins)
  29. ## Ralph: where is bugs.devuan.org? and roles re it?
  30. **************************************************************************
  31. Each component section is listed in the order it appears in the workflow
  32. **************************************************************************
  33. gdo (GitLab/GL)
  34. ===============
  35. gdo is currently functioning as:
  36. - VCS hosting (git)
  37. - Authentication source
  38. - Authorisation source
  39. ## Ralph: what about gdo issues?
  40. ## Ralph: what about gdo wiki?
  41. TODO / Wishlist
  42. ----------------
  43. - move authentication/authorisation to a dedicated service
  44. - use webhooks for triggering
  45. - move to a lighter weight git repo service
  46. ## Ralph: - well documented organizational structure wrt authorization
  47. Releasebot
  48. ==========
  49. (this section is based on code audit of releasebot by evilham)
  50. https://git.devuan.org/devuan-infrastructure/devuan-releasebot
  51. Definitions
  52. -----------
  53. GROUP: git.devuan.org/$GROUP/$REPO
  54. PERMISSIONS: nomenclature amd:msb
  55. 1st half Add, Modify, Delete: permission on job associated with a repository
  56. 2nd half Master, Security, Build suite: permission to build branches in each repository
  57. Here Master refers to "master", "Security" to "-security" and "Build suite" to "suites/" branches.
  58. Any entity starts with permissions ---:---; permissions are additive on these criteria:
  59. MEMBERS GROUP: 'cfg.get("jobs", "members_projectid")' --> /devuan/devuan-repository-masters
  60. Being member with master permissions adds amd:m-b permissions on all repositories
  61. SECURITY GROUP: 'cfg.get("jobs", "security_projectid")' --> ?
  62. Being member adds ---:-s- permissions on all repositories
  63. MASTER+ (on a given REPO):
  64. Having these privileges adds ---:--b permissions on a given REPO
  65. Permissions audit
  66. -----------------
  67. (so-called security group is still unknown. Does not exist?)
  68. Permissions per-package
  69. http://devuan.evilham.com/perms.json
  70. Permissions where "amd:m-b" has been filtered out:
  71. http://devuan.evilham.com/perms_not_full.json
  72. Issuing a build (no pun intended)
  73. ---------------------------------
  74. * A GROUP is enabled in releasebot's config file
  75. * A MEMBERS GROUP entity creates a GitLab Issue(*) "buildadd" for REPO
  76. - It is possible to "buildadd/modify/del" repositories that cannot be built (whose GROUP is not enabled)
  77. + "build" commands can only be executed against repositories whose GROUP is enabled
  78. * A MASTER+ entity on REPO creates a GitLab Issue(*) "build" for REPO
  79. (*) Issues must be assigned to autobuild and the labels correspond to the architecture
  80. Labels can also be used to setup the build (sequential, pbuilder_network, ...)
  81. Builds can also be manually triggered on jenkins / the jenkins machine by *selected few*
  82. (*this could be rectified by m)
  83. Release bot is run by a cron job every 5 minutes checking for issues reported to "autobuild", checks permissions based on the reporter and hooks that with jenkins.
  84. - There are no error checks in the gitlab or jenkins integration libraries:
  85. + if jenkins fails to respond the script reports a job not found error
  86. + if gitlab fails there is a deafening silence ...typically happens once every couple of months.
  87. - If the job doesn't exist in jenkins or a command is not recognized, an appropriate response is reported back to the originating issue and the issue is closed.
  88. The jenkins jobs are created using xml templates found in jenkins\_tpl, one template each for source, binary, and repo jobs.
  89. For buildadd and buildmodify commands, Releasebot uses the templates to create each job by submit
  90. Actual build job
  91. ----------------
  92. Release bot passes info to jenkins, where each job is handled by jenkins-debian-glue
  93. "various bits of script provided via jenkins-debian-glue that do the actual work of building sources and packages."
  94. "The built packages are deployed to the respective repo in the repo job that has a script that signs and uploads the files to dak and then tells dak to get to work."
  95. jenkins-debian-glue and jenkins-debian-glue-buildenv and jenkins-debian-glue-buildenv provide various hooks used during the builds.
  96. TODO / Wishlist
  97. ----------------
  98. Jenkins
  99. =======
  100. jenkins-debian-glue
  101. -------------------
  102. TODO / Wishlist
  103. ----------------
  104. TODO / Wishlist
  105. ================
  106. Authentication system
  107. ---------------------
  108. If we are interested in building a federated authentication system that must be considered on a project wide scope not just at a build system scope, otherwise we get 2 or more separate authentication systems, each with their own risks & weaknesses.
  109. More accountability (and accessiblity) for permissions:
  110. The way we administer the permissions, the fact that there is no way of telling who has permissions to do what is an issue (quickly, without checking all repos)
  111. Giving permissions should be practical and not discourage secure values