Adds packaging docs on readthedocs #1062
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
onurctirtir
reviewed
Jul 13, 2023
Comment on lines
+15
to
+18
| 4. **Scalability**: Docker environments can be easily scaled up or down to meet the packaging requirements. This scalability ensures that the packaging process can be efficiently adjusted to handle different workload sizes. | ||
|
|
||
| 5. **Other Benefits**: Docker environments provide a secure, reliable, consistent, flexible, efficient, simple, and maintainable packaging environment. Docker isolates the packaging process from the underlying host system, ensuring a controlled and stable environment. | ||
|
|
There was a problem hiding this comment.
I feel like those two are not something that we really benefit from, we can remove them altogether?
Suggested change
| 4. **Scalability**: Docker environments can be easily scaled up or down to meet the packaging requirements. This scalability ensures that the packaging process can be efficiently adjusted to handle different workload sizes. | |
| 5. **Other Benefits**: Docker environments provide a secure, reliable, consistent, flexible, efficient, simple, and maintainable packaging environment. Docker isolates the packaging process from the underlying host system, ensuring a controlled and stable environment. |
Comment on lines
+22
to
+30
| We utilize Docker images for each OS/release/PostgreSQL version combination to create a reproducible and consistent packaging environment. | ||
|
|
||
| For Debian-based packages, we can use the same Docker image for all PostgreSQL versions. | ||
|
|
||
| For example, the Debian/10 Docker image can be used for both PostgreSQL 13 and PostgreSQL 14 packages. | ||
|
|
||
| However, for RPM-based packages, we need a dedicated Docker image for each PostgreSQL version. | ||
|
|
||
| For instance, we require a Docker image for CentOS/7 with PostgreSQL 13 and another Docker image for CentOS/7 with PostgreSQL 14 packages. |
There was a problem hiding this comment.
Suggested change
| We utilize Docker images for each OS/release/PostgreSQL version combination to create a reproducible and consistent packaging environment. | |
| For Debian-based packages, we can use the same Docker image for all PostgreSQL versions. | |
| For example, the Debian/10 Docker image can be used for both PostgreSQL 13 and PostgreSQL 14 packages. | |
| However, for RPM-based packages, we need a dedicated Docker image for each PostgreSQL version. | |
| For instance, we require a Docker image for CentOS/7 with PostgreSQL 13 and another Docker image for CentOS/7 with PostgreSQL 14 packages. | |
| We utilize Docker images for each OS/release/PostgreSQL version combination to create a reproducible and consistent packaging environment. | |
| For Debian-based packages, we use the same Docker image for all PostgreSQL versions. | |
| For example, the Debian/10 Docker image is used for both PostgreSQL 13 and PostgreSQL 14 packages. | |
| However, for RPM-based packages, we need a dedicated Docker image for each PostgreSQL version. | |
| For instance, we require a Docker image for CentOS/7 with PostgreSQL 13 and another Docker image for CentOS/7 with PostgreSQL 14 packages. |
Comment on lines
+6
to
+9
| * `PackageCloud <https://packagecloud.io/citusdata>`_: Stores deb and rpm packages. Community users and Marlin uses this repository. This is the main repo for all users | ||
| * PGDG: Stores only latest Citus rpm packages. You can not find older versions here. We do not manage the packages here directly. We open a ticket with PGDG to create a new package. | ||
| * `PGXN <https://pgxn.org/dist/citus/>`_: The PostgreSQL Extension Network (PGXN) is a community-driven effort aimed at providing a centralized platform for distributing and sharing extensions for the PostgreSQL database management system. We use this repository to store the citus extension. | ||
| * `Docker Hub <https://hub.docker.com/repository/docker/citusdata/citus>`_: Stores docker images. We use this repository to store the docker images for citus. |
There was a problem hiding this comment.
Suggested change
| * `PackageCloud <https://packagecloud.io/citusdata>`_: Stores deb and rpm packages. Community users and Marlin uses this repository. This is the main repo for all users | |
| * PGDG: Stores only latest Citus rpm packages. You can not find older versions here. We do not manage the packages here directly. We open a ticket with PGDG to create a new package. | |
| * `PGXN <https://pgxn.org/dist/citus/>`_: The PostgreSQL Extension Network (PGXN) is a community-driven effort aimed at providing a centralized platform for distributing and sharing extensions for the PostgreSQL database management system. We use this repository to store the citus extension. | |
| * `Docker Hub <https://hub.docker.com/repository/docker/citusdata/citus>`_: Stores docker images. We use this repository to store the docker images for citus. | |
| * `PackageCloud <https://packagecloud.io/citusdata>`_: Stores deb and rpm packages. Community users and our managed service uses this repository. This is the main repo for all users | |
| * PGDG: Stores only latest Citus rpm packages. You can not find older versions here. We do not manage the packages here directly. We open a ticket with PGDG to create a new package. | |
| * `PGXN <https://pgxn.org/dist/citus/>`_: The PostgreSQL Extension Network (PGXN) is a community-driven effort aimed at providing a centralized platform for distributing and sharing extensions for the PostgreSQL database management system. We use this repository to store the Citus extension. | |
| * `Docker Hub <https://hub.docker.com/repository/docker/citusdata/citus>`_: Stores docker images. We use this repository to store the docker images for Citus. |
Comment on lines
+24
to
+28
| * citus_package: This is the main script responsible for building the packages. | ||
| * update_docker: This script generates Citus Docker images for end users. | ||
| * upload_to_package_cloud: Publishes the packages created by citus_package to Package Cloud. | ||
| * publish_docker: Publishes the Docker images created by the update_docker script. | ||
| * delete_packages_on_package_cloud: Removes nightly packages older than 10 days from Package Cloud to reduce repository size and lower Package Cloud bills. |
There was a problem hiding this comment.
Should we mention other release scripts too (such as prepare_release), or should we maybe simply add a link to https://github.com/citusdata/tools/blob/develop/packaging_automation/README.md and say:
We have some other scripts used in the release process before the phase of baking the packages, see https://github.com/citusdata/tools/blob/develop/packaging_automation/README.md.
|
|
||
| To create docker images for patch releases, a tag needs to be added to the latest commit on the master branch and pushed. | ||
|
|
||
| The `The tag pipeline <https://github.com/citusdata/docker/blob/master/.github/workflows/publish_docker_images_on_tag.yml>`_ pipeline will create the docker images for the tag and push them to the docker repository. |
There was a problem hiding this comment.
Suggested change
| The `The tag pipeline <https://github.com/citusdata/docker/blob/master/.github/workflows/publish_docker_images_on_tag.yml>`_ pipeline will create the docker images for the tag and push them to the docker repository. | |
| The `tag pipeline <https://github.com/citusdata/docker/blob/master/.github/workflows/publish_docker_images_on_tag.yml>`_ pipeline will create the docker images for the tag and push them to the docker repository. |
|
@gurkanindibay do you plan to address reviews in here and make some progress? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is the main documentation for packaging. Below is the compiled version
https://citusdata-packaging.readthedocs.io/en/latest/