Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

security(platform): Add sealed secrets #8342

Merged
merged 6 commits into from
Oct 16, 2024
Merged

security(platform): Add sealed secrets #8342

merged 6 commits into from
Oct 16, 2024

Conversation

aarushik93
Copy link
Contributor

@aarushik93 aarushik93 commented Oct 15, 2024 •
edited
Loading

Background

Adding sealed secrets for all of our secrets. This encrypts the secrets and ensures only the cluster can decrypt internally. This way we aren't locked into any vendor with a vault or secret manager. We can also store all our secrets publically since no one can decrypt them.

Changes 🏗️

Encrypted all secrets and added to appropriate values

Testing 🔍

Note

Only for the new autogpt platform, currently in autogpt_platform/

  • Create from scratch and execute an agent with at least 3 blocks
  • Import an agent from file upload, and confirm it executes correctly
  • Upload agent to marketplace
  • Import an agent from marketplace and confirm it executes correctly
  • Edit an agent from monitor, and confirm it executes correctly

@github-actions github-actions bot added documentation Improvements or additions to documentation platform/frontend AutoGPT Platform - Front end platform/backend AutoGPT Platform - Back end size/l labels Oct 15, 2024
@netlify Netlify
Copy link

netlify bot commented Oct 15, 2024 •
edited
Loading

✅ Deploy Preview for auto-gpt-docs canceled.

Name Link
🔨 Latest commit e8721d7
🔍 Latest deploy log https://app.netlify.com/sites/auto-gpt-docs/deploys/670e6ad96e94ea00094a1fd5

@aarushik93 aarushik93 changed the base branch from master to dev October 15, 2024 13:15
@github-actions github-actions bot removed documentation Improvements or additions to documentation platform/frontend AutoGPT Platform - Front end labels Oct 15, 2024
@aarushik93 aarushik93 marked this pull request as ready for review October 15, 2024 13:40
@aarushik93 aarushik93 requested a review from a team as a code owner October 15, 2024 13:40
@qodo-merge-pro Qodo Merge Pro
Copy link

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
🧪 No relevant tests
🔒 Security concerns

Sensitive information exposure:
The PR introduces sealed secrets and encrypted sensitive data across multiple files (e.g., autogpt_platform/infra/helm/autogpt-market/values.dev.yaml, autogpt_platform/infra/helm/autogpt-server/values.dev.yaml). While using sealed secrets is a good practice for protecting sensitive information, it's crucial to ensure that the encryption method is robust and that access to these secrets is tightly controlled. Additionally, the presence of encrypted API keys, database URLs, and other credentials in the codebase requires careful management to prevent any potential leaks or unauthorized access.

⚡ Recommended focus areas for review

Security Concern
The new sealed-secrets.yaml file contains sensitive information. Ensure that the encryption method used for sealing secrets is secure and that access to these secrets is properly restricted.

Sensitive Data Exposure
The file contains encrypted sensitive data such as API keys and database URLs. Verify that these secrets are properly encrypted and that the encryption keys are securely managed.

Configuration Change
Several environment variables have been moved to a new 'secrets' section. Ensure that all applications and services that depend on these variables are updated to use the new secret management system.

ntindle
ntindle approved these changes Oct 15, 2024
View reviewed changes
Copy link
Member

@ntindle ntindle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add docs on how someone can generate new ones of these so you aren’t the only one

@aarushik93 aarushik93 merged commit 9fe3fed into dev Oct 16, 2024
6 checks passed
@aarushik93 aarushik93 deleted the aarushikansal/install-sealed-secrets-k8s branch October 16, 2024 08:59
aarushik93 added a commit that referenced this pull request Oct 16, 2024
@aarushik93 @kcze
@Pwuts @majdyz @Swiftyos
fix(infra): Update backend server health check endpoint (#8351)
405dd16 
* feat(platform): List and revoke credentials in user profile (#8207)

Display existing credentials (OAuth and API keys) for all current providers: Google, Github, Notion and allow user to remove them. For providers that support it, we also revoke the tokens through the API: of the providers we currently have, Google and GitHub support it; Notion doesn't.

- Add credentials list and `Delete` button in `/profile`
- Add `revoke_tokens` abstract method to `BaseOAuthHandler` and implement it in each provider
- Revoke OAuth tokens for providers on `DELETE` `/{provider}/credentials/{cred_id}`, and return whether tokens could be revoked
   - Update `autogpt-server-api/baseClient.ts:deleteCredentials` with `CredentialsDeleteResponse` return type

Bonus:
- Update `autogpt-server-api/baseClient.ts:_request` to properly handle empty server responses

* fix(backend): Lower the number of node workers to save DB connections (#8331)

Change [graph]×[node] worker limit from 10×5 to 10×3

---------

Co-authored-by: Reinier van der Leer <[email protected]>

* fix(ci,platform): Add dev branch trigger to all ci (#8339)

* update ci for dev

* update classic

* remove duplicate dev

* fix(frontend): Fix styling inconsistencies in input elements (#8337)

- Apply consistent border styling to `Input`, `Select`, and `Textarea`
   - Remove `rounded-xl` from node input elements

- Add `whitespace-nowrap` to `CustomNode` header category tags

---------

Co-authored-by: Zamil Majdy <[email protected]>

* feat(builder): Use configmap for builder (#8343)

use configmap in builder

* fix(platform,infra): Checkin non secret values  (#8344)

checkin non secrets

* security(platform): Add sealed secrets (#8342)

* add sealed secrets

* add encrypted secrets

* remove extra space

* Tf public media buckets (#8324)

* fix(infra): Fix sealed secret names  (#8350)

* fix sealed secret names

* fix names and add annotation

* feat(backend): Introduce executors shared DB connection (#8340)

* update health checkendpoint

---------

Co-authored-by: Krzysztof Czerwinski <[email protected]>
Co-authored-by: Reinier van der Leer <[email protected]>
Co-authored-by: Zamil Majdy <[email protected]>
Co-authored-by: Swifty <[email protected]>
aarushik93 added a commit that referenced this pull request Oct 16, 2024
@aarushik93 @kcze
@Pwuts @majdyz @Swiftyos
fix(infra): Update backend server health check endpoint (#8351)
f621d39 
* feat(platform): List and revoke credentials in user profile (#8207)

Display existing credentials (OAuth and API keys) for all current providers: Google, Github, Notion and allow user to remove them. For providers that support it, we also revoke the tokens through the API: of the providers we currently have, Google and GitHub support it; Notion doesn't.

- Add credentials list and `Delete` button in `/profile`
- Add `revoke_tokens` abstract method to `BaseOAuthHandler` and implement it in each provider
- Revoke OAuth tokens for providers on `DELETE` `/{provider}/credentials/{cred_id}`, and return whether tokens could be revoked
   - Update `autogpt-server-api/baseClient.ts:deleteCredentials` with `CredentialsDeleteResponse` return type

Bonus:
- Update `autogpt-server-api/baseClient.ts:_request` to properly handle empty server responses

* fix(backend): Lower the number of node workers to save DB connections (#8331)

Change [graph]×[node] worker limit from 10×5 to 10×3

---------

Co-authored-by: Reinier van der Leer <[email protected]>

* fix(ci,platform): Add dev branch trigger to all ci (#8339)

* update ci for dev

* update classic

* remove duplicate dev

* fix(frontend): Fix styling inconsistencies in input elements (#8337)

- Apply consistent border styling to `Input`, `Select`, and `Textarea`
   - Remove `rounded-xl` from node input elements

- Add `whitespace-nowrap` to `CustomNode` header category tags

---------

Co-authored-by: Zamil Majdy <[email protected]>

* feat(builder): Use configmap for builder (#8343)

use configmap in builder

* fix(platform,infra): Checkin non secret values  (#8344)

checkin non secrets

* security(platform): Add sealed secrets (#8342)

* add sealed secrets

* add encrypted secrets

* remove extra space

* Tf public media buckets (#8324)

* fix(infra): Fix sealed secret names  (#8350)

* fix sealed secret names

* fix names and add annotation

* feat(backend): Introduce executors shared DB connection (#8340)

* update health checkendpoint

---------

Co-authored-by: Krzysztof Czerwinski <[email protected]>
Co-authored-by: Reinier van der Leer <[email protected]>
Co-authored-by: Zamil Majdy <[email protected]>
Co-authored-by: Swifty <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ntindle ntindle ntindle approved these changes

Assignees
No one assigned
Labels
platform/backend AutoGPT Platform - Back end Review effort [1-5]: 4 size/l
Projects
AutoGPT development kanban
Status: ✅ Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@aarushik93 @ntindle