Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate Python FileStorageService into EntityService #86

Merged
merged 37 commits into from
Aug 26, 2021

Conversation

tamim-khan
Copy link
Member

@tamim-khan tamim-khan commented Jul 7, 2021

Ticket link

Closes #81

Implementation description

  • Add file name field to MongoDB and Postgres schemas
  • Update entity DTO with file name and file fields for create and update operations
  • Update entity service for MongoDB and Postgres to call FileStorageService methods
  • Update entity REST routes and add get endpoint to retrieve file by UUID
  • Add file type validation and update validator to work with multipart form data

Steps to test

Same as #88
REST:
(using Postman)

  1. Set the body to form-data and add the body field (as stringified JSON) and file field (postman lets you upload files).

image

{
    "stringField": "test 1",
    "intField": 1,
    "enumField": "A",
    "stringArrayField": ["test1"],
    "boolField": true
}

(if you want an example)
2. Send the POST request to entities endpoint to create an entity. Verify in firebase console that the file is created.
3. Try creating another entity whose mime type does not belong to text/plain, application/pdf, image/png, image/jpeg, image/gif. Verify that there is a validation error.
4. Try creating another entity without file parameter. Verify that this works (file field is optional).
5. Send a GET request to /entities endpoint. Verify that the file name is returned.
6. Send a GET request to /entities/files/{fileUUID}. Verify that you get the signed URL.
7. Send a PUT request to this entity's endpoint, with a different file. Verify that the file is updated.
8. Send a PUT request to this entity's endpoint, with no file field. Verify that the file is deleted in firebase console.
9. Send a PUT request to this entity's endpoint, with a file field. Verify that a new file UUID is generated and that it appears in firebase console.
10. Send a DELETE request to this entity's endpoint. Verify that the file is deleted in firebase console.
11. Repeat with the other EntityService (mongo or Postgres)

What should reviewers focus on?

  • Cases where there is no file
  • Update logic

Checklist

  • My PR name is descriptive and in imperative tense
  • My commit messages are descriptive and in imperative tense. My commits are atomic and trivial commits are squashed or fixup'd into non-trivial commits
  • I have run the appropriate linter(s)
  • I have requested a review from the PL, as well as other devs who have background knowledge on this PR or who will be building on top of this PR

@tamim-khan tamim-khan changed the title Update EntityService Mongo Integrate Python FileStorageService into EntityService Jul 7, 2021
@zm2he zm2he requested a review from sherryhli August 10, 2021 06:03
Copy link
Member

@sherryhli sherryhli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost there! A couple more minor issues

backend/python/app/rest/entity_routes.py Outdated Show resolved Hide resolved
backend/python/app/rest/entity_routes.py Outdated Show resolved Hide resolved
backend/python/app/rest/entity_routes.py Outdated Show resolved Hide resolved
backend/python/app/rest/entity_routes.py Show resolved Hide resolved
@zm2he zm2he requested a review from sherryhli August 15, 2021 05:10
sherryhli
sherryhli previously approved these changes Aug 16, 2021
Copy link
Member

@sherryhli sherryhli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for both of your patience in addressing all the changes! I'll take care of merging this

@sherryhli sherryhli merged commit e421cd2 into main Aug 26, 2021
@sherryhli sherryhli deleted the tamim/file-service-integration branch August 26, 2021 05:30
@sherryhli sherryhli mentioned this pull request Aug 29, 2021
4 tasks
sherryhli added a commit that referenced this pull request Aug 29, 2021
* Add example context components for frontend state management (#37)

* Add example context components

* Add ts root dir to fix eslint errors

* Fix typo, add return type to reducer

* Add react plugins to eslint

* Update README.md (#46)

* Linting fixes (#42)

* Add ts plugins to eslint, fix eslint errors

* Remove deep copy of form data

* Add more return types, change prettier errors to warn

* Type decoded JWT, move AuthenticatedUser to types file

* Create AuthTypes

* Fix token expiry

Co-authored-by: Sherry Li <[email protected]>

* Refactor auth functions that use localStorage logic into AuthAPIClient (#45)

* duplicate code to authapiclient.ts

* refactor functions with localstorage to AuthAPIClient

* Update with removed REST tags

* update with pr fixes

* Update mongoose version to remove warnings (#47)

* Configure Sequelize migrations (#48)

* Configure Sequelize migrations using umzug

* Add defaultValue and allowNull options to add column example

* Add swagger for typescript backend (#51)

* Add Python REST Validators (#50)

* add initial validate decorator

* update with modified middleware and add decorators to the respective functions

* add decorator and validate method for entity dto

* Update README.md and add nodemailer config path to secret.config (#49)

* Update README.md

* Add nodemailer config path to secret.config

* Add frontend stack to README.md

* Add minor edits to README.md

* Add TypeScript REST API validation (#52)

* Add auth validators

* Add Entity and User Validators

* Refactor validation utils

* Add API error messages

* Swagger For Python (#54)

* Swagger For Python

* Minor fixes

* Include file name in log using wrapper for Winston typescript logger (#55)

Co-authored-by: Bruce He <[email protected]>

* Create Python FileStorageService (#62)

* Add initial file storage service

* Add initial file storage service

* Create and Update

* Add exception handling and more docstrings

* Make expiration time a parameter, clean up code

* Add content type parameter, update interface

Co-authored-by: Richard Chen <[email protected]>

* Create TypeScript FileStorageService (#59)

* add initial files

* Fix firebase bucket init

* add create file implementation

* add interface notes and add checking if file already exists

* Add updateFile and deleteFile implementations

* update create file method

* update create method notes

* Refactor getFile

* remove import in server.ts

* Update error message on failure to get file

Co-authored-by: Sherry Li <[email protected]>

* add contentType param

* fix formatting in storageService.ts

* Make bucketName a param

* Fix bucket name format

Co-authored-by: tamim-khan <[email protected]>
Co-authored-by: Sherry Li <[email protected]>

* Add testing frameworks (#57)

* Delete repeated requirements

* Add pytest to requirements

* Create python test examples

* Add typescript BE examples

* Create frontend sample tests

* Add test-db to docker-compose

* Update tests for docker containers

* Update psql conn string

* Add mongodb-memory-server dependency libcurl3 to docker container

* Add mongomock and pytest-mock to py requirements

* Create test_user_service

* Add pg user service test

* Configure SQLAlchemy migrations (#58)

* Configure SQLAlchemy migrations

* Fix create_app configs when calling from Flask CLI

* Update README.md (#64)

* Update README.md

* Apply suggestions from code review

Co-authored-by: Sherry Li <[email protected]>

Co-authored-by: Sherry Li <[email protected]>

* Set up frontend deployment and previews with Firebase Hosting and GitHub Actions (#87)

* Set up frontend hosting using Firebase

* Set up GitHub Actions for frontend deploy preview and automated prod deployment

* Set working directory in frontend deployment GitHub Actions

* Add yarn install step to deployment scripts

* Add entryPoint to action-hosting-deploy step

* Use separate backend urls for previews and prod

* Fix several minor issues (#63)

* Set extended option in express.urlencoded in server.ts

* Fix user_id casing in Python openapi.json

* Simplify Python DTO validators

* Add newline in Logout.tsx

* Rename TypeScript storageService to fileStorageService

* Integrate TS FileStorageService into EntityService (#88)

* temp changes

* add initial changes to EntityServiceMg

* Fix update function in entityservice

* Add file crud methods to postgres entity service

* Add file content type validation

* Fix create endpoints

* Add local file storing to graphql resolvers

* Add endpoint for getting file based on fileUUID

* Change storage service to store bucketName instead of the bucket

* Minor fixes

* Fix update functions in entity service

* Minor fixes

* Add content type to entity request DTO and filestorageservice calls

* Fix graphql

* Fix imports

* Refactor create/update to prevent dangling file references

* Address comments

* Wait for file to finish writing before continuing

* Delete file after uploading

* Add awaits to filestorageservice calls

Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Emily Louie <[email protected]>

* Add frontend csv utility (#90)

* Integrate Python FileStorageService into EntityService (#86)

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Fix python file get endpoint file storage service call

* Remove EntityDTO.file_name from Mongo Impl

* Update validator to check for missing body

* fix

* Fix file get endpoint auth, rename entity DTO variable

* Remove EntityDTO.file_name from Postgres Impl

* Update get_file to return object

* Fix bug in mongo service update logic

* Fix Postgres update route and logic

* Fix python postgres pop file and lint code.

* Postgres migration to add entity file name

Co-authored-by: Bruce He <[email protected]>

* Remove extra DecodedJWT import in BaseAPIClient.ts

* Add file-storage scrubber tags

* Fix scrubber tags spacing

Co-authored-by: Xin Hao Zhang <[email protected]>
Co-authored-by: alexguo8 <[email protected]>
Co-authored-by: Emily Louie <[email protected]>
Co-authored-by: Tamim Khan <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Dinu Wijetunga <[email protected]>
sherryhli added a commit that referenced this pull request Aug 31, 2021
* Add example context components for frontend state management (#37)

* Add example context components

* Add ts root dir to fix eslint errors

* Fix typo, add return type to reducer

* Add react plugins to eslint

* Update README.md (#46)

* Linting fixes (#42)

* Add ts plugins to eslint, fix eslint errors

* Remove deep copy of form data

* Add more return types, change prettier errors to warn

* Type decoded JWT, move AuthenticatedUser to types file

* Create AuthTypes

* Fix token expiry

Co-authored-by: Sherry Li <[email protected]>

* Refactor auth functions that use localStorage logic into AuthAPIClient (#45)

* duplicate code to authapiclient.ts

* refactor functions with localstorage to AuthAPIClient

* Update with removed REST tags

* update with pr fixes

* Update mongoose version to remove warnings (#47)

* Configure Sequelize migrations (#48)

* Configure Sequelize migrations using umzug

* Add defaultValue and allowNull options to add column example

* Add swagger for typescript backend (#51)

* Add Python REST Validators (#50)

* add initial validate decorator

* update with modified middleware and add decorators to the respective functions

* add decorator and validate method for entity dto

* Update README.md and add nodemailer config path to secret.config (#49)

* Update README.md

* Add nodemailer config path to secret.config

* Add frontend stack to README.md

* Add minor edits to README.md

* Add TypeScript REST API validation (#52)

* Add auth validators

* Add Entity and User Validators

* Refactor validation utils

* Add API error messages

* Swagger For Python (#54)

* Swagger For Python

* Minor fixes

* Include file name in log using wrapper for Winston typescript logger (#55)

Co-authored-by: Bruce He <[email protected]>

* Create Python FileStorageService (#62)

* Add initial file storage service

* Add initial file storage service

* Create and Update

* Add exception handling and more docstrings

* Make expiration time a parameter, clean up code

* Add content type parameter, update interface

Co-authored-by: Richard Chen <[email protected]>

* Create TypeScript FileStorageService (#59)

* add initial files

* Fix firebase bucket init

* add create file implementation

* add interface notes and add checking if file already exists

* Add updateFile and deleteFile implementations

* update create file method

* update create method notes

* Refactor getFile

* remove import in server.ts

* Update error message on failure to get file

Co-authored-by: Sherry Li <[email protected]>

* add contentType param

* fix formatting in storageService.ts

* Make bucketName a param

* Fix bucket name format

Co-authored-by: tamim-khan <[email protected]>
Co-authored-by: Sherry Li <[email protected]>

* Add testing frameworks (#57)

* Delete repeated requirements

* Add pytest to requirements

* Create python test examples

* Add typescript BE examples

* Create frontend sample tests

* Add test-db to docker-compose

* Update tests for docker containers

* Update psql conn string

* Add mongodb-memory-server dependency libcurl3 to docker container

* Add mongomock and pytest-mock to py requirements

* Create test_user_service

* Add pg user service test

* Configure SQLAlchemy migrations (#58)

* Configure SQLAlchemy migrations

* Fix create_app configs when calling from Flask CLI

* Update README.md (#64)

* Update README.md

* Apply suggestions from code review

Co-authored-by: Sherry Li <[email protected]>

Co-authored-by: Sherry Li <[email protected]>

* Set up frontend deployment and previews with Firebase Hosting and GitHub Actions (#87)

* Set up frontend hosting using Firebase

* Set up GitHub Actions for frontend deploy preview and automated prod deployment

* Set working directory in frontend deployment GitHub Actions

* Add yarn install step to deployment scripts

* Add entryPoint to action-hosting-deploy step

* Use separate backend urls for previews and prod

* Fix several minor issues (#63)

* Set extended option in express.urlencoded in server.ts

* Fix user_id casing in Python openapi.json

* Simplify Python DTO validators

* Add newline in Logout.tsx

* Rename TypeScript storageService to fileStorageService

* Integrate TS FileStorageService into EntityService (#88)

* temp changes

* add initial changes to EntityServiceMg

* Fix update function in entityservice

* Add file crud methods to postgres entity service

* Add file content type validation

* Fix create endpoints

* Add local file storing to graphql resolvers

* Add endpoint for getting file based on fileUUID

* Change storage service to store bucketName instead of the bucket

* Minor fixes

* Fix update functions in entity service

* Minor fixes

* Add content type to entity request DTO and filestorageservice calls

* Fix graphql

* Fix imports

* Refactor create/update to prevent dangling file references

* Address comments

* Wait for file to finish writing before continuing

* Delete file after uploading

* Add awaits to filestorageservice calls

Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Emily Louie <[email protected]>

* Add frontend csv utility (#90)

* Integrate Python FileStorageService into EntityService (#86)

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Fix python file get endpoint file storage service call

* Remove EntityDTO.file_name from Mongo Impl

* Update validator to check for missing body

* fix

* Fix file get endpoint auth, rename entity DTO variable

* Remove EntityDTO.file_name from Postgres Impl

* Update get_file to return object

* Fix bug in mongo service update logic

* Fix Postgres update route and logic

* Fix python postgres pop file and lint code.

* Postgres migration to add entity file name

Co-authored-by: Bruce He <[email protected]>

* Add backend TS csv generation utility (#91)

* initial files

* add a responseutil

* update rest api

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

Co-authored-by: Alex Guo <[email protected]>

* Add download csv functionality from frontend utility (#95)

* initial files

* add a responseutil

* update rest api

* Add download csv functionality from frontend utility

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

* Add backend API integration

Co-authored-by: Emily Louie <[email protected]>

* Refactor route strings into constants (#111)

* Backend CSV generation Python (#108)

* Initial change

Linter and tests

Update default options

csv default settings

Examples and minor format changes

Update docker-compose.yml

Moved tests to unit

* Add minor fixes

Co-authored-by: Sherry Li <[email protected]>

* Make minor corrections and formatting changes

Co-authored-by: Xin Hao Zhang <[email protected]>
Co-authored-by: alexguo8 <[email protected]>
Co-authored-by: Emily Louie <[email protected]>
Co-authored-by: Tamim Khan <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Dinu Wijetunga <[email protected]>
Co-authored-by: Alex Guo <[email protected]>
sherryhli added a commit that referenced this pull request Sep 3, 2021
* Add example context components for frontend state management (#37)

* Add example context components

* Add ts root dir to fix eslint errors

* Fix typo, add return type to reducer

* Add react plugins to eslint

* Update README.md (#46)

* Linting fixes (#42)

* Add ts plugins to eslint, fix eslint errors

* Remove deep copy of form data

* Add more return types, change prettier errors to warn

* Type decoded JWT, move AuthenticatedUser to types file

* Create AuthTypes

* Fix token expiry

Co-authored-by: Sherry Li <[email protected]>

* Refactor auth functions that use localStorage logic into AuthAPIClient (#45)

* duplicate code to authapiclient.ts

* refactor functions with localstorage to AuthAPIClient

* Update with removed REST tags

* update with pr fixes

* Update mongoose version to remove warnings (#47)

* Configure Sequelize migrations (#48)

* Configure Sequelize migrations using umzug

* Add defaultValue and allowNull options to add column example

* Add swagger for typescript backend (#51)

* Add Python REST Validators (#50)

* add initial validate decorator

* update with modified middleware and add decorators to the respective functions

* add decorator and validate method for entity dto

* Update README.md and add nodemailer config path to secret.config (#49)

* Update README.md

* Add nodemailer config path to secret.config

* Add frontend stack to README.md

* Add minor edits to README.md

* Add TypeScript REST API validation (#52)

* Add auth validators

* Add Entity and User Validators

* Refactor validation utils

* Add API error messages

* Swagger For Python (#54)

* Swagger For Python

* Minor fixes

* Include file name in log using wrapper for Winston typescript logger (#55)

Co-authored-by: Bruce He <[email protected]>

* Create Python FileStorageService (#62)

* Add initial file storage service

* Add initial file storage service

* Create and Update

* Add exception handling and more docstrings

* Make expiration time a parameter, clean up code

* Add content type parameter, update interface

Co-authored-by: Richard Chen <[email protected]>

* Create TypeScript FileStorageService (#59)

* add initial files

* Fix firebase bucket init

* add create file implementation

* add interface notes and add checking if file already exists

* Add updateFile and deleteFile implementations

* update create file method

* update create method notes

* Refactor getFile

* remove import in server.ts

* Update error message on failure to get file

Co-authored-by: Sherry Li <[email protected]>

* add contentType param

* fix formatting in storageService.ts

* Make bucketName a param

* Fix bucket name format

Co-authored-by: tamim-khan <[email protected]>
Co-authored-by: Sherry Li <[email protected]>

* Add testing frameworks (#57)

* Delete repeated requirements

* Add pytest to requirements

* Create python test examples

* Add typescript BE examples

* Create frontend sample tests

* Add test-db to docker-compose

* Update tests for docker containers

* Update psql conn string

* Add mongodb-memory-server dependency libcurl3 to docker container

* Add mongomock and pytest-mock to py requirements

* Create test_user_service

* Add pg user service test

* Configure SQLAlchemy migrations (#58)

* Configure SQLAlchemy migrations

* Fix create_app configs when calling from Flask CLI

* Update README.md (#64)

* Update README.md

* Apply suggestions from code review

Co-authored-by: Sherry Li <[email protected]>

Co-authored-by: Sherry Li <[email protected]>

* Set up frontend deployment and previews with Firebase Hosting and GitHub Actions (#87)

* Set up frontend hosting using Firebase

* Set up GitHub Actions for frontend deploy preview and automated prod deployment

* Set working directory in frontend deployment GitHub Actions

* Add yarn install step to deployment scripts

* Add entryPoint to action-hosting-deploy step

* Use separate backend urls for previews and prod

* Fix several minor issues (#63)

* Set extended option in express.urlencoded in server.ts

* Fix user_id casing in Python openapi.json

* Simplify Python DTO validators

* Add newline in Logout.tsx

* Rename TypeScript storageService to fileStorageService

* Integrate TS FileStorageService into EntityService (#88)

* temp changes

* add initial changes to EntityServiceMg

* Fix update function in entityservice

* Add file crud methods to postgres entity service

* Add file content type validation

* Fix create endpoints

* Add local file storing to graphql resolvers

* Add endpoint for getting file based on fileUUID

* Change storage service to store bucketName instead of the bucket

* Minor fixes

* Fix update functions in entity service

* Minor fixes

* Add content type to entity request DTO and filestorageservice calls

* Fix graphql

* Fix imports

* Refactor create/update to prevent dangling file references

* Address comments

* Wait for file to finish writing before continuing

* Delete file after uploading

* Add awaits to filestorageservice calls

Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Emily Louie <[email protected]>

* Add frontend csv utility (#90)

* Integrate Python FileStorageService into EntityService (#86)

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Fix python file get endpoint file storage service call

* Remove EntityDTO.file_name from Mongo Impl

* Update validator to check for missing body

* fix

* Fix file get endpoint auth, rename entity DTO variable

* Remove EntityDTO.file_name from Postgres Impl

* Update get_file to return object

* Fix bug in mongo service update logic

* Fix Postgres update route and logic

* Fix python postgres pop file and lint code.

* Postgres migration to add entity file name

Co-authored-by: Bruce He <[email protected]>

* Add backend TS csv generation utility (#91)

* initial files

* add a responseutil

* update rest api

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

Co-authored-by: Alex Guo <[email protected]>

* Add download csv functionality from frontend utility (#95)

* initial files

* add a responseutil

* update rest api

* Add download csv functionality from frontend utility

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

* Add backend API integration

Co-authored-by: Emily Louie <[email protected]>

* Refactor route strings into constants (#111)

* Backend CSV generation Python (#108)

* Initial change

Linter and tests

Update default options

csv default settings

Examples and minor format changes

Update docker-compose.yml

Moved tests to unit

* Add minor fixes

Co-authored-by: Sherry Li <[email protected]>

* Create Typescript registration flow with email verification (#104)

* Add endpoints for register user with email verification

* Check email verified on the firebase user instead of the token

* Create Python registration flow with email verification (#105)

* Add endpoints to register user using email verification

* fix import and add validator

* Add validation for RegisterUserDTO

* Fix accessing the email verification from token

* Change the verification email title

* check email verified in firebase user rather than token

* Frontend User Sign-Up (#115)

* Add sign-up page & auth client register functions

* Rebase on main and make minor routing changes

Co-authored-by: Sherry Li <[email protected]>

* Delete extra DecodedJWT import in BaseAPIClient.ts

* Delete extra getCSV definition in EntityAPIClient.ts

* Rearrange import in backend/python/app/rest/user_routes.py

* Uncomment REST implementation in onSignupClick

* Fix tag spacing

* Fix email_verified check in auth_service

Co-authored-by: Xin Hao Zhang <[email protected]>
Co-authored-by: alexguo8 <[email protected]>
Co-authored-by: Emily Louie <[email protected]>
Co-authored-by: Tamim Khan <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Dinu Wijetunga <[email protected]>
Co-authored-by: Alex Guo <[email protected]>
sherryhli added a commit that referenced this pull request Sep 4, 2021
* Add example context components for frontend state management (#37)

* Add example context components

* Add ts root dir to fix eslint errors

* Fix typo, add return type to reducer

* Add react plugins to eslint

* Update README.md (#46)

* Linting fixes (#42)

* Add ts plugins to eslint, fix eslint errors

* Remove deep copy of form data

* Add more return types, change prettier errors to warn

* Type decoded JWT, move AuthenticatedUser to types file

* Create AuthTypes

* Fix token expiry

Co-authored-by: Sherry Li <[email protected]>

* Refactor auth functions that use localStorage logic into AuthAPIClient (#45)

* duplicate code to authapiclient.ts

* refactor functions with localstorage to AuthAPIClient

* Update with removed REST tags

* update with pr fixes

* Update mongoose version to remove warnings (#47)

* Configure Sequelize migrations (#48)

* Configure Sequelize migrations using umzug

* Add defaultValue and allowNull options to add column example

* Add swagger for typescript backend (#51)

* Add Python REST Validators (#50)

* add initial validate decorator

* update with modified middleware and add decorators to the respective functions

* add decorator and validate method for entity dto

* Update README.md and add nodemailer config path to secret.config (#49)

* Update README.md

* Add nodemailer config path to secret.config

* Add frontend stack to README.md

* Add minor edits to README.md

* Add TypeScript REST API validation (#52)

* Add auth validators

* Add Entity and User Validators

* Refactor validation utils

* Add API error messages

* Swagger For Python (#54)

* Swagger For Python

* Minor fixes

* Include file name in log using wrapper for Winston typescript logger (#55)

Co-authored-by: Bruce He <[email protected]>

* Create Python FileStorageService (#62)

* Add initial file storage service

* Add initial file storage service

* Create and Update

* Add exception handling and more docstrings

* Make expiration time a parameter, clean up code

* Add content type parameter, update interface

Co-authored-by: Richard Chen <[email protected]>

* Create TypeScript FileStorageService (#59)

* add initial files

* Fix firebase bucket init

* add create file implementation

* add interface notes and add checking if file already exists

* Add updateFile and deleteFile implementations

* update create file method

* update create method notes

* Refactor getFile

* remove import in server.ts

* Update error message on failure to get file

Co-authored-by: Sherry Li <[email protected]>

* add contentType param

* fix formatting in storageService.ts

* Make bucketName a param

* Fix bucket name format

Co-authored-by: tamim-khan <[email protected]>
Co-authored-by: Sherry Li <[email protected]>

* Add testing frameworks (#57)

* Delete repeated requirements

* Add pytest to requirements

* Create python test examples

* Add typescript BE examples

* Create frontend sample tests

* Add test-db to docker-compose

* Update tests for docker containers

* Update psql conn string

* Add mongodb-memory-server dependency libcurl3 to docker container

* Add mongomock and pytest-mock to py requirements

* Create test_user_service

* Add pg user service test

* Configure SQLAlchemy migrations (#58)

* Configure SQLAlchemy migrations

* Fix create_app configs when calling from Flask CLI

* Update README.md (#64)

* Update README.md

* Apply suggestions from code review

Co-authored-by: Sherry Li <[email protected]>

Co-authored-by: Sherry Li <[email protected]>

* Set up frontend deployment and previews with Firebase Hosting and GitHub Actions (#87)

* Set up frontend hosting using Firebase

* Set up GitHub Actions for frontend deploy preview and automated prod deployment

* Set working directory in frontend deployment GitHub Actions

* Add yarn install step to deployment scripts

* Add entryPoint to action-hosting-deploy step

* Use separate backend urls for previews and prod

* Fix several minor issues (#63)

* Set extended option in express.urlencoded in server.ts

* Fix user_id casing in Python openapi.json

* Simplify Python DTO validators

* Add newline in Logout.tsx

* Rename TypeScript storageService to fileStorageService

* Integrate TS FileStorageService into EntityService (#88)

* temp changes

* add initial changes to EntityServiceMg

* Fix update function in entityservice

* Add file crud methods to postgres entity service

* Add file content type validation

* Fix create endpoints

* Add local file storing to graphql resolvers

* Add endpoint for getting file based on fileUUID

* Change storage service to store bucketName instead of the bucket

* Minor fixes

* Fix update functions in entity service

* Minor fixes

* Add content type to entity request DTO and filestorageservice calls

* Fix graphql

* Fix imports

* Refactor create/update to prevent dangling file references

* Address comments

* Wait for file to finish writing before continuing

* Delete file after uploading

* Add awaits to filestorageservice calls

Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Emily Louie <[email protected]>

* Add frontend csv utility (#90)

* Integrate Python FileStorageService into EntityService (#86)

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Fix python file get endpoint file storage service call

* Remove EntityDTO.file_name from Mongo Impl

* Update validator to check for missing body

* fix

* Fix file get endpoint auth, rename entity DTO variable

* Remove EntityDTO.file_name from Postgres Impl

* Update get_file to return object

* Fix bug in mongo service update logic

* Fix Postgres update route and logic

* Fix python postgres pop file and lint code.

* Postgres migration to add entity file name

Co-authored-by: Bruce He <[email protected]>

* Add backend TS csv generation utility (#91)

* initial files

* add a responseutil

* update rest api

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

Co-authored-by: Alex Guo <[email protected]>

* Add download csv functionality from frontend utility (#95)

* initial files

* add a responseutil

* update rest api

* Add download csv functionality from frontend utility

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

* Add backend API integration

Co-authored-by: Emily Louie <[email protected]>

* Refactor route strings into constants (#111)

* Backend CSV generation Python (#108)

* Initial change

Linter and tests

Update default options

csv default settings

Examples and minor format changes

Update docker-compose.yml

Moved tests to unit

* Add minor fixes

Co-authored-by: Sherry Li <[email protected]>

* Create Typescript registration flow with email verification (#104)

* Add endpoints for register user with email verification

* Check email verified on the firebase user instead of the token

* Create Python registration flow with email verification (#105)

* Add endpoints to register user using email verification

* fix import and add validator

* Add validation for RegisterUserDTO

* Fix accessing the email verification from token

* Change the verification email title

* check email verified in firebase user rather than token

* Frontend User Sign-Up (#115)

* Add sign-up page & auth client register functions

* Rebase on main and make minor routing changes

Co-authored-by: Sherry Li <[email protected]>

* Integrate Google OAuth into Python backend (#112)

* Add google oauth signin to firebase rest client

* Add google oauth login and signup functionality

* Add frontend for google oauth signin

* Fix backend

* Add comment for google user id token

* Integrate Google OAuth into TS (#114)

* Add REST Impl

* Fix try catch

Co-authored-by: Alex Guo <[email protected]>

* Fix email_verified syntax in Python auth_service (#123)

* Add TS GraphQL support for Google OAuth (#122)

* GraphQL OAuth

* rename id_token

* Validators

* Update authValidators.ts

* Remove duplicated code

Co-authored-by: Xin Hao Zhang <[email protected]>
Co-authored-by: alexguo8 <[email protected]>
Co-authored-by: Emily Louie <[email protected]>
Co-authored-by: Tamim Khan <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Dinu Wijetunga <[email protected]>
Co-authored-by: Alex Guo <[email protected]>
sherryhli added a commit that referenced this pull request Sep 8, 2021
* Add example context components for frontend state management (#37)

* Add example context components

* Add ts root dir to fix eslint errors

* Fix typo, add return type to reducer

* Add react plugins to eslint

* Update README.md (#46)

* Linting fixes (#42)

* Add ts plugins to eslint, fix eslint errors

* Remove deep copy of form data

* Add more return types, change prettier errors to warn

* Type decoded JWT, move AuthenticatedUser to types file

* Create AuthTypes

* Fix token expiry

Co-authored-by: Sherry Li <[email protected]>

* Refactor auth functions that use localStorage logic into AuthAPIClient (#45)

* duplicate code to authapiclient.ts

* refactor functions with localstorage to AuthAPIClient

* Update with removed REST tags

* update with pr fixes

* Update mongoose version to remove warnings (#47)

* Configure Sequelize migrations (#48)

* Configure Sequelize migrations using umzug

* Add defaultValue and allowNull options to add column example

* Add swagger for typescript backend (#51)

* Add Python REST Validators (#50)

* add initial validate decorator

* update with modified middleware and add decorators to the respective functions

* add decorator and validate method for entity dto

* Update README.md and add nodemailer config path to secret.config (#49)

* Update README.md

* Add nodemailer config path to secret.config

* Add frontend stack to README.md

* Add minor edits to README.md

* Add TypeScript REST API validation (#52)

* Add auth validators

* Add Entity and User Validators

* Refactor validation utils

* Add API error messages

* Swagger For Python (#54)

* Swagger For Python

* Minor fixes

* Include file name in log using wrapper for Winston typescript logger (#55)

Co-authored-by: Bruce He <[email protected]>

* Create Python FileStorageService (#62)

* Add initial file storage service

* Add initial file storage service

* Create and Update

* Add exception handling and more docstrings

* Make expiration time a parameter, clean up code

* Add content type parameter, update interface

Co-authored-by: Richard Chen <[email protected]>

* Create TypeScript FileStorageService (#59)

* add initial files

* Fix firebase bucket init

* add create file implementation

* add interface notes and add checking if file already exists

* Add updateFile and deleteFile implementations

* update create file method

* update create method notes

* Refactor getFile

* remove import in server.ts

* Update error message on failure to get file

Co-authored-by: Sherry Li <[email protected]>

* add contentType param

* fix formatting in storageService.ts

* Make bucketName a param

* Fix bucket name format

Co-authored-by: tamim-khan <[email protected]>
Co-authored-by: Sherry Li <[email protected]>

* Add testing frameworks (#57)

* Delete repeated requirements

* Add pytest to requirements

* Create python test examples

* Add typescript BE examples

* Create frontend sample tests

* Add test-db to docker-compose

* Update tests for docker containers

* Update psql conn string

* Add mongodb-memory-server dependency libcurl3 to docker container

* Add mongomock and pytest-mock to py requirements

* Create test_user_service

* Add pg user service test

* Configure SQLAlchemy migrations (#58)

* Configure SQLAlchemy migrations

* Fix create_app configs when calling from Flask CLI

* Update README.md (#64)

* Update README.md

* Apply suggestions from code review

Co-authored-by: Sherry Li <[email protected]>

Co-authored-by: Sherry Li <[email protected]>

* Set up frontend deployment and previews with Firebase Hosting and GitHub Actions (#87)

* Set up frontend hosting using Firebase

* Set up GitHub Actions for frontend deploy preview and automated prod deployment

* Set working directory in frontend deployment GitHub Actions

* Add yarn install step to deployment scripts

* Add entryPoint to action-hosting-deploy step

* Use separate backend urls for previews and prod

* Fix several minor issues (#63)

* Set extended option in express.urlencoded in server.ts

* Fix user_id casing in Python openapi.json

* Simplify Python DTO validators

* Add newline in Logout.tsx

* Rename TypeScript storageService to fileStorageService

* Integrate TS FileStorageService into EntityService (#88)

* temp changes

* add initial changes to EntityServiceMg

* Fix update function in entityservice

* Add file crud methods to postgres entity service

* Add file content type validation

* Fix create endpoints

* Add local file storing to graphql resolvers

* Add endpoint for getting file based on fileUUID

* Change storage service to store bucketName instead of the bucket

* Minor fixes

* Fix update functions in entity service

* Minor fixes

* Add content type to entity request DTO and filestorageservice calls

* Fix graphql

* Fix imports

* Refactor create/update to prevent dangling file references

* Address comments

* Wait for file to finish writing before continuing

* Delete file after uploading

* Add awaits to filestorageservice calls

Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Emily Louie <[email protected]>

* Add frontend csv utility (#90)

* Integrate Python FileStorageService into EntityService (#86)

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Fix python file get endpoint file storage service call

* Remove EntityDTO.file_name from Mongo Impl

* Update validator to check for missing body

* fix

* Fix file get endpoint auth, rename entity DTO variable

* Remove EntityDTO.file_name from Postgres Impl

* Update get_file to return object

* Fix bug in mongo service update logic

* Fix Postgres update route and logic

* Fix python postgres pop file and lint code.

* Postgres migration to add entity file name

Co-authored-by: Bruce He <[email protected]>

* Add backend TS csv generation utility (#91)

* initial files

* add a responseutil

* update rest api

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

Co-authored-by: Alex Guo <[email protected]>

* Add download csv functionality from frontend utility (#95)

* initial files

* add a responseutil

* update rest api

* Add download csv functionality from frontend utility

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

* Add backend API integration

Co-authored-by: Emily Louie <[email protected]>

* Refactor route strings into constants (#111)

* Backend CSV generation Python (#108)

* Initial change

Linter and tests

Update default options

csv default settings

Examples and minor format changes

Update docker-compose.yml

Moved tests to unit

* Add minor fixes

Co-authored-by: Sherry Li <[email protected]>

* Create Typescript registration flow with email verification (#104)

* Add endpoints for register user with email verification

* Check email verified on the firebase user instead of the token

* Create Python registration flow with email verification (#105)

* Add endpoints to register user using email verification

* fix import and add validator

* Add validation for RegisterUserDTO

* Fix accessing the email verification from token

* Change the verification email title

* check email verified in firebase user rather than token

* Frontend User Sign-Up (#115)

* Add sign-up page & auth client register functions

* Rebase on main and make minor routing changes

Co-authored-by: Sherry Li <[email protected]>

* Integrate Google OAuth into Python backend (#112)

* Add google oauth signin to firebase rest client

* Add google oauth login and signup functionality

* Add frontend for google oauth signin

* Fix backend

* Add comment for google user id token

* Integrate Google OAuth into TS (#114)

* Add REST Impl

* Fix try catch

Co-authored-by: Alex Guo <[email protected]>

* Fix email_verified syntax in Python auth_service (#123)

* Add TS GraphQL support for Google OAuth (#122)

* GraphQL OAuth

* rename id_token

* Validators

* Update authValidators.ts

* Fix generate_csv_from_list call in entity and user routes (#119)

* Integrate file uploads and downloads into frontend  (#116)

* add initial upload button and onChange function

* Add field to display table

* Fix mutation and add console.logs to debug

* Update graphql queries in create and update

* Add call in EntityAPIClient, graphql in DisplayTableContainer, and setup FileUtil

* Fix display of buttons to only be shown if the fileName is not null

* Add suggested changes to remove circular JSON errors

* Uncomment python part in docker-compose.yml

* Address PR comments and fix Graphql get in DisplayTableContainer

* Fix REST file upload

* Add REST file update to frontend

* Decamelize formData when using Python backend

* Fix fileUrl access in EntityAPIClient.getFile for Python backend

Co-authored-by: Alex Guo <[email protected]>
Co-authored-by: Sherry Li <[email protected]>

* Delete duplicate DecodedJWT import in BaseAPIClient.ts

* Fix tag spacing in CRUD components

* Fix field_names logic in csv_utils when file_name is optional

Co-authored-by: Xin Hao Zhang <[email protected]>
Co-authored-by: alexguo8 <[email protected]>
Co-authored-by: Emily Louie <[email protected]>
Co-authored-by: Tamim Khan <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Dinu Wijetunga <[email protected]>
Co-authored-by: Alex Guo <[email protected]>
sherryhli added a commit that referenced this pull request Sep 9, 2021
* Add example context components for frontend state management (#37)

* Add example context components

* Add ts root dir to fix eslint errors

* Fix typo, add return type to reducer

* Add react plugins to eslint

* Update README.md (#46)

* Linting fixes (#42)

* Add ts plugins to eslint, fix eslint errors

* Remove deep copy of form data

* Add more return types, change prettier errors to warn

* Type decoded JWT, move AuthenticatedUser to types file

* Create AuthTypes

* Fix token expiry

Co-authored-by: Sherry Li <[email protected]>

* Refactor auth functions that use localStorage logic into AuthAPIClient (#45)

* duplicate code to authapiclient.ts

* refactor functions with localstorage to AuthAPIClient

* Update with removed REST tags

* update with pr fixes

* Update mongoose version to remove warnings (#47)

* Configure Sequelize migrations (#48)

* Configure Sequelize migrations using umzug

* Add defaultValue and allowNull options to add column example

* Add swagger for typescript backend (#51)

* Add Python REST Validators (#50)

* add initial validate decorator

* update with modified middleware and add decorators to the respective functions

* add decorator and validate method for entity dto

* Update README.md and add nodemailer config path to secret.config (#49)

* Update README.md

* Add nodemailer config path to secret.config

* Add frontend stack to README.md

* Add minor edits to README.md

* Add TypeScript REST API validation (#52)

* Add auth validators

* Add Entity and User Validators

* Refactor validation utils

* Add API error messages

* Swagger For Python (#54)

* Swagger For Python

* Minor fixes

* Include file name in log using wrapper for Winston typescript logger (#55)

Co-authored-by: Bruce He <[email protected]>

* Create Python FileStorageService (#62)

* Add initial file storage service

* Add initial file storage service

* Create and Update

* Add exception handling and more docstrings

* Make expiration time a parameter, clean up code

* Add content type parameter, update interface

Co-authored-by: Richard Chen <[email protected]>

* Create TypeScript FileStorageService (#59)

* add initial files

* Fix firebase bucket init

* add create file implementation

* add interface notes and add checking if file already exists

* Add updateFile and deleteFile implementations

* update create file method

* update create method notes

* Refactor getFile

* remove import in server.ts

* Update error message on failure to get file

Co-authored-by: Sherry Li <[email protected]>

* add contentType param

* fix formatting in storageService.ts

* Make bucketName a param

* Fix bucket name format

Co-authored-by: tamim-khan <[email protected]>
Co-authored-by: Sherry Li <[email protected]>

* Add testing frameworks (#57)

* Delete repeated requirements

* Add pytest to requirements

* Create python test examples

* Add typescript BE examples

* Create frontend sample tests

* Add test-db to docker-compose

* Update tests for docker containers

* Update psql conn string

* Add mongodb-memory-server dependency libcurl3 to docker container

* Add mongomock and pytest-mock to py requirements

* Create test_user_service

* Add pg user service test

* Configure SQLAlchemy migrations (#58)

* Configure SQLAlchemy migrations

* Fix create_app configs when calling from Flask CLI

* Update README.md (#64)

* Update README.md

* Apply suggestions from code review

Co-authored-by: Sherry Li <[email protected]>

Co-authored-by: Sherry Li <[email protected]>

* Set up frontend deployment and previews with Firebase Hosting and GitHub Actions (#87)

* Set up frontend hosting using Firebase

* Set up GitHub Actions for frontend deploy preview and automated prod deployment

* Set working directory in frontend deployment GitHub Actions

* Add yarn install step to deployment scripts

* Add entryPoint to action-hosting-deploy step

* Use separate backend urls for previews and prod

* Fix several minor issues (#63)

* Set extended option in express.urlencoded in server.ts

* Fix user_id casing in Python openapi.json

* Simplify Python DTO validators

* Add newline in Logout.tsx

* Rename TypeScript storageService to fileStorageService

* Integrate TS FileStorageService into EntityService (#88)

* temp changes

* add initial changes to EntityServiceMg

* Fix update function in entityservice

* Add file crud methods to postgres entity service

* Add file content type validation

* Fix create endpoints

* Add local file storing to graphql resolvers

* Add endpoint for getting file based on fileUUID

* Change storage service to store bucketName instead of the bucket

* Minor fixes

* Fix update functions in entity service

* Minor fixes

* Add content type to entity request DTO and filestorageservice calls

* Fix graphql

* Fix imports

* Refactor create/update to prevent dangling file references

* Address comments

* Wait for file to finish writing before continuing

* Delete file after uploading

* Add awaits to filestorageservice calls

Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Emily Louie <[email protected]>

* Add frontend csv utility (#90)

* Integrate Python FileStorageService into EntityService (#86)

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Fix python file get endpoint file storage service call

* Remove EntityDTO.file_name from Mongo Impl

* Update validator to check for missing body

* fix

* Fix file get endpoint auth, rename entity DTO variable

* Remove EntityDTO.file_name from Postgres Impl

* Update get_file to return object

* Fix bug in mongo service update logic

* Fix Postgres update route and logic

* Fix python postgres pop file and lint code.

* Postgres migration to add entity file name

Co-authored-by: Bruce He <[email protected]>

* Add backend TS csv generation utility (#91)

* initial files

* add a responseutil

* update rest api

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

Co-authored-by: Alex Guo <[email protected]>

* Add download csv functionality from frontend utility (#95)

* initial files

* add a responseutil

* update rest api

* Add download csv functionality from frontend utility

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

* Add backend API integration

Co-authored-by: Emily Louie <[email protected]>

* Refactor route strings into constants (#111)

* Backend CSV generation Python (#108)

* Initial change

Linter and tests

Update default options

csv default settings

Examples and minor format changes

Update docker-compose.yml

Moved tests to unit

* Add minor fixes

Co-authored-by: Sherry Li <[email protected]>

* Create Typescript registration flow with email verification (#104)

* Add endpoints for register user with email verification

* Check email verified on the firebase user instead of the token

* Create Python registration flow with email verification (#105)

* Add endpoints to register user using email verification

* fix import and add validator

* Add validation for RegisterUserDTO

* Fix accessing the email verification from token

* Change the verification email title

* check email verified in firebase user rather than token

* Frontend User Sign-Up (#115)

* Add sign-up page & auth client register functions

* Rebase on main and make minor routing changes

Co-authored-by: Sherry Li <[email protected]>

* Integrate Google OAuth into Python backend (#112)

* Add google oauth signin to firebase rest client

* Add google oauth login and signup functionality

* Add frontend for google oauth signin

* Fix backend

* Add comment for google user id token

* Integrate Google OAuth into TS (#114)

* Add REST Impl

* Fix try catch

Co-authored-by: Alex Guo <[email protected]>

* Fix email_verified syntax in Python auth_service (#123)

* Add TS GraphQL support for Google OAuth (#122)

* GraphQL OAuth

* rename id_token

* Validators

* Update authValidators.ts

* Fix generate_csv_from_list call in entity and user routes (#119)

* Integrate file uploads and downloads into frontend  (#116)

* add initial upload button and onChange function

* Add field to display table

* Fix mutation and add console.logs to debug

* Update graphql queries in create and update

* Add call in EntityAPIClient, graphql in DisplayTableContainer, and setup FileUtil

* Fix display of buttons to only be shown if the fileName is not null

* Add suggested changes to remove circular JSON errors

* Uncomment python part in docker-compose.yml

* Address PR comments and fix Graphql get in DisplayTableContainer

* Fix REST file upload

* Add REST file update to frontend

* Decamelize formData when using Python backend

* Fix fileUrl access in EntityAPIClient.getFile for Python backend

Co-authored-by: Alex Guo <[email protected]>
Co-authored-by: Sherry Li <[email protected]>

* Create HooksDemo page (#125)

Co-authored-by: Xin Hao Zhang <[email protected]>
Co-authored-by: alexguo8 <[email protected]>
Co-authored-by: Emily Louie <[email protected]>
Co-authored-by: Tamim Khan <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Dinu Wijetunga <[email protected]>
Co-authored-by: Alex Guo <[email protected]>
sherryhli added a commit that referenced this pull request Sep 10, 2021
* Add example context components for frontend state management (#37)

* Add example context components

* Add ts root dir to fix eslint errors

* Fix typo, add return type to reducer

* Add react plugins to eslint

* Update README.md (#46)

* Linting fixes (#42)

* Add ts plugins to eslint, fix eslint errors

* Remove deep copy of form data

* Add more return types, change prettier errors to warn

* Type decoded JWT, move AuthenticatedUser to types file

* Create AuthTypes

* Fix token expiry

Co-authored-by: Sherry Li <[email protected]>

* Refactor auth functions that use localStorage logic into AuthAPIClient (#45)

* duplicate code to authapiclient.ts

* refactor functions with localstorage to AuthAPIClient

* Update with removed REST tags

* update with pr fixes

* Update mongoose version to remove warnings (#47)

* Configure Sequelize migrations (#48)

* Configure Sequelize migrations using umzug

* Add defaultValue and allowNull options to add column example

* Add swagger for typescript backend (#51)

* Add Python REST Validators (#50)

* add initial validate decorator

* update with modified middleware and add decorators to the respective functions

* add decorator and validate method for entity dto

* Update README.md and add nodemailer config path to secret.config (#49)

* Update README.md

* Add nodemailer config path to secret.config

* Add frontend stack to README.md

* Add minor edits to README.md

* Add TypeScript REST API validation (#52)

* Add auth validators

* Add Entity and User Validators

* Refactor validation utils

* Add API error messages

* Swagger For Python (#54)

* Swagger For Python

* Minor fixes

* Include file name in log using wrapper for Winston typescript logger (#55)

Co-authored-by: Bruce He <[email protected]>

* Create Python FileStorageService (#62)

* Add initial file storage service

* Add initial file storage service

* Create and Update

* Add exception handling and more docstrings

* Make expiration time a parameter, clean up code

* Add content type parameter, update interface

Co-authored-by: Richard Chen <[email protected]>

* Create TypeScript FileStorageService (#59)

* add initial files

* Fix firebase bucket init

* add create file implementation

* add interface notes and add checking if file already exists

* Add updateFile and deleteFile implementations

* update create file method

* update create method notes

* Refactor getFile

* remove import in server.ts

* Update error message on failure to get file

Co-authored-by: Sherry Li <[email protected]>

* add contentType param

* fix formatting in storageService.ts

* Make bucketName a param

* Fix bucket name format

Co-authored-by: tamim-khan <[email protected]>
Co-authored-by: Sherry Li <[email protected]>

* Add testing frameworks (#57)

* Delete repeated requirements

* Add pytest to requirements

* Create python test examples

* Add typescript BE examples

* Create frontend sample tests

* Add test-db to docker-compose

* Update tests for docker containers

* Update psql conn string

* Add mongodb-memory-server dependency libcurl3 to docker container

* Add mongomock and pytest-mock to py requirements

* Create test_user_service

* Add pg user service test

* Configure SQLAlchemy migrations (#58)

* Configure SQLAlchemy migrations

* Fix create_app configs when calling from Flask CLI

* Update README.md (#64)

* Update README.md

* Apply suggestions from code review

Co-authored-by: Sherry Li <[email protected]>

Co-authored-by: Sherry Li <[email protected]>

* Set up frontend deployment and previews with Firebase Hosting and GitHub Actions (#87)

* Set up frontend hosting using Firebase

* Set up GitHub Actions for frontend deploy preview and automated prod deployment

* Set working directory in frontend deployment GitHub Actions

* Add yarn install step to deployment scripts

* Add entryPoint to action-hosting-deploy step

* Use separate backend urls for previews and prod

* Fix several minor issues (#63)

* Set extended option in express.urlencoded in server.ts

* Fix user_id casing in Python openapi.json

* Simplify Python DTO validators

* Add newline in Logout.tsx

* Rename TypeScript storageService to fileStorageService

* Integrate TS FileStorageService into EntityService (#88)

* temp changes

* add initial changes to EntityServiceMg

* Fix update function in entityservice

* Add file crud methods to postgres entity service

* Add file content type validation

* Fix create endpoints

* Add local file storing to graphql resolvers

* Add endpoint for getting file based on fileUUID

* Change storage service to store bucketName instead of the bucket

* Minor fixes

* Fix update functions in entity service

* Minor fixes

* Add content type to entity request DTO and filestorageservice calls

* Fix graphql

* Fix imports

* Refactor create/update to prevent dangling file references

* Address comments

* Wait for file to finish writing before continuing

* Delete file after uploading

* Add awaits to filestorageservice calls

Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Emily Louie <[email protected]>

* Add frontend csv utility (#90)

* Integrate Python FileStorageService into EntityService (#86)

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Fix python file get endpoint file storage service call

* Remove EntityDTO.file_name from Mongo Impl

* Update validator to check for missing body

* fix

* Fix file get endpoint auth, rename entity DTO variable

* Remove EntityDTO.file_name from Postgres Impl

* Update get_file to return object

* Fix bug in mongo service update logic

* Fix Postgres update route and logic

* Fix python postgres pop file and lint code.

* Postgres migration to add entity file name

Co-authored-by: Bruce He <[email protected]>

* Add backend TS csv generation utility (#91)

* initial files

* add a responseutil

* update rest api

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

Co-authored-by: Alex Guo <[email protected]>

* Add download csv functionality from frontend utility (#95)

* initial files

* add a responseutil

* update rest api

* Add download csv functionality from frontend utility

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

* Add backend API integration

Co-authored-by: Emily Louie <[email protected]>

* Refactor route strings into constants (#111)

* Backend CSV generation Python (#108)

* Initial change

Linter and tests

Update default options

csv default settings

Examples and minor format changes

Update docker-compose.yml

Moved tests to unit

* Add minor fixes

Co-authored-by: Sherry Li <[email protected]>

* Create Typescript registration flow with email verification (#104)

* Add endpoints for register user with email verification

* Check email verified on the firebase user instead of the token

* Create Python registration flow with email verification (#105)

* Add endpoints to register user using email verification

* fix import and add validator

* Add validation for RegisterUserDTO

* Fix accessing the email verification from token

* Change the verification email title

* check email verified in firebase user rather than token

* Frontend User Sign-Up (#115)

* Add sign-up page & auth client register functions

* Rebase on main and make minor routing changes

Co-authored-by: Sherry Li <[email protected]>

* Integrate Google OAuth into Python backend (#112)

* Add google oauth signin to firebase rest client

* Add google oauth login and signup functionality

* Add frontend for google oauth signin

* Fix backend

* Add comment for google user id token

* Integrate Google OAuth into TS (#114)

* Add REST Impl

* Fix try catch

Co-authored-by: Alex Guo <[email protected]>

* Fix email_verified syntax in Python auth_service (#123)

* Add TS GraphQL support for Google OAuth (#122)

* GraphQL OAuth

* rename id_token

* Validators

* Update authValidators.ts

* Fix generate_csv_from_list call in entity and user routes (#119)

* Integrate file uploads and downloads into frontend  (#116)

* add initial upload button and onChange function

* Add field to display table

* Fix mutation and add console.logs to debug

* Update graphql queries in create and update

* Add call in EntityAPIClient, graphql in DisplayTableContainer, and setup FileUtil

* Fix display of buttons to only be shown if the fileName is not null

* Add suggested changes to remove circular JSON errors

* Uncomment python part in docker-compose.yml

* Address PR comments and fix Graphql get in DisplayTableContainer

* Fix REST file upload

* Add REST file update to frontend

* Decamelize formData when using Python backend

* Fix fileUrl access in EntityAPIClient.getFile for Python backend

Co-authored-by: Alex Guo <[email protected]>
Co-authored-by: Sherry Li <[email protected]>

* Create HooksDemo page (#125)

* Update README.md (#128)

* Fix various bugs (#127)

* Add type check on self.enum_field before calling .upper() in validator

* Set erase_db_and_sync in Python backend based on TESTING config value

* Change DB dialect name from postgres to postgresql in Python connection string

* Update testDb connect method for compatibility with mongo-memory-server's updated API

* Change setData to setEntities in DisplayTableContainer.tsx

* Add comments to AuthService's generateTokenOAuth method

* Separate SignInResponse types for password and OAuth sign ins

* Fix Python csv_utils to account for optional fields

* Add *.egg-info to .gitignore

* Remove commented out code in backend/typescript/utilities/CSVUtils.ts

* Lint and reformat

* Fix form reset issue after setting file field in CreateForm and UpdateForm

* Fix casing in CSVUtils import path

Co-authored-by: Xin Hao Zhang <[email protected]>
Co-authored-by: alexguo8 <[email protected]>
Co-authored-by: Emily Louie <[email protected]>
Co-authored-by: Tamim Khan <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Dinu Wijetunga <[email protected]>
Co-authored-by: Alex Guo <[email protected]>
sherryhli added a commit that referenced this pull request Sep 11, 2021
* Add example context components for frontend state management (#37)

* Add example context components

* Add ts root dir to fix eslint errors

* Fix typo, add return type to reducer

* Add react plugins to eslint

* Update README.md (#46)

* Linting fixes (#42)

* Add ts plugins to eslint, fix eslint errors

* Remove deep copy of form data

* Add more return types, change prettier errors to warn

* Type decoded JWT, move AuthenticatedUser to types file

* Create AuthTypes

* Fix token expiry

Co-authored-by: Sherry Li <[email protected]>

* Refactor auth functions that use localStorage logic into AuthAPIClient (#45)

* duplicate code to authapiclient.ts

* refactor functions with localstorage to AuthAPIClient

* Update with removed REST tags

* update with pr fixes

* Update mongoose version to remove warnings (#47)

* Configure Sequelize migrations (#48)

* Configure Sequelize migrations using umzug

* Add defaultValue and allowNull options to add column example

* Add swagger for typescript backend (#51)

* Add Python REST Validators (#50)

* add initial validate decorator

* update with modified middleware and add decorators to the respective functions

* add decorator and validate method for entity dto

* Update README.md and add nodemailer config path to secret.config (#49)

* Update README.md

* Add nodemailer config path to secret.config

* Add frontend stack to README.md

* Add minor edits to README.md

* Add TypeScript REST API validation (#52)

* Add auth validators

* Add Entity and User Validators

* Refactor validation utils

* Add API error messages

* Swagger For Python (#54)

* Swagger For Python

* Minor fixes

* Include file name in log using wrapper for Winston typescript logger (#55)

Co-authored-by: Bruce He <[email protected]>

* Create Python FileStorageService (#62)

* Add initial file storage service

* Add initial file storage service

* Create and Update

* Add exception handling and more docstrings

* Make expiration time a parameter, clean up code

* Add content type parameter, update interface

Co-authored-by: Richard Chen <[email protected]>

* Create TypeScript FileStorageService (#59)

* add initial files

* Fix firebase bucket init

* add create file implementation

* add interface notes and add checking if file already exists

* Add updateFile and deleteFile implementations

* update create file method

* update create method notes

* Refactor getFile

* remove import in server.ts

* Update error message on failure to get file

Co-authored-by: Sherry Li <[email protected]>

* add contentType param

* fix formatting in storageService.ts

* Make bucketName a param

* Fix bucket name format

Co-authored-by: tamim-khan <[email protected]>
Co-authored-by: Sherry Li <[email protected]>

* Add testing frameworks (#57)

* Delete repeated requirements

* Add pytest to requirements

* Create python test examples

* Add typescript BE examples

* Create frontend sample tests

* Add test-db to docker-compose

* Update tests for docker containers

* Update psql conn string

* Add mongodb-memory-server dependency libcurl3 to docker container

* Add mongomock and pytest-mock to py requirements

* Create test_user_service

* Add pg user service test

* Configure SQLAlchemy migrations (#58)

* Configure SQLAlchemy migrations

* Fix create_app configs when calling from Flask CLI

* Update README.md (#64)

* Update README.md

* Apply suggestions from code review

Co-authored-by: Sherry Li <[email protected]>

Co-authored-by: Sherry Li <[email protected]>

* Set up frontend deployment and previews with Firebase Hosting and GitHub Actions (#87)

* Set up frontend hosting using Firebase

* Set up GitHub Actions for frontend deploy preview and automated prod deployment

* Set working directory in frontend deployment GitHub Actions

* Add yarn install step to deployment scripts

* Add entryPoint to action-hosting-deploy step

* Use separate backend urls for previews and prod

* Fix several minor issues (#63)

* Set extended option in express.urlencoded in server.ts

* Fix user_id casing in Python openapi.json

* Simplify Python DTO validators

* Add newline in Logout.tsx

* Rename TypeScript storageService to fileStorageService

* Integrate TS FileStorageService into EntityService (#88)

* temp changes

* add initial changes to EntityServiceMg

* Fix update function in entityservice

* Add file crud methods to postgres entity service

* Add file content type validation

* Fix create endpoints

* Add local file storing to graphql resolvers

* Add endpoint for getting file based on fileUUID

* Change storage service to store bucketName instead of the bucket

* Minor fixes

* Fix update functions in entity service

* Minor fixes

* Add content type to entity request DTO and filestorageservice calls

* Fix graphql

* Fix imports

* Refactor create/update to prevent dangling file references

* Address comments

* Wait for file to finish writing before continuing

* Delete file after uploading

* Add awaits to filestorageservice calls

Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Emily Louie <[email protected]>

* Add frontend csv utility (#90)

* Integrate Python FileStorageService into EntityService (#86)

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Fix python file get endpoint file storage service call

* Remove EntityDTO.file_name from Mongo Impl

* Update validator to check for missing body

* fix

* Fix file get endpoint auth, rename entity DTO variable

* Remove EntityDTO.file_name from Postgres Impl

* Update get_file to return object

* Fix bug in mongo service update logic

* Fix Postgres update route and logic

* Fix python postgres pop file and lint code.

* Postgres migration to add entity file name

Co-authored-by: Bruce He <[email protected]>

* Add backend TS csv generation utility (#91)

* initial files

* add a responseutil

* update rest api

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

Co-authored-by: Alex Guo <[email protected]>

* Add download csv functionality from frontend utility (#95)

* initial files

* add a responseutil

* update rest api

* Add download csv functionality from frontend utility

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

* Add backend API integration

Co-authored-by: Emily Louie <[email protected]>

* Refactor route strings into constants (#111)

* Backend CSV generation Python (#108)

* Initial change

Linter and tests

Update default options

csv default settings

Examples and minor format changes

Update docker-compose.yml

Moved tests to unit

* Add minor fixes

Co-authored-by: Sherry Li <[email protected]>

* Create Typescript registration flow with email verification (#104)

* Add endpoints for register user with email verification

* Check email verified on the firebase user instead of the token

* Create Python registration flow with email verification (#105)

* Add endpoints to register user using email verification

* fix import and add validator

* Add validation for RegisterUserDTO

* Fix accessing the email verification from token

* Change the verification email title

* check email verified in firebase user rather than token

* Frontend User Sign-Up (#115)

* Add sign-up page & auth client register functions

* Rebase on main and make minor routing changes

Co-authored-by: Sherry Li <[email protected]>

* Integrate Google OAuth into Python backend (#112)

* Add google oauth signin to firebase rest client

* Add google oauth login and signup functionality

* Add frontend for google oauth signin

* Fix backend

* Add comment for google user id token

* Integrate Google OAuth into TS (#114)

* Add REST Impl

* Fix try catch

Co-authored-by: Alex Guo <[email protected]>

* Fix email_verified syntax in Python auth_service (#123)

* Add TS GraphQL support for Google OAuth (#122)

* GraphQL OAuth

* rename id_token

* Validators

* Update authValidators.ts

* Fix generate_csv_from_list call in entity and user routes (#119)

* Integrate file uploads and downloads into frontend  (#116)

* add initial upload button and onChange function

* Add field to display table

* Fix mutation and add console.logs to debug

* Update graphql queries in create and update

* Add call in EntityAPIClient, graphql in DisplayTableContainer, and setup FileUtil

* Fix display of buttons to only be shown if the fileName is not null

* Add suggested changes to remove circular JSON errors

* Uncomment python part in docker-compose.yml

* Address PR comments and fix Graphql get in DisplayTableContainer

* Fix REST file upload

* Add REST file update to frontend

* Decamelize formData when using Python backend

* Fix fileUrl access in EntityAPIClient.getFile for Python backend

Co-authored-by: Alex Guo <[email protected]>
Co-authored-by: Sherry Li <[email protected]>

* Create HooksDemo page (#125)

* Update README.md (#128)

* Fix various bugs (#127)

* Add type check on self.enum_field before calling .upper() in validator

* Set erase_db_and_sync in Python backend based on TESTING config value

* Change DB dialect name from postgres to postgresql in Python connection string

* Update testDb connect method for compatibility with mongo-memory-server's updated API

* Change setData to setEntities in DisplayTableContainer.tsx

* Add comments to AuthService's generateTokenOAuth method

* Separate SignInResponse types for password and OAuth sign ins

* Fix Python csv_utils to account for optional fields

* Add *.egg-info to .gitignore

* Remove commented out code in backend/typescript/utilities/CSVUtils.ts

* Lint and reformat

* Fix form reset issue after setting file field in CreateForm and UpdateForm

* Fix casing in CSVUtils import path

* Update Swagger docs with new endpoints and request body types (#131)

* Import EntityRequest in UpdateForm.tsx

* Include Firebase initialization when no-auth and file-storage are selected

Co-authored-by: Xin Hao Zhang <[email protected]>
Co-authored-by: alexguo8 <[email protected]>
Co-authored-by: Emily Louie <[email protected]>
Co-authored-by: Tamim Khan <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Dinu Wijetunga <[email protected]>
Co-authored-by: Alex Guo <[email protected]>
sherryhli added a commit that referenced this pull request Dec 19, 2021
* Add example context components for frontend state management (#37)

* Add example context components

* Add ts root dir to fix eslint errors

* Fix typo, add return type to reducer

* Add react plugins to eslint

* Update README.md (#46)

* Linting fixes (#42)

* Add ts plugins to eslint, fix eslint errors

* Remove deep copy of form data

* Add more return types, change prettier errors to warn

* Type decoded JWT, move AuthenticatedUser to types file

* Create AuthTypes

* Fix token expiry

Co-authored-by: Sherry Li <[email protected]>

* Refactor auth functions that use localStorage logic into AuthAPIClient (#45)

* duplicate code to authapiclient.ts

* refactor functions with localstorage to AuthAPIClient

* Update with removed REST tags

* update with pr fixes

* Update mongoose version to remove warnings (#47)

* Configure Sequelize migrations (#48)

* Configure Sequelize migrations using umzug

* Add defaultValue and allowNull options to add column example

* Add swagger for typescript backend (#51)

* Add Python REST Validators (#50)

* add initial validate decorator

* update with modified middleware and add decorators to the respective functions

* add decorator and validate method for entity dto

* Update README.md and add nodemailer config path to secret.config (#49)

* Update README.md

* Add nodemailer config path to secret.config

* Add frontend stack to README.md

* Add minor edits to README.md

* Add TypeScript REST API validation (#52)

* Add auth validators

* Add Entity and User Validators

* Refactor validation utils

* Add API error messages

* Swagger For Python (#54)

* Swagger For Python

* Minor fixes

* Include file name in log using wrapper for Winston typescript logger (#55)

Co-authored-by: Bruce He <[email protected]>

* Create Python FileStorageService (#62)

* Add initial file storage service

* Add initial file storage service

* Create and Update

* Add exception handling and more docstrings

* Make expiration time a parameter, clean up code

* Add content type parameter, update interface

Co-authored-by: Richard Chen <[email protected]>

* Create TypeScript FileStorageService (#59)

* add initial files

* Fix firebase bucket init

* add create file implementation

* add interface notes and add checking if file already exists

* Add updateFile and deleteFile implementations

* update create file method

* update create method notes

* Refactor getFile

* remove import in server.ts

* Update error message on failure to get file

Co-authored-by: Sherry Li <[email protected]>

* add contentType param

* fix formatting in storageService.ts

* Make bucketName a param

* Fix bucket name format

Co-authored-by: tamim-khan <[email protected]>
Co-authored-by: Sherry Li <[email protected]>

* Add testing frameworks (#57)

* Delete repeated requirements

* Add pytest to requirements

* Create python test examples

* Add typescript BE examples

* Create frontend sample tests

* Add test-db to docker-compose

* Update tests for docker containers

* Update psql conn string

* Add mongodb-memory-server dependency libcurl3 to docker container

* Add mongomock and pytest-mock to py requirements

* Create test_user_service

* Add pg user service test

* Configure SQLAlchemy migrations (#58)

* Configure SQLAlchemy migrations

* Fix create_app configs when calling from Flask CLI

* Update README.md (#64)

* Update README.md

* Apply suggestions from code review

Co-authored-by: Sherry Li <[email protected]>

Co-authored-by: Sherry Li <[email protected]>

* Set up frontend deployment and previews with Firebase Hosting and GitHub Actions (#87)

* Set up frontend hosting using Firebase

* Set up GitHub Actions for frontend deploy preview and automated prod deployment

* Set working directory in frontend deployment GitHub Actions

* Add yarn install step to deployment scripts

* Add entryPoint to action-hosting-deploy step

* Use separate backend urls for previews and prod

* Fix several minor issues (#63)

* Set extended option in express.urlencoded in server.ts

* Fix user_id casing in Python openapi.json

* Simplify Python DTO validators

* Add newline in Logout.tsx

* Rename TypeScript storageService to fileStorageService

* Integrate TS FileStorageService into EntityService (#88)

* temp changes

* add initial changes to EntityServiceMg

* Fix update function in entityservice

* Add file crud methods to postgres entity service

* Add file content type validation

* Fix create endpoints

* Add local file storing to graphql resolvers

* Add endpoint for getting file based on fileUUID

* Change storage service to store bucketName instead of the bucket

* Minor fixes

* Fix update functions in entity service

* Minor fixes

* Add content type to entity request DTO and filestorageservice calls

* Fix graphql

* Fix imports

* Refactor create/update to prevent dangling file references

* Address comments

* Wait for file to finish writing before continuing

* Delete file after uploading

* Add awaits to filestorageservice calls

Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Emily Louie <[email protected]>

* Add frontend csv utility (#90)

* Integrate Python FileStorageService into EntityService (#86)

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Fix python file get endpoint file storage service call

* Remove EntityDTO.file_name from Mongo Impl

* Update validator to check for missing body

* fix

* Fix file get endpoint auth, rename entity DTO variable

* Remove EntityDTO.file_name from Postgres Impl

* Update get_file to return object

* Fix bug in mongo service update logic

* Fix Postgres update route and logic

* Fix python postgres pop file and lint code.

* Postgres migration to add entity file name

Co-authored-by: Bruce He <[email protected]>

* Add backend TS csv generation utility (#91)

* initial files

* add a responseutil

* update rest api

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

Co-authored-by: Alex Guo <[email protected]>

* Add download csv functionality from frontend utility (#95)

* initial files

* add a responseutil

* update rest api

* Add download csv functionality from frontend utility

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

* Add backend API integration

Co-authored-by: Emily Louie <[email protected]>

* Refactor route strings into constants (#111)

* Backend CSV generation Python (#108)

* Initial change

Linter and tests

Update default options

csv default settings

Examples and minor format changes

Update docker-compose.yml

Moved tests to unit

* Add minor fixes

Co-authored-by: Sherry Li <[email protected]>

* Create Typescript registration flow with email verification (#104)

* Add endpoints for register user with email verification

* Check email verified on the firebase user instead of the token

* Create Python registration flow with email verification (#105)

* Add endpoints to register user using email verification

* fix import and add validator

* Add validation for RegisterUserDTO

* Fix accessing the email verification from token

* Change the verification email title

* check email verified in firebase user rather than token

* Frontend User Sign-Up (#115)

* Add sign-up page & auth client register functions

* Rebase on main and make minor routing changes

Co-authored-by: Sherry Li <[email protected]>

* Integrate Google OAuth into Python backend (#112)

* Add google oauth signin to firebase rest client

* Add google oauth login and signup functionality

* Add frontend for google oauth signin

* Fix backend

* Add comment for google user id token

* Integrate Google OAuth into TS (#114)

* Add REST Impl

* Fix try catch

Co-authored-by: Alex Guo <[email protected]>

* Fix email_verified syntax in Python auth_service (#123)

* Add TS GraphQL support for Google OAuth (#122)

* GraphQL OAuth

* rename id_token

* Validators

* Update authValidators.ts

* Fix generate_csv_from_list call in entity and user routes (#119)

* Integrate file uploads and downloads into frontend  (#116)

* add initial upload button and onChange function

* Add field to display table

* Fix mutation and add console.logs to debug

* Update graphql queries in create and update

* Add call in EntityAPIClient, graphql in DisplayTableContainer, and setup FileUtil

* Fix display of buttons to only be shown if the fileName is not null

* Add suggested changes to remove circular JSON errors

* Uncomment python part in docker-compose.yml

* Address PR comments and fix Graphql get in DisplayTableContainer

* Fix REST file upload

* Add REST file update to frontend

* Decamelize formData when using Python backend

* Fix fileUrl access in EntityAPIClient.getFile for Python backend

Co-authored-by: Alex Guo <[email protected]>
Co-authored-by: Sherry Li <[email protected]>

* Create HooksDemo page (#125)

* Update README.md (#128)

* Fix various bugs (#127)

* Add type check on self.enum_field before calling .upper() in validator

* Set erase_db_and_sync in Python backend based on TESTING config value

* Change DB dialect name from postgres to postgresql in Python connection string

* Update testDb connect method for compatibility with mongo-memory-server's updated API

* Change setData to setEntities in DisplayTableContainer.tsx

* Add comments to AuthService's generateTokenOAuth method

* Separate SignInResponse types for password and OAuth sign ins

* Fix Python csv_utils to account for optional fields

* Add *.egg-info to .gitignore

* Remove commented out code in backend/typescript/utilities/CSVUtils.ts

* Lint and reformat

* Fix form reset issue after setting file field in CreateForm and UpdateForm

* Fix casing in CSVUtils import path

* Update Swagger docs with new endpoints and request body types (#131)

* Fix error when deleting an entity with no file in Python (#133)

* Fix error when deleting an entity with no file

* Add id return for delete entity endpoint in TypeScript

* Address comment

* Modify and clean up infra setup (#134)

* Add db health check, db init script, and container names

* Make scripts executable

* Fix formatting in backend/python/app/__init__.py

* Fix whitespace

* Fix ApolloClient link when using file storage without auth

* Change user_id param type in Python update user endpoint

Co-authored-by: Xin Hao Zhang <[email protected]>
Co-authored-by: alexguo8 <[email protected]>
Co-authored-by: Emily Louie <[email protected]>
Co-authored-by: Tamim Khan <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Dinu Wijetunga <[email protected]>
Co-authored-by: Alex Guo <[email protected]>
sherryhli added a commit that referenced this pull request Jan 11, 2022
* Add example context components for frontend state management (#37)

* Add example context components

* Add ts root dir to fix eslint errors

* Fix typo, add return type to reducer

* Add react plugins to eslint

* Update README.md (#46)

* Linting fixes (#42)

* Add ts plugins to eslint, fix eslint errors

* Remove deep copy of form data

* Add more return types, change prettier errors to warn

* Type decoded JWT, move AuthenticatedUser to types file

* Create AuthTypes

* Fix token expiry

Co-authored-by: Sherry Li <[email protected]>

* Refactor auth functions that use localStorage logic into AuthAPIClient (#45)

* duplicate code to authapiclient.ts

* refactor functions with localstorage to AuthAPIClient

* Update with removed REST tags

* update with pr fixes

* Update mongoose version to remove warnings (#47)

* Configure Sequelize migrations (#48)

* Configure Sequelize migrations using umzug

* Add defaultValue and allowNull options to add column example

* Add swagger for typescript backend (#51)

* Add Python REST Validators (#50)

* add initial validate decorator

* update with modified middleware and add decorators to the respective functions

* add decorator and validate method for entity dto

* Update README.md and add nodemailer config path to secret.config (#49)

* Update README.md

* Add nodemailer config path to secret.config

* Add frontend stack to README.md

* Add minor edits to README.md

* Add TypeScript REST API validation (#52)

* Add auth validators

* Add Entity and User Validators

* Refactor validation utils

* Add API error messages

* Swagger For Python (#54)

* Swagger For Python

* Minor fixes

* Include file name in log using wrapper for Winston typescript logger (#55)

Co-authored-by: Bruce He <[email protected]>

* Create Python FileStorageService (#62)

* Add initial file storage service

* Add initial file storage service

* Create and Update

* Add exception handling and more docstrings

* Make expiration time a parameter, clean up code

* Add content type parameter, update interface

Co-authored-by: Richard Chen <[email protected]>

* Create TypeScript FileStorageService (#59)

* add initial files

* Fix firebase bucket init

* add create file implementation

* add interface notes and add checking if file already exists

* Add updateFile and deleteFile implementations

* update create file method

* update create method notes

* Refactor getFile

* remove import in server.ts

* Update error message on failure to get file

Co-authored-by: Sherry Li <[email protected]>

* add contentType param

* fix formatting in storageService.ts

* Make bucketName a param

* Fix bucket name format

Co-authored-by: tamim-khan <[email protected]>
Co-authored-by: Sherry Li <[email protected]>

* Add testing frameworks (#57)

* Delete repeated requirements

* Add pytest to requirements

* Create python test examples

* Add typescript BE examples

* Create frontend sample tests

* Add test-db to docker-compose

* Update tests for docker containers

* Update psql conn string

* Add mongodb-memory-server dependency libcurl3 to docker container

* Add mongomock and pytest-mock to py requirements

* Create test_user_service

* Add pg user service test

* Configure SQLAlchemy migrations (#58)

* Configure SQLAlchemy migrations

* Fix create_app configs when calling from Flask CLI

* Update README.md (#64)

* Update README.md

* Apply suggestions from code review

Co-authored-by: Sherry Li <[email protected]>

Co-authored-by: Sherry Li <[email protected]>

* Set up frontend deployment and previews with Firebase Hosting and GitHub Actions (#87)

* Set up frontend hosting using Firebase

* Set up GitHub Actions for frontend deploy preview and automated prod deployment

* Set working directory in frontend deployment GitHub Actions

* Add yarn install step to deployment scripts

* Add entryPoint to action-hosting-deploy step

* Use separate backend urls for previews and prod

* Fix several minor issues (#63)

* Set extended option in express.urlencoded in server.ts

* Fix user_id casing in Python openapi.json

* Simplify Python DTO validators

* Add newline in Logout.tsx

* Rename TypeScript storageService to fileStorageService

* Integrate TS FileStorageService into EntityService (#88)

* temp changes

* add initial changes to EntityServiceMg

* Fix update function in entityservice

* Add file crud methods to postgres entity service

* Add file content type validation

* Fix create endpoints

* Add local file storing to graphql resolvers

* Add endpoint for getting file based on fileUUID

* Change storage service to store bucketName instead of the bucket

* Minor fixes

* Fix update functions in entity service

* Minor fixes

* Add content type to entity request DTO and filestorageservice calls

* Fix graphql

* Fix imports

* Refactor create/update to prevent dangling file references

* Address comments

* Wait for file to finish writing before continuing

* Delete file after uploading

* Add awaits to filestorageservice calls

Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Emily Louie <[email protected]>

* Add frontend csv utility (#90)

* Integrate Python FileStorageService into EntityService (#86)

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Fix python file get endpoint file storage service call

* Remove EntityDTO.file_name from Mongo Impl

* Update validator to check for missing body

* fix

* Fix file get endpoint auth, rename entity DTO variable

* Remove EntityDTO.file_name from Postgres Impl

* Update get_file to return object

* Fix bug in mongo service update logic

* Fix Postgres update route and logic

* Fix python postgres pop file and lint code.

* Postgres migration to add entity file name

Co-authored-by: Bruce He <[email protected]>

* Add backend TS csv generation utility (#91)

* initial files

* add a responseutil

* update rest api

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

Co-authored-by: Alex Guo <[email protected]>

* Add download csv functionality from frontend utility (#95)

* initial files

* add a responseutil

* update rest api

* Add download csv functionality from frontend utility

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

* Add backend API integration

Co-authored-by: Emily Louie <[email protected]>

* Refactor route strings into constants (#111)

* Backend CSV generation Python (#108)

* Initial change

Linter and tests

Update default options

csv default settings

Examples and minor format changes

Update docker-compose.yml

Moved tests to unit

* Add minor fixes

Co-authored-by: Sherry Li <[email protected]>

* Create Typescript registration flow with email verification (#104)

* Add endpoints for register user with email verification

* Check email verified on the firebase user instead of the token

* Create Python registration flow with email verification (#105)

* Add endpoints to register user using email verification

* fix import and add validator

* Add validation for RegisterUserDTO

* Fix accessing the email verification from token

* Change the verification email title

* check email verified in firebase user rather than token

* Frontend User Sign-Up (#115)

* Add sign-up page & auth client register functions

* Rebase on main and make minor routing changes

Co-authored-by: Sherry Li <[email protected]>

* Integrate Google OAuth into Python backend (#112)

* Add google oauth signin to firebase rest client

* Add google oauth login and signup functionality

* Add frontend for google oauth signin

* Fix backend

* Add comment for google user id token

* Integrate Google OAuth into TS (#114)

* Add REST Impl

* Fix try catch

Co-authored-by: Alex Guo <[email protected]>

* Fix email_verified syntax in Python auth_service (#123)

* Add TS GraphQL support for Google OAuth (#122)

* GraphQL OAuth

* rename id_token

* Validators

* Update authValidators.ts

* Fix generate_csv_from_list call in entity and user routes (#119)

* Integrate file uploads and downloads into frontend  (#116)

* add initial upload button and onChange function

* Add field to display table

* Fix mutation and add console.logs to debug

* Update graphql queries in create and update

* Add call in EntityAPIClient, graphql in DisplayTableContainer, and setup FileUtil

* Fix display of buttons to only be shown if the fileName is not null

* Add suggested changes to remove circular JSON errors

* Uncomment python part in docker-compose.yml

* Address PR comments and fix Graphql get in DisplayTableContainer

* Fix REST file upload

* Add REST file update to frontend

* Decamelize formData when using Python backend

* Fix fileUrl access in EntityAPIClient.getFile for Python backend

Co-authored-by: Alex Guo <[email protected]>
Co-authored-by: Sherry Li <[email protected]>

* Create HooksDemo page (#125)

* Update README.md (#128)

* Fix various bugs (#127)

* Add type check on self.enum_field before calling .upper() in validator

* Set erase_db_and_sync in Python backend based on TESTING config value

* Change DB dialect name from postgres to postgresql in Python connection string

* Update testDb connect method for compatibility with mongo-memory-server's updated API

* Change setData to setEntities in DisplayTableContainer.tsx

* Add comments to AuthService's generateTokenOAuth method

* Separate SignInResponse types for password and OAuth sign ins

* Fix Python csv_utils to account for optional fields

* Add *.egg-info to .gitignore

* Remove commented out code in backend/typescript/utilities/CSVUtils.ts

* Lint and reformat

* Fix form reset issue after setting file field in CreateForm and UpdateForm

* Fix casing in CSVUtils import path

* Update Swagger docs with new endpoints and request body types (#131)

* Fix error when deleting an entity with no file in Python (#133)

* Fix error when deleting an entity with no file

* Add id return for delete entity endpoint in TypeScript

* Address comment

* Modify and clean up infra setup (#134)

* Add db health check, db init script, and container names

* Make scripts executable

* Fix formatting in backend/python/app/__init__.py

* Return string id from TypeScript PostgreSQL user and entity services (#136)

* Add end-to-end tests for the backend (#137)

* Add end-to-end tests for the backend

* Rename folder

Co-authored-by: Xin Hao Zhang <[email protected]>
Co-authored-by: alexguo8 <[email protected]>
Co-authored-by: Emily Louie <[email protected]>
Co-authored-by: Tamim Khan <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Dinu Wijetunga <[email protected]>
Co-authored-by: Alex Guo <[email protected]>
sherryhli added a commit that referenced this pull request Jan 14, 2022
* Add example context components for frontend state management (#37)

* Add example context components

* Add ts root dir to fix eslint errors

* Fix typo, add return type to reducer

* Add react plugins to eslint

* Update README.md (#46)

* Linting fixes (#42)

* Add ts plugins to eslint, fix eslint errors

* Remove deep copy of form data

* Add more return types, change prettier errors to warn

* Type decoded JWT, move AuthenticatedUser to types file

* Create AuthTypes

* Fix token expiry

Co-authored-by: Sherry Li <[email protected]>

* Refactor auth functions that use localStorage logic into AuthAPIClient (#45)

* duplicate code to authapiclient.ts

* refactor functions with localstorage to AuthAPIClient

* Update with removed REST tags

* update with pr fixes

* Update mongoose version to remove warnings (#47)

* Configure Sequelize migrations (#48)

* Configure Sequelize migrations using umzug

* Add defaultValue and allowNull options to add column example

* Add swagger for typescript backend (#51)

* Add Python REST Validators (#50)

* add initial validate decorator

* update with modified middleware and add decorators to the respective functions

* add decorator and validate method for entity dto

* Update README.md and add nodemailer config path to secret.config (#49)

* Update README.md

* Add nodemailer config path to secret.config

* Add frontend stack to README.md

* Add minor edits to README.md

* Add TypeScript REST API validation (#52)

* Add auth validators

* Add Entity and User Validators

* Refactor validation utils

* Add API error messages

* Swagger For Python (#54)

* Swagger For Python

* Minor fixes

* Include file name in log using wrapper for Winston typescript logger (#55)

Co-authored-by: Bruce He <[email protected]>

* Create Python FileStorageService (#62)

* Add initial file storage service

* Add initial file storage service

* Create and Update

* Add exception handling and more docstrings

* Make expiration time a parameter, clean up code

* Add content type parameter, update interface

Co-authored-by: Richard Chen <[email protected]>

* Create TypeScript FileStorageService (#59)

* add initial files

* Fix firebase bucket init

* add create file implementation

* add interface notes and add checking if file already exists

* Add updateFile and deleteFile implementations

* update create file method

* update create method notes

* Refactor getFile

* remove import in server.ts

* Update error message on failure to get file

Co-authored-by: Sherry Li <[email protected]>

* add contentType param

* fix formatting in storageService.ts

* Make bucketName a param

* Fix bucket name format

Co-authored-by: tamim-khan <[email protected]>
Co-authored-by: Sherry Li <[email protected]>

* Add testing frameworks (#57)

* Delete repeated requirements

* Add pytest to requirements

* Create python test examples

* Add typescript BE examples

* Create frontend sample tests

* Add test-db to docker-compose

* Update tests for docker containers

* Update psql conn string

* Add mongodb-memory-server dependency libcurl3 to docker container

* Add mongomock and pytest-mock to py requirements

* Create test_user_service

* Add pg user service test

* Configure SQLAlchemy migrations (#58)

* Configure SQLAlchemy migrations

* Fix create_app configs when calling from Flask CLI

* Update README.md (#64)

* Update README.md

* Apply suggestions from code review

Co-authored-by: Sherry Li <[email protected]>

Co-authored-by: Sherry Li <[email protected]>

* Set up frontend deployment and previews with Firebase Hosting and GitHub Actions (#87)

* Set up frontend hosting using Firebase

* Set up GitHub Actions for frontend deploy preview and automated prod deployment

* Set working directory in frontend deployment GitHub Actions

* Add yarn install step to deployment scripts

* Add entryPoint to action-hosting-deploy step

* Use separate backend urls for previews and prod

* Fix several minor issues (#63)

* Set extended option in express.urlencoded in server.ts

* Fix user_id casing in Python openapi.json

* Simplify Python DTO validators

* Add newline in Logout.tsx

* Rename TypeScript storageService to fileStorageService

* Integrate TS FileStorageService into EntityService (#88)

* temp changes

* add initial changes to EntityServiceMg

* Fix update function in entityservice

* Add file crud methods to postgres entity service

* Add file content type validation

* Fix create endpoints

* Add local file storing to graphql resolvers

* Add endpoint for getting file based on fileUUID

* Change storage service to store bucketName instead of the bucket

* Minor fixes

* Fix update functions in entity service

* Minor fixes

* Add content type to entity request DTO and filestorageservice calls

* Fix graphql

* Fix imports

* Refactor create/update to prevent dangling file references

* Address comments

* Wait for file to finish writing before continuing

* Delete file after uploading

* Add awaits to filestorageservice calls

Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Emily Louie <[email protected]>

* Add frontend csv utility (#90)

* Integrate Python FileStorageService into EntityService (#86)

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Fix python file get endpoint file storage service call

* Remove EntityDTO.file_name from Mongo Impl

* Update validator to check for missing body

* fix

* Fix file get endpoint auth, rename entity DTO variable

* Remove EntityDTO.file_name from Postgres Impl

* Update get_file to return object

* Fix bug in mongo service update logic

* Fix Postgres update route and logic

* Fix python postgres pop file and lint code.

* Postgres migration to add entity file name

Co-authored-by: Bruce He <[email protected]>

* Add backend TS csv generation utility (#91)

* initial files

* add a responseutil

* update rest api

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

Co-authored-by: Alex Guo <[email protected]>

* Add download csv functionality from frontend utility (#95)

* initial files

* add a responseutil

* update rest api

* Add download csv functionality from frontend utility

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

* Add backend API integration

Co-authored-by: Emily Louie <[email protected]>

* Refactor route strings into constants (#111)

* Backend CSV generation Python (#108)

* Initial change

Linter and tests

Update default options

csv default settings

Examples and minor format changes

Update docker-compose.yml

Moved tests to unit

* Add minor fixes

Co-authored-by: Sherry Li <[email protected]>

* Create Typescript registration flow with email verification (#104)

* Add endpoints for register user with email verification

* Check email verified on the firebase user instead of the token

* Create Python registration flow with email verification (#105)

* Add endpoints to register user using email verification

* fix import and add validator

* Add validation for RegisterUserDTO

* Fix accessing the email verification from token

* Change the verification email title

* check email verified in firebase user rather than token

* Frontend User Sign-Up (#115)

* Add sign-up page & auth client register functions

* Rebase on main and make minor routing changes

Co-authored-by: Sherry Li <[email protected]>

* Integrate Google OAuth into Python backend (#112)

* Add google oauth signin to firebase rest client

* Add google oauth login and signup functionality

* Add frontend for google oauth signin

* Fix backend

* Add comment for google user id token

* Integrate Google OAuth into TS (#114)

* Add REST Impl

* Fix try catch

Co-authored-by: Alex Guo <[email protected]>

* Fix email_verified syntax in Python auth_service (#123)

* Add TS GraphQL support for Google OAuth (#122)

* GraphQL OAuth

* rename id_token

* Validators

* Update authValidators.ts

* Fix generate_csv_from_list call in entity and user routes (#119)

* Integrate file uploads and downloads into frontend  (#116)

* add initial upload button and onChange function

* Add field to display table

* Fix mutation and add console.logs to debug

* Update graphql queries in create and update

* Add call in EntityAPIClient, graphql in DisplayTableContainer, and setup FileUtil

* Fix display of buttons to only be shown if the fileName is not null

* Add suggested changes to remove circular JSON errors

* Uncomment python part in docker-compose.yml

* Address PR comments and fix Graphql get in DisplayTableContainer

* Fix REST file upload

* Add REST file update to frontend

* Decamelize formData when using Python backend

* Fix fileUrl access in EntityAPIClient.getFile for Python backend

Co-authored-by: Alex Guo <[email protected]>
Co-authored-by: Sherry Li <[email protected]>

* Create HooksDemo page (#125)

* Update README.md (#128)

* Fix various bugs (#127)

* Add type check on self.enum_field before calling .upper() in validator

* Set erase_db_and_sync in Python backend based on TESTING config value

* Change DB dialect name from postgres to postgresql in Python connection string

* Update testDb connect method for compatibility with mongo-memory-server's updated API

* Change setData to setEntities in DisplayTableContainer.tsx

* Add comments to AuthService's generateTokenOAuth method

* Separate SignInResponse types for password and OAuth sign ins

* Fix Python csv_utils to account for optional fields

* Add *.egg-info to .gitignore

* Remove commented out code in backend/typescript/utilities/CSVUtils.ts

* Lint and reformat

* Fix form reset issue after setting file field in CreateForm and UpdateForm

* Fix casing in CSVUtils import path

* Update Swagger docs with new endpoints and request body types (#131)

* Fix error when deleting an entity with no file in Python (#133)

* Fix error when deleting an entity with no file

* Add id return for delete entity endpoint in TypeScript

* Address comment

* Modify and clean up infra setup (#134)

* Add db health check, db init script, and container names

* Make scripts executable

* Fix formatting in backend/python/app/__init__.py

* Return string id from TypeScript PostgreSQL user and entity services (#136)

* Add end-to-end tests for the backend (#137)

* Add end-to-end tests for the backend

* Rename folder

* Add files and modifications for backend deployment to Heroku (#138)

* Add files and modifications for backend deployment to Heroku

* Require migrations in TypeScript backend

* Stringify Google login error object

* Fix formatting in frontend

* Remove nodemailer config path from secret.config

* Fix typo in DATABASE_URL

* Check if MONGODB_URL is present before reading from app.config

* Delete demo migrations

* Remove newline from backend/typescript/models/index.ts

* Change model file path pattern in umzug.ts

* Add tags to GitHub Actions workflow files

* Fix id type in TypeScript PostgreSQL backend

* Add file storage tags to entities table migration file

* Add auth tags for users table creation in Python initial migration

Co-authored-by: Xin Hao Zhang <[email protected]>
Co-authored-by: alexguo8 <[email protected]>
Co-authored-by: Emily Louie <[email protected]>
Co-authored-by: Tamim Khan <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Dinu Wijetunga <[email protected]>
Co-authored-by: Alex Guo <[email protected]>
sherryhli added a commit that referenced this pull request Jan 16, 2022
* Add example context components for frontend state management (#37)

* Add example context components

* Add ts root dir to fix eslint errors

* Fix typo, add return type to reducer

* Add react plugins to eslint

* Update README.md (#46)

* Linting fixes (#42)

* Add ts plugins to eslint, fix eslint errors

* Remove deep copy of form data

* Add more return types, change prettier errors to warn

* Type decoded JWT, move AuthenticatedUser to types file

* Create AuthTypes

* Fix token expiry

Co-authored-by: Sherry Li <[email protected]>

* Refactor auth functions that use localStorage logic into AuthAPIClient (#45)

* duplicate code to authapiclient.ts

* refactor functions with localstorage to AuthAPIClient

* Update with removed REST tags

* update with pr fixes

* Update mongoose version to remove warnings (#47)

* Configure Sequelize migrations (#48)

* Configure Sequelize migrations using umzug

* Add defaultValue and allowNull options to add column example

* Add swagger for typescript backend (#51)

* Add Python REST Validators (#50)

* add initial validate decorator

* update with modified middleware and add decorators to the respective functions

* add decorator and validate method for entity dto

* Update README.md and add nodemailer config path to secret.config (#49)

* Update README.md

* Add nodemailer config path to secret.config

* Add frontend stack to README.md

* Add minor edits to README.md

* Add TypeScript REST API validation (#52)

* Add auth validators

* Add Entity and User Validators

* Refactor validation utils

* Add API error messages

* Swagger For Python (#54)

* Swagger For Python

* Minor fixes

* Include file name in log using wrapper for Winston typescript logger (#55)

Co-authored-by: Bruce He <[email protected]>

* Create Python FileStorageService (#62)

* Add initial file storage service

* Add initial file storage service

* Create and Update

* Add exception handling and more docstrings

* Make expiration time a parameter, clean up code

* Add content type parameter, update interface

Co-authored-by: Richard Chen <[email protected]>

* Create TypeScript FileStorageService (#59)

* add initial files

* Fix firebase bucket init

* add create file implementation

* add interface notes and add checking if file already exists

* Add updateFile and deleteFile implementations

* update create file method

* update create method notes

* Refactor getFile

* remove import in server.ts

* Update error message on failure to get file

Co-authored-by: Sherry Li <[email protected]>

* add contentType param

* fix formatting in storageService.ts

* Make bucketName a param

* Fix bucket name format

Co-authored-by: tamim-khan <[email protected]>
Co-authored-by: Sherry Li <[email protected]>

* Add testing frameworks (#57)

* Delete repeated requirements

* Add pytest to requirements

* Create python test examples

* Add typescript BE examples

* Create frontend sample tests

* Add test-db to docker-compose

* Update tests for docker containers

* Update psql conn string

* Add mongodb-memory-server dependency libcurl3 to docker container

* Add mongomock and pytest-mock to py requirements

* Create test_user_service

* Add pg user service test

* Configure SQLAlchemy migrations (#58)

* Configure SQLAlchemy migrations

* Fix create_app configs when calling from Flask CLI

* Update README.md (#64)

* Update README.md

* Apply suggestions from code review

Co-authored-by: Sherry Li <[email protected]>

Co-authored-by: Sherry Li <[email protected]>

* Set up frontend deployment and previews with Firebase Hosting and GitHub Actions (#87)

* Set up frontend hosting using Firebase

* Set up GitHub Actions for frontend deploy preview and automated prod deployment

* Set working directory in frontend deployment GitHub Actions

* Add yarn install step to deployment scripts

* Add entryPoint to action-hosting-deploy step

* Use separate backend urls for previews and prod

* Fix several minor issues (#63)

* Set extended option in express.urlencoded in server.ts

* Fix user_id casing in Python openapi.json

* Simplify Python DTO validators

* Add newline in Logout.tsx

* Rename TypeScript storageService to fileStorageService

* Integrate TS FileStorageService into EntityService (#88)

* temp changes

* add initial changes to EntityServiceMg

* Fix update function in entityservice

* Add file crud methods to postgres entity service

* Add file content type validation

* Fix create endpoints

* Add local file storing to graphql resolvers

* Add endpoint for getting file based on fileUUID

* Change storage service to store bucketName instead of the bucket

* Minor fixes

* Fix update functions in entity service

* Minor fixes

* Add content type to entity request DTO and filestorageservice calls

* Fix graphql

* Fix imports

* Refactor create/update to prevent dangling file references

* Address comments

* Wait for file to finish writing before continuing

* Delete file after uploading

* Add awaits to filestorageservice calls

Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Emily Louie <[email protected]>

* Add frontend csv utility (#90)

* Integrate Python FileStorageService into EntityService (#86)

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Fix python file get endpoint file storage service call

* Remove EntityDTO.file_name from Mongo Impl

* Update validator to check for missing body

* fix

* Fix file get endpoint auth, rename entity DTO variable

* Remove EntityDTO.file_name from Postgres Impl

* Update get_file to return object

* Fix bug in mongo service update logic

* Fix Postgres update route and logic

* Fix python postgres pop file and lint code.

* Postgres migration to add entity file name

Co-authored-by: Bruce He <[email protected]>

* Add backend TS csv generation utility (#91)

* initial files

* add a responseutil

* update rest api

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

Co-authored-by: Alex Guo <[email protected]>

* Add download csv functionality from frontend utility (#95)

* initial files

* add a responseutil

* update rest api

* Add download csv functionality from frontend utility

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

* Add backend API integration

Co-authored-by: Emily Louie <[email protected]>

* Refactor route strings into constants (#111)

* Backend CSV generation Python (#108)

* Initial change

Linter and tests

Update default options

csv default settings

Examples and minor format changes

Update docker-compose.yml

Moved tests to unit

* Add minor fixes

Co-authored-by: Sherry Li <[email protected]>

* Create Typescript registration flow with email verification (#104)

* Add endpoints for register user with email verification

* Check email verified on the firebase user instead of the token

* Create Python registration flow with email verification (#105)

* Add endpoints to register user using email verification

* fix import and add validator

* Add validation for RegisterUserDTO

* Fix accessing the email verification from token

* Change the verification email title

* check email verified in firebase user rather than token

* Frontend User Sign-Up (#115)

* Add sign-up page & auth client register functions

* Rebase on main and make minor routing changes

Co-authored-by: Sherry Li <[email protected]>

* Integrate Google OAuth into Python backend (#112)

* Add google oauth signin to firebase rest client

* Add google oauth login and signup functionality

* Add frontend for google oauth signin

* Fix backend

* Add comment for google user id token

* Integrate Google OAuth into TS (#114)

* Add REST Impl

* Fix try catch

Co-authored-by: Alex Guo <[email protected]>

* Fix email_verified syntax in Python auth_service (#123)

* Add TS GraphQL support for Google OAuth (#122)

* GraphQL OAuth

* rename id_token

* Validators

* Update authValidators.ts

* Fix generate_csv_from_list call in entity and user routes (#119)

* Integrate file uploads and downloads into frontend  (#116)

* add initial upload button and onChange function

* Add field to display table

* Fix mutation and add console.logs to debug

* Update graphql queries in create and update

* Add call in EntityAPIClient, graphql in DisplayTableContainer, and setup FileUtil

* Fix display of buttons to only be shown if the fileName is not null

* Add suggested changes to remove circular JSON errors

* Uncomment python part in docker-compose.yml

* Address PR comments and fix Graphql get in DisplayTableContainer

* Fix REST file upload

* Add REST file update to frontend

* Decamelize formData when using Python backend

* Fix fileUrl access in EntityAPIClient.getFile for Python backend

Co-authored-by: Alex Guo <[email protected]>
Co-authored-by: Sherry Li <[email protected]>

* Create HooksDemo page (#125)

* Update README.md (#128)

* Fix various bugs (#127)

* Add type check on self.enum_field before calling .upper() in validator

* Set erase_db_and_sync in Python backend based on TESTING config value

* Change DB dialect name from postgres to postgresql in Python connection string

* Update testDb connect method for compatibility with mongo-memory-server's updated API

* Change setData to setEntities in DisplayTableContainer.tsx

* Add comments to AuthService's generateTokenOAuth method

* Separate SignInResponse types for password and OAuth sign ins

* Fix Python csv_utils to account for optional fields

* Add *.egg-info to .gitignore

* Remove commented out code in backend/typescript/utilities/CSVUtils.ts

* Lint and reformat

* Fix form reset issue after setting file field in CreateForm and UpdateForm

* Fix casing in CSVUtils import path

* Update Swagger docs with new endpoints and request body types (#131)

* Fix error when deleting an entity with no file in Python (#133)

* Fix error when deleting an entity with no file

* Add id return for delete entity endpoint in TypeScript

* Address comment

* Modify and clean up infra setup (#134)

* Add db health check, db init script, and container names

* Make scripts executable

* Fix formatting in backend/python/app/__init__.py

* Return string id from TypeScript PostgreSQL user and entity services (#136)

* Add end-to-end tests for the backend (#137)

* Add end-to-end tests for the backend

* Rename folder

* Add files and modifications for backend deployment to Heroku (#138)

* Add files and modifications for backend deployment to Heroku

* Require migrations in TypeScript backend

* Stringify Google login error object

* Fix formatting in frontend

* Remove nodemailer config path from secret.config

* Fix typo in DATABASE_URL

* Check if MONGODB_URL is present before reading from app.config

* Create GitHub Actions linting workflow and fix lint errors (#149)

* Create GitHub Actions file for running linters

* Fix lint and formatting errors in backend/typescript

* Fix lint and formatting errors in frontend

* Fix quotes in .github/workflows/lint.yml

* Fix newlines and lint errors

Co-authored-by: Xin Hao Zhang <[email protected]>
Co-authored-by: alexguo8 <[email protected]>
Co-authored-by: Emily Louie <[email protected]>
Co-authored-by: Tamim Khan <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Dinu Wijetunga <[email protected]>
Co-authored-by: Alex Guo <[email protected]>
sherryhli added a commit that referenced this pull request Jan 19, 2022
* Add example context components for frontend state management (#37)

* Add example context components

* Add ts root dir to fix eslint errors

* Fix typo, add return type to reducer

* Add react plugins to eslint

* Update README.md (#46)

* Linting fixes (#42)

* Add ts plugins to eslint, fix eslint errors

* Remove deep copy of form data

* Add more return types, change prettier errors to warn

* Type decoded JWT, move AuthenticatedUser to types file

* Create AuthTypes

* Fix token expiry

Co-authored-by: Sherry Li <[email protected]>

* Refactor auth functions that use localStorage logic into AuthAPIClient (#45)

* duplicate code to authapiclient.ts

* refactor functions with localstorage to AuthAPIClient

* Update with removed REST tags

* update with pr fixes

* Update mongoose version to remove warnings (#47)

* Configure Sequelize migrations (#48)

* Configure Sequelize migrations using umzug

* Add defaultValue and allowNull options to add column example

* Add swagger for typescript backend (#51)

* Add Python REST Validators (#50)

* add initial validate decorator

* update with modified middleware and add decorators to the respective functions

* add decorator and validate method for entity dto

* Update README.md and add nodemailer config path to secret.config (#49)

* Update README.md

* Add nodemailer config path to secret.config

* Add frontend stack to README.md

* Add minor edits to README.md

* Add TypeScript REST API validation (#52)

* Add auth validators

* Add Entity and User Validators

* Refactor validation utils

* Add API error messages

* Swagger For Python (#54)

* Swagger For Python

* Minor fixes

* Include file name in log using wrapper for Winston typescript logger (#55)

Co-authored-by: Bruce He <[email protected]>

* Create Python FileStorageService (#62)

* Add initial file storage service

* Add initial file storage service

* Create and Update

* Add exception handling and more docstrings

* Make expiration time a parameter, clean up code

* Add content type parameter, update interface

Co-authored-by: Richard Chen <[email protected]>

* Create TypeScript FileStorageService (#59)

* add initial files

* Fix firebase bucket init

* add create file implementation

* add interface notes and add checking if file already exists

* Add updateFile and deleteFile implementations

* update create file method

* update create method notes

* Refactor getFile

* remove import in server.ts

* Update error message on failure to get file

Co-authored-by: Sherry Li <[email protected]>

* add contentType param

* fix formatting in storageService.ts

* Make bucketName a param

* Fix bucket name format

Co-authored-by: tamim-khan <[email protected]>
Co-authored-by: Sherry Li <[email protected]>

* Add testing frameworks (#57)

* Delete repeated requirements

* Add pytest to requirements

* Create python test examples

* Add typescript BE examples

* Create frontend sample tests

* Add test-db to docker-compose

* Update tests for docker containers

* Update psql conn string

* Add mongodb-memory-server dependency libcurl3 to docker container

* Add mongomock and pytest-mock to py requirements

* Create test_user_service

* Add pg user service test

* Configure SQLAlchemy migrations (#58)

* Configure SQLAlchemy migrations

* Fix create_app configs when calling from Flask CLI

* Update README.md (#64)

* Update README.md

* Apply suggestions from code review

Co-authored-by: Sherry Li <[email protected]>

Co-authored-by: Sherry Li <[email protected]>

* Set up frontend deployment and previews with Firebase Hosting and GitHub Actions (#87)

* Set up frontend hosting using Firebase

* Set up GitHub Actions for frontend deploy preview and automated prod deployment

* Set working directory in frontend deployment GitHub Actions

* Add yarn install step to deployment scripts

* Add entryPoint to action-hosting-deploy step

* Use separate backend urls for previews and prod

* Fix several minor issues (#63)

* Set extended option in express.urlencoded in server.ts

* Fix user_id casing in Python openapi.json

* Simplify Python DTO validators

* Add newline in Logout.tsx

* Rename TypeScript storageService to fileStorageService

* Integrate TS FileStorageService into EntityService (#88)

* temp changes

* add initial changes to EntityServiceMg

* Fix update function in entityservice

* Add file crud methods to postgres entity service

* Add file content type validation

* Fix create endpoints

* Add local file storing to graphql resolvers

* Add endpoint for getting file based on fileUUID

* Change storage service to store bucketName instead of the bucket

* Minor fixes

* Fix update functions in entity service

* Minor fixes

* Add content type to entity request DTO and filestorageservice calls

* Fix graphql

* Fix imports

* Refactor create/update to prevent dangling file references

* Address comments

* Wait for file to finish writing before continuing

* Delete file after uploading

* Add awaits to filestorageservice calls

Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Emily Louie <[email protected]>

* Add frontend csv utility (#90)

* Integrate Python FileStorageService into EntityService (#86)

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Fix python file get endpoint file storage service call

* Remove EntityDTO.file_name from Mongo Impl

* Update validator to check for missing body

* fix

* Fix file get endpoint auth, rename entity DTO variable

* Remove EntityDTO.file_name from Postgres Impl

* Update get_file to return object

* Fix bug in mongo service update logic

* Fix Postgres update route and logic

* Fix python postgres pop file and lint code.

* Postgres migration to add entity file name

Co-authored-by: Bruce He <[email protected]>

* Add backend TS csv generation utility (#91)

* initial files

* add a responseutil

* update rest api

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

Co-authored-by: Alex Guo <[email protected]>

* Add download csv functionality from frontend utility (#95)

* initial files

* add a responseutil

* update rest api

* Add download csv functionality from frontend utility

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

* Add backend API integration

Co-authored-by: Emily Louie <[email protected]>

* Refactor route strings into constants (#111)

* Backend CSV generation Python (#108)

* Initial change

Linter and tests

Update default options

csv default settings

Examples and minor format changes

Update docker-compose.yml

Moved tests to unit

* Add minor fixes

Co-authored-by: Sherry Li <[email protected]>

* Create Typescript registration flow with email verification (#104)

* Add endpoints for register user with email verification

* Check email verified on the firebase user instead of the token

* Create Python registration flow with email verification (#105)

* Add endpoints to register user using email verification

* fix import and add validator

* Add validation for RegisterUserDTO

* Fix accessing the email verification from token

* Change the verification email title

* check email verified in firebase user rather than token

* Frontend User Sign-Up (#115)

* Add sign-up page & auth client register functions

* Rebase on main and make minor routing changes

Co-authored-by: Sherry Li <[email protected]>

* Integrate Google OAuth into Python backend (#112)

* Add google oauth signin to firebase rest client

* Add google oauth login and signup functionality

* Add frontend for google oauth signin

* Fix backend

* Add comment for google user id token

* Integrate Google OAuth into TS (#114)

* Add REST Impl

* Fix try catch

Co-authored-by: Alex Guo <[email protected]>

* Fix email_verified syntax in Python auth_service (#123)

* Add TS GraphQL support for Google OAuth (#122)

* GraphQL OAuth

* rename id_token

* Validators

* Update authValidators.ts

* Fix generate_csv_from_list call in entity and user routes (#119)

* Integrate file uploads and downloads into frontend  (#116)

* add initial upload button and onChange function

* Add field to display table

* Fix mutation and add console.logs to debug

* Update graphql queries in create and update

* Add call in EntityAPIClient, graphql in DisplayTableContainer, and setup FileUtil

* Fix display of buttons to only be shown if the fileName is not null

* Add suggested changes to remove circular JSON errors

* Uncomment python part in docker-compose.yml

* Address PR comments and fix Graphql get in DisplayTableContainer

* Fix REST file upload

* Add REST file update to frontend

* Decamelize formData when using Python backend

* Fix fileUrl access in EntityAPIClient.getFile for Python backend

Co-authored-by: Alex Guo <[email protected]>
Co-authored-by: Sherry Li <[email protected]>

* Create HooksDemo page (#125)

* Update README.md (#128)

* Fix various bugs (#127)

* Add type check on self.enum_field before calling .upper() in validator

* Set erase_db_and_sync in Python backend based on TESTING config value

* Change DB dialect name from postgres to postgresql in Python connection string

* Update testDb connect method for compatibility with mongo-memory-server's updated API

* Change setData to setEntities in DisplayTableContainer.tsx

* Add comments to AuthService's generateTokenOAuth method

* Separate SignInResponse types for password and OAuth sign ins

* Fix Python csv_utils to account for optional fields

* Add *.egg-info to .gitignore

* Remove commented out code in backend/typescript/utilities/CSVUtils.ts

* Lint and reformat

* Fix form reset issue after setting file field in CreateForm and UpdateForm

* Fix casing in CSVUtils import path

* Update Swagger docs with new endpoints and request body types (#131)

* Fix error when deleting an entity with no file in Python (#133)

* Fix error when deleting an entity with no file

* Add id return for delete entity endpoint in TypeScript

* Address comment

* Modify and clean up infra setup (#134)

* Add db health check, db init script, and container names

* Make scripts executable

* Fix formatting in backend/python/app/__init__.py

* Return string id from TypeScript PostgreSQL user and entity services (#136)

* Add end-to-end tests for the backend (#137)

* Add end-to-end tests for the backend

* Rename folder

* Add files and modifications for backend deployment to Heroku (#138)

* Add files and modifications for backend deployment to Heroku

* Require migrations in TypeScript backend

* Stringify Google login error object

* Fix formatting in frontend

* Remove nodemailer config path from secret.config

* Fix typo in DATABASE_URL

* Check if MONGODB_URL is present before reading from app.config

* Create GitHub Actions linting workflow and fix lint errors (#149)

* Create GitHub Actions file for running linters

* Fix lint and formatting errors in backend/typescript

* Fix lint and formatting errors in frontend

* Fix quotes in .github/workflows/lint.yml

* Fix error message typing in backend/typescript (#152)

* Fix error message typing in backend/typescript

* Add eslint-disable comments

* Fix content_type check in Python validate middleware (#153)

* Fix newline

* Fix GraphQL file upload bug

Co-authored-by: Xin Hao Zhang <[email protected]>
Co-authored-by: alexguo8 <[email protected]>
Co-authored-by: Emily Louie <[email protected]>
Co-authored-by: Tamim Khan <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Dinu Wijetunga <[email protected]>
Co-authored-by: Alex Guo <[email protected]>
sherryhli added a commit that referenced this pull request Jan 19, 2022
* Add example context components for frontend state management (#37)

* Add example context components

* Add ts root dir to fix eslint errors

* Fix typo, add return type to reducer

* Add react plugins to eslint

* Update README.md (#46)

* Linting fixes (#42)

* Add ts plugins to eslint, fix eslint errors

* Remove deep copy of form data

* Add more return types, change prettier errors to warn

* Type decoded JWT, move AuthenticatedUser to types file

* Create AuthTypes

* Fix token expiry

Co-authored-by: Sherry Li <[email protected]>

* Refactor auth functions that use localStorage logic into AuthAPIClient (#45)

* duplicate code to authapiclient.ts

* refactor functions with localstorage to AuthAPIClient

* Update with removed REST tags

* update with pr fixes

* Update mongoose version to remove warnings (#47)

* Configure Sequelize migrations (#48)

* Configure Sequelize migrations using umzug

* Add defaultValue and allowNull options to add column example

* Add swagger for typescript backend (#51)

* Add Python REST Validators (#50)

* add initial validate decorator

* update with modified middleware and add decorators to the respective functions

* add decorator and validate method for entity dto

* Update README.md and add nodemailer config path to secret.config (#49)

* Update README.md

* Add nodemailer config path to secret.config

* Add frontend stack to README.md

* Add minor edits to README.md

* Add TypeScript REST API validation (#52)

* Add auth validators

* Add Entity and User Validators

* Refactor validation utils

* Add API error messages

* Swagger For Python (#54)

* Swagger For Python

* Minor fixes

* Include file name in log using wrapper for Winston typescript logger (#55)

Co-authored-by: Bruce He <[email protected]>

* Create Python FileStorageService (#62)

* Add initial file storage service

* Add initial file storage service

* Create and Update

* Add exception handling and more docstrings

* Make expiration time a parameter, clean up code

* Add content type parameter, update interface

Co-authored-by: Richard Chen <[email protected]>

* Create TypeScript FileStorageService (#59)

* add initial files

* Fix firebase bucket init

* add create file implementation

* add interface notes and add checking if file already exists

* Add updateFile and deleteFile implementations

* update create file method

* update create method notes

* Refactor getFile

* remove import in server.ts

* Update error message on failure to get file

Co-authored-by: Sherry Li <[email protected]>

* add contentType param

* fix formatting in storageService.ts

* Make bucketName a param

* Fix bucket name format

Co-authored-by: tamim-khan <[email protected]>
Co-authored-by: Sherry Li <[email protected]>

* Add testing frameworks (#57)

* Delete repeated requirements

* Add pytest to requirements

* Create python test examples

* Add typescript BE examples

* Create frontend sample tests

* Add test-db to docker-compose

* Update tests for docker containers

* Update psql conn string

* Add mongodb-memory-server dependency libcurl3 to docker container

* Add mongomock and pytest-mock to py requirements

* Create test_user_service

* Add pg user service test

* Configure SQLAlchemy migrations (#58)

* Configure SQLAlchemy migrations

* Fix create_app configs when calling from Flask CLI

* Update README.md (#64)

* Update README.md

* Apply suggestions from code review

Co-authored-by: Sherry Li <[email protected]>

Co-authored-by: Sherry Li <[email protected]>

* Set up frontend deployment and previews with Firebase Hosting and GitHub Actions (#87)

* Set up frontend hosting using Firebase

* Set up GitHub Actions for frontend deploy preview and automated prod deployment

* Set working directory in frontend deployment GitHub Actions

* Add yarn install step to deployment scripts

* Add entryPoint to action-hosting-deploy step

* Use separate backend urls for previews and prod

* Fix several minor issues (#63)

* Set extended option in express.urlencoded in server.ts

* Fix user_id casing in Python openapi.json

* Simplify Python DTO validators

* Add newline in Logout.tsx

* Rename TypeScript storageService to fileStorageService

* Integrate TS FileStorageService into EntityService (#88)

* temp changes

* add initial changes to EntityServiceMg

* Fix update function in entityservice

* Add file crud methods to postgres entity service

* Add file content type validation

* Fix create endpoints

* Add local file storing to graphql resolvers

* Add endpoint for getting file based on fileUUID

* Change storage service to store bucketName instead of the bucket

* Minor fixes

* Fix update functions in entity service

* Minor fixes

* Add content type to entity request DTO and filestorageservice calls

* Fix graphql

* Fix imports

* Refactor create/update to prevent dangling file references

* Address comments

* Wait for file to finish writing before continuing

* Delete file after uploading

* Add awaits to filestorageservice calls

Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Emily Louie <[email protected]>

* Add frontend csv utility (#90)

* Integrate Python FileStorageService into EntityService (#86)

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Update EntityService Mongo

* Update Python Entity models for file storage.

* Add file get request to routes, work on entity services

* Add entity request dto for python

* Add file support in entity routes

* Add file to update route

* Update validator and DTO

* Add content type check to python entity dto validator

* Add file storage support to Python Postgres Entity service

* Add PUT logic to Python Postgres entity service update function

* Change update logic for mongo

* lint

* Fix python file get endpoint file storage service call

* Remove EntityDTO.file_name from Mongo Impl

* Update validator to check for missing body

* fix

* Fix file get endpoint auth, rename entity DTO variable

* Remove EntityDTO.file_name from Postgres Impl

* Update get_file to return object

* Fix bug in mongo service update logic

* Fix Postgres update route and logic

* Fix python postgres pop file and lint code.

* Postgres migration to add entity file name

Co-authored-by: Bruce He <[email protected]>

* Add backend TS csv generation utility (#91)

* initial files

* add a responseutil

* update rest api

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

Co-authored-by: Alex Guo <[email protected]>

* Add download csv functionality from frontend utility (#95)

* initial files

* add a responseutil

* update rest api

* Add download csv functionality from frontend utility

* Add new Graphql querires

* Address PR comments and remove CSV option for requests that return single items

* Change generateCSV function to match frontends

* update yarn.lock

* Fix yarn.lock

* Add backend API integration

Co-authored-by: Emily Louie <[email protected]>

* Refactor route strings into constants (#111)

* Backend CSV generation Python (#108)

* Initial change

Linter and tests

Update default options

csv default settings

Examples and minor format changes

Update docker-compose.yml

Moved tests to unit

* Add minor fixes

Co-authored-by: Sherry Li <[email protected]>

* Create Typescript registration flow with email verification (#104)

* Add endpoints for register user with email verification

* Check email verified on the firebase user instead of the token

* Create Python registration flow with email verification (#105)

* Add endpoints to register user using email verification

* fix import and add validator

* Add validation for RegisterUserDTO

* Fix accessing the email verification from token

* Change the verification email title

* check email verified in firebase user rather than token

* Frontend User Sign-Up (#115)

* Add sign-up page & auth client register functions

* Rebase on main and make minor routing changes

Co-authored-by: Sherry Li <[email protected]>

* Integrate Google OAuth into Python backend (#112)

* Add google oauth signin to firebase rest client

* Add google oauth login and signup functionality

* Add frontend for google oauth signin

* Fix backend

* Add comment for google user id token

* Integrate Google OAuth into TS (#114)

* Add REST Impl

* Fix try catch

Co-authored-by: Alex Guo <[email protected]>

* Fix email_verified syntax in Python auth_service (#123)

* Add TS GraphQL support for Google OAuth (#122)

* GraphQL OAuth

* rename id_token

* Validators

* Update authValidators.ts

* Fix generate_csv_from_list call in entity and user routes (#119)

* Integrate file uploads and downloads into frontend  (#116)

* add initial upload button and onChange function

* Add field to display table

* Fix mutation and add console.logs to debug

* Update graphql queries in create and update

* Add call in EntityAPIClient, graphql in DisplayTableContainer, and setup FileUtil

* Fix display of buttons to only be shown if the fileName is not null

* Add suggested changes to remove circular JSON errors

* Uncomment python part in docker-compose.yml

* Address PR comments and fix Graphql get in DisplayTableContainer

* Fix REST file upload

* Add REST file update to frontend

* Decamelize formData when using Python backend

* Fix fileUrl access in EntityAPIClient.getFile for Python backend

Co-authored-by: Alex Guo <[email protected]>
Co-authored-by: Sherry Li <[email protected]>

* Create HooksDemo page (#125)

* Update README.md (#128)

* Fix various bugs (#127)

* Add type check on self.enum_field before calling .upper() in validator

* Set erase_db_and_sync in Python backend based on TESTING config value

* Change DB dialect name from postgres to postgresql in Python connection string

* Update testDb connect method for compatibility with mongo-memory-server's updated API

* Change setData to setEntities in DisplayTableContainer.tsx

* Add comments to AuthService's generateTokenOAuth method

* Separate SignInResponse types for password and OAuth sign ins

* Fix Python csv_utils to account for optional fields

* Add *.egg-info to .gitignore

* Remove commented out code in backend/typescript/utilities/CSVUtils.ts

* Lint and reformat

* Fix form reset issue after setting file field in CreateForm and UpdateForm

* Fix casing in CSVUtils import path

* Update Swagger docs with new endpoints and request body types (#131)

* Fix error when deleting an entity with no file in Python (#133)

* Fix error when deleting an entity with no file

* Add id return for delete entity endpoint in TypeScript

* Address comment

* Modify and clean up infra setup (#134)

* Add db health check, db init script, and container names

* Make scripts executable

* Fix formatting in backend/python/app/__init__.py

* Return string id from TypeScript PostgreSQL user and entity services (#136)

* Add end-to-end tests for the backend (#137)

* Add end-to-end tests for the backend

* Rename folder

* Add files and modifications for backend deployment to Heroku (#138)

* Add files and modifications for backend deployment to Heroku

* Require migrations in TypeScript backend

* Stringify Google login error object

* Fix formatting in frontend

* Remove nodemailer config path from secret.config

* Fix typo in DATABASE_URL

* Check if MONGODB_URL is present before reading from app.config

* Create GitHub Actions linting workflow and fix lint errors (#149)

* Create GitHub Actions file for running linters

* Fix lint and formatting errors in backend/typescript

* Fix lint and formatting errors in frontend

* Fix quotes in .github/workflows/lint.yml

* Fix error message typing in backend/typescript (#152)

* Fix error message typing in backend/typescript

* Add eslint-disable comments

* Fix content_type check in Python validate middleware (#153)

* Ignore tests under build folder in backend/typescript (#157)

Co-authored-by: Xin Hao Zhang <[email protected]>
Co-authored-by: alexguo8 <[email protected]>
Co-authored-by: Emily Louie <[email protected]>
Co-authored-by: Tamim Khan <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Bruce He <[email protected]>
Co-authored-by: Richard Chen <[email protected]>
Co-authored-by: Dinu Wijetunga <[email protected]>
Co-authored-by: Alex Guo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Integrate Python FileStorageService into EntityService
4 participants