Create release v1.9.3 (#135)

* Add example context components for frontend state management (#37) * Add example context components * Add ts root dir to fix eslint errors * Fix typo, add return type to reducer * Add react plugins to eslint * Update README.md (#46) * Linting fixes (#42) * Add ts plugins to eslint, fix eslint errors * Remove deep copy of form data * Add more return types, change prettier errors to warn * Type decoded JWT, move AuthenticatedUser to types file * Create AuthTypes * Fix token expiry Co-authored-by: Sherry Li <[email protected]> * Refactor auth functions that use localStorage logic into AuthAPIClient (#45) * duplicate code to authapiclient.ts * refactor functions with localstorage to AuthAPIClient * Update with removed REST tags * update with pr fixes * Update mongoose version to remove warnings (#47) * Configure Sequelize migrations (#48) * Configure Sequelize migrations using umzug * Add defaultValue and allowNull options to add column example * Add swagger for typescript backend (#51) * Add Python REST Validators (#50) * add initial validate decorator * update with modified middleware and add decorators to the respective functions * add decorator and validate method for entity dto * Update README.md and add nodemailer config path to secret.config (#49) * Update README.md * Add nodemailer config path to secret.config * Add frontend stack to README.md * Add minor edits to README.md * Add TypeScript REST API validation (#52) * Add auth validators * Add Entity and User Validators * Refactor validation utils * Add API error messages * Swagger For Python (#54) * Swagger For Python * Minor fixes * Include file name in log using wrapper for Winston typescript logger (#55) Co-authored-by: Bruce He <[email protected]> * Create Python FileStorageService (#62) * Add initial file storage service * Add initial file storage service * Create and Update * Add exception handling and more docstrings * Make expiration time a parameter, clean up code * Add content type parameter, update interface Co-authored-by: Richard Chen <[email protected]> * Create TypeScript FileStorageService (#59) * add initial files * Fix firebase bucket init * add create file implementation * add interface notes and add checking if file already exists * Add updateFile and deleteFile implementations * update create file method * update create method notes * Refactor getFile * remove import in server.ts * Update error message on failure to get file Co-authored-by: Sherry Li <[email protected]> * add contentType param * fix formatting in storageService.ts * Make bucketName a param * Fix bucket name format Co-authored-by: tamim-khan <[email protected]> Co-authored-by: Sherry Li <[email protected]> * Add testing frameworks (#57) * Delete repeated requirements * Add pytest to requirements * Create python test examples * Add typescript BE examples * Create frontend sample tests * Add test-db to docker-compose * Update tests for docker containers * Update psql conn string * Add mongodb-memory-server dependency libcurl3 to docker container * Add mongomock and pytest-mock to py requirements * Create test_user_service * Add pg user service test * Configure SQLAlchemy migrations (#58) * Configure SQLAlchemy migrations * Fix create_app configs when calling from Flask CLI * Update README.md (#64) * Update README.md * Apply suggestions from code review Co-authored-by: Sherry Li <[email protected]> Co-authored-by: Sherry Li <[email protected]> * Set up frontend deployment and previews with Firebase Hosting and GitHub Actions (#87) * Set up frontend hosting using Firebase * Set up GitHub Actions for frontend deploy preview and automated prod deployment * Set working directory in frontend deployment GitHub Actions * Add yarn install step to deployment scripts * Add entryPoint to action-hosting-deploy step * Use separate backend urls for previews and prod * Fix several minor issues (#63) * Set extended option in express.urlencoded in server.ts * Fix user_id casing in Python openapi.json * Simplify Python DTO validators * Add newline in Logout.tsx * Rename TypeScript storageService to fileStorageService * Integrate TS FileStorageService into EntityService (#88) * temp changes * add initial changes to EntityServiceMg * Fix update function in entityservice * Add file crud methods to postgres entity service * Add file content type validation * Fix create endpoints * Add local file storing to graphql resolvers * Add endpoint for getting file based on fileUUID * Change storage service to store bucketName instead of the bucket * Minor fixes * Fix update functions in entity service * Minor fixes * Add content type to entity request DTO and filestorageservice calls * Fix graphql * Fix imports * Refactor create/update to prevent dangling file references * Address comments * Wait for file to finish writing before continuing * Delete file after uploading * Add awaits to filestorageservice calls Co-authored-by: Richard Chen <[email protected]> Co-authored-by: Emily Louie <[email protected]> * Add frontend csv utility (#90) * Integrate Python FileStorageService into EntityService (#86) * Update EntityService Mongo * Update Python Entity models for file storage. * Add file get request to routes, work on entity services * Add entity request dto for python * Add file support in entity routes * Add file to update route * Update validator and DTO * Add content type check to python entity dto validator * Add file storage support to Python Postgres Entity service * Add PUT logic to Python Postgres entity service update function * Change update logic for mongo * lint * Update EntityService Mongo * Update Python Entity models for file storage. * Add file get request to routes, work on entity services * Add entity request dto for python * Add file support in entity routes * Add file to update route * Update validator and DTO * Add content type check to python entity dto validator * Add file storage support to Python Postgres Entity service * Add PUT logic to Python Postgres entity service update function * Change update logic for mongo * lint * Fix python file get endpoint file storage service call * Remove EntityDTO.file_name from Mongo Impl * Update validator to check for missing body * fix * Fix file get endpoint auth, rename entity DTO variable * Remove EntityDTO.file_name from Postgres Impl * Update get_file to return object * Fix bug in mongo service update logic * Fix Postgres update route and logic * Fix python postgres pop file and lint code. * Postgres migration to add entity file name Co-authored-by: Bruce He <[email protected]> * Add backend TS csv generation utility (#91) * initial files * add a responseutil * update rest api * Add new Graphql querires * Address PR comments and remove CSV option for requests that return single items * Change generateCSV function to match frontends * update yarn.lock * Fix yarn.lock Co-authored-by: Alex Guo <[email protected]> * Add download csv functionality from frontend utility (#95) * initial files * add a responseutil * update rest api * Add download csv functionality from frontend utility * Add new Graphql querires * Address PR comments and remove CSV option for requests that return single items * Change generateCSV function to match frontends * update yarn.lock * Fix yarn.lock * Add backend API integration Co-authored-by: Emily Louie <[email protected]> * Refactor route strings into constants (#111) * Backend CSV generation Python (#108) * Initial change Linter and tests Update default options csv default settings Examples and minor format changes Update docker-compose.yml Moved tests to unit * Add minor fixes Co-authored-by: Sherry Li <[email protected]> * Create Typescript registration flow with email verification (#104) * Add endpoints for register user with email verification * Check email verified on the firebase user instead of the token * Create Python registration flow with email verification (#105) * Add endpoints to register user using email verification * fix import and add validator * Add validation for RegisterUserDTO * Fix accessing the email verification from token * Change the verification email title * check email verified in firebase user rather than token * Frontend User Sign-Up (#115) * Add sign-up page & auth client register functions * Rebase on main and make minor routing changes Co-authored-by: Sherry Li <[email protected]> * Integrate Google OAuth into Python backend (#112) * Add google oauth signin to firebase rest client * Add google oauth login and signup functionality * Add frontend for google oauth signin * Fix backend * Add comment for google user id token * Integrate Google OAuth into TS (#114) * Add REST Impl * Fix try catch Co-authored-by: Alex Guo <[email protected]> * Fix email_verified syntax in Python auth_service (#123) * Add TS GraphQL support for Google OAuth (#122) * GraphQL OAuth * rename id_token * Validators * Update authValidators.ts * Fix generate_csv_from_list call in entity and user routes (#119) * Integrate file uploads and downloads into frontend (#116) * add initial upload button and onChange function * Add field to display table * Fix mutation and add console.logs to debug * Update graphql queries in create and update * Add call in EntityAPIClient, graphql in DisplayTableContainer, and setup FileUtil * Fix display of buttons to only be shown if the fileName is not null * Add suggested changes to remove circular JSON errors * Uncomment python part in docker-compose.yml * Address PR comments and fix Graphql get in DisplayTableContainer * Fix REST file upload * Add REST file update to frontend * Decamelize formData when using Python backend * Fix fileUrl access in EntityAPIClient.getFile for Python backend Co-authored-by: Alex Guo <[email protected]> Co-authored-by: Sherry Li <[email protected]> * Create HooksDemo page (#125) * Update README.md (#128) * Fix various bugs (#127) * Add type check on self.enum_field before calling .upper() in validator * Set erase_db_and_sync in Python backend based on TESTING config value * Change DB dialect name from postgres to postgresql in Python connection string * Update testDb connect method for compatibility with mongo-memory-server's updated API * Change setData to setEntities in DisplayTableContainer.tsx * Add comments to AuthService's generateTokenOAuth method * Separate SignInResponse types for password and OAuth sign ins * Fix Python csv_utils to account for optional fields * Add *.egg-info to .gitignore * Remove commented out code in backend/typescript/utilities/CSVUtils.ts * Lint and reformat * Fix form reset issue after setting file field in CreateForm and UpdateForm * Fix casing in CSVUtils import path * Update Swagger docs with new endpoints and request body types (#131) * Fix error when deleting an entity with no file in Python (#133) * Fix error when deleting an entity with no file * Add id return for delete entity endpoint in TypeScript * Address comment * Modify and clean up infra setup (#134) * Add db health check, db init script, and container names * Make scripts executable * Fix formatting in backend/python/app/__init__.py * Fix whitespace * Fix ApolloClient link when using file storage without auth * Change user_id param type in Python update user endpoint Co-authored-by: Xin Hao Zhang <[email protected]> Co-authored-by: alexguo8 <[email protected]> Co-authored-by: Emily Louie <[email protected]> Co-authored-by: Tamim Khan <[email protected]> Co-authored-by: Richard Chen <[email protected]> Co-authored-by: Bruce He <[email protected]> Co-authored-by: Bruce He <[email protected]> Co-authored-by: Richard Chen <[email protected]> Co-authored-by: Dinu Wijetunga <[email protected]> Co-authored-by: Alex Guo <[email protected]>
uwblueprint · Dec 19, 2021 · 1a79fd7 · 1a79fd7
1 parent 267a034
commit 1a79fd7
Show file tree

Hide file tree

Showing 15 changed files with 74 additions and 34 deletions.
diff --git a/.gitattributes b/.gitattributes
@@ -0,0 +1 @@
+db-init/create-multiple-dbs.sh eol=lf
diff --git a/README.md b/README.md
@@ -117,10 +117,10 @@ docker ps
 
 ```bash
 # run a bash shell in the container
-docker exec -it <container-name> /bin/bash
+docker exec -it scv2_db /bin/bash
 
 # in container now
-psql -U postgres -d starter-code-v2
+psql -U postgres -d scv2
 
 # in postgres shell, some common commands:
 # display all table names
@@ -134,27 +134,27 @@ SELECT * FROM <table-name>;
 ### Linting & Formatting
 Python backend:
 ```bash
-docker exec -it <container-name> /bin/bash -c "black ."
+docker exec -it scv2_py_backend /bin/bash -c "black ."
 ```
 
 TypeScript backend and frontend:
 ```bash
 # linting & formatting warnings only
-docker exec -it <container-name> /bin/bash -c "yarn lint"
+docker exec -it scv2_ts_backend /bin/bash -c "yarn lint"
 
 # linting with fix & formatting
-docker exec -it <container-name> /bin/bash -c "yarn fix"
+docker exec -it scv2_ts_backend /bin/bash -c "yarn fix"
 ```
 
 ### Running Tests
 Python backend:
 ```bash
-docker exec -it <container-name> /bin/bash -c "pip install -e . && pytest"
+docker exec -it scv2_py_backend /bin/bash -c "pip install -e . && pytest"
 ```
 
 TypeScript backend and frontend:
 ```bash
-docker exec -it <container-name> /bin/bash -c "yarn test"
+docker exec -it scv2_ts_backend /bin/bash -c "yarn test"
 ```
 
 

diff --git a/backend/python/app/__init__.py b/backend/python/app/__init__.py
@@ -54,10 +54,12 @@ def create_app(config_name):
     ] = "postgresql://{username}:{password}@{host}:5432/{db}".format(
         username=os.getenv("POSTGRES_USER"),
         password=os.getenv("POSTGRES_PASSWORD"),
-        host=(
-            os.getenv("DB_TEST_HOST") if app.config["TESTING"] else os.getenv("DB_HOST")
+        host=os.getenv("DB_HOST"),
+        db=(
+            os.getenv("POSTGRES_DB_TEST")
+            if app.config["TESTING"]
+            else os.getenv("POSTGRES_DB_DEV")
         ),
-        db=os.getenv("POSTGRES_DB"),
     )
     app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False
     # } postgresql

diff --git a/backend/python/app/rest/user_routes.py b/backend/python/app/rest/user_routes.py
@@ -112,7 +112,12 @@ def create_user():
         return jsonify({"error": (error_message if error_message else str(e))}), 500
 
 
+# mongodb {
 @blueprint.route("/<string:user_id>", methods=["PUT"], strict_slashes=False)
+# } mongodb
+# postgresql {
+@blueprint.route("/<int:user_id>", methods=["PUT"], strict_slashes=False)
+# } postgresql
 @require_authorization_by_role({"User", "Admin"})
 @validate_request("UpdateUserDTO")
 def update_user(user_id):

diff --git a/backend/python/app/services/implementations/entity_service.py b/backend/python/app/services/implementations/entity_service.py
@@ -223,7 +223,7 @@ def delete_entity(self, id):
             # } file-storage
             deleted = Entity.objects(id=id).modify(remove=True)
             # file-storage {
-            if deleted:
+            if deleted and file_name:
                 self.file_storage_service.delete_file(file_name)
             # } file-storage
             return id

diff --git a/backend/typescript/models/index.ts b/backend/typescript/models/index.ts
@@ -3,7 +3,7 @@ import * as path from "path";
 import { Sequelize } from "sequelize-typescript";
 
 export const sequelize = new Sequelize(
-  `postgres://${process.env.POSTGRES_USER}:${process.env.POSTGRES_PASSWORD}@${process.env.DB_HOST}:5432/${process.env.POSTGRES_DB}`,
+  `postgres://${process.env.POSTGRES_USER}:${process.env.POSTGRES_PASSWORD}@${process.env.DB_HOST}:5432/${process.env.POSTGRES_DB_DEV}`,
   { models: [path.join(__dirname, "/*.model.ts")] },
 );
 

diff --git a/backend/typescript/rest/entityRoutes.ts b/backend/typescript/rest/entityRoutes.ts
@@ -152,8 +152,8 @@ entityRouter.delete("/:id", async (req, res) => {
   const { id } = req.params;
 
   try {
-    await entityService.deleteEntity(id);
-    res.status(204).send();
+    const deletedId = await entityService.deleteEntity(id);
+    res.status(200).json({ id: deletedId });
   } catch (e) {
     res.status(500).send(e.message);
   }

diff --git a/backend/typescript/services/implementations/entityService.ts b/backend/typescript/services/implementations/entityService.ts
@@ -172,7 +172,7 @@ class EntityService implements IEntityService {
     };
   }
 
-  async deleteEntity(id: string): Promise<void> {
+  async deleteEntity(id: string): Promise<string> {
     try {
       const deletedEntity: Entity | null = await MgEntity.findByIdAndDelete(id);
       if (!deletedEntity) {
@@ -183,6 +183,7 @@ class EntityService implements IEntityService {
         await this.storageService.deleteFile(deletedEntity.fileName);
       }
       // } file-storage
+      return id;
     } catch (error) {
       Logger.error(`Failed to delete entity. Reason = ${error.message}`);
       throw error;
@@ -378,7 +379,7 @@ class EntityService implements IEntityService {
     };
   }
 
-  async deleteEntity(id: string): Promise<void> {
+  async deleteEntity(id: string): Promise<string> {
     try {
       // file-storage {
       const entityToDelete = await PgEntity.findByPk(id, { raw: true });
@@ -402,6 +403,7 @@ class EntityService implements IEntityService {
         await this.storageService.deleteFile(entityToDelete.file_name);
       }
       // } file-storage
+      return id;
     } catch (error) {
       Logger.error(`Failed to delete entity. Reason = ${error.message}`);
       throw error;

diff --git a/backend/typescript/services/interfaces/IEntityService.ts b/backend/typescript/services/interfaces/IEntityService.ts
@@ -62,7 +62,8 @@ export interface IEntityService {
   /**
    * delete the entity with the given id
    * @param id entity id
+   * @returns id of the entity deleted
    * @throws Error if deletion fails
    */
-  deleteEntity(id: string): Promise<void>;
+  deleteEntity(id: string): Promise<string>;
 }
diff --git a/backend/typescript/swagger.yml b/backend/typescript/swagger.yml
@@ -284,7 +284,7 @@ paths:
         - Entity
       description: Delete an entity based on its ID
       responses:
-        '204':
+        '200':
           description: Successfully deleted entity based on ID
         '401':
           description: Unauthorized

diff --git a/backend/typescript/testUtils/testDb.ts b/backend/typescript/testUtils/testDb.ts
@@ -48,7 +48,7 @@ export default mongoTest;
 // postgresql {
 /* eslint-disable-next-line import/prefer-default-export */
 export const testSql = new Sequelize(
-  `postgres://${process.env.POSTGRES_USER}:${process.env.POSTGRES_PASSWORD}@${process.env.DB_TEST_HOST}:5432/${process.env.POSTGRES_DB}`,
+  `postgres://${process.env.POSTGRES_USER}:${process.env.POSTGRES_PASSWORD}@${process.env.DB_HOST}:5432/${process.env.POSTGRES_DB_TEST}`,
   { models: [resolve(__dirname, "../models/*.model.ts")], logging: false },
 );
 // } postgresql
diff --git a/db-init/create-multiple-dbs.sh b/db-init/create-multiple-dbs.sh
@@ -0,0 +1,22 @@
+#!/bin/bash
+
+set -e
+set -u
+
+function create_user_and_database() {
+	local database=$1
+	echo "  Creating user and database '$database'"
+	psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" <<-EOSQL
+	    CREATE USER $database;
+	    CREATE DATABASE $database;
+	    GRANT ALL PRIVILEGES ON DATABASE $database TO $database;
+EOSQL
+}
+
+if [ -n "$POSTGRES_MULTIPLE_DATABASES" ]; then
+	echo "Multiple database creation requested: $POSTGRES_MULTIPLE_DATABASES"
+	for db in $(echo $POSTGRES_MULTIPLE_DATABASES | tr ',' ' '); do
+		create_user_and_database $db
+	done
+	echo "Multiple databases created"
+fi
diff --git a/docker-compose.yml b/docker-compose.yml
@@ -2,6 +2,8 @@ version: "3.7"
 
 services:
   frontend:
+    # TODO: rename container for your project
+    container_name: scv2_frontend
     build:
       context: ./frontend
       dockerfile: Dockerfile
@@ -16,6 +18,8 @@ services:
       - ./frontend/.env
   typescript {
   ts-backend:
+    # TODO: rename container for your project
+    container_name: scv2_ts_backend
     build:
       context: ./backend/typescript
       dockerfile: Dockerfile
@@ -28,14 +32,16 @@ services:
       - 8.8.8.8
     postgresql {
     depends_on:
-      - db
-      - db-test
+      db:
+        condition: service_healthy
     } postgresql
     env_file:
       - ./.env
   } typescript
   python {
   py-backend:
+    # TODO: rename container for your project
+    container_name: scv2_py_backend
     restart: always
     build:
       context: ./backend/python
@@ -48,33 +54,34 @@ services:
       - ./backend/python:/app
     postgresql {
     depends_on:
-      - db
-      - db-test
+      db:
+        condition: service_healthy
     } postgresql
     env_file:
       - ./.env
   } python 
   postgresql {
   db:
+    # TODO: rename container for your project
+    container_name: scv2_db
     image: postgres:12-alpine
     ports:
       - 5432:5432
     volumes:
       - postgres_data:/var/lib/postgresql/data/
+      - ./db-init:/docker-entrypoint-initdb.d
     env_file:
       - ./.env
-  db-test:
-    image: postgres:12-alpine
-    ports:
-      - 5430:5432
-    volumes:
-      - postgres_data_test:/var/lib/postgresql/data/
-    env_file:
-      - ./.env
-    } postgresql
+    environment:
+      - POSTGRES_MULTIPLE_DATABASES=${POSTGRES_DB_DEV},${POSTGRES_DB_TEST}
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -U postgres"]
+      interval: 5s
+      timeout: 5s
+      retries: 5
+  } postgresql
 
 postgresql {
 volumes:
   postgres_data:
-  postgres_data_test:
 } postgresql
diff --git a/frontend/src/index.tsx b/frontend/src/index.tsx
@@ -110,7 +110,7 @@ const apolloClient = new ApolloClient({
   link,
   // } no-file-storage
   // file-storage {
-  link as any,
+  link: (link as any),
   // } file-storage
   // } no-auth
   cache: new InMemoryCache(),

diff --git a/setup.sh b/setup.sh