This repository has been archived by the owner on Sep 16, 2022. It is now read-only.
Cscfairmeta 847 relax draft validation #740
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add extra validation to publishing method to validate against the given data catalog
- One old tests needed to be changed to match the new functionality
EarthModule
pushed a commit
that referenced
this pull request
Jan 27, 2021
* add error handling to migration 27 in case of missing syke.csv file * split settings with django-split-settings, enable root logger, add .env support * replace redis client with new implementation, replace yml file with settings * integrate refdata loading as a task * remove unnecessary 403 error on flush_records * remove the yaml loading blocks * replace configs with env variables * fix some unnecessary asserts in tests * remove unnecessary request logging logic * update .gitignore, deps, and travis. Create .env.template * delete settings.py * update readme * fix ENV_VARS instructions * CSCFAIRADM-375: [ADD] instructions for mac and for db creation (#679) * add watchman system monitoring, bind elasticsearch hosts to env * disable tests testing file permissions, make elasticsearch take its host from env vars, add Docker as an option for local development setup * Increase logging on redis and rabbitmq connections, fix rabbitmq retry behaviour to be faster * optimize imports * CSCFAIRADM-375: [FIX] typos in variables and ignore log directory * CSCFAIRMETA-701: [FIX] crash on v2 dataset rpc api - Prevents creating a new dataset version if there exists a draft * CSCFAIRMETA-701: trigger travis * Update-test-db-with-migrations - Add migrations to correspond production db * Bump django from 3.1.2 to 3.1.4 Bumps [django](https://github.com/django/django) from 3.1.2 to 3.1.4. - [Release notes](https://github.com/django/django/releases) - [Commits](django/django@3.1.2...3.1.4) Signed-off-by: dependabot[bot] <[email protected]> * Modify-JYU-catalog-logo - Remove URL from JYU catalog logo name * Document env vars (#720) Co-authored-by: Toni Nurmi <[email protected]> * add OAI env vars OAI_REPOSITORY_NAME, OAI_ETSIN_URL_TEMPLATE and OAI_ADMIN_EMAIL. Add new remote environment (#724) Co-authored-by: Toni Nurmi <[email protected]> * Add env var to control if rabbitmq should use vhost. Change RabbitMQ service to use VHOST only if env var allows it (#726) Co-authored-by: Toni Nurmi <[email protected]> * CSCFAIRMETA-799-RabbitMQ-TTV (#722) * CSCFAIRMETA-799-RabbitMQ-TTV - Add new connection with new vhost to TTV - Add new exchange and routing_key for TTV - Publish messages to 1 queue for TTV * CSCFAIRMETA-799-RabbitMQ-TTV - Add rabbitmq consumer for ttv * CSCFAIRMETA-799: [FIX] simplify publish process - Use the same virtual host for ttv and etsin - Adding exchanges should now only need to define it in settings - Define queues in settings - Move rabbitmq consumers to local environment since they are not used in other places Co-authored-by: Katri Tegel <[email protected]> Co-authored-by: Tommi Pulli <[email protected]> * CSCFAIRMETA-799-RabbitMQ-TTV-fix-settings (#728) -Rename django_settings to settings Co-authored-by: Katri Tegel <[email protected]> * Add-files-to-att-dataset (#732) Modify single dataset a0651f93-1826-4264-8fe4-df39f1ee72ac in prod Related to changing catalog Co-authored-by: Katri Tegel <[email protected]> * Add ENVs to travis.yml and fixed related settings - added SSL for elasticsearch in ENV - rabbitmq vhost is name is not needed if it's not used - travis environment settings are the same as in test environment * linting and removed unnecessary setting variable * fix flake8 problems and ones that are consequent - api_access settings were defined too permissive for unit tests - uncommented previously commented auth tests because of the api_access change - deleted a file permission related test that tests trivial issue and causes a headache in the new docker local env * add directory for logs during travis before_script * add rabbitmq hosts to default env * start using real redis in travis - Ignore console logging in Travis - test/stable deploys were removed since they aren't used anyways * Add-files-to-att-dataset (#734) Add dataset_version_set to datasets Co-authored-by: Katri Tegel <[email protected]> * Att test dataset (#736) * Add-files-to-att-dataset -Add test dataset that belongs to IDA-project Co-authored-by: Katri Tegel <[email protected]> * Cscfairmeta 847 relax draft validation (#740) * CSCFAIRMETA-847: [ADD] needed changes in draft json schema * CSCFAIRMETA-847: [FIX] draft datasets use dft schema - Add extra validation to publishing method to validate against the given data catalog * CSCFAIRMETA-847: [ADD|FIX] tests * CSCFAIRMETA-890-DOI-as-pref-identifier (#741) * CSCFAIRMETA-890-DOI-as-pref-identifier - Add a check for DOIs from remote datasets - Add a check that remote datasets with DOI are validated against datacite * CSCFAIRMETA-890-DOI-as-pref-identifier - Modify check for which DOIs are sent for registration Co-authored-by: Katri Tegel <[email protected]> * CSCFAIRMETA-776: [ADD] Bulk delete (#725) * CSCFAIRMETA-776: [ADD] Bulk delete * CSCFAIRMETA-776-bulk-delete -Enable deleting of drafts Co-authored-by: genie9 <[email protected]> Co-authored-by: Katri Tegel <[email protected]> * CSCFAIRMETA-847: [FIX] bug where files could not be added to drafts * Api endpoints env (#743) Co-authored-by: Toni Nurmi <[email protected]> * remove test comparing datetimes down to seconds as not needed (#744) Co-authored-by: Toni Nurmi <[email protected]> * Bump djangorestframework from 3.12.1 to 3.12.2 (#718) Bumps [djangorestframework](https://github.com/encode/django-rest-framework) from 3.12.1 to 3.12.2. - [Release notes](https://github.com/encode/django-rest-framework/releases) - [Commits](encode/django-rest-framework@3.12.1...3.12.2) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Toni Nurmi <[email protected]> * Bump lxml from 4.6.1 to 4.6.2 (#733) Bumps [lxml](https://github.com/lxml/lxml) from 4.6.1 to 4.6.2. - [Release notes](https://github.com/lxml/lxml/releases) - [Changelog](https://github.com/lxml/lxml/blob/master/CHANGES.txt) - [Commits](lxml/lxml@lxml-4.6.1...lxml-4.6.2) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Toni Nurmi <[email protected]> * Bump pytz from 2020.1 to 2020.5 (#730) Bumps [pytz](https://github.com/stub42/pytz) from 2020.1 to 2020.5. - [Release notes](https://github.com/stub42/pytz/releases) - [Commits](stub42/pytz@release_2020.1...release_2020.5) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Toni Nurmi <[email protected]> Co-authored-by: Toni Nurmi <[email protected]> Co-authored-by: Tommi Pulli <[email protected]> Co-authored-by: Tommi Pulli <[email protected]> Co-authored-by: Katri Tegel <[email protected]> Co-authored-by: Toni Nurmi <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Evgenia Lyjina <[email protected]> Co-authored-by: genie9 <[email protected]>
EarthModule
pushed a commit
that referenced
this pull request
Mar 10, 2021
* Bump django from 3.1.2 to 3.1.4 Bumps [django](https://github.com/django/django) from 3.1.2 to 3.1.4. - [Release notes](https://github.com/django/django/releases) - [Commits](django/django@3.1.2...3.1.4) Signed-off-by: dependabot[bot] <[email protected]> * include comparison values in logging when comparing auth password if debug is on * Update-test-db-with-migrations - Add migrations to correspond production db * Add-files-to-att-dataset (#732) Modify single dataset a0651f93-1826-4264-8fe4-df39f1ee72ac in prod Related to changing catalog Co-authored-by: Katri Tegel <[email protected]> * Add-files-to-att-dataset (#734) Add dataset_version_set to datasets Co-authored-by: Katri Tegel <[email protected]> * Att test dataset (#736) * Add-files-to-att-dataset -Add test dataset that belongs to IDA-project * att-test-dataset Add att-test-dataset i# with '#' will be ignored, and an empty message aborts the commit. Co-authored-by: Katri Tegel <[email protected]> * Bump djangorestframework from 3.12.1 to 3.12.2 (#718) Bumps [djangorestframework](https://github.com/encode/django-rest-framework) from 3.12.1 to 3.12.2. - [Release notes](https://github.com/encode/django-rest-framework/releases) - [Commits](encode/django-rest-framework@3.12.1...3.12.2) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Toni Nurmi <[email protected]> * Bump lxml from 4.6.1 to 4.6.2 (#733) Bumps [lxml](https://github.com/lxml/lxml) from 4.6.1 to 4.6.2. - [Release notes](https://github.com/lxml/lxml/releases) - [Changelog](https://github.com/lxml/lxml/blob/master/CHANGES.txt) - [Commits](lxml/lxml@lxml-4.6.1...lxml-4.6.2) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Toni Nurmi <[email protected]> * Bump pytz from 2020.1 to 2020.5 (#730) Bumps [pytz](https://github.com/stub42/pytz) from 2020.1 to 2020.5. - [Release notes](https://github.com/stub42/pytz/releases) - [Commits](stub42/pytz@release_2020.1...release_2020.5) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Toni Nurmi <[email protected]> * Merge Test to Stable (#737) * add error handling to migration 27 in case of missing syke.csv file * split settings with django-split-settings, enable root logger, add .env support * replace redis client with new implementation, replace yml file with settings * integrate refdata loading as a task * remove unnecessary 403 error on flush_records * remove the yaml loading blocks * replace configs with env variables * fix some unnecessary asserts in tests * remove unnecessary request logging logic * update .gitignore, deps, and travis. Create .env.template * delete settings.py * update readme * fix ENV_VARS instructions * CSCFAIRADM-375: [ADD] instructions for mac and for db creation (#679) * add watchman system monitoring, bind elasticsearch hosts to env * disable tests testing file permissions, make elasticsearch take its host from env vars, add Docker as an option for local development setup * Increase logging on redis and rabbitmq connections, fix rabbitmq retry behaviour to be faster * optimize imports * CSCFAIRADM-375: [FIX] typos in variables and ignore log directory * CSCFAIRMETA-701: [FIX] crash on v2 dataset rpc api - Prevents creating a new dataset version if there exists a draft * CSCFAIRMETA-701: trigger travis * Update-test-db-with-migrations - Add migrations to correspond production db * Bump django from 3.1.2 to 3.1.4 Bumps [django](https://github.com/django/django) from 3.1.2 to 3.1.4. - [Release notes](https://github.com/django/django/releases) - [Commits](django/django@3.1.2...3.1.4) Signed-off-by: dependabot[bot] <[email protected]> * Modify-JYU-catalog-logo - Remove URL from JYU catalog logo name * Document env vars (#720) Co-authored-by: Toni Nurmi <[email protected]> * add OAI env vars OAI_REPOSITORY_NAME, OAI_ETSIN_URL_TEMPLATE and OAI_ADMIN_EMAIL. Add new remote environment (#724) Co-authored-by: Toni Nurmi <[email protected]> * Add env var to control if rabbitmq should use vhost. Change RabbitMQ service to use VHOST only if env var allows it (#726) Co-authored-by: Toni Nurmi <[email protected]> * CSCFAIRMETA-799-RabbitMQ-TTV (#722) * CSCFAIRMETA-799-RabbitMQ-TTV - Add new connection with new vhost to TTV - Add new exchange and routing_key for TTV - Publish messages to 1 queue for TTV * CSCFAIRMETA-799-RabbitMQ-TTV - Add rabbitmq consumer for ttv * CSCFAIRMETA-799: [FIX] simplify publish process - Use the same virtual host for ttv and etsin - Adding exchanges should now only need to define it in settings - Define queues in settings - Move rabbitmq consumers to local environment since they are not used in other places Co-authored-by: Katri Tegel <[email protected]> Co-authored-by: Tommi Pulli <[email protected]> * CSCFAIRMETA-799-RabbitMQ-TTV-fix-settings (#728) -Rename django_settings to settings Co-authored-by: Katri Tegel <[email protected]> * Add-files-to-att-dataset (#732) Modify single dataset a0651f93-1826-4264-8fe4-df39f1ee72ac in prod Related to changing catalog Co-authored-by: Katri Tegel <[email protected]> * Add ENVs to travis.yml and fixed related settings - added SSL for elasticsearch in ENV - rabbitmq vhost is name is not needed if it's not used - travis environment settings are the same as in test environment * linting and removed unnecessary setting variable * fix flake8 problems and ones that are consequent - api_access settings were defined too permissive for unit tests - uncommented previously commented auth tests because of the api_access change - deleted a file permission related test that tests trivial issue and causes a headache in the new docker local env * add directory for logs during travis before_script * add rabbitmq hosts to default env * start using real redis in travis - Ignore console logging in Travis - test/stable deploys were removed since they aren't used anyways * Add-files-to-att-dataset (#734) Add dataset_version_set to datasets Co-authored-by: Katri Tegel <[email protected]> * Att test dataset (#736) * Add-files-to-att-dataset -Add test dataset that belongs to IDA-project Co-authored-by: Katri Tegel <[email protected]> * Cscfairmeta 847 relax draft validation (#740) * CSCFAIRMETA-847: [ADD] needed changes in draft json schema * CSCFAIRMETA-847: [FIX] draft datasets use dft schema - Add extra validation to publishing method to validate against the given data catalog * CSCFAIRMETA-847: [ADD|FIX] tests * CSCFAIRMETA-890-DOI-as-pref-identifier (#741) * CSCFAIRMETA-890-DOI-as-pref-identifier - Add a check for DOIs from remote datasets - Add a check that remote datasets with DOI are validated against datacite * CSCFAIRMETA-890-DOI-as-pref-identifier - Modify check for which DOIs are sent for registration Co-authored-by: Katri Tegel <[email protected]> * CSCFAIRMETA-776: [ADD] Bulk delete (#725) * CSCFAIRMETA-776: [ADD] Bulk delete * CSCFAIRMETA-776-bulk-delete -Enable deleting of drafts Co-authored-by: genie9 <[email protected]> Co-authored-by: Katri Tegel <[email protected]> * CSCFAIRMETA-847: [FIX] bug where files could not be added to drafts * Api endpoints env (#743) Co-authored-by: Toni Nurmi <[email protected]> * remove test comparing datetimes down to seconds as not needed (#744) Co-authored-by: Toni Nurmi <[email protected]> * Bump djangorestframework from 3.12.1 to 3.12.2 (#718) Bumps [djangorestframework](https://github.com/encode/django-rest-framework) from 3.12.1 to 3.12.2. - [Release notes](https://github.com/encode/django-rest-framework/releases) - [Commits](encode/django-rest-framework@3.12.1...3.12.2) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Toni Nurmi <[email protected]> * Bump lxml from 4.6.1 to 4.6.2 (#733) Bumps [lxml](https://github.com/lxml/lxml) from 4.6.1 to 4.6.2. - [Release notes](https://github.com/lxml/lxml/releases) - [Changelog](https://github.com/lxml/lxml/blob/master/CHANGES.txt) - [Commits](lxml/lxml@lxml-4.6.1...lxml-4.6.2) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Toni Nurmi <[email protected]> * Bump pytz from 2020.1 to 2020.5 (#730) Bumps [pytz](https://github.com/stub42/pytz) from 2020.1 to 2020.5. - [Release notes](https://github.com/stub42/pytz/releases) - [Commits](stub42/pytz@release_2020.1...release_2020.5) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Toni Nurmi <[email protected]> Co-authored-by: Toni Nurmi <[email protected]> Co-authored-by: Tommi Pulli <[email protected]> Co-authored-by: Tommi Pulli <[email protected]> Co-authored-by: Katri Tegel <[email protected]> Co-authored-by: Toni Nurmi <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Evgenia Lyjina <[email protected]> Co-authored-by: genie9 <[email protected]> * fix near production api access scheme (#747) Co-authored-by: Toni Nurmi <[email protected]> * add migration to fix incorrect file counts on directories (#752) Co-authored-by: Toni Nurmi <[email protected]> * hotfixes to stable before production update (#748) * fix near production api access scheme (#747) Co-authored-by: Toni Nurmi <[email protected]> * add migration to fix incorrect file counts on directories (#752) Co-authored-by: Toni Nurmi <[email protected]> * make new management command to verify filecount for directories (#755) * make new management command to verify filecount for directories * fix/directory_file Fix typos s Co-authored-by: Toni Nurmi <[email protected]> Co-authored-by: Katri Tegel <[email protected]> * rewrite load initial data management command (#758) * rewrite load initial data management command Co-authored-by: Toni Nurmi <[email protected]> * delete flake8 check from travis. (#760) * delete flake8 check from travis. * cd into src before calling manage.py Co-authored-by: Toni Nurmi <[email protected]> * fix service role issues (#761) * fix service role issues * add draft perms Co-authored-by: Toni Nurmi <[email protected]> * Add_metax_env_variable_to_components (#756) Add METAX_ENV to components via version control Co-authored-by: Katri Tegel <[email protected]> Co-authored-by: Toni Nurmi <[email protected]> * CSCFAIRMETA-912-recalculate-bytesize-filecount-for-cr (#765) - Recalculate byte_size and file_count for catalog record Co-authored-by: Katri Tegel <[email protected]> * CSCFAIRMETA-954-publish-records-to-TTV (#767) - Fetch and publish all existing datasets into, and only to, TTV-operations queue Co-authored-by: Katri Tegel <[email protected]> * include comparison values in logging when comparing auth password if debug is on * clear API_USERS setting if server is restarted Co-authored-by: Katri Tegel <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Toni Nurmi <[email protected]> Co-authored-by: Katri Tegel <[email protected]> Co-authored-by: Tommi Pulli <[email protected]> Co-authored-by: Tommi Pulli <[email protected]> Co-authored-by: Evgenia Lyjina <[email protected]> Co-authored-by: genie9 <[email protected]>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.