This repository has been archived by the owner on Sep 16, 2022. It is now read-only.
Merge stable to master: v0.9.7 #424
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…is info when populating org object: if user has not provided an is_part_of relation for an org obj and if the org obj itself is a sub org from ref data (as indicated by the cache), populate parent org automatically to sub org obj with a new is_part_of relation org object.
CSCMETAX-531: [ADD] Add parent org code to org ref data cache. Use th…
…log object's id field contained a string
- Add new field to catalog record: preservation_identifier. The field value can be modified only by Metax. It is set 1) on cr creation time if cr is ida type and user has requested a doi pid for the cr or 2) if the previous cr version had a doi pid or 3) separately by the new RPC endpoint described below. - Change the way convert_catalog_record_to_datacite_xml method chooses the doi to use: preservation_identifier field is now preferred over preferred identifier. Only if neither of the mentioned fields contain a valid doi, use the dummy doi. The methods calling this method need to manually add preservation_identifier to the json provided, if it has a value. - Add new RPC endpoint 'set_preservation_identifier' to be used only by tpas user. The endpoint returns preservation identifier when it runs successfully. Preservation identifier is created for the cr if it does not exist. - Refactor DataciteDOIUpdate class to be agnostic regarding doi location within cr: doi is now given to it. Check that the doi is in fact generated by Metax.
CSCMETAX-524: structured logging
- Add preservation_identifier to all dataset schemas - Add preservation_identifier to catalogrecord.json in api_schemas - Add set_preservation_identifier API endpoint doc to swagger - Add warning to set_preservation_identifier if a code block, which is not supposed to be reached, is reached - On saving cr in set_preservation_identifier, only update one field - Add tests for set_preservation_identifier and datacite xml conversion for the doi part - Add ida, att and legacy data catalog identifiers to settings and use them wherever possible - Change dataset creation parameter 'pid_type' so that doi value can be applied only for ida datasets - Fix a bug found in doi identifier generation
…CSCfi/metax-api into CSCMETAX-526-pas-doi-generation
- settings.py bug
…atalogs-from-app-config CSCMETAX-61: [REF] Load END_USER_ALLOWED_DATA_CATALOGS from app_confi…
Cscmetax 526 pas doi generation
…fier in one catalog record gets its own oai-pmh record.
CSCMETAX-524: [ADD] Refine and add more structured logging entries
- OAI Server and datasets REST API (using dataset_format parameter) now offer two datacite formats (OAI: oai_datacite, oai_fairdata_datacite / datasets REST: datacite, fairdata_datacite) - datacite is the strict version, which requires datacite required fields to be present and which is validated against datacite schema - fairdata-datacite is the less strict version, which allows urn as datacite identifier and uses fallback values on some required but missing datacite field values - Datacite xml creation separated from datacite validation - This is done due to the fact that when doi creation is requested for a dataset, the dataset must adhere to the strict datacite schema - otherwise the datacite xml along with the doi could not be sent to Datacite API as it would fail the validation. So, catalog record needs to be validated against datacite schema before it can be created or updated - Fix a bug in request_logging
- Change the order in which datacite-required fields are checked, since tpas might benefit from having identifier checked last
- Move code to its right place
- Do not allow datacite harvesting using ListRecords OAI-PMH verb, since resumptionToken gets confused when some records cannot be displayed - Refix the thought-to-be-fixed request_logging bug
- Flake8
- Enable oai_datacite and oai_fairdata_datacite metadata prefixes for OAI-OMH ListRecords verb. Does not display those records that would not be shown either with GetRecord verb.
…n already existing record. Fix tests accordingly
…ogs-on-update CSCMETAX-537: [FIX] Check permitted data catalogs also when editing a…
CSCMETAX-540: [ADD] PAS file storage to initialdata/
- Remove useless code
…Cfi/metax-api into CSCMETAX-529-530-541-datacite
CSCMETAX-529-530-541: [ADD|REF]
- When debug is false and json is requested from api using Accept-header 'text/html' (e.g. browser), fix encoding of some characters.
CSCMETAX-535: [REF]
…ore reference data in a python var in process, when reference data is first needed in that process. Uses a bit more memory, but gunicorn processes are restarted periodically anyway
CSCMETAX-61: [REF] Optimize ref data loading
…research_dataset.files and .directories, automatically populate them with titles
… to total_files_byte_size
CSCMETAX-528: [ADD] Populate file and dir metadata title if omitted by user
…-size CSCMETAX-533: [REF] Rename research_dataset field total_ida_byte_size…
Merge pull request #423 from CSCfi/test
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.