Release Notes for Data Sprint 22.08

The following list of new features, updates and bug fixes are included in the latest Sprint 22.08 release to FDW/FDE.

Updates

A new collect_datasourcedocument permission allows more granular access permissions to data source documents in FDW than the existing manage_datasourcedocument permission. The new permission grants users access to add and edit data collections and has enabled a new filter. When uploading data on the data collection page, the new filter only shows the data source documents for which the user has the collect_datasourcedocument permission.
This update includes the following Jira issues:
- FDW data source filter on data collection screen (DATA-1462)
- Add collect_datasourcedocument permission to fdw. (DATA-1638)

All instances of using the IPC31 value in FDW have been be replaced with the choices label IPC 3.1.

Two new pipelines have been established to ingest data collections from the field via MagPi and KoBoToolbox into the data platform. Metadata match reports are being automatically generated as part of the new pipelines.
This update includes the following Jira issues:
- XBT Ingestion pipeline from Kobo toolbox (DATA-1669)
- Price Ingestion pipeline from MagPi to FDW for Ethiopia (DATA-1671)

Food insecurity classification data and shapefiles have been updated in several ways. For example, empty data sets were removed, the list of viewable fields was updated for data.fews.net and fews.net, and more standardized tags were incorporated. In addition, data sets for historic IPC shapefiles were re-organized.
This update includes the following Jira issues:
- Update IPC FIC Shapefile publishing to publish to data.fews.net and HDX as well as website (DATA-1721)
- Publish historic IPC FIC Shapefiles to data.fews.net and HDX (DATA-1723)

The list of default fields for Food Insecure Population Estimates data in the API, FDE and on fews.net and data.fews.net has been updated to the following: source_organization, source_document, country, country_code, geographic_group, fewsnet_region, admin_0, admin_1, admin_2, admin_3, admin_4, phase, scenario_name, indicator_group, indicator_name, start_date, period_date, scenario, data_usage_policy, collection_schedule, phase_name, and population_range.

Price projection spreadsheets can now be uploaded to an existing data source document using Add Data Collection in FDW.

Attempting to open the Semi structured data points page in the FDW was resulting in an error caused by data points from the Source Document with values of NaN instead of a numeric or a null value. This issue was resolved and the page can now be accessed.

The link in the footer of FDE to the FEWS NET privacy policy was broken. It is now linked to the correct page.

Trying to edit IPC Population data collections was resulting in an error message due to data series with empty values being set with a default status of Collected. Setting the default status to Not collected has resolved the error and users are now able to edit the data collections.

Error pages in FDW were incorrectly showing a {% trans %} tag. This issue has been fixed and error pages are now showing correct messages.

The IPC 3.1 migration caused duplicate names to be produced due to the way the geographic unit name was calculated. This has been fixed and the IPC migration now runs correctly.

Trying to search in the News Articles admin screen in FDW was resulting in an error message. This issue has been fixed and the search function on the News Articles admin page now works correctly.

Trying to edit the data point values of the survey data domains (i.e., Crop, Nutrition, and Population) was resulting in an error message. The issue has been fixed and survey domain data points can now be successfully edited.