Release Notes for Data Sprint 21.05

Introduction

The following list of new features, tasks, and resolved bugs are included in the latest Sprint 21.05 release to FDW/FDE:

Features

• Add FDW vector tiles to COVID-19 map https://fewsnet.atlassian.net/browse/DATA-529

• Capture admin_1 and admin_2 relationship for IDP camps where they exist https://fewsnet.atlassian.net/browse/DATA-596

• Add availability data to /admin views https://fewsnet.atlassian.net/browse/DATA-865

• Support as-is ingestion of semi-structured data https://fewsnet.atlassian.net/browse/DATA-884

• Use dataset parameter in preference to dataseries list in FDE where possible https://fewsnet.atlassian.net/browse/DATA-900

• Support fields parameter on semi-structured values API endpoint https://fewsnet.atlassian.net/browse/DATA-918

• Add Options tab to Other Domain https://fewsnet.atlassian.net/browse/DATA-923

• On Trade Domain, Add filter for Trade Type (i.e. formal, informal) https://fewsnet.atlassian.net/browse/DATA-927

• Data Collection and Data Collection Period Country filters should work against expected Data Series https://fewsnet.atlassian.net/browse/DATA-930

• Add FDW vector tiles to IPC PNG maps https://fewsnet.atlassian.net/browse/DATA-936

Tasks

• Update to main django-autocomplete-light https://fewsnet.atlassian.net/browse/DATA-649

• Upload global admin0 units and world ocean https://fewsnet.atlassian.net/browse/DATA-943

Bugs

• Fix query count for Geographic Unit Export https://fewsnet.atlassian.net/browse/DATA-818

• Calculated Index Value should return Not Collected if any component is missing https://fewsnet.atlassian.net/browse/DATA-844

• Search filter working incorrectly on Indicator API Endpoint https://fewsnet.atlassian.net/browse/DATA-925

Features

Add FDW vector tiles to COVID-19 map https://fewsnet.atlassian.net/browse/DATA-529

• This feature is about implementing the FDW vector tiles to COVID-19 map by changing the API calls of geojson to vector tiles for both admin units and IPC data.

Capture admin_1 and admin_2 relationship for IDP camps where they exist https://fewsnet.atlassian.net/browse/DATA-596

• This enhancement is about creating admin_1 and admin_2 geographic unit relationships of the IDP camps, as seen here:

https://fdw.fews.net/en/admin/spatial/geographicunitrelationshiptype/idp_5Fcamp_5Fadmin2/change/

https://fdw.fews.net/en/admin/spatial/geographicunitrelationshiptype/idp_5Fcamp_5Fadmin1/change/

Add availability data to /admin views https://fewsnet.atlassian.net/browse/DATA-865

• This enhancement is about adding the availability data to the datacollectionperiod change admin screen, which enables the user to view and update the Availability Data in numeric form

  (i.e. https://fdw.fews.net/en/admin/warehouse/datacollectionperiod/163139/change/?_changelist_filters=country%3DMG%26data_domain%3DMarketProduct ).

Support as-is ingestion of semi-structured data https://fewsnet.atlassian.net/browse/DATA-884

• This feature is about allowing the as-is ingestion of semi-structured data (i.e. data which does not inherently belong to an existing domain) which makes it easy to load new generic data sources using the offline data upload functionality of the FDW, without performing any data or spreadsheet manipulation (i.e.  https://fdw.fews.net/en/admin/warehouse/dataupload/add/?_changelist_filters=subtype__exact%3DAsIsDataPoint ).

Use dataset parameter in preference to dataseries list in FDE where possible https://fewsnet.atlassian.net/browse/DATA-900

• This feature is about enabling the ability to open an existing DataSet in FDE, check that the correct Data Series are included, and then copy the refreshable link from the Export tab and use it in Excel.

Support fields parameter on semi-structured values API endpoint https://fewsnet.atlassian.net/browse/DATA-918

• This enhancement is about incorporating fields parameter on semi-structured API endpoints which helps to filter the data returned from the API end points, for instance:

  https://fdw.fews.net/api/remittancesvalue/?format=xml&dataseries=6450072&fields=simple   

  https://fdw.fews.net/api/days_work_availablevalue/?format=xml&dataseries=6466282&fields=simple .

Add Options tab to Other Domain https://fewsnet.atlassian.net/browse/DATA-923

• This feature is about implementing the Options tab to the Other data domain of the FDE, which enables the user to view and analyze the data by Reporting schedule parameter. Users can select a Daily, Weekly, Monthly, Quarterly or Annual reporting schedule.

On Trade Domain, Add filter for Trade Type (i.e. formal, informal) https://fewsnet.atlassian.net/browse/DATA-927

• This enhancement is about adding the Trade type filter (i.e formal, informal) to the Trade data domain of the FDE.

Data Collection and Data Collection Period Country filters should work against expected Data Series https://fewsnet.atlassian.net/browse/DATA-930

• This enhancement allows users to use the country filter on the DataCollectionPeriod and DataCollection admin screens to filter all relevant countries data series for a specific country data collection.

Add FDW vector tiles to IPC PNG maps https://fewsnet.atlassian.net/browse/DATA-936

• This feature is about applying the FDW vector tiles, instead of geojson, to the IPC PNG maps within the system to improve the performance of rending of maps for users.

Tasks

Update to main django-autocomplete-light https://fewsnet.atlassian.net/browse/DATA-649

• This task was about the upgrading of Django with auto-complete-light (a new Django feature) which results in allowing the use of auto-complete within fields of the admin screens of the FDW, such as:

https://fdw.fews.net/en/admin/warehouse/datacollection/add/

https://fdw.fews.net/en/admin/price/marketproduct/add/

https://fdw.fews.net/en/admin/price/exchangerate/add/

https://fdw.fews.net/en/admin/price/priceindex/add/

https://fdw.fews.net/en/admin/price/calculatedindex/add/

https://fdw.fews.net/en/admin/warehouse/datasourcedocument/add/

Upload global admin0 units and world ocean https://fewsnet.atlassian.net/browse/DATA-943

• This task was about uploading the global Admin0 geographic unit boundaries, world ocean layer and lakes to FDW, so that they show up correctly on maps rendered from FDW.

Bugs

Fix query count for Geographic Unit Export https://fewsnet.atlassian.net/browse/DATA-818

• There was an issue of query count in the GeographicUnit export and this was resolved.

Calculated Index Value should return Not Collected if any component is missing https://fewsnet.atlassian.net/browse/DATA-844

• There was an issue in the calculated index values which had missing components and this was resolved. It is returning Not Collected values for an index which contains missing components.

Search filter working incorrectly on Indicator API Endpoint https://fewsnet.atlassian.net/browse/DATA-925

• There was an issue in using the search parameter of the Indicator API endpoint and this was resolved. The functionality is working as expected, as can be seen here:

https://fdw.fews.net/api/indicator/?search=measles vaccination&data_domain=NutritionIndicator

https://fdw.fews.net/api/indicator/?search=latrines&data_domain=NutritionIndicator