Release Notes for Data Sprint 21.08

Introduction

The following list of new features, tasks, and resolved bugs are included in the latest Sprint 21.08.4 release to FDW/FDE:

Validate Geographic Unit for Calculated Price Index https://fewsnet.atlassian.net/browse/DATA-748
DocumentTypeViewSet should only return document types with visible data https://fewsnet.atlassian.net/browse/DATA-986
Add an option to include Narrow Ranges in FIPE data domain extracts https://fewsnet.atlassian.net/browse/DATA-1033
Review Livelihood zones code column to use FNID instead of the LZ_CODE in uploads https://fewsnet.atlassian.net/browse/DATA-1035
Add admin 1 and admin 2 to the price time series extracts for both the price and CPI columns https://fewsnet.atlassian.net/browse/DATA-1047
Include Admin1 and admin2 in Market extracts in both the api end point and the admin extracts https://fewsnet.atlassian.net/browse/DATA-1057
Include datasource document in ExchangeRateResource and review all other dataseries resource classes to ensure they have source document in the import_fields list https://fewsnet.atlassian.net/browse/DATA-1059
Updated website link for Staple Food Price data download https://fewsnet.atlassian.net/browse/DATA-1063
Linear interpolation formulas for data points https://fewsnet.atlassian.net/browse/DATA-1065
Pull in Data from Colombia Price API https://fewsnet.atlassian.net/browse/DATA-1068
Market Price pipeline from KoboToolbox to FDW https://fewsnet.atlassian.net/browse/DATA-1074

Market screen doesn't allow to make a change on a country name https://fewsnet.atlassian.net/browse/DATA-1058
Fix Resource Class to Return Lat/Long in Market Extracts https://fewsnet.atlassian.net/browse/DATA-1093
Error in "other" domain of FDE https://fewsnet.atlassian.net/browse/DATA-1095
FDE filter not working for labor "products" https://fewsnet.atlassian.net/browse/DATA-1098
Cross Border Trade (XBT) Transit recognition error https://fewsnet.atlassian.net/browse/DATA-1100

This enhancement is about adding a validation error message , i.e The {icm.name} Index Component is for {icm.geographic_unit} Geographic Unit which cannot be aggregated to the {dsr.geographic_unit} Geographic Unit for this Calculated Index , for when an Index Component is added to a Calculated Price Index or if the Geographic Unit for a Calculated Price Index is changed.

This enhancement is about showing the document types of the OTHER data domain (https://fdw.fews.net/data-explorer/#/other) which contains data for the external users.

The /api/ipcpopulationsize/ API end point has been enhanced to allow use of the range parameter which as a result accepts the point, narrow and public values; for instance:

This feature is about enhancing the livelihood zones code column to use the FNID; i.e https://fdw.fews.net/en/admin/spatial/livelihoodzone/.

This enhancement is about adding admin 1 and admin 2 fields to the price timeseries extract, (i.e http://fdw.fews.net/api/marketpricefacts/?format=csv&timeseries=true&country_code=CM) and the metadata fields are revised based on the suggestions of the EWT i.e base_period_start - base_period_end is shown as a concatenated field and in addition to this, the period dates are shown in the newly added column.

This feature is about allowing the inclusion of Admin1 and Admin2 fields in the market API end point (https://fdw.fews.net/api/market/) and Admin (https://fdw.fews.net/en/admin/spatial/market/export/?) extracts.

This feature is about introducing datasourcedocument in all the import_fields of the data series resource classes, for instance: https://fdw.fews.net/en/admin/price/exchangerate/import/.

This enhancement is about implementing a linear interpolation calculation formula for a maximum of four consecutive missing data point values of a data series.

This feature is about introducing a pipeline that imports daily data collections from the Colombia DANE API.

This feature is about establishing the ingestion of the price data using Kobo into FDW. The weekly data collected in kobo (forms.fews.net) and its status approved is automatically imported into FDW.

There was an issue in updating the country name of the market screen (https://fdw.fews.net/en/admin/spatial/market/) and this was resolved.

The market extract was returning an empty columns for the metadata fields of Latitude and Longitude and this was fixed and users can now access the values through the extract, (i.e https://fdw.fews.net/en/admin/spatial/market/).

Trying to select data series from the OTHER data domain of the FDE resulted in a blank page and this was resolved and the functionality is working as expected.

There was a problem with filtering specific products of Labor in the price data domain of the FDE, (i.e https://fdw.fews.net/data-explorer/#/price) and this was resolved.

There was a problem in recognizing the Transit flow_type in the cross border trade data upload page and this was resolved and the metadata successfully loaded.