Release Notes for Data Sprint 21.08

Introduction

The following list of new features, tasks, and resolved bugs are included in the latest Sprint 21.08.4 release to FDW/FDE:

Features

• Validate Geographic Unit for Calculated Price Index https://fewsnet.atlassian.net/browse/DATA-748

• DocumentTypeViewSet should only return document types with visible data https://fewsnet.atlassian.net/browse/DATA-986

• Add an option to include Narrow Ranges in FIPE data domain extracts https://fewsnet.atlassian.net/browse/DATA-1033

• Review Livelihood zones code column to use FNID instead of the LZ_CODE in uploads https://fewsnet.atlassian.net/browse/DATA-1035

• Add admin 1 and admin 2 to the price time series extracts for both the price and CPI columns https://fewsnet.atlassian.net/browse/DATA-1047

• Include Admin1 and admin2 in Market extracts in both the api end point and the admin extracts https://fewsnet.atlassian.net/browse/DATA-1057

• Include datasource document in ExchangeRateResource and review all other dataseries resource classes to ensure they have source document in the import_fields list https://fewsnet.atlassian.net/browse/DATA-1059

• Updated website link for Staple Food Price data download https://fewsnet.atlassian.net/browse/DATA-1063

• Linear interpolation formulas for data points https://fewsnet.atlassian.net/browse/DATA-1065

• Pull in Data from Colombia Price API https://fewsnet.atlassian.net/browse/DATA-1068

• Market Price pipeline from KoboToolbox to FDW https://fewsnet.atlassian.net/browse/DATA-1074

Tasks

• Unable to load NG_LHZ_2018 livelihood file https://fewsnet.atlassian.net/browse/DATA-446

Bugs

• Market screen doesn't allow to make a change on a country name https://fewsnet.atlassian.net/browse/DATA-1058

• Fix Resource Class to Return Lat/Long in Market Extracts https://fewsnet.atlassian.net/browse/DATA-1093

• Error in "other" domain of FDE https://fewsnet.atlassian.net/browse/DATA-1095

• FDE filter not working for labor "products" https://fewsnet.atlassian.net/browse/DATA-1098

• Cross Border Trade (XBT) Transit recognition error https://fewsnet.atlassian.net/browse/DATA-1100

Features

Validate Geographic Unit for Calculated Price Index https://fewsnet.atlassian.net/browse/DATA-748

• This enhancement is about adding a validation error message , i.e The {icm.name} Index Component is for {icm.geographic_unit} Geographic Unit which cannot be aggregated to the {dsr.geographic_unit} Geographic Unit for this Calculated Index , for when  an Index Component is added to a Calculated Price Index or if the Geographic Unit for a Calculated Price Index is changed.

DocumentTypeViewSet should only return document types with visible data https://fewsnet.atlassian.net/browse/DATA-986

• This enhancement is about showing the document types of the OTHER data domain ( https://fdw.fews.net/data-explorer/#/other ) which contains data for the external users.

Add an option to include Narrow Ranges in FIPE data domain extracts https://fewsnet.atlassian.net/browse/DATA-1033

• The /api/ipcpopulationsize/ API end point has been enhanced to allow use of the range parameter which as a result accepts the point, narrow and public values; for instance:

https://fdw.fews.net/api/ipcpopulationsize/?format=html&country_code=NG&range=point

https://fdw.fews.net/api/ipcpopulationsize/?format=html&country_code=NG&range=narrow and

https://fdw.fews.net/api/ipcpopulationsize/?format=html&country_code=NG&range=public

Review Livelihood zones code column to use FNID instead of the LZ_CODE in uploads https://fewsnet.atlassian.net/browse/DATA-1035

• This feature is about enhancing the livelihood zones code column to use the FNID; i.e https://fdw.fews.net/en/admin/spatial/livelihoodzone/ .

Add admin 1 and admin 2 to the price time series extracts for both the price and CPI columns https://fewsnet.atlassian.net/browse/DATA-1047

• This enhancement is about adding admin 1 and admin 2 fields to the price timeseries extract, (i.e http://fdw.fews.net/api/marketpricefacts/?format=csv&timeseries=true&country_code=CM ) and the metadata fields are revised based on the suggestions of the EWT i.e base_period_start - base_period_end is shown as a concatenated field and in addition to this, the period dates are shown in the newly added column.

Include Admin1 and admin2 in Market extracts in both the api end point and the admin extracts https://fewsnet.atlassian.net/browse/DATA-1057

• This feature is about allowing the inclusion of Admin1 and Admin2 fields in the market API end point ( https://fdw.fews.net/api/market/ ) and Admin ( https://fdw.fews.net/en/admin/spatial/market/export/ ?) extracts.

Include datasource document in ExchangeRateResource and review all other dataseries resource classes to ensure they have source document in the import_fields list https://fewsnet.atlassian.net/browse/DATA-1059

• This feature is about introducing datasourcedocument in all the import_fields of the data series resource classes, for instance: https://fdw.fews.net/en/admin/price/exchangerate/import/ .

Updated website link for Staple Food Price data download https://fewsnet.atlassian.net/browse/DATA-1063

• This feature is about updating the website link by introducing additional parameters such as end_date=second_last_month or end_date=second_last_month, for instance: https://fdw.fews.net/api/marketpricefacts/?format=xlsx&country_code=HN&fields=website&end_date=last_month or

  https://fdw.fews.net/api/marketpricefacts/?format=xlsx&country_code=HN&fields=website& end_date=second_last_month .

Linear interpolation formulas for data points https://fewsnet.atlassian.net/browse/DATA-1065

• This enhancement is about implementing a linear interpolation calculation formula for a maximum of four consecutive missing data point values of a data series.

Pull in Data from Colombia Price API https://fewsnet.atlassian.net/browse/DATA-1068

• This feature is about introducing a pipeline that imports daily data collections from the Colombia DANE API.

Market Price pipeline from KoboToolbox to FDW https://fewsnet.atlassian.net/browse/DATA-1074

• This feature is about establishing the ingestion of the price data using Kobo into FDW. The weekly data collected in kobo ( forms.fews.net ) and its status approved is automatically imported into FDW.

Tasks

Unable to load NG_LHZ_2018 livelihood file https://fewsnet.atlassian.net/browse/DATA-446

• This task was about loading the NG_LHZ_2018 livelihood file and it was successfully loaded (i.e. https://fdw.fews.net/en/admin/spatial/geographicunitsetversion/3475/change/?_changelist_filters=geographicunitset__country__iso3166a2__exact%3DNG ).

Bugs

Market screen doesn't allow to make a change on a country name https://fewsnet.atlassian.net/browse/DATA-1058

• There was an issue in updating the country name of the market screen ( https://fdw.fews.net/en/admin/spatial/market/ ) and this was resolved.

Fix Resource Class to Return Lat/Long in Market Extracts https://fewsnet.atlassian.net/browse/DATA-1093

• The market extract was returning an empty columns for the metadata fields of Latitude and Longitude and this was fixed and users can now access the values through the extract, (i.e https://fdw.fews.net/en/admin/spatial/market/ ).

Error in "other" domain of FDE https://fewsnet.atlassian.net/browse/DATA-1095

• Trying to select data series from the OTHER data domain of the FDE resulted in a blank page and this was resolved and the functionality is working as expected.

FDE filter not working for labor "products" https://fewsnet.atlassian.net/browse/DATA-1098

• There was a problem with filtering specific products of Labor in the price data domain of the FDE, (i.e https://fdw.fews.net/data-explorer/#/price ) and this was resolved.

Cross Border Trade (XBT) Transit recognition error https://fewsnet.atlassian.net/browse/DATA-1100

• There was a problem in recognizing the Transit flow_type in the cross border trade data upload page and this was resolved and the metadata successfully loaded.