Release Notes for Data Sprint 21.11

Dated: 17 December, 2021

Introduction

The following list of new features, tasks, and resolved bugs are included in the latest Sprint 21.11.2 release to FDW/FDE:

Features

• Add option to FDE to produce URL that includes GeoUnit feature attributes https://fewsnet.atlassian.net/browse/DATA-1073

• Additional Filters for the Other Domain - Exchange Rate https://fewsnet.atlassian.net/browse/DATA-1171

• Prototype TopoJSON data extraction support https://fewsnet.atlassian.net/browse/DATA-1188

• Data ingestion pipeline for areal estimates of specific remote sensing data sets https://fewsnet.atlassian.net/browse/DATA-1205

• Validate existing Somalia Crop Data https://fewsnet.atlassian.net/browse/DATA-1226

• Update Data inventory dashboard to include a breakdown of the new data points added each month by domain https://fewsnet.atlassian.net/browse/DATA-1232

• Calculated price index & Price Ratio end points of the FDE should point to their specific end points https://fewsnet.atlassian.net/browse/DATA-1247

• Add data usage policy to data point extracts from all domains https://fewsnet.atlassian.net/browse/DATA-1250

Tasks

• Investigate the automated API documentation https://fewsnet.atlassian.net/browse/DATA-1111

• Uploading of Ethiopia crop production data https://fewsnet.atlassian.net/browse/DATA-1202

Bugs

• Investigate and fix price projection spreadsheet linkage to FDW API https://fewsnet.atlassian.net/browse/DATA-891

• Fix exchange rate endpoint in FDE https://fewsnet.atlassian.net/browse/DATA-1224

• Fix Country api endpoint XML format bug https://fewsnet.atlassian.net/browse/DATA-1228

• Unable to see exchange data in chart view on FDE https://fewsnet.atlassian.net/browse/DATA-1240

• Error in Average Price Change Map visualization https://fewsnet.atlassian.net/browse/DATA-1242

• Vector tiles caching is not working properly in production https://fewsnet.atlassian.net/browse/DATA-1249

Features

Add option to FDE to produce URL that includes GeoUnit feature attributes https://fewsnet.atlassian.net/browse/DATA-1073

• This enhancement is about making changes to the spatial data domain of the FDE by introducing the field: include feature attributes to the Options tab which displays all the GeoUnit feature attributes.

Additional Filters for the Other Domain - Exchange Rate https://fewsnet.atlassian.net/browse/DATA-1171

• This feature is about incorporating the Exchange rate type and Market filters to the Other data domain of the exchange rate subtype.

Prototype TopoJSON data extraction support https://fewsnet.atlassian.net/browse/DATA-1188

• This enhancement is about incorporating the format=topojson to the data point APIs and spatial data domain, for instance: https://fdw.fews.net/api/ipcphase/?format=topojson&dataseries=6530952 . With the ability to export as TropoJSON, these data can be brought into Power-BI for analysis.

Data ingestion pipeline for areal estimates of specific remote sensing data sets https://fewsnet.atlassian.net/browse/DATA-1205

• This feature is about developing a data ingestion pipeline for areal estimates of specific remote sensing data sets. This will help users to understand how time series of remote sensing data affect specific geographic units of interest.

Validate existing Somalia Crop Data https://fewsnet.atlassian.net/browse/DATA-1226

• This enhancement is about making changes to the existing fewsnet_crop_ingestion pipeline for authenticating the Somalia crop production data.

Update Data inventory dashboard to include a breakdown of the new data points added each month by domain https://fewsnet.atlassian.net/browse/DATA-1232

• This feature is about adding a new chart type to the data inventory dashboard ( https://fdw.fews.net/dashboard/inventory/ ) which shows the data points added by month and data domain type.

Calculated Price Index & Price Ratio end points of the FDE should point to their specific end points https://fewsnet.atlassian.net/browse/DATA-1247

• This enhancement is about changing the FDE API endpoints of the calculated price index and price ratio  from /api/datapointfacts to /api/calculatedpriceindexvaluefacts/ and /api/priceratiovaluefacts/ respectively, so that users are brought to the correct views as expected.

Add data usage policy to data point extracts from all domains https://fewsnet.atlassian.net/browse/DATA-1250

• This enhancement is about adding the data_usage_policy field to the exchange rate ( https://fdw.fews.net/api/exchangeratevaluefacts/?format=csv&subtype=ExchangeRateValue&fields=simple ) and secondary price index ( https://fdw.fews.net/api/priceindexvaluefacts/?format=csv&subtype=PriceIndexValue&fields=simple )  to data points extracts of the FDE .

Tasks

Investigate the automated API documentation https://fewsnet.atlassian.net/browse/DATA-1111

• This task was about automating the API documentation ( https://fdw.fews.net/docs/rapidoc/ ) which runs actual requests against the API.

Uploading of Ethiopia crop production data https://fewsnet.atlassian.net/browse/DATA-1202

• This task was about uploading the Ethiopia crop production historic data and it was successfully loaded into the FDW.

Bugs

Investigate and fix price projection spreadsheet linkage to FDW API https://fewsnet.atlassian.net/browse/DATA-891

• The system was throwing an error during the API connection between the FDW and the excel TPPT spreadsheet and this was resolved.

Fix exchange rate endpoint in FDE https://fewsnet.atlassian.net/browse/DATA-1224

• There was an issue in accessing the API:/api/exchangeratevaluefacts, and this was resolved and the functionality is working as expected.

Fix Country api endpoint XML format bug https://fewsnet.atlassian.net/browse/DATA-1228

• There was an issue with the country API (for instance, https://fdw.fews.net/api/country/DK.xml ) where the XML only shows the attribute names but not the actual values and this was resolved.

Unable to see exchange data in chart view on FDE https://fewsnet.atlassian.net/browse/DATA-1240

• There was an issue in rendering charts for exchange rate data of the FDE ( https://fdw.fews.net/data-explorer/#/other ) and this was resolved and the functionality is working as expected.

Error in Average Price Change Map visualization https://fewsnet.atlassian.net/browse/DATA-1242

• There was an error in average price change map visualization and this was resolved.

Vector tiles caching is not working properly in production https://fewsnet.atlassian.net/browse/DATA-1249

• There was an issue in caching the vector tiles which results in performance issues of endpoints and this was resolved.