Release Notes for Data Sprint 21.03

Introduction

The following list of new features, tasks, and resolved bugs are included in the latest Sprint 21.03.1 release to FDW/FDE:

Features

• The Spatial domain has been enhanced to include an Options tab that allows users to control display of geographic information, age and sex population estimates, population source and year. An About tab has also been included to display data source information. https://fewsnet.atlassian.net/browse/DATA-430

• Read-Only APIs for Data Series can be used to insert or update user privileges and permit automated metadata management where possible. https://fewsnet.atlassian.net/browse/DATA-740

• The following endpoints have been enhanced: datacollection, datacollectionperiod and datapoint to allow users to filter by those metadata selections. https://fewsnet.atlassian.net/browse/DATA-760

• In continuation of Commodity Balance domain development, next steps include gathering API output requirements, identifying common language definitions, period dates and template resources. https://fewsnet.atlassian.net/browse/DATA-769

• The geographic unit type filter has been updated to enable shapefiles to be filtered by type. https://fewsnet.atlassian.net/browse/DATA-773

• Users will now receive a descriptive error report when they are unable to upload shapefiles. https://fewsnet.atlassian.net/browse/DATA-782

• Upon creating the calculated index, the FDW data source document is automatically populated as FEWS NET datawarehouse and the indicator type is automatically populated as calculated index. https://fewsnet.atlassian.net/browse/DATA-803

• In FDW, the Currency.iso4217a3 code has been expanded to 20 characters to accomodate the transaction type with a currency in countries where it is required. https://fewsnet.atlassian.net/browse/DATA-811

• In FDW, Data Sets have been updated to display the Data Series Count column. https://fewsnet.atlassian.net/browse/DATA-849 Navigation: FDW: Data Analysis > Datasets

• The Consumer Price Index Name field of Consumer Price Index screen has been updated to allow additional characters. https://fewsnet.atlassian.net/browse/DATA-856

Tasks

• Commodity Balances historic data has been uploaded. https://fewsnet.atlassian.net/browse/DATA-820

• Support to Southern Africa Crop Tours are complete. https://fewsnet.atlassian.net/browse/DATA-833

• FSO Internal document upload for IPC Classifications have been uploaded. https://fewsnet.atlassian.net/browse/DATA-836

Bugs

• Performance issues on export of TradeFlowQuantity were investigated and resolved. https://fewsnet.atlassian.net/browse/DATA-710

• Performance issues on export of IPC ClassificationResource were investigated and resolved. https://fewsnet.atlassian.net/browse/DATA-711

• Performance issues for data source document exports were investigated and resolved. https://fewsnet.atlassian.net/browse/DATA-752

• Errors in /api/tradeflowquantityvalue were investigated and resolved. https://fewsnet.atlassian.net/browse/DATA-805

• Country was not set/derived from the selected layer of the market that is going to be created. This issue has been resolved. https://fewsnet.atlassian.net/browse/DATA-812

• MarketPriceFacts extracts issues using currency parameter were investigated and resolved. https://fewsnet.atlassian.net/browse/DATA-839

• Failed MarketPriceFacts time series extract with empty Price Index were resolved. https://fewsnet.atlassian.net/browse/DATA-840

• /ipcmap, /ipcpngmap and /api/ipcpackage are no longer accept standalone parameters https://fewsnet.atlassian.net/browse/DATA-842

• Trade data extract issues for a specific source document i.e (Famine Early Warning Systems Network (FEWS NET) were resolved. https://fewsnet.atlassian.net/browse/DATA-845

• Issues uploading new exchange rate metadata file were investigated and resolved. https://fewsnet.atlassian.net/browse/DATA-846

• /api/ipcpackage were broken because of missing IPC Legend file. These issues were resolved. https://fewsnet.atlassian.net/browse/DATA-851

• Failed timeseries marketpricefacts extracts for Nigeria were resolved https://fewsnet.atlassian.net/browse/DATA-861

Features

Add demographic information, such as age and sex to spatial, and allow users to be able to toggle that information being displayed or not https://fewsnet.atlassian.net/browse/DATA-430

• This enhancement is about making changes to the spatial data domain of the FDE by introducing the Options tab which contains or shows the Geographic information , population estimates , demography estimates , Population Source and Population Year and also the About tab that displays the necessary data source information required for a user to understand the data and create an attribution.

Enhance existing Read-Only APIs for Data Series to allow them to be used for inserting and updating by users with correct privileges, allowing for automated metadata management where possible https://fewsnet.atlassian.net/browse/DATA-740

• This feature is about enhancing the operations of creating and updating the existing dataseries APIs from back end using the tool postman .

Enhancement to datacollection, datacollectionperiod and datapoint endpoints https://fewsnet.atlassian.net/browse/DATA-760

• This feature enhances the filtering capability of the APIs datacollection and datacollectionperiod, for instance,( https://fdw.fews.net/api/exchangeratevalue/?fields=with_previous_value&datacollection=18063 or https://fdw.fews.net/api/marketprice/?fields=with_previous_value&datacollection=81521 ) and in addition to this the parameter with_previous_value has been added to dataPoint API which returns the previous_value and percentage_changes of the datapoint values , for instance , ( https://fdw.fews.net/api/marketprice/?fields=with_previous_value ).

Adding geographic unit type filter https://fewsnet.atlassian.net/browse/DATA-773

• This feature is about introducing geographic unit type filter to the geographic unit set version i.e ( https://fdw.fews.net/en/admin/spatial/geographicunitsetversion/ ).

Provide descriptive error report when users are unable to upload shapefiles https://fewsnet.atlassian.net/browse/DATA-782

• This feature is about making descriptive error messages for the failed shapefiles upload.

Data source document and indicator type should automatically be populated upon creating calculated index https://fewsnet.atlassian.net/browse/DATA-803

• This enhancement is about automatically setting the Datasourcedocument as FEWS NET datawarehouse and the  Indicator as Calculated index when creating the calculated indices i.e ( https://fdw.fews.net/en/admin/price/calculatedindex/add/ ).

Expand Currency.iso4217a3 code to 20 characters https://fewsnet.atlassian.net/browse/DATA-811

• This feature is about lengthening the currency code of  ISO 4217 Alpha-3 to 20 characters. i.e ( https://fdw.fews.net/en/admin/common/currency/add/ ).

Add Data Series Count to Data Set admin screens https://fewsnet.atlassian.net/browse/DATA-849

• This enhancement is about adding the column information of Data Series Count to all data Set admin screens.

Allow more characters in Consumer Price Index Name field of Consumer Price Index screen https://fewsnet.atlassian.net/browse/DATA-856

• This feature is about lengthening the Consumer Price Index Name field of the Consumer Price Index screen ( https://fdw.fews.net/en/admin/price/priceindex/add/ ).

Tasks

Commodity Balances historic data upload https://fewsnet.atlassian.net/browse/DATA-820

• This task was about loading commodity balances historic data into the FDW Development environment i.e ( https://fdwdev.fews.net/en/admin/balance/commoditybalance/ ) which is ready for review by EWT.

Support to Southern Africa Crop Tours https://fewsnet.atlassian.net/browse/DATA-833

• This task was about providing support to the south African crop tour, which involves collecting data about crops in fields using mobile devices, designing the data collection forms in Kobo, rearranging and rephrasing the questions, and extracting the collected data in a format which is easier to do analysis later.

Enable FSO Internal document upload for IPC Classifications https://fewsnet.atlassian.net/browse/DATA-836

• This task was about enabling the uploading of the FSO Internal documents for the IPC Classification data domain .

Bugs

Investigate performance issues on export of TradeFlowQuantity and fix https://fewsnet.atlassian.net/browse/DATA-710

• There was a performance issue in exporting trade flow quantity data i.e

  ( https://fdw.fews.net/en/admin/trade/tradeflowquantity/export/ ?) and this was resolved.

Investigate performance issues on export of IPCClassificationResource and fix https://fewsnet.atlassian.net/browse/DATA-711

• There was a performance issue in exporting the IPC Classification data  i.e

  ( https://fdw.fews.net/en/admin/ipc/ipcclassification/export/ ?)  into different formats and this was resolved.

Investigate performance for data source document export https://fewsnet.atlassian.net/browse/DATA-752

• There was a performance issue in exporting the source document data ( https://fdw.fews.net/en/admin/warehouse/datasourcedocument/export/ ?) and this was resolved.

Error in /api/tradeflowquantityvalue https://fewsnet.atlassian.net/browse/DATA-805

• The system was throwing an error upon accessing /api/tradeflowquantityvalue and this was resolved.

Country is not set/derived from the selected layer of the market that is going to be created https://fewsnet.atlassian.net/browse/DATA-812

• There was no drop down list of the Country field in the market screen i.e ( https://fdw.fews.net/en/admin/spatial/market/add/ ) and in addition to this , the Layer field was not showing the country of the created market and this was resolved.

MarketPriceFacts extracts using currency parameter are not working https://fewsnet.atlassian.net/browse/DATA-839

• The system was throwing an error when trying to extract a marketpricefacts using the currency parameter and this was resolved.

Failed MarketPriceFacts timeseries extract with empty Price Index https://fewsnet.atlassian.net/browse/DATA-840

• There was a problem when trying to extract a marketpricefacts with an empty price index and this was resolved.

/ipcmap, /ipcpngmap and /api/ipcpackage no longer accept standalone parameters https://fewsnet.atlassian.net/browse/DATA-842

• There was an issue in accessing the query parameters of the APIs i.e

  (ipcmap, ipcpngmap and api/ipcpackage) and this was resolved and the functionality is working as expected.

Trade data extract for a specific source document i.e (Famine Early Warning Systems Network (FEWS NET) is not working https://fewsnet.atlassian.net/browse/DATA-845

• There was an issue in extracting a specific source document data i.e( https://fdw.fews.net/api/tradeflowquantityvalue/format=csv&datasourcedocument=6824&fields=simple ) and this was resolved.

Issue of uploading new exchange rate metadata file https://fewsnet.atlassian.net/browse/DATA-846

• There was a problem in uploading exchange rate metadata for the country Nigeria and this was resolved.  

/api/ipcpackageis broken because of missing IPC Legend file https://fewsnet.atlassian.net/browse/DATA-851

• There was an issue in using the API i.e (/api/ipcpackage) and this was resolved and the functionality is working as expected.

Failing timeseries marketpricefacts extracts for Nigeria https://fewsnet.atlassian.net/browse/DATA-861

• There was a problem in extracting the timeseries marketpricefacts  for the country Nigeria  and this was resolved.