Glossary and Policies
Glossary and Policies
Breadcrumbs

Release Notes for Data Sprint 20.07

Introduction

Date: 11 August 2020

The following list of new features, tasks and resolved bugs are included in the Data Sprint 20.07 release which is running version 20.07.8:

Features

Tasks

Bugs

The aliases are dropped off at the creation of the product codes - https://fewsnet.atlassian.net/browse/DATA-73

Crop data extract missing data area population and per capita - https://fewsnet.atlassian.net/browse/DATA-344

https://fdw.fews.net/en/admin/spatial/geographicunit/export/ ? results in 500 Internal server error - https://fewsnet.atlassian.net/browse/DATA-345

Price dataset extracts extracts produce wrong CPI values - https://fewsnet.atlassian.net/browse/DATA-424

Geography filter not displaying - https://fewsnet.atlassian.net/browse/DATA-435

Landscan Population failure with pre-2000 years - https://fewsnet.atlassian.net/browse/DATA-442

Update warehouse/templates/warehouse/datapointchart.html to use /api/datapoint instead of /api/datapointfacts - https://fewsnet.atlassian.net/browse/DATA-455

Features

Ingestion pipeline for Crop Production data. https://fewsnet.atlassian.net/browse/DATA-212

  • This effort produced a prototype for implementing an ingestion pipeline for Crop Production data. With more than 4 million rows of data, this domain was a clear candidate to demonstrate the flexibility of FDW’s ingestion pipeline capabilities.

Price Charts Dashboard Enhanced Version. https://fewsnet.atlassian.net/browse/DATA-273

  • This feature is about implementing an enhanced Price Charts dashboard to include the Price Watch and the Price Bulletin, and can be accessed at: https://fdw.fews.net/dashboard/price/ .

DATA-273.png

Add a "Best" option to the Scenario Filter for IPCPhaseViewSet. https://fewsnet.atlassian.net/browse/DATA-295

  • This feature is about adding the “Best Case options” scenario types i.e (Best Case in 3 months & Best Case in 9 months) to the Scenario filter in FDW.

Incorporate M&T Formatting Guidance into FDE Market Price and Cross-Border Trade Charts. https://fewsnet.atlassian.net/browse/DATA-306

  • This enhancement is about implementing the Markets and Trade formatting standards in the Market Price and Cross-Border Trade data domains of the FDE charts.

Enhance the Crop Production Dashboard to show Data Points by Product. https://fewsnet.atlassian.net/browse/DATA-324

  • This feature is about enhancing the Crop Production Dashboard ( https://fdw.fews.net/dashboard/crop-production/ ) to show the crop data by product , Indicator, Admin level for all the countries of all the years. Each of these points are filterable as well.

DATA-324.png

Population Estimates for ad hoc shapefiles. https://fewsnet.atlassian.net/browse/DATA-343

  • This feature is about generating population estimates per density area from the ad hoc shapefiles uploaded into the system.

DATA-343.png

Add Totals to the Cumulative Number of Data Points by Month. https://fewsnet.atlassian.net/browse/DATA-384

  • This enhancement is about adding total number of data points to the Data Inventory Dashboard. This is now shown in Cumulative Number of Data Points section to the right of the graphic, and broken down by month.

DATA-384.png

Add as_of_date filter to DataPointViewSet. https://fewsnet.atlassian.net/browse/DATA-393

Better Support to ADS-579 Compliance within FDW. https://fewsnet.atlassian.net/browse/DATA-402

Introduce duplicate period date within the same Data Source Document for forecast domains. https://fewsnet.atlassian.net/browse/DATA-413

  • This feature is about allowing a duplicate period date within the same Data Source Document for the forecast data within the IPC data domains. This was previously prevented by the system, but is now allowable.

Include data domain and dataseries count in DataSourceDocumentSerializer. https://fewsnet.atlassian.net/browse/DATA-438

Apply pagination / Infinite Scroll to Semistructured data in Other domain. https://fewsnet.atlassian.net/browse/DATA-451

  • This feature is about introducing the infinite scroll or pagination functionality for the Other Data Domain.

DATA-451.png

Tasks

Complete Bulk Upload from the S3 Bucket Livelihood Zones into FDW. https://fewsnet.atlassian.net/browse/DATA-168

  • This task was about giving support on uploading the Livelihood Zones data and the data was successfully loaded. However, the remaining data to upload is being addressed in singular tickets by country.

In the map view while selecting a country, add a zoom-in feature so it takes you directly to that country, and the other countries around it are mostly hidden. https://fewsnet.atlassian.net/browse/DATA-426

Layers need to be sorted alphabetically when viewing the Spatial domain. https://fewsnet.atlassian.net/browse/DATA-427

  • This task was about sorting the Admin layers alphabetically on the map view of the Spatial data domain.

DATA-427.png

Add export format for .csv on the Export tab in the FDE Spatial view. https://fewsnet.atlassian.net/browse/DATA-428

  • This task was about adding the .csv format to the Export tab of the FDE Spatial Data Domain.

DATA-428.png

Add a preview url to the the documents_list in mayan. https://fewsnet.atlassian.net/browse/DATA-434

Load remittances data into production. https://fewsnet.atlassian.net/browse/DATA-440

  • This task was about loading the remittance data into FDW via the data ingestion pipelines. It was successfully loaded and can also be extracted from the FDE end point once it is published.

Bugs

The aliases are dropped off at the creation of the product codes. https://fewsnet.atlassian.net/browse/DATA-73

  • The aliases were dropped off at the creation of the CPC Product codes and now the functionality is working as expected.

Crop data extract missing data area population and per capita. https://fewsnet.atlassian.net/browse/DATA-344

  • On the crop data extract, the column of information for population and value_per_capita were shown empty and this was resolved.

https://fdw.fews.net/en/admin/spatial/geographicunit/export/? results in 500 Internal server error. https://fewsnet.atlassian.net/browse/DATA-345

Price dataset extracts extracts produce wrong CPI values. https://fewsnet.atlassian.net/browse/DATA-424

  • There were incorrect CPI values that were shown in Price datasets extracts and this was resolved.

Geography filter not displaying. https://fewsnet.atlassian.net/browse/DATA-435

  • There was an issue in loading the data values of the geographic filter of the crop data domain and this was resolved.

Landscan Population failure with pre-2000 years. https://fewsnet.atlassian.net/browse/DATA-442

  • There was a problem in loading the crop extracts for the years prior to 2000 and this was resolved.

Update warehouse/templates/warehouse/datapointchart.html to use /api/datapoint instead of /api/datapointfacts. https://fewsnet.atlassian.net/browse/DATA-455

  • There was an issue in viewing charts on the single data series page of FDW and this was resolved. The functionality is now working as expected.