Prices

This page discusses the processes for preparing and uploading prices data to the FEWS NET Data Warehouse (FDW). It covers the steps for adding historical data to FDW for a new data source, as well as the routine processes for uploading monthly or weekly data collections.

Historical data

Before FDW can ingest a new prices data source, you must do the following: 

• Identify new metadata, such as new markets or products, and add it to FDW.

• Prepare a file containing the historical data that has been cleaned and organized using an upload template built for price recognition. 

  • The historical data file will be used as the basis for a new upload file for the field to begin adding their prices monthly or weekly.

Add new metadata

1. Determine if any new metadata needs to be added to the FDW using the metadata refreshable workbook.  

2. Add new metadata to FDW. For example, markets can be added through the markets admin screen and products through the market products admin screen.

Upload historical data 

1. Prepare the Microsoft Excel historical data upload file. An example of fields required for uploading historical data are as follows:

   1. country

   2. market

   3. product

   4. product_name

   5. price_type

   6. product_source

   7. unit

   8. currency

   9. period_date

   10. price

   11. data_source_document

   12. data_source_organization

2. When entering a date in the upload spreadsheet, it is important that the date column is set using the custom format YYYY-MM-DD with no leading or trailing spaces. 

3. Enter the prices in the Price column of the Excel upload file.

   1. Any cell without a price should be entered as NC or NA. Do not add zeroes or #N/A.

      1. Use NC to indicate the product was available, but the price was not collected. 

      2. Use NA to indicate the product was not at the market. 

   2. Always change “Missing historical data” to NC or NA.

   3. If FEWS NET enumerators are using a mobile app in their regularly scheduled upload, all empty fields are marked NA in historical uploads for those countries.

4. After preparing the historical data file, upload by way of the import screen in the FDW: https://fdw.fews.net/en/admin/price/marketprice/import

Monthly and weekly data collection

The field’s weekly/monthly scheduled data will be uploaded using the file that is prepared using the set of market products and metadata from the historical file.

The upload files for each country vary in format. When there are variables in the metadata, it is important to include the field in the upload file (e.g., the data’s product sources include local and import).

Example: Upload file headers for WFP, Guinea-Bissau include the following fields:

If you prefer, you may include all fields in an upload file as in the following example:

Prepare the data file

If you have information on multiple tabs, the data to be uploaded must be in the leftmost tab.

The prices reported are always the month before the publication month.

Period date

A period date for a monthly data collection is always the last day of a month; not the date you enter the prices.

The date must be in the custom format YYYY-MM-DD with no leading or trailing spaces.

The period date should always be included as a column when uploading in bulk (e.g., FEWS NET (weekly), Nigeria).

In some cases, the date column is included in a monthly or weekly upload file. Otherwise, the period date should be recorded on the spreadsheet tab and in the file name instead.

Source Document

When selecting the Source Document, be sure to select the price data source and not the production source.

Product codes

You may use the product code instead of the product name in the Product column (e.g., replace Maize (Corn), unspecified with R01122AA).

If product codes are not used in an upload file, the commodity column header is “product.” If the product code is used in the upload file, the product code field is named “product,” and the product name header is “product_name.”

Spelling of product and market names

The spelling of the product and market names should not be changed from how they appear in the FDW. They match the names in the FDW, with the following exceptions: 

• The product_source International Import should be entered as Import in the upload file. 

• If the unit for livestock in the FDW is head, enter it into the upload file as ea.

Entries without prices

In any cell without a price, enter NC or NA:

• NC indicates the product was available, but the price was not collected.

• NA indicates the product was not on the market.

Do not add zeroes or #N/A. We strongly recommend that a price with a value of “0” be updated to NC or NA. A zero in the original file should be taken to indicate that the product was either not available or was not collected, versus having a price of “0” and being sold for free.

Upload to FDW

The Data Entry screen can be accessed through either of the following options: 

• From the main toolbar, select Data Entry > Add a new Data Collection (manually or by uploading a file).

• From the main toolbar, select Data Processing > Display or edit an existing Data Collection (including Submit, Review and Publish). From the Data Collections screen, click the Add Data Collection button. 

  • The benefit of this option is that you can see Data Collections already added to the FDW. This will indicate which collection date needs to be added next or whether there are any unpublished Data Collections you need to investigate. 

These instructions will proceed using the Add a new Data Collection (manually or by uploading a file) option.

1. Navigate to the Add data collection page by selecting Data Entry > Add a new Data Collection (manually or by uploading a file).

2. Fill in your data collection page:

   1. Select the Data Source Document by searching in the text box. You can enter part of the source name in the text box and a list will appear from which you can select your source name (e.g., WFP, Guinea-Bissau).

   2. The Collection Date should populate automatically to show the next monthly collection date. If the date does not appear, enter the date manually or use the calendar icon. 

   3. Click the Choose File button and select the Excel spreadsheet.

   4. At the bottom of the screen, click the Save button.

1. Check for errors: If you find any prices missing in your price collection that still have a status of Collected or Missing Historic Data, change them to Not Collected or Not Available as necessary.   

   If you have prices in your data file that are not loading into the markets in your Data Collection, confirm the spellings and market names. Remove any trailing or leading spaces in the metadata names. Do not select Missing Historic Data.

2. Click the Save button at the bottom right-hand corner of your browser. 

1. Comments: Write any necessary comment about the data in the Comments box at the bottom of the Change Data Collection page and click Send. As the FDW will not send a notification, you should send an email to the person you want to share this information with, notifying them that there is a comment in the Data Collection. They should reply within the Data Collection comment box so that others (e.g., the Home Office) will see the reply.

2. For reviewers: If you are the person to review and submit the Data Collection, you can access Data Collections by navigating to Data Processing > Display or edit an existing Data Collection (including Submit, Review and Publish).

   1. Use the search box, sortable columns, and filters to find the Data Collection you want to review. See Data Processing for additional guidance.

   2. Click on a Data Collection row to open the Change Data Collection screen.

   3. Scroll down to the Collection Period section and select Edit to open the Change collection period screen.

   4. Check that all null values are identified in the Status column as Not collected or Not available.

   5. Check the Change column percentages from the month before for any large or small increases you may need to question.

   6. Return to the Change data collection screen (use the back button in your browser or click the Collection name).

   7. If you have questions and the Data Collection was not added by you, enter your question in the Comments box. Then, send an email to the person to whom the comment was intended to let them know the comment is there.

   8. If all data are reviewed and ready to move to the next approval level, select Submit.

3. When you have reviewed each Data Collection and are satisfied with your data, click the Source Document link, e.g., WFP, Guinea-Bissau.

   1. Enter a comment in the comments box to let the Home Office know that the Data Collection is ready for their review for publishing, then select Send.

   2. Click the Submit button at the top of your screen.

4. If you are at the regional level, have reviewed the data and are satisfied with the final results, please enter “Okay” or “Ready for Home Office” in the Comments box.