AgML - Machine Learning for Agricultural Modeling

AgML is the AgMIP transdisciplinary community of agricultural and machine learning modelers.

AgML aspires to

• identify key research gaps and opportunities at the intersection of agricultural modelling and machine learning
  research,
• support enhanced collaboration and engagement between experts in these disciplines, and
• conduct and publish protocol-based studies to establish best practices for robust machine learning use in agricultural
  modelling.

AgML Crop Yield Forecasting

The objective of AgML Crop Yield Forecasting task is to create a benchmark to compare models for crop yield forecasting
across countries and crops. The models and forecasts can be used for food security planning or famine early warning. The
benchmark is called CY-Bench (crop yield benchmark).

Table of contents

• Overview
• Getting started
• Dataset
• Leaderboard
• How to cite
• How to contribute

Overview

Early in-season predictions of crop yields can inform decisions at multiple levels of the food value chain from
late-season agricultural management such as fertilization, harvest, and storage to import or export of produce.
Anticipating crop yields is also important to ensure market transparency at the global level (
e.g. Agriculture Market Information System, GEOGLAM Crop Monitor)
and to plan response actions in food insecure countries at risk of food production shortfalls.

We propose CY-Bench, a dataset and benchmark for subnational crop yield forecasting, with coverage of major crop
growing countries and underrepresented countries of the world for maize and wheat. By subnational, we mean the
administrative level where yield statistics are published. When statistics are available for multiple levels, we
pick the highest resolution. By yield, we mean end-of-season yield statistics as published by national statistics
offices or similar entities representing a group of countries. By forecasting, we mean prediction is made ahead of
harvest. The task is also called in-season crop yield forecasting. In-season forecasting is done at a number of
time points during the growing season from start of season (SOS) to end of season (EOS) or harvest. The first
forecast is made at middle-of-season (EOS - SOS)/2. Other options are quarter-of-season (EOS - SOS)/4
and n-day(s) before harvest. The exact time point or time step when forecast is made depends on the crop calendar
for the selected crop and country (or region). All time series inputs are truncated up to the forecast or
inference time point, i.e. data from the remaining part of the season is not used. Since yield statistics may not
be available for the current season, we evaluate models using predictors and yield statistics for all available
years. The models and forecasts can be used for food security planning or famine early warning. We compare models,
algorithms and architectures by keeping other parts of the workflow as similar as possible. For example: the
dataset includes same source for each type of predictor (e.g. weather variables, soil moisture, evapotranspiration,
remote sensing biomass indicators, soil properties), and selected data are preprocessed using the same pipeline
(use the crop mask, crop calendar; use the same boundary files and approach for spatial aggregation) and (for
algorithms that require feature design) and same feature design protocol.

Coverage for maize

Undifferentiated Maize or Grain Maize where differentiated

Coverage for wheat

Undifferentiated Wheat or Winter Wheat where differentiated

Deciphering crop names

The terms used to reference different varieties or seasons of maize/wheat has been simplified in CY-Bench. The following
table describes the representative crop name as provided in the crop statistics

Getting started

cybench is an open source python library to load CY-Bench dataset and run the CY-Bench tasks.

Installation

git clone https://github.com/BigDataWUR/AgML-CY-Bench

Requirements

The benchmark results were produced in the following test environment:

Operating system: Ubuntu 18.04
CPU: Intel Xeon Gold 6448Y (32 Cores)
memory (RAM): 256GB
disk storage: 2TB
GPU: NVIDIA RTX A6000

Benchmark run time

During the benchmark run with the baseline models, several countries were run in parallel, each in a GPU in a
distributed cluster.
The larger countries took approximately 18 hours to complete.
If run sequentially in a single capable GPU, the whole benchmark should take 50-60 hours to complete.

Software requirements: Python 3.9.4, scikit-learn 1.4.2, PyTorch 2.3.0+cu118.

Downloading dataset

Get the dataset
from Zenodo.

Running the benchmark

First write a model class your_model that extends the BaseModel class. The base model class definition is
inside models.model.

from cybench.models.model import BaseModel
from cybench.runs.run_benchmark import run_benchmark

class MyModel(BaseModel): 
    pass


run_name = <run_name>
dataset_name = "maize_US"
run_benchmark(run_name=run_name, 
              model_name="my_model",
              model_constructor=MyModel,
              model_init_kwargs: <int args>,
              model_fit_kwargs: <fit params>,
              dataset_name=dataset_name)

Dataset

Dataset can be loaded by crop and (optionally by country).

For example

dataset = Dataset.load("maize")

will load data for countries covered by the maize dataset. Maize data for the US can be loaded as follows:

dataset = Dataset.load("maize_US")

Data sources

1: Mali data at admin level 3. Mali data is also included in the FEWSNET Africa dataset, but at admin level 1 only.

2: Germany data is also included in the EU dataset, but there most of the data fails coherence tests (e.g. yield =
production / harvest_area)

Leaderboard

See baseline results

How to cite

Please cite CY-bench as follows:

How to contribute

Thank you for your interest in contributing to AgML Crop Yield Forecasting. Please
check contributing guidelines for how to get involved and contribute.

Additional information

For more information please visit the AgML website.