A a large subset of the WWRP/WCRP S2S Project Database available online in the IRI/LDEO Climate Data Library: http://iridl.ldeo.columbia.edu/SOURCES/.ECMWF/.S2S/ We hope this will provide a valuable addition to the two official archiving centers at ECMWF & CMA, and we plan in future to make various derived products available there too.

In addition to the S2S project data, the SubX project data are archived in IRIDL here: http://iridl.ldeo.columbia.edu/SOURCES/.Models/.SubX/

Please see the README file at the top of the page for notes on accessing & manipulating the data. We are still ironing out some of the kinks, so please let us know if you encounter any problems. Most of the data should be there, though there may be delays in updating the data.

Data holdings status: (21 Dec 2018) 72TB Detailed breakdown
Data holdings status: (30 Mar 2021) 155TB Detailed breakdown
Data access stats: https://dlstats.iri.columbia.edu/awstats/awstats.pl?config=ECMWF-S2S

This page contains Ingrid scripts as additional resources for accessing and manipulating the data.

Climatologies

IRI has now developed a whole set of daily climatologies with different methods for all S2S models, available here:

Daily climatologies

The models used are those available in the Subseasonal-to-Seasonal Prediction Project Database. Total precipitation and 2-meter temperature fields were accessed via the International Research Institute for Climate and Society (IRI) Data Library. Observational climatologies were computed using CHIRPS and the CPC temperature dataset (https://psl.noaa.gov/data/gridded/data.cpc.globaltemp.html). Climatologies were produced considering the common period Jan 1st-Dec 31st 1999-2010. The time coordinate of the produced climatological fields contain a total of 366 days, to be able to consider leap years. Three different methods have been implemented.

The na�ve method involves computing daily averages using exactly the same calendar day in all years in the climatological period. Hence, for example, the climatological day Feb 14th is computed as the average of all Feb 14th in the 1999-2010 period. This approach, although simple, has some drawbacks, as has been discussed in detail by Tippet et al (2018).

The triangular method uses a moving average with a 30-day triangular kernel (weighting function), centered on the target day. For example, the climatological day March 15th has contributions from the previous and following 15 days, following a triangular weighting with the maximum weight located on March 15th. This method is the one used by the Subseasonal Experiment (SubX) project, and is described by Pegion et al. (2019).

The harmonic method fits a Fourier series to the na�ve climatology to produce the smoothed daily series. After testing with multiple truncations to identify the configuration that best capture the characteristics of local seasonal cycles, the final climatology uses N=16 harmonics for total precipitation, and N=6 harmonics for 2-meter temperature, where N is the number of harmonics. The truncation is determined by finding the value of N for each variable that minimizes the spatially-averaged Root Mean Square Error (RMSE) between the harmonic and the triangular methods. A RMSE comparison considering a daily climatology produced via a simple 5-day running average as reference was also conducted. An example of the comparison between the approaches is presented in the figure below (for only three models, but climatologies have been computed for all models).

Example from before producing the daily climatologies at the IRI:

ECMWF weekly ensemble avg precip THURSDAY starts

Script access from unix command line, Matlab, R and Python

An access key is required in order to download S2S data from the command line. To obtain one, please:

Firstly sign the S2S terms and conditions by downloading any test data via the "Data Files" tab on the data library page, e.g. dataURL.
Secondly, send an email to help@iri.columbia.edu, stating that you are requesting an access key for S2S data, and including the email address that you used to sign the terms and conditions. You will be sent an access key.

Follow this example using curl to download the data:

curl -k -b '__dlauth_id=xxxyyyyzzz' 'dataURL' > file.nc

where xxxyyyyzzz is your key, and file.nc is your preferred name of the output NETCDF file.

The system command can be used to execute curl from MATLAB and R. [Important: use one � at the beginning and at the end and two � (i.e., " �no space between them) in the middle cases.].

Example:

MATLAB:

system('curl -k -b "__dlauth_id=xxxyyyzzz" "dataURL" >hgt_ecmwf_ctrl.nc')

R: (here it's better to just define a string with the entire curl command, and then make the system call)

pepe<-'curl -k -b "__dlauth_id=xxxyyyzzz" "dataURL" >hgt_ecmwf_ctrl.nc'

system(pepe)

Python:

For Jupyter Notebooks, you can do:

!curl -k -b "__dlauth_id=xxxyyyzzz" "dataURL" >hgt_ecmwf_ctrl.nc

If you are using "pure" Python, you can use something like (thanks for Dr. Thea Turkington -- NEA, Singapore) :

#----------------------------------

from subprocess import call

call('curl �k ' + 'whatever the url is' + ' -o ' + 'output file name' , shell = True)

#------------------------

Recipes for constructing weekly averages of model forecast precipitation

The lead grid (L1) for the accumulated precipitation variable varies from model to model based upon the leads that each model makes available. L1 starts at 0. for ECMWF, HMCR, and ISAC. For the other models the 0. lead is not available -- the first lead is 1.0 (or 0.5 for JMA -- a 12-hour forecast).

ECMWF weekly forecast ensemble avg precip THURSDAY starts
Advanced example (not for the faint of heart!) for NCEP model, in which we insert an L1 grid at time 0. to make it like the ECMWF model. Weekly precipitation (perturbed runs average)
Example plot of ECMWF week-1 precip anomaly forecast, mean of perturbed members

Observed precipitation anomalies for comparing against

Recipes for constructing derived variables

10hPa Zonal Wind Anomaly Index at 60N

[[https://tinyurl.com/y5mbe876 |From reanalysis, example for Oct 2008 - Mar 2009.

Vertically integrated moisture fluxes

Zonal component for an example forecast

Recipes for using S2S Data with CPT

Example for the first week of Dec, using forecasts from Nov 14 (1996-2015)

Forecast anomaly maps - ensemble & lead averaged

After IRI developed daily climatologies for all S2S models, the anomaly calculation in Ingrid can be (hugely) simplified. An example follows:

ECMWF forecast anomalies

Spatial domain, start, and lead range can be selected.

Obs estimates for verification:

MJO RMM indices

The RMM indices from the models are available for each model through IRIDL (computed by Frederic Vitart, with work by Steve Woolnough & Tetsuo Nakazawa on the validation and format definition).

MJO composites of circulation variables using reanalysis

This link shows composites of the stream function (psi) using MJO phases. It is possible to quickly change the season of interest in the upper boxes.

Other circulation variables in the reanalysis could be used, changing the name of the field in the "var" box on the top of the canvas.

Finally, the wind field can also be added, as it is shown in this link.

Including lead times in MJO composites of circulation variables (reanalysis) and precipitation

MJO composites using an S2S model (CFSv2)

This takes a while to compute (go prepare a coffee):

CFSv2 rainfall composite

The season of interest can be selected using the boxes on the top of the canvas.

Resources for using the WWRP/WCRP S2S Project Database from IRI Data Library