Tutorial 1: Creating DataArrays and Datasets to Assess Global Climate Data#

Week 1, Day 1, Climate System Overview

Content creators: Sloane Garelick, Julia Kent

Content reviewers: Yosmely Bermúdez, Katrina Dobson, Younkap Nina Duplex, Danika Gupta, Maria Gonzalez, Will Gregory, Nahid Hasan, Sherry Mi, Beatriz Cosenza Muralles, Jenna Pearson, Chi Zhang, Ohad Zivan

Content editors: Jenna Pearson, Chi Zhang, Ohad Zivan

Production editors: Wesley Banfield, Jenna Pearson, Chi Zhang, Ohad Zivan

Our 2023 Sponsors: NASA TOPS, Google DeepMind, NFDI4Earth!

#

Pythia credit: Rose, B. E. J., Kent, J., Tyle, K., Clyne, J., Banihirwe, A., Camron, D., May, R., Grover, M., Ford, R. R., Paul, K., Morley, J., Eroglu, O., Kailyn, L., & Zacharias, A. (2023). Pythia Foundations (Version v2023.05.01) https://zenodo.org/record/8065851

#

Tutorial Objectives#

As you just learned in the Introduction to Climate video, variations in global climate involve various forcings, feedbacks, and interactions between multiple processes and systems. Because of this complexity, global climate datasets are often very large with multiple dimensions and variables.

One useful computational tool for organizing, analyzing and interpreting large global datasets is Xarray, an open source project and Python package that makes working with labelled multi-dimensional arrays simple and efficient.

In this first tutorial, we will use the DataArray and Dataset objects, which are used to represent and manipulate spatial data, to practice organizing large global climate datasets and to understand variations in Earth’s climate system.

Setup#

Similar to numpy, np; pandas, pd; you may often encounter xarray imported within a shortened namespace as xr.

# imports
import numpy as np
import pandas as pd
import xarray as xr
import matplotlib.pyplot as plt

Figure Settings#

# @title Figure Settings
import ipywidgets as widgets  # interactive display

%config InlineBackend.figure_format = 'retina'
plt.style.use(
    "https://raw.githubusercontent.com/ClimateMatchAcademy/course-content/main/cma.mplstyle"
)

Video 1: Introduction to Climate#

Tutorial slides#

These are the slides for the videos in all tutorials today

Introducing the `DataArray` and `Dataset`#

Xarray expands on the capabilities on NumPy arrays, providing a lot of streamlined data manipulation. It is similar in that respect to Pandas, but whereas Pandas excels at working with tabular data, Xarray is focused on N-dimensional arrays of data (i.e. grids). Its interface is based largely on the netCDF data model (variables, attributes, and dimensions), but it goes beyond the traditional netCDF interfaces to provide functionality similar to netCDF-java’s Common Data Model (CDM).

Section 1: Creation of a `DataArray` Object#

The DataArray is one of the basic building blocks of Xarray (see docs here). It provides a numpy.ndarray-like object that expands to provide two critical pieces of functionality:

Coordinate names and values are stored with the data, making slicing and indexing much more powerful
It has a built-in container for attributes

Here we’ll initialize a DataArray object by wrapping a plain NumPy array, and explore a few of its properties.

Section 1.1: Generate a Random Numpy Array#

For our first example, we’ll just create a random array of “temperature” data in units of Kelvin:

rand_data = 283 + 5 * np.random.randn(5, 3, 4)
rand_data

array([[[284.01981799, 288.60416752, 280.164391  , 276.63442792],
        [290.44831894, 282.59011219, 284.5380792 , 290.42857619],
        [287.15236197, 287.51933151, 275.01154359, 293.17913947]],

       [[274.39799007, 282.79978578, 283.32264251, 268.9884763 ],
        [288.95809125, 284.92039125, 278.51384981, 287.85576971],
        [277.65689332, 278.43746795, 282.13393524, 285.93729069]],

       [[280.46408865, 283.60850813, 285.89537151, 273.99139699],
        [284.64515918, 291.18653695, 279.78812068, 278.50048079],
        [276.85608999, 286.21271191, 272.07878347, 282.29925238]],

       [[282.01366621, 288.77296074, 281.34770578, 293.89862591],
        [285.13819336, 285.68505849, 292.15036797, 285.08722618],
        [286.87037646, 275.77827207, 281.43967177, 273.66207177]],

       [[292.40567693, 278.70365543, 284.09449836, 289.30927515],
        [284.78650261, 278.98630279, 284.09550471, 289.80034213],
        [285.88524684, 280.67820484, 281.88705931, 277.28511325]]])

Section 1.2: Wrap the Array: First Attempt#

Now we create a basic DataArray just by passing our plain data as an input:

temperature = xr.DataArray(rand_data)
temperature

<xarray.DataArray (dim_0: 5, dim_1: 3, dim_2: 4)>
array([[[284.01981799, 288.60416752, 280.164391  , 276.63442792],
        [290.44831894, 282.59011219, 284.5380792 , 290.42857619],
        [287.15236197, 287.51933151, 275.01154359, 293.17913947]],

       [[274.39799007, 282.79978578, 283.32264251, 268.9884763 ],
        [288.95809125, 284.92039125, 278.51384981, 287.85576971],
        [277.65689332, 278.43746795, 282.13393524, 285.93729069]],

       [[280.46408865, 283.60850813, 285.89537151, 273.99139699],
        [284.64515918, 291.18653695, 279.78812068, 278.50048079],
        [276.85608999, 286.21271191, 272.07878347, 282.29925238]],

       [[282.01366621, 288.77296074, 281.34770578, 293.89862591],
        [285.13819336, 285.68505849, 292.15036797, 285.08722618],
        [286.87037646, 275.77827207, 281.43967177, 273.66207177]],

       [[292.40567693, 278.70365543, 284.09449836, 289.30927515],
        [284.78650261, 278.98630279, 284.09550471, 289.80034213],
        [285.88524684, 280.67820484, 281.88705931, 277.28511325]]])
Dimensions without coordinates: dim_0, dim_1, dim_2

Note two things:

Xarray generates some basic dimension names for us (dim_0, dim_1, dim_2). We’ll improve this with better names in the next example.
Wrapping the numpy array in a DataArray gives us a rich display in the notebook! (Try clicking the array symbol to expand or collapse the view)

Section 1.3: Assign Dimension Names#

Much of the power of Xarray comes from making use of named dimensions. So let’s add some more useful names! We can do that by passing an ordered list of names using the keyword argument dims:

temperature = xr.DataArray(rand_data, dims=["time", "lat", "lon"])
temperature

<xarray.DataArray (time: 5, lat: 3, lon: 4)>
array([[[284.01981799, 288.60416752, 280.164391  , 276.63442792],
        [290.44831894, 282.59011219, 284.5380792 , 290.42857619],
        [287.15236197, 287.51933151, 275.01154359, 293.17913947]],

       [[274.39799007, 282.79978578, 283.32264251, 268.9884763 ],
        [288.95809125, 284.92039125, 278.51384981, 287.85576971],
        [277.65689332, 278.43746795, 282.13393524, 285.93729069]],

       [[280.46408865, 283.60850813, 285.89537151, 273.99139699],
        [284.64515918, 291.18653695, 279.78812068, 278.50048079],
        [276.85608999, 286.21271191, 272.07878347, 282.29925238]],

       [[282.01366621, 288.77296074, 281.34770578, 293.89862591],
        [285.13819336, 285.68505849, 292.15036797, 285.08722618],
        [286.87037646, 275.77827207, 281.43967177, 273.66207177]],

       [[292.40567693, 278.70365543, 284.09449836, 289.30927515],
        [284.78650261, 278.98630279, 284.09550471, 289.80034213],
        [285.88524684, 280.67820484, 281.88705931, 277.28511325]]])
Dimensions without coordinates: time, lat, lon

This is already an improvement over a NumPy array because we have names for each of the dimensions (or axes). Even better, we can associate arrays representing the values for the coordinates for each of these dimensions with the data when we create the DataArray. We’ll see this in the next example.

Section 2: Create a `DataArray` with Named Coordinates#

Section 3: The `Dataset`: a Container for `DataArray`s with Shared Coordinates#

Along with DataArray, the other key object type in Xarray is the Dataset, which is a dictionary-like container that holds one or more DataArrays, which can also optionally share coordinates (see docs here).

The most common way to create a Dataset object is to load data from a file (which we will practice in a later tutorial). Here, instead, we will create another DataArray and combine it with our temperature data.

This will illustrate how the information about common coordinate axes is used.

Summary#

In this initial tutorial, the DataArray and Dataset objects were utilized to create and explore synthetic examples of climate data.

Resources#

Code and data for this tutorial is based on existing content from Project Pythia.

Climatematch Academy: Computational Tools for Climate Science

Tutorial 1: Creating DataArrays and Datasets to Assess Global Climate Data

Contents

Tutorial 1: Creating DataArrays and Datasets to Assess Global Climate Data#

#

#

Tutorial Objectives#

Setup#

Figure Settings#

Video 1: Introduction to Climate#

Tutorial slides#

Introducing the `DataArray` and `Dataset`#

Section 1: Creation of a `DataArray` Object#

Section 1.1: Generate a Random Numpy Array#

Section 1.2: Wrap the Array: First Attempt#

Section 1.3: Assign Dimension Names#

Section 2: Create a `DataArray` with Named Coordinates#

Section 2.1: Make Time and Space Coordinates#

Section 2.1.1: Initialize the `DataArray` with Complete Coordinate Info#

Section 2.1.2: Set Useful Attributes#

Section 2.1.3: Attributes Are Not Preserved by Default!#

Section 3: The `Dataset`: a Container for `DataArray`s with Shared Coordinates#

Section 3.1: Create a Pressure `DataArray` Using the Same Coordinates#

Section 3.2: Create a `Dataset` Object#

Section 3.3: Access Data Variables and Coordinates in a `Dataset`#

Summary#

Resources#

Climatematch Academy: Computational Tools for Climate Science

Tutorial 1: Creating DataArrays and Datasets to Assess Global Climate Data

Contents

Tutorial 1: Creating DataArrays and Datasets to Assess Global Climate Data#

#

#

Tutorial Objectives#

Setup#

Figure Settings#

Video 1: Introduction to Climate#

Tutorial slides#

Introducing the DataArray and Dataset#

Section 1: Creation of a DataArray Object#

Section 1.1: Generate a Random Numpy Array#

Section 1.2: Wrap the Array: First Attempt#

Section 1.3: Assign Dimension Names#

Section 2: Create a DataArray with Named Coordinates#

Section 2.1: Make Time and Space Coordinates#

Section 2.1.1: Initialize the DataArray with Complete Coordinate Info#

Section 2.1.2: Set Useful Attributes#

Section 2.1.3: Attributes Are Not Preserved by Default!#

Section 3: The Dataset: a Container for DataArrays with Shared Coordinates#

Section 3.1: Create a Pressure DataArray Using the Same Coordinates#

Section 3.2: Create a Dataset Object#

Section 3.3: Access Data Variables and Coordinates in a Dataset#

Summary#

Resources#

Introducing the `DataArray` and `Dataset`#

Section 1: Creation of a `DataArray` Object#

Section 2: Create a `DataArray` with Named Coordinates#

Section 2.1.1: Initialize the `DataArray` with Complete Coordinate Info#

Section 3: The `Dataset`: a Container for `DataArray`s with Shared Coordinates#

Section 3.1: Create a Pressure `DataArray` Using the Same Coordinates#

Section 3.2: Create a `Dataset` Object#

Section 3.3: Access Data Variables and Coordinates in a `Dataset`#