Python API

Data Store Framework

Functions

Classes

Cube generation

xcube.core.gen.gen.gen_cube(input_paths: Sequence[str] = None, input_processor_name: str = None, input_processor_params: dict = None, input_reader_name: str = None, input_reader_params: dict[str, Any] = None, output_region: tuple[float, float, float, float] = None, output_size: tuple[int, int] = [512, 512], output_resampling: str = 'Nearest', output_path: str = 'out.zarr', output_writer_name: str = None, output_writer_params: dict[str, Any] = None, output_metadata: dict[str, Any] = None, output_variables: list[tuple[str, dict[str, Any] | None]] = None, processed_variables: list[tuple[str, dict[str, Any] | None]] = None, profile_mode: bool = False, no_sort_mode: bool = False, append_mode: bool = None, dry_run: bool = False, monitor: Callable[[...], None] = None) → bool[source]

Generate a xcube dataset from one or more input files.

Parameters:

no_sort_mode
input_paths – The input paths.
input_processor_name – Name of a registered input processor (xcube.core.gen.inputprocessor.InputProcessor) to be used to transform the inputs.
input_processor_params – Parameters to be passed to the input processor.
input_reader_name – Name of a registered input reader (xcube.core.util.dsio.DatasetIO).
input_reader_params – Parameters passed to the input reader.
output_region – Output region as tuple of floats: (lon_min, lat_min, lon_max, lat_max).
output_size – The spatial dimensions of the output as tuple of ints: (width, height).
output_resampling – The resampling method for the output.
output_path – The output directory.
output_writer_name – Name of an output writer (xcube.core.util.dsio.DatasetIO) used to write the cube.
output_writer_params – Parameters passed to the output writer.
output_metadata – Extra metadata passed to output cube.
output_variables – Output variables.
processed_variables – Processed variables computed on-the-fly.
profile_mode – Whether profiling should be enabled.
append_mode – Deprecated. The function will always either insert, replace, or append new time slices.
dry_run – Doesn’t write any data. For testing.
monitor – A progress monitor.

Returns:

True for success.

xcube.core.new.new_cube(title='Test Cube', width=360, height=180, x_name='lon', y_name='lat', x_dtype='float64', y_dtype=None, x_units='degrees_east', y_units='degrees_north', x_res=1.0, y_res=None, x_start=-180.0, y_start=-90.0, inverse_y=False, time_name='time', time_dtype='datetime64[ns]', time_units='seconds since 1970-01-01T00:00:00', time_calendar='proleptic_gregorian', time_periods=5, time_freq='D', time_start='2010-01-01T00:00:00', use_cftime=False, drop_bounds=False, variables=None, crs=None, crs_name=None, time_encoding_dtype='int64')[source]

Create a new empty cube. Useful for creating cubes templates with predefined coordinate variables and metadata. The function is also heavily used by xcube’s unit tests.

The values of the variables dictionary can be either constants, array-like objects, or functions that compute their return value from passed coordinate indexes. The expected signature is::

def my_func(time: int, y: int, x: int) -> Union[bool, int, float]

Parameters:

title – A title. Defaults to ‘Test Cube’.
width – Horizontal number of grid cells. Defaults to 360.
height – Vertical number of grid cells. Defaults to 180.
x_name – Name of the x coordinate variable. Defaults to ‘lon’.
y_name – Name of the y coordinate variable. Defaults to ‘lat’.
x_dtype – Data type of x coordinates. Defaults to ‘float64’.
y_dtype – Data type of y coordinates. Defaults to ‘float64’.
x_units – Units of the x coordinates. Defaults to ‘degrees_east’.
y_units – Units of the y coordinates. Defaults to ‘degrees_north’.
x_start – Minimum x value. Defaults to -180.
y_start – Minimum y value. Defaults to -90.
x_res – Spatial resolution in x-direction. Defaults to 1.0.
y_res – Spatial resolution in y-direction. Defaults to 1.0.
inverse_y – Whether to create an inverse y axis. Defaults to False.
time_name – Name of the time coordinate variable. Defaults to ‘time’.
time_periods – Number of time steps. Defaults to 5.
time_freq – Duration of each time step. Defaults to `1D’.
time_start – First time value. Defaults to ‘2010-01-01T00:00:00’.
time_dtype – Numpy data type for time coordinates. Defaults to ‘datetime64[s]’. If used, parameter ‘use_cftime’ must be False.
time_units – Units for time coordinates. Defaults to ‘seconds since 1970-01-01T00:00:00’.
time_calendar – Calender for time coordinates. Defaults to ‘proleptic_gregorian’.
use_cftime – If True, the time will be given as data types according to the ‘cftime’ package. If used, the time_calendar parameter must be also be given with an appropriate value such as ‘gregorian’ or ‘julian’. If used, parameter ‘time_dtype’ must be None.
drop_bounds – If True, coordinate bounds variables are not created. Defaults to False.
variables – Dictionary of data variables to be added. None by default.
crs – pyproj-compatible CRS string or instance of pyproj.CRS or None
crs_name – Name of the variable that will hold the CRS information. Ignored, if crs is not given.
time_encoding_dtype – data type used to encode the time variable when serializing the dataset

Returns:

A cube instance

Cube computation

xcube.core.evaluate.evaluate_dataset(dataset: Dataset, processed_variables: list[tuple[str, dict[str, Any] | None]] = None, errors: str = 'raise') → Dataset[source]

Compute new variables or mask existing variables in dataset by the evaluation of Python expressions, that may refer to other existing or new variables. Returns a new dataset that contains the old and new variables, where both may bew now masked.

Expressions may be given by attributes of existing variables in dataset or passed a via the processed_variables argument which is a sequence of variable name / attributes tuples.

Two types of expression attributes are recognized in the attributes:

The attribute expression generates a new variable computed from its attribute value.
The attribute valid_pixel_expression masks out invalid variable values.

In both cases the attribute value must be a string that forms a valid Python expression that can reference any other preceding variables by name. The expression can also reference any flags defined by another variable according to their CF attributes flag_meaning and flag_values.

Invalid variable values may be masked out using the value the valid_pixel_expression attribute whose value should form a Boolean Python expression. In case, the expression returns zero or false, the value of the _FillValue attribute or NaN will be used in the new variable.

Other attributes will be stored as variable metadata as-is.

Parameters:

dataset – A dataset.
processed_variables – Optional list of variable name-attributes pairs that will be processed in the given order.
errors – How to deal with errors while evaluating expressions. May be be one of “raise”, “warn”, or “ignore”.

Returns:

new dataset with computed variables

Cube data extraction

xcube.core.extract.get_cube_values_for_points(cube: Dataset, points: Dataset | DataFrame | Mapping[str, ndarray | Array | DataArray | Series | Sequence[int | float]], var_names: Sequence[str] = None, include_coords: bool = False, include_bounds: bool = False, include_indexes: bool = False, index_name_pattern: str = '{name}_index', include_refs: bool = False, ref_name_pattern: str = '{name}_ref', method: str = 'nearest', cube_asserted: bool = False) → Dataset[source]

Extract values from cube variables at given coordinates in points.

Returns a new dataset with values of variables from cube selected by the coordinate columns provided in points. All variables will be 1-D and have the same order as the rows in points.

Parameters:

cube – The cube dataset.
points – Dictionary that maps dimension name to coordinate arrays.
var_names – An optional list of names of data variables in cube whose values shall be extracted.
include_coords – Whether to include the cube coordinates for each point in return value.
include_bounds – Whether to include the cube coordinate boundaries (if any) for each point in return value.
include_indexes – Whether to include computed indexes into the cube for each point in return value.
index_name_pattern – A naming pattern for the computed index columns. Must include “{name}” which will be replaced by the index’ dimension name.
include_refs – Whether to include point (reference) values from points in return value.
ref_name_pattern – A naming pattern for the computed point data columns. Must include “{name}” which will be replaced by the point’s attribute name.
method – “nearest” or “linear”.
cube_asserted – If False, cube will be verified, otherwise it is expected to be a valid cube.

Returns:

A new data frame whose columns are values from cube variables at given points.

xcube.core.extract.get_cube_point_indexes(cube: ~xarray.core.dataset.Dataset, points: ~xarray.core.dataset.Dataset | ~pandas.core.frame.DataFrame | ~collections.abc.Mapping[str, ~numpy.ndarray | ~dask.array.core.Array | ~xarray.core.dataarray.DataArray | ~pandas.core.series.Series | ~collections.abc.Sequence[int | float]], dim_name_mapping: ~collections.abc.Mapping[str, str] = None, index_name_pattern: str = '{name}_index', index_dtype=<class 'numpy.float64'>, cube_asserted: bool = False) → Dataset[source]

Get indexes of given point coordinates points into the given dataset.

Parameters:

cube – The cube dataset.
points – A mapping from column names to column data arrays, which must all have the same length.
dim_name_mapping – A mapping from dimension names in cube to column names in points.
index_name_pattern – A naming pattern for the computed indexes columns. Must include “{name}” which will be replaced by the dimension name.
index_dtype – Numpy data type for the indexes. If it is a floating point type (default), then indexes will contain fractions, which may be used for interpolation. For out-of- range coordinates in points, indexes will be -1 if index_dtype is an integer type, and NaN, if index_dtype is a floating point types.
cube_asserted – If False, cube will be verified, otherwise it is expected to be a valid cube.

Returns:

A dataset containing the index columns.

xcube.core.extract.get_cube_values_for_indexes(cube: Dataset, indexes: Dataset | DataFrame | Mapping[str, Any], include_coords: bool = False, include_bounds: bool = False, data_var_names: Sequence[str] = None, index_name_pattern: str = '{name}_index', method: str = 'nearest', cube_asserted: bool = False) → Dataset[source]

Get values from the cube at given indexes.

Parameters:

cube – A cube dataset.
indexes – A mapping from column names to index and fraction arrays for all cube dimensions.
include_coords – Whether to include the cube coordinates for each point in return value.
include_bounds – Whether to include the cube coordinate boundaries (if any) for each point in return value.
data_var_names – An optional list of names of data variables in cube whose values shall be extracted.
index_name_pattern – A naming pattern for the computed indexes columns. Must include “{name}” which will be replaced by the dimension name.
method – “nearest” or “linear”.
cube_asserted – If False, cube will be verified, otherwise it is expected to be a valid cube.

Returns:

A new data frame whose columns are values from cube variables at given indexes.

xcube.core.extract.get_dataset_indexes(dataset: ~xarray.core.dataset.Dataset, coord_var_name: str, coord_values: ~numpy.ndarray | ~dask.array.core.Array | ~xarray.core.dataarray.DataArray | ~pandas.core.series.Series | ~collections.abc.Sequence[int | float], index_dtype=<class 'numpy.float64'>) → DataArray | ndarray[source]

Compute the indexes and their fractions into a coordinate variable coord_var_name of a dataset for the given coordinate values coord_values.

The coordinate variable’s labels must be monotonic increasing or decreasing, otherwise the result will be nonsense.

For any value in coord_values that is out of the bounds of the coordinate variable’s values, the index depends on the value of index_dtype. If index_dtype is an integer type, invalid indexes are encoded as -1 while for floating point types, NaN will be used.

Returns a tuple of indexes as int64 array and fractions as float64 array.

Parameters:

dataset – A cube dataset.
coord_var_name – Name of a coordinate variable.
coord_values – Array-like coordinate values.
index_dtype – Numpy data type for the indexes. If it is floating point type (default), then indexes contain fractions, which may be used for interpolation. If dtype is an integer type out-of-range coordinates are indicated by index -1, and NaN if it is is a floating point type.

Returns:

The indexes and their fractions as a tuple of numpy int64 and float64 arrays.

Get a time series dataset from a data cube.

geometry may be provided as a (shapely) geometry object, a valid GeoJSON object, a valid WKT string, a sequence of box coordinates (x1, y1, x2, y2), or point coordinates (x, y). If geometry covers an area, i.e. is not a point, the function aggregates the variables to compute a mean value and if desired, the number of valid observations and the standard deviation.

start_date and end_date may be provided as a numpy.datetime64 or an ISO datetime string.

Returns a time-series dataset whose data variables have a time dimension but no longer have spatial dimensions, hence the resulting dataset’s variables will only have N-2 dimensions. A global attribute max_number_of_observations will be set to the maximum number of observations that could have been made in each time step. If the given geometry does not overlap the cube’s boundaries, or if not output variables remain, the function returns None.

Parameters:

dataset – The dataset
grid_mapping – Grid mapping of cube.
geometry – Optional geometry
var_names – Optional sequence of names of variables to be included.
start_date – Optional start date.
end_date – Optional end date.
agg_methods – Aggregation methods. May be single string or sequence of strings. Possible values are ‘mean’, ‘median’, ‘min’, ‘max’, ‘std’, ‘count’. Defaults to ‘mean’. Ignored if geometry is a point.
use_groupby – Use group-by operation. May increase or decrease runtime performance and/or memory consumption.
cube_asserted – Deprecated and ignored since xcube 0.11.0. No replacement.

Cube Resampling

xcube.core.resampling.affine_transform_dataset(source_ds: Dataset, /, source_gm: GridMapping | None = None, target_gm: GridMapping | None = None, ref_ds: Dataset | None = None, var_configs: Mapping[Hashable, Mapping[str, Any]] | None = None, encode_cf: bool = True, gm_name: str | None = None, reuse_coords: bool = False) → Dataset[source]

Resample dataset according to an affine transformation.

The affine transformation will be applied only if the CRS of source_gm and the CRS of target_gm are both geographic or equal. Otherwise, a ValueError will be raised.

New in 1.6: If target_ds is given, its coordinate variables are copied by reference into the returned dataset.

Parameters:

source_ds – The source dataset
source_gm – Optional source grid mapping of dataset. If not provided, computed from source_ds. Must be regular and must have same CRS as target_gm.
target_gm – Optional target grid mapping. If not provided, computed from target_ds or source grid mapping. Must be regular and must have same CRS as source grid mapping.
ref_ds – An optional dataset that provides the target grid mapping if target_gm is not provided. If ref_ds is given, its coordinate variables are copied by reference into the returned dataset.
var_configs – Optional resampling configurations for individual variables.
encode_cf – Whether to encode the target grid mapping into the resampled dataset in a CF-compliant way. Defaults to True.
gm_name – Name for the grid mapping variable. Defaults to “crs”. Used only if encode_cf is True.
reuse_coords – Whether to either reuse target coordinate arrays from target_gm or to compute new ones.

Returns:

The resampled target dataset.

xcube.core.resampling.resample_ndimage(image: ~numpy.ndarray | ~dask.array.core.Array, scale: float | tuple[float, float] = 1, offset: float | tuple[float, float] = None, shape: int | tuple[int, int] = None, chunks: ~collections.abc.Sequence[int] = None, spline_order: int = 1, aggregator: ~typing.Callable[[~numpy.ndarray | ~dask.array.core.Array], ~numpy.ndarray | ~dask.array.core.Array] | None = <function nanmean>, recover_nan: bool = False) → Array[source]

xcube.core.resampling.encode_grid_mapping(ds: Dataset, gm: GridMapping, gm_name: str | None = None, force: bool | None = None) → Dataset[source]

Encode the given grid mapping gm into a copy of ds in a CF-compliant way and return the dataset copy. The function removes any existing grid mappings.

If the CRS of gm is geographic and the spatial dimension and coordinate names are “lat”, “lon” and force is False, or force is None and no former grid mapping was encoded in ds, then nothing else is done and the dataset copy is returned without further action.

Otherwise, for every spatial data variable with dims=(…, y, x), the function sets the attribute “grid_mapping” to gm_name. The grid mapping CRS is encoded in a new 0-D variable named gm_name.

Parameters:

ds – The dataset.
gm – The dataset’s grid mapping.
gm_name – Name for the grid mapping variable. Defaults to “crs”.
force – Whether to force encoding of grid mapping even if CRS is geographic and spatial dimension names are “lon”, “lat”. Optional value, if not provided, force will be assumed True if a former grid mapping was encoded in ds.

Returns:

A copy of ds with gm encoded into it.

Reproject dataset source_ds using its per-pixel x,y coordinates or the given source_gm.

The function expects source_ds or the given source_gm to have either one- or two-dimensional coordinate variables that provide spatial x,y coordinates for every data variable with the same spatial dimensions.

For example, a dataset may comprise variables with spatial dimensions var(..., y_dim, x_dim), then the function expects coordinates to be provided in two forms:

One-dimensional x_var(x_dim) and y_var(y_dim) (coordinate) variables.
Two-dimensional x_var(y_dim, x_dim) and y_var(y_dim, x_dim) (coordinate) variables.

If target_gm is given, and it defines a tile size, or tile_size is given and the number of tiles is greater than one in the output’s x- or y-direction, then the returned dataset will be composed of lazy, chunked dask arrays. Otherwise, the returned dataset will be composed of ordinary numpy arrays.

New in 1.6: If target_ds is given, its coordinate variables are copied by reference into the returned dataset.

Parameters:

source_ds – Source dataset.
source_gm – Source dataset grid mapping.
target_gm – Optional target geometry. If not given, output geometry will be computed to spatially fit dataset and to retain its spatial resolution.
ref_ds – An optional dataset that provides the target grid mapping if target_gm is not provided. If ref_ds is given, its coordinate variables are copied by reference into the returned dataset.
var_names – Optional variable name or sequence of variable names.
encode_cf – Whether to encode the target grid mapping into the resampled dataset in a CF-compliant way. Defaults to True.
gm_name – Name for the grid mapping variable. Defaults to “crs”. Used only if encode_cf is True.
tile_size – Optional tile size for the output.
is_j_axis_up – Whether y coordinates are increasing with positive image j axis.
output_ij_names – If given, a tuple of variable names in which to store the computed source pixel coordinates in the returned output.
compute_subset – Whether to compute a spatial subset from source_ds using the boundary of the target grid mapping. If set, the function may return None in case there is no overlap.
uv_delta – A normalized value that is used to determine whether x,y coordinates in the output are contained in the triangles defined by the input x,y coordinates. The higher this value, the more inaccurate the rectification will be.
interpolation – Interpolation method for computing output pixels. If given, must be “nearest”, “triangular”, or “bilinear”. The default is “nearest”. The “triangular” interpolation is performed between 3 and “bilinear” between 4 adjacent source pixels. Both are applied only to variables of floating point type. If you need to interpolate between integer data you should cast it to float first.
xy_var_names – Deprecated. No longer used since 1.0.0, no replacement.

Returns:

A reprojected dataset, or None if the requested output does not intersect with dataset.

For implementation details refer to Spatial Rectification Algorithm.

xcube.core.resampling.resample_in_space(source_ds: Dataset, /, source_gm: GridMapping | None = None, target_gm: GridMapping | None = None, ref_ds: Dataset | None = None, var_configs: Mapping[Hashable, Mapping[str, Any]] | None = None, encode_cf: bool = True, gm_name: str | None = None, rectify_kwargs: dict | None = None)[source]

Resample a dataset source_ds in the spatial dimensions.

If the source grid mapping source_gm is not given, it is derived from dataset: source_gm = GridMapping.from_dataset(source_ds).

If the target grid mapping target_gm is not given, it is derived from source_gm as target_gm = source_gm.to_regular(), or if target dataset ref_ds is given as target_gm = GridMapping.from_dataset(ref_ds).

New in 1.6: If ref_ds is given, its coordinate variables are copied by reference into the returned dataset.

If source_gm is almost equal to target_gm, this function is a no-op and dataset is returned unchanged.

Otherwise, the function computes a spatially resampled version of dataset and returns it.

Using var_configs, the resampling of individual variables can be configured. If given, var_configs must be a mapping from variable names to configuration dictionaries which can have the following properties:

spline_order (int) - The order of spline polynomials
used for interpolating. It is used for up-sampling only. Possible values are 0 to 5. Default is 1 (bi-linear) for floating point variables, and 0 (= nearest neighbor) for integer and bool variables.
aggregator (str) - An optional aggregating
function. It is used for down-sampling only. Examples are numpy.nanmean, numpy.nanmin, numpy.nanmax. Default is numpy.nanmean for floating point variables, and None (= nearest neighbor) for integer and bool variables.
recover_nan (bool) - whether a special algorithm
shall be used that is able to recover values that would otherwise yield NaN during resampling. Default is False for all variable types since this may require considerable CPU resources on top.

Note that var_configs is only used if the resampling involves an affine transformation. This is true if the CRS of source_gm and CRS of target_gm are equal and one of two cases is given:

source_gm is regular. In this case the resampling is the affine transformation. and the result is returned directly.
source_gm is not regular and has a lower resolution than target_cm. In this case dataset is down-sampled first using an affine transformation. Then the result is rectified.

In all other cases, no affine transformation is applied and the resampling is a direct rectification.

Parameters:

source_ds – The source dataset.
source_gm – The source grid mapping.
target_gm – The target grid mapping. Must be regular.
ref_ds – An optional dataset that provides the target grid mapping if target_gm is not provided. If ref_ds is given, its coordinate variables are copied by reference into the returned dataset.
var_configs – Optional resampling configurations for individual variables.
encode_cf – Whether to encode the target grid mapping into the resampled dataset in a CF-compliant way. Defaults to True.
gm_name – Name for the grid mapping variable. Defaults to “crs”. Used only if encode_cf is True.
rectify_kwargs – Keyword arguments passed func:rectify_dataset should a rectification be required.

Returns: The spatially resampled dataset, or None if the requested: output area does not intersect with dataset.

xcube.core.resampling.resample_in_time(dataset: Dataset, frequency: str, method: str | Sequence[str], offset=None, tolerance=None, interp_kind=None, time_chunk_size=None, var_names: Sequence[str] = None, metadata: dict[str, Any] = None, cube_asserted: bool = False) → Dataset[source]

Resample a dataset in the time dimension.

The argument method may be one or a sequence of 'all', 'any', 'argmax', 'argmin', 'count', 'first', 'last', 'max', 'min', 'mean', 'median', 'percentile_<p>', 'std', 'sum', 'var'.

In value 'percentile_<p>' is a placeholder, where '<p>' must be replaced by an integer percentage value, e.g. 'percentile_90' is the 90%-percentile.

Important note: As of xarray 0.14 and dask 2.8, the methods 'median' and 'percentile_<p>'` cannot be used if the variables in *cube* comprise chunked dask arrays. In this case, use the ``compute() or load() method to convert dask arrays into numpy arrays.

Parameters:

dataset – The xcube dataset.
frequency – Temporal aggregation frequency. Use format “<count><offset>” where <offset> is one of ‘H’, ‘D’, ‘W’, ‘M’, ‘Q’, ‘Y’.
method – Resampling method or sequence of resampling methods.
offset – Offset used to adjust the resampled time labels. Uses same syntax as frequency.
time_chunk_size – If not None, the chunk size to be used for the “time” dimension.
var_names – Variable names to include.
tolerance – Time tolerance for selective upsampling methods. Defaults to frequency.
interp_kind – Kind of interpolation if method is ‘interpolation’.
metadata – Output metadata.
cube_asserted – If False, cube will be verified, otherwise it is expected to be a valid cube.

Returns:

A new xcube dataset resampled in time.

Cube Manipulation

xcube.core.vars2dim.vars_to_dim(cube: Dataset, dim_name: str = 'var', var_name='data', cube_asserted: bool = False)[source]

Convert data variables into a dimension.

Parameters:

cube – The xcube dataset.
dim_name – The name of the new dimension and coordinate variable. Defaults to ‘var’.
var_name – The name of the new, single data variable. Defaults to ‘data’.
cube_asserted – If False, cube will be verified, otherwise it is expected to be a valid cube.

Returns:

A new xcube dataset with data variables turned into a new dimension.

xcube.core.chunk.chunk_dataset(dataset: Dataset, chunk_sizes: dict[str, int] = None, format_name: str = None, data_vars_only: bool = False) → Dataset[source]

Chunk dataset using chunk_sizes and optionally update encodings for given format_name.

Parameters:

dataset – input dataset
chunk_sizes – mapping from dimension name to new chunk size
format_name – optional format, e.g. “zarr” or “netcdf4”
data_vars_only – only chunk data variables, not coordinates

Returns:

the (re)chunked dataset

xcube.core.unchunk.unchunk_dataset(dataset_path: str, var_names: Sequence[str] = None, coords_only: bool = False)[source]

Unchunk dataset variables in-place.

Parameters:

dataset_path – Path to ZARR dataset directory.
var_names – Optional list of variable names.
coords_only – Un-chunk coordinate variables only.

xcube.core.optimize.optimize_dataset(input_path: str, output_path: str = None, in_place: bool = False, unchunk_coords: bool | str | ~collections.abc.Sequence[str] = False, exception_type: type[Exception] = <class 'ValueError'>)[source]

Optimize a dataset for faster access.

Reduces the number of metadata and coordinate data files in xcube dataset given by given by dataset_path. Consolidated cubes open much faster from remote locations, e.g. in object storage, because obviously much less HTTP requests are required to fetch initial cube meta information. That is, it merges all metadata files into a single top-level JSON file “.zmetadata”.

If unchunk_coords is given, it also removes any chunking of coordinate variables so they comprise a single binary data file instead of one file per data chunk. The primary usage of this function is to optimize data cubes for cloud object storage. The function currently works only for data cubes using Zarr format. unchunk_coords can be a name, or list of names of the coordinate variable(s) to be consolidated. If boolean True is used, coordinate all variables will be consolidated.

Parameters:

input_path – Path to input dataset with ZARR format.
output_path – Path to output dataset with ZARR format. May contain “{input}” template string, which is replaced by the input path’s file name without file name extension.
in_place – Whether to modify the dataset in place. If False, a copy is made and output_path must be given.
unchunk_coords – The name of a coordinate variable or a list of coordinate variables whose chunks should be consolidated. Pass True to consolidate chunks of all coordinate variables.
exception_type – Type of exception to be used on value errors.

Cube Subsetting

xcube.core.select.select_variables_subset(dataset: Dataset, var_names: Collection[str] | None = None) → Dataset[source]

Select data variable from given dataset and create new dataset.

Parameters:

dataset – The dataset from which to select variables.
var_names – The names of data variables to select.

Returns:

A new dataset. It is empty, if var_names is empty. It is dataset, if var_names is None.

Spatially clip a dataset according to the bounding box of a given geometry.

Parameters:

dataset – The dataset
geometry – A geometry-like object, see normalize_geometry().
update_attrs – Weather to update (spatial) CF attributes of the returned dataset. The default is True.
save_geometry_wkt – If the value is a string, the effective intersection geometry is stored as a Geometry WKT string in the global attribute named by save_geometry. If the value is True, the name “geometry_wkt” is used.

Returns:

The dataset spatial subset, or None if the bounding box of the dataset has a no or a zero area intersection with the bounding box of the geometry.

Cube Masking

xcube.core.geom.mask_dataset_by_geometry(dataset: Dataset, geometry: BaseGeometry | dict[str, Any] | str | Sequence[float | int], tile_size: int | tuple[int, int] = None, excluded_vars: Sequence[str] = None, all_touched: bool = False, no_clip: bool = False, update_attrs: bool = True, save_geometry_mask: str | bool = False, save_geometry_wkt: str | bool = False) → Dataset | None[source]

Mask a dataset according to the given geometry. The cells of variables of the returned dataset will have NaN-values where their spatial coordinates are not intersecting the given geometry.

Parameters:

dataset – The dataset
geometry – A geometry-like object, see normalize_geometry().
tile_size – If given, the unconditional spatial chunk sizes in x- and y-direction in pixels. May be given as integer scalar or x,y-pair of integers.
excluded_vars – Optional sequence of names of data variables that should not be masked (but still may be clipped).
all_touched – If True, all pixels intersected by geometry outlines will be included in the mask. If False, only pixels whose center is within the polygon or that are selected by Bresenham’s line algorithm will be included in the mask. The default value is set to False.
no_clip – If True, the function will not clip the dataset before masking, this is, the returned dataset will have the same dimension size as the given dataset.
update_attrs – If no_clip is False, weather to update (spatial) CF attributes of the returned dataset. The default is True.
save_geometry_mask – If the value is a string, the effective geometry mask array is stored as a 2D data variable named by save_geometry_mask. If the value is True, the name “geometry_mask” is used.
save_geometry_wkt – If the value is a string, the effective intersection geometry is stored as a Geometry WKT string in the global attribute named by save_geometry. If the value is True, the name “geometry_wkt” is used.

Returns:

The dataset spatial subset, or None if the bounding box of the dataset has a no or a zero area intersection with the bounding box of the geometry.

class xcube.core.maskset.MaskSet(flag_var: DataArray)[source]

A set of mask variables derived from a variable flag_var with the following CF attributes:

One or both of flag_masks and flag_values

flag_meanings (always required)

See https://cfconventions.org/Data/cf-conventions/cf-conventions-1.9/cf-conventions.html#flags for details on the use of these attributes.

Each mask is represented by an xarray.DataArray, has the name of the flag, is of type numpy.unit8, and has the dimensions of the given flag_var.

Parameters:: flag_var – an xarray.DataArray that defines flag values. The CF attributes flag_meanings and one or both of flag_masks and flag_values are expected to exist and be valid.

classmethod get_mask_sets(dataset: Dataset) → dict[str, MaskSet][source]

For each “flag” variable in given dataset, turn it into a MaskSet, store it in a dictionary.

Parameters:: dataset – The dataset
Returns:: A mapping of flag names to MaskSet. Will be empty if there are no flag variables in dataset.

get_cmap(default: str = 'viridis') → tuple[Colormap, BoundaryNorm | None][source]

Get a suitable color mapping for use with matplotlib.

Parameters:: default – Default color map name in case a color mapping cannot be created, e.g., flag_values are not defined or if the flag values are not in the range [0, 2**16 - 1).
Returns:: A suitable instance of matplotlib.colors.Colormap and the corresponding matplotlib.colors.BoundaryNorm if applicable

Rasterisation of Features

xcube.core.geom.rasterize_features(dataset: Dataset, features: pandas.geodataframe.GeoDataFrame | Sequence[Mapping[str, Any]], feature_props: Sequence[str], var_props: dict[str, Mapping[str, Mapping[str, Any]]] = None, tile_size: int | tuple[int, int] = None, all_touched: bool = False, in_place: bool = False) → Dataset | None[source]

Rasterize feature properties given by feature_props of vector-data features as new variables into dataset.

dataset must have two spatial 1-D coordinates, either lon and lat in degrees, reprojected coordinates, x and y, or similar.

feature_props is a sequence of names of feature properties that must exists in each feature of features.

features may be passed as pandas.GeoDataFrame`` or as an iterable of GeoJSON features.

Using the optional var_props, the properties of newly created variables from feature properties can be specified. It is a mapping of feature property names to mappings of variable properties. Here is an example variable properties mapping::

‘name’: ‘land_class’, # (str) - the variable’s name,
# default is the feature property name;

‘dtype’ np.int16, # (str|np.dtype) - the variable’s dtype,
# default is np.float64;

‘fill_value’: -999, # (bool|int|float|np.nparray) -
# the variable’s fill value, # default is np.nan;

‘attrs’: {}, # (Mapping[str, Any]) -
# the variable’s fill value, default is {};

‘converter’: int, # (Callable[[Any], Any]) -
# a converter function used to convert # from property feature value to variable # value, default is float. # Deprecated, no longer used.

}

Note that newly created variables will have data type np.float64 because np.nan is used to encode missing values. fill_value and dtype are used to encode the variables when persisting the data.

Currently, the coordinates of the geometries in the given features must use the same CRS as the given dataset.

Parameters:

dataset – The xarray dataset.
features – A geopandas.GeoDataFrame instance or a sequence of GeoJSON features.
feature_props – Sequence of names of numeric feature properties to be rasterized.
var_props – Optional mapping of feature property name to a name or a 5-tuple (name, dtype, fill_value, attributes, converter) for the new variable.
tile_size – If given, the unconditional spatial chunk sizes in x- and y-direction in pixels. May be given as integer scalar or x,y-pair of integers.
all_touched – If True, all pixels intersected by a feature’s geometry outlines will be included. If False, only pixels whose center is within the feature polygon or that are selected by Bresenham’s line algorithm will be included in the mask. The default is False.
in_place – Whether to add new variables to dataset. If False, a copy will be created and returned.

Returns:

dataset with rasterized feature_property

Cube Metadata

xcube.core.update.update_dataset_attrs(dataset: Dataset, global_attrs: dict[str, Any] = None, update_existing: bool = False, in_place: bool = False) → Dataset[source]

Update spatio-temporal CF/THREDDS attributes given dataset according to spatio-temporal coordinate variables time, lat, and lon.

Parameters:

dataset – The dataset.
global_attrs – Optional global attributes.
update_existing – If True, any existing attributes will be updated.
in_place – If True, dataset will be modified in place and returned.

Returns:

A new dataset, if in_place if False (default), else the passed and modified dataset.

xcube.core.update.update_dataset_spatial_attrs(dataset: Dataset, update_existing: bool = False, in_place: bool = False) → Dataset[source]

Update spatial CF/THREDDS attributes of given dataset.

Parameters:

dataset – The dataset.
update_existing – If True, any existing attributes will be updated.
in_place – If True, dataset will be modified in place and returned.

Returns:

A new dataset, if in_place if False (default), else the passed and modified dataset.

xcube.core.update.update_dataset_temporal_attrs(dataset: Dataset, update_existing: bool = False, in_place: bool = False) → Dataset[source]

Update temporal CF/THREDDS attributes of given dataset.

Parameters:

dataset – The dataset.
update_existing – If True, any existing attributes will be updated.
in_place – If True, dataset will be modified in place and returned.

Returns:

A new dataset, if in_place is False (default), else the passed and modified dataset.

Cube verification

xcube.core.verify.assert_cube(dataset: Dataset, name=None) → Dataset[source]

Assert that the given dataset is a valid xcube dataset.

Parameters:

dataset – The dataset to be validated.
name – Optional parameter name.

Raises:

ValueError, if dataset is not a valid xcube dataset –

xcube.core.verify.verify_cube(dataset: Dataset) → list[str][source]

Verify the given dataset for being a valid xcube dataset.

The tool verifies that dataset * defines two spatial x,y coordinate variables, that are 1D, non-empty, using correct units; * defines a time coordinate variables, that are 1D, non-empty, using correct units; * has valid bounds variables for spatial x,y and time coordinate variables, if any; * has any data variables and that they are valid, e.g. min. 3-D, all have

same dimensions, have at least the dimensions dim(time), dim(y), dim(x) in that order.

Returns a list of issues, which is empty if dataset is a valid xcube dataset.

Parameters:: dataset – A dataset to be verified.
Returns:: List of issues or empty list.

Multi-Resolution Datasets

Zarr Store

Utilities

An abstract base class for grid mappings that define an image grid and a transformation from image pixel coordinates to spatial Earth coordinates defined in a well-known coordinate reference system (CRS).

This class cannot be instantiated directly. Use one of its factory methods to create instances:

regular()
from_dataset()
from_coords()

Some instance methods can be used to derive new instances:

derive()
scale()
transform()
to_regular()

This class is thread-safe.

derive(xy_var_names: tuple[str, str] = None, xy_dim_names: tuple[str, str] = None, tile_size: int | tuple[int, int] = None, is_j_axis_up: bool = None)[source]

Derive a new grid mapping from this one with some properties changed.

Parameters:

xy_var_names – The new x-, and y-variable names.
xy_dim_names – The new x-, and y-dimension names.
tile_size – The new tile size
is_j_axis_up – Whether j-axis points up.

Returns:

A new, derived grid mapping.

Derive a scaled version of this regular grid mapping.

Scaling factors larger than one correspond to up-scaling (pixels sizes decrease, image size increases).

Scaling factors lower than one correspond to down-scaling. (pixels sizes increase, image size decreases).

Parameters:

xy_scale – The x-, and y-scaling factors. May be a single number or tuple.
tile_size – The new tile size

Returns:

A new, scaled grid mapping.

property size: tuple[int, int]: Image size (width, height) in pixels.

property width: int: Image width in pixels.

property height: int: Image height in pixels.

property tile_size: tuple[int, int]: Image tile size (width, height) in pixels.

property is_tiled: bool: Whether the image is tiled.

property tile_width: int: Image tile width in pixels.

property tile_height: int: Image tile height in pixels.

property x_coords: The 1D or 2D x-coordinate array of shape (width,) or (height, width).

property y_coords: The 1D or 2D y-coordinate array of shape (width,) or (height, width).

property xy_coords: DataArray: The x,y coordinates as data array of shape (2, height, width). Coordinates are given in units of the CRS.

property xy_coords_chunks: tuple[int, int, int]: Get the chunks for the xy_coords array.

property xy_var_names: tuple[str, str]: The variable names of the x,y coordinates as tuple (x_var_name, y_var_name).

property xy_dim_names: tuple[str, str]: The dimension names of the x,y coordinates as tuple (x_dim_name, y_dim_name).

property xy_bbox: tuple[float, float, float, float]: The image’s bounding box in CRS coordinates.

property x_min: int | float: Minimum x-coordinate in CRS units.

property y_min: int | float: Minimum y-coordinate in CRS units.

property x_max: int | float: Maximum x-coordinate in CRS units.

property y_max: int | float: Maximum y-coordinate in CRS units.

property xy_res: tuple[int | float, int | float]: Pixel size in x and y direction.

property x_res: int | float: Pixel size in CRS units per pixel in x-direction.

property y_res: int | float: Pixel size in CRS units per pixel in y-direction.

property crs: CRS: The coordinate reference system.

property is_lon_360: bool | None: Check whether x_max is greater than 180 degrees. Effectively tests whether the range x_min, x_max crosses the anti-meridian at 180 degrees. Works only for geographical coordinate reference systems.

property is_regular: bool | None

Do the x,y coordinates for a regular grid? A regular grid has a constant delta in both x- and y-directions of the x- and y-coordinates.

Returns: None, if this property cannot be determined,: True or False otherwise.

property is_j_axis_up: bool | None

Does the positive image j-axis point up? By default, the positive image j-axis points down.

Returns: None, if this property cannot be determined,: True or False otherwise.

property ij_to_xy_transform: tuple[tuple[int | float, int | float, int | float], tuple[int | float, int | float, int | float]]: The affine transformation matrix from image to CRS coordinates. Defined only for grid mappings with rectified x,y coordinates.

property xy_to_ij_transform: tuple[tuple[int | float, int | float, int | float], tuple[int | float, int | float, int | float]]: The affine transformation matrix from CRS to image coordinates. Defined only for grid mappings with rectified x,y coordinates.

Get the affine transformation matrix that transforms image coordinates of other into image coordinates of this image geometry.

Defined only for grid mappings with rectified x,y coordinates.

Parameters:: other – The other image geometry
Returns:: Affine transformation matrix

Get the affine transformation matrix that transforms image coordinates of this image geometry to image coordinates of other.

Defined only for grid mappings with rectified x,y coordinates.

Parameters:: other – The other image geometry
Returns:: Affine transformation matrix

property ij_bbox: tuple[int, int, int, int]: The image’s bounding box in pixel coordinates.

property ij_bboxes: ndarray: The image tiles’ bounding boxes in image pixel coordinates.

property xy_bboxes: ndarray: The image tiles’ bounding boxes in CRS coordinates.

ij_bbox_from_xy_bbox(xy_bbox: tuple[float, float, float, float], xy_border: float = 0.0, ij_border: int = 0) → tuple[int, int, int, int][source]

Compute bounding box in i,j pixel coordinates given a bounding box xy_bbox in x,y coordinates.

Parameters:

xy_bbox – Box (x_min, y_min, x_max, y_max) given in the same CS as x and y.
xy_border – If non-zero, grows the bounding box xy_bbox before using it for comparisons. Defaults to 0.
ij_border – If non-zero, grows the returned i,j bounding box and clips it to size. Defaults to 0.

Returns:

Bounding box in (i_min, j_min, i_max, j_max) in pixel coordinates. Returns (-1, -1, -1, -1) if xy_bbox isn’t intersecting any of the x,y coordinates.

ij_bboxes_from_xy_bboxes(xy_bboxes: ndarray, xy_border: float = 0.0, ij_border: int = 0, ij_bboxes: ndarray = None) → ndarray[source]

Compute bounding boxes in pixel coordinates given bounding boxes xy_bboxes [[x_min, y_min, x_max, y_max], …] in x,y coordinates.

The returned array in i,j pixel coordinates has the same shape as xy_bboxes. The value ranges in the returned array [[i_min, j_min, i_max, j_max], ..]] are:

i_min from 0 to width-1, i_max from 1 to width;
j_min from 0 to height-1, j_max from 1 to height;

so the i,j pixel coordinates can be used as array index slices.

Parameters:

xy_bboxes – Numpy array of x,y bounding boxes [[x_min, y_min, x_max, y_max], …] given in the same CS as x and y.
xy_border – If non-zero, grows the bounding box xy_bbox before using it for comparisons. Defaults to 0.
ij_border – If non-zero, grows the returned i,j bounding box and clips it to size. Defaults to 0.
ij_bboxes – Numpy array of pixel i,j bounding boxes [[x_min, y_min, x_max, y_max], …]. If given, must have same shape as xy_bboxes.

Returns:

Bounding boxes in [[i_min, j_min, i_max, j_max], ..]] in pixel coordinates.

to_dataset_attrs() → Mapping[str, Any][source]

Get spatial dataset attributes as recommended by https://wiki.esipfed.org/Attribute_Convention_for_Data_Discovery_1-3#Recommended

Returns:: dictionary with dataset coordinate attributes.

to_coords(xy_var_names: tuple[str, str] = None, xy_dim_names: tuple[str, str] = None, exclude_bounds: bool = False, reuse_coords: bool = False) → Mapping[str, DataArray][source]

Get CF-compliant axis coordinate variables and cell boundary coordinate variables.

Defined only for grid mappings with regular x,y coordinates.

Parameters:

xy_var_names – Optional coordinate variable names (x_var_name, y_var_name).
xy_dim_names – Optional coordinate dimensions names (x_dim_name, y_dim_name).
exclude_bounds – If True, do not create bounds coordinates. Defaults to False.
reuse_coords – Whether to either reuse target coordinate arrays from target_gm or to compute new ones.

Returns:

dictionary with coordinate variables

Transform this grid mapping so it uses the given spatial coordinate reference system into another crs.

Parameters:

crs – The new spatial coordinate reference system.
xy_res – Optional resolution in x- and y-directions. If given, speeds up the method by avoiding time-consuming spatial resolution estimation.
tile_size – Optional new tile size.
xy_var_names – Optional new coordinate names.
tolerance – Absolute tolerance used when comparing coordinates with each other. Must be in the units of the crs and must be greater zero.

Returns:

A new grid mapping that uses crs.

classmethod regular(size: int | tuple[int, int], xy_min: tuple[float, float], xy_res: float | tuple[float, float], crs: str | CRS, *, tile_size: int | tuple[int, int] = None, is_j_axis_up: bool = False) → GridMapping[source]

Create a new regular grid mapping.

Parameters:

size – Size in pixels.
xy_min – Minimum x- and y-coordinates.
xy_res – Resolution in x- and y-directions.
crs – Spatial coordinate reference system.
tile_size – Optional tile size.
is_j_axis_up – Whether positive j-axis points up. Defaults to false.

Returns:

A new regular grid mapping.

to_regular(tile_size: int | tuple[int, int] = None, is_j_axis_up: bool = False) → GridMapping[source]

Transform this grid mapping into one that is regular.

Parameters:

tile_size – Optional tile size.
is_j_axis_up – Whether positive j-axis points up. Defaults to false.

Returns:

A new regular grid mapping or this grid mapping, if it is already regular.

classmethod from_dataset(dataset: Dataset, *, crs: str | CRS = None, tile_size: int | tuple[int, int] = None, prefer_is_regular: bool = True, prefer_crs: str | CRS = None, emit_warnings: bool = False, tolerance: float = 1e-05) → GridMapping[source]

Create a grid mapping for the given dataset.

Parameters:

dataset – The dataset.
crs – Optional spatial coordinate reference system.
tile_size – Optional tile size
prefer_is_regular – Whether to prefer a regular grid mapping if multiple found. Default is True.
prefer_crs – The preferred CRS of a grid mapping if multiple found.
emit_warnings – Whether to emit warning for non-CF compliant datasets.
tolerance – Absolute tolerance used when comparing coordinates with each other. Must be in the units of the crs and must be greater zero.

Returns:

a new grid mapping instance.

classmethod from_coords(x_coords: DataArray, y_coords: DataArray, crs: str | CRS, *, tile_size: int | tuple[int, int] = None, tolerance: float = 1e-05) → GridMapping[source]

Create a grid mapping from given x- and y-coordinates x_coords, y_coords and spatial coordinate reference system crs.

Parameters:

x_coords – The x-coordinates.
y_coords – The y-coordinates.
crs – The spatial coordinate reference system.
tile_size – Optional tile size.
tolerance – Absolute tolerance used when comparing coordinates with each other. Must be in the units of the crs and must be greater zero.

Returns:

A new grid mapping.

is_close(other: GridMapping, tolerance: float = 1e-05) → bool[source]

Tests whether this grid mapping is close to other.

Parameters:

other – The other grid mapping.
tolerance – Absolute tolerance used when comparing coordinates with each other. Must be in the units of the crs and must be greater zero.

Returns:

True, if so, False otherwise.

Convert a geometry-like object into a shapely geometry object (shapely.geometry.BaseGeometry).

A geometry-like object may be any shapely geometry object, * a dictionary that can be serialized to valid GeoJSON, * a WKT string, * a box given by a string of the form “<x1>,<y1>,<x2>,<y2>”

or by a sequence of four numbers x1, y1, x2, y2,

a point by a string of the form “<x>,<y>” or by a sequence of two numbers x, y.

Handling of geometries crossing the anti-meridian:

If box coordinates are given, it is allowed to pass x1, x2 where x1 > x2, which is interpreted as a box crossing the anti-meridian. In this case the function splits the box along the anti-meridian and returns a multi-polygon.
In all other cases, 2D geometries are assumed to _not cross the anti-meridian at all_.

Parameters:: geometry – A geometry-like object
Returns:: Shapely geometry object or None.

class xcube.core.schema.CubeSchema(shape: Sequence[int], coords: Mapping[str, DataArray], x_name: str = 'lon', y_name: str = 'lat', time_name: str = 'time', dims: Sequence[str] = None, chunks: Sequence[int] = None)[source]

A schema that can be used to create new xcube datasets. The given shape, dims, and chunks, coords apply to all data variables.

Parameters:

shape – A tuple of dimension sizes.
coords – A dictionary of coordinate variables. Must have values for all dims.
dims – A sequence of dimension names. Defaults to ('time', 'lat', 'lon').
chunks – A tuple of chunk sizes in each dimension.

property ndim: int: Number of dimensions.

property dims: tuple[str, ...]: Tuple of dimension names.

property x_name: str: Name of the spatial x coordinate variable.

property y_name: str: Name of the spatial y coordinate variable.

property time_name: str: Name of the time coordinate variable.

property x_var: DataArray: Spatial x coordinate variable.

property y_var: DataArray: Spatial y coordinate variable.

property time_var: DataArray: Time coordinate variable.

property x_dim: str: Name of the spatial x dimension.

property y_dim: str: Name of the spatial y dimension.

property time_dim: str: Name of the time dimension.

property x_size: int: Size of the spatial x dimension.

property y_size: int: Size of the spatial y dimension.

property time_size: int: Size of the time dimension.

property shape: tuple[int, ...]: Tuple of dimension sizes.

property chunks: tuple[int] | None: Tuple of dimension chunk sizes.

property coords: dict[str, DataArray]: Dictionary of coordinate variables.

classmethod new(cube: Dataset) → CubeSchema[source]: Create a cube schema from given cube.

xcube.util.dask.new_cluster(provider: str = 'coiled', name: str | None = None, software: str | None = None, n_workers: int = 4, resource_tags: dict[str, str] | None = None, account: str = None, region: str = 'eu-central-1', **kwargs) → Cluster[source]

Create a new Dask cluster.

Cloud resource tags can be specified in an environment variable XCUBE_DASK_CLUSTER_TAGS in the format tag_1=value_1:tag_2=value_2:...:tag_n=value_n. In case of conflicts, tags specified in resource_tags will override tags specified by the environment variable.

The cluster provider account name can be specified in an environment variable XCUBE_DASK_CLUSTER_ACCOUNT. If the account argument is given to new_cluster, it will override the value from the environment variable.

Parameters:

provider – identifier of the provider to use. Currently, only ‘coiled’ is supported.
name – name to use as an identifier for the cluster
software – identifier for the software environment to be used.
n_workers – number of workers in the cluster
resource_tags – tags to apply to the cloud resources forming the cluster
account – cluster provider account name
**kwargs – further named arguments will be passed on to the cluster creation function
region – default region where workers of the cluster will be deployed set to eu-central-1

Plugin Development

class xcube.util.extension.ExtensionRegistry[source]

A registry of extensions. Typically used by plugins to register extensions.

has_extension(point: str, name: str) → bool[source]

Test if an extension with given point and name is registered.

Parameters:

point – extension point identifier
name – extension name

Returns:

True, if extension exists

get_extension(point: str, name: str) → Extension | None[source]

Get registered extension for given point and name.

Parameters:

point – extension point identifier
name – extension name

Returns:

the extension or None, if no such exists

get_component(point: str, name: str) → Any[source]

Get extension component for given point and name. Raises a ValueError if no such extension exists.

Parameters:

point – extension point identifier
name – extension name

Returns:

extension component

find_extensions(point: str, predicate: Callable[[Extension], bool] = None) → list[Extension][source]

Find extensions for point and optional filter function predicate.

The filter function is called with an extension and should return a truth value to indicate a match or mismatch.

Parameters:

point – extension point identifier
predicate – optional filter function

Returns:

list of matching extensions

find_components(point: str, predicate: Callable[[Extension], bool] = None) → list[Any][source]

Find extension components for point and optional filter function predicate.

The filter function is called with an extension and should return a truth value to indicate a match or mismatch.

Parameters:

point – extension point identifier
predicate – optional filter function

Returns:

list of matching extension components

add_extension(point: str, name: str, component: Any = None, loader: Callable[[Extension], Any] = None, **metadata) → Extension[source]

Register an extension component or an extension component loader for the given extension point, name, and additional metadata.

Either component or loader must be specified, but not both.

A given loader must be a callable with one positional argument extension of type Extension and is expected to return the actual extension component, which may be of any type. The loader will only be called once and only when the actual extension component is requested for the first time. Consider using the function import_component() to create a loader that lazily imports a component from a module and optionally executes it.

Parameters:

point – extension point identifier
name – extension name
component – extension component
loader – extension component loader function
**metadata – extension metadata

Returns:

a registered extension

remove_extension(point: str, name: str)[source]

Remove registered extension name from given point.

Parameters:

point – extension point identifier
name – extension name

to_dict()[source]: Get a JSON-serializable dictionary representation of this extension registry.

class xcube.util.extension.Extension(point: str, name: str, component: Any = None, loader: Callable[[Extension], Any] = None, **metadata)[source]

An extension that provides a component of any type.

Extensions are registered in a ExtensionRegistry.

Extension objects are not meant to be instantiated directly. Instead, ExtensionRegistry#add_extension() is used to register extensions.

Parameters:

point – extension point identifier
name – extension name
component – extension component
loader – extension component loader function
metadata – extension metadata

property is_lazy: bool: Whether this is a lazy extension that uses a loader.

property component: Any: Extension component.

property point: str: Extension point identifier.

property name: str: Extension name.

property metadata: dict[str, Any]: Extension metadata.

to_dict() → dict[str, Any][source]: Get a JSON-serializable dictionary representation of this extension.

xcube.util.extension.import_component(spec: str, transform: Callable[[Any, Extension], Any] = None, call: bool = False, call_args: Sequence[Any] = None, call_kwargs: Mapping[str, Any] = None) → Callable[[Extension], Any][source]

Return a component loader that imports a module or module component from spec. To import a module, spec should be the fully qualified module name. To import a component, spec must also append the component name to the fully qualified module name separated by a color (“:”) character.

An optional transform callable my be used to transform the imported component. If given, a new component is computed:

component = transform(component, extension)

If the call flag is set, the component is expected to be a callable which will be called using the given call_args and call_kwargs to produce a new component:

component = component(*call_kwargs, **call_kwargs)

Finally, the component is returned.

Parameters:

spec – String of the form “module_path” or “module_path:component_name”
transform – callable that takes two positional arguments, the imported component and the extension of type Extension
call – Whether to finally call the component with given call_args and call_kwargs
call_args – arguments passed to a callable component if call flag is set
call_kwargs – keyword arguments passed to callable component if call flag is set

Returns:

a component loader

xcube.constants.EXTENSION_POINT_INPUT_PROCESSORS = 'xcube.core.gen.iproc': The extension point identifier for input processor extensions

xcube.constants.EXTENSION_POINT_DATASET_IOS = 'xcube.core.dsio': The extension point identifier for dataset I/O extensions

xcube.constants.EXTENSION_POINT_CLI_COMMANDS = 'xcube.cli': The extension point identifier for CLI command extensions

xcube.util.plugin.get_extension_registry() → ExtensionRegistry[source]: Get populated extension registry.

xcube.util.plugin.get_plugins() → dict[str, dict][source]: Get mapping of “xcube_plugins” entry point names to JSON-serializable plugin meta-information.