Python API¶
Cube I/O¶
Cube generation¶
-
xcube.core.new.
new_cube
(title='Test Cube', width=360, height=180, x_name='lon', y_name='lat', x_dtype='float64', y_dtype=None, x_units='degrees_east', y_units='degrees_north', x_res=1.0, y_res=None, x_start=-180.0, y_start=-90.0, inverse_y=False, time_name='time', time_dtype='datetime64[s]', time_units='seconds since 1970-01-01T00:00:00', time_calendar='proleptic_gregorian', time_periods=5, time_freq='D', time_start='2010-01-01T00:00:00', drop_bounds=False, variables=None)¶ Create a new empty cube. Useful for creating cubes templates with predefined coordinate variables and metadata. The function is also heavily used by xcube’s unit tests.
The values of the variables dictionary can be either constants, array-like objects, or functions that compute their return value from passed coordinate indexes. The expected signature is::
def my_func(time: int, y: int, x: int) -> Union[bool, int, float]
- Parameters
title – A title. Defaults to ‘Test Cube’.
width – Horizontal number of grid cells. Defaults to 360.
height – Vertical number of grid cells. Defaults to 180.
x_name – Name of the x coordinate variable. Defaults to ‘lon’.
y_name – Name of the y coordinate variable. Defaults to ‘lat’.
x_dtype – Data type of x coordinates. Defaults to ‘float64’.
y_dtype – Data type of y coordinates. Defaults to ‘float64’.
x_units – Units of the x coordinates. Defaults to ‘degrees_east’.
y_units – Units of the y coordinates. Defaults to ‘degrees_north’.
x_start – Minimum x value. Defaults to -180.
y_start – Minimum y value. Defaults to -90.
x_res – Spatial resolution in x-direction. Defaults to 1.0.
y_res – Spatial resolution in y-direction. Defaults to 1.0.
inverse_y – Whether to create an inverse y axis. Defaults to False.
time_name – Name of the time coordinate variable. Defaults to ‘time’.
time_periods – Number of time steps. Defaults to 5.
time_freq – Duration of each time step. Defaults to `1D’.
time_start – First time value. Defaults to ‘2010-01-01T00:00:00’.
time_dtype – Numpy data type for time coordinates. Defaults to ‘datetime64[s]’.
time_units – Units for time coordinates. Defaults to ‘seconds since 1970-01-01T00:00:00’.
time_calendar – Calender for time coordinates. Defaults to ‘proleptic_gregorian’.
drop_bounds – If True, coordinate bounds variables are not created. Defaults to False.
variables – Dictionary of data variables to be added. None by default.
- Returns
A cube instance
Cube computation¶
Cube data extraction¶
Cube manipulation¶
-
xcube.core.unchunk.
unchunk_dataset
(dataset_path: str, var_names: Sequence[str] = None, coords_only: bool = False)¶ Unchunk dataset variables in-place.
- Parameters
dataset_path – Path to ZARR dataset directory.
var_names – Optional list of variable names.
coords_only – Un-chunk coordinate variables only.
-
xcube.core.optimize.
optimize_dataset
(input_path: str, output_path: str = None, in_place: bool = False, unchunk_coords: bool = False, exception_type: Type[Exception] = <class 'ValueError'>)¶ Optimize a dataset for faster access.
Reduces the number of metadata and coordinate data files in xcube dataset given by given by dataset_path. Consolidated cubes open much faster from remote locations, e.g. in object storage, because obviously much less HTTP requests are required to fetch initial cube meta information. That is, it merges all metadata files into a single top-level JSON file “.zmetadata”. If unchunk_coords is set, it also removes any chunking of coordinate variables so they comprise a single binary data file instead of one file per data chunk. The primary usage of this function is to optimize data cubes for cloud object storage. The function currently works only for data cubes using ZARR format.
- Parameters
input_path – Path to input dataset with ZARR format.
output_path – Path to output dataset with ZARR format. May contain “{input}” template string, which is replaced by the input path’s file name without file name extentsion.
in_place – Whether to modify the dataset in place. If False, a copy is made and output_path must be given.
unchunk_coords – Whether to also consolidate coordinate chunk files.
exception_type – Type of exception to be used on value errors.
Cube subsetting¶
-
xcube.core.select.
select_vars
(dataset: xarray.Dataset, var_names: Collection[str] = None) → xarray.Dataset¶ Select data variable from given dataset and create new dataset.
- Parameters
dataset – The dataset from which to select variables.
var_names – The names of data variables to select.
- Returns
A new dataset. It is empty, if var_names is empty. It is dataset, if var_names is None.
Cube masking¶
-
class
xcube.core.maskset.
MaskSet
(flag_var: xarray.DataArray)¶ A set of mask variables derived from a variable flag_var with CF attributes “flag_masks” and “flag_meanings”.
Each mask is represented by an xarray.DataArray and has the name of the flag, is of type numpy.unit8, and has the dimensions of the given flag_var.
- Parameters
flag_var – an xarray.DataArray that defines flag values. The CF attributes “flag_masks” and “flag_meanings” are expected to exists and be valid.
-
classmethod
get_mask_sets
(dataset: xarray.Dataset) → Dict[str, xcube.core.maskset.MaskSet]¶ For each “flag” variable in given dataset, turn it into a
MaskSet
, store it in a dictionary.- Parameters
dataset – The dataset
- Returns
A mapping of flag names to
MaskSet
. Will be empty if there are no flag variables in dataset.
Rasterisation of Features¶
Cube metadata¶
Cube verification¶
Multi-resolution pyramids¶
Utilities¶
-
class
xcube.core.store.
CubeStore
(dims: Sequence[str], shape: Sequence[int], chunks: Sequence[int], attrs: Dict[str, Any] = None, get_chunk: Callable[[CubeStore, str, Tuple[int, ...]], bytes] = None, trace_store_calls: bool = False)¶ A Zarr Store that generates data cubes by allowing data variables to fetch or compute their chunks by a user-defined function get_chunk. Implements the standard Python
MutableMapping
interface.This is how the get_chunk function is called::
data = get_chunk(cube_store, var_name, chunk_indexes)
where
cube_store
is this store,var_name
is the name of the variable for which data is fetched, andchunk_indexes
is a tuple of zero-based, integer chunk indexes. The result must be a Python bytes object.- Parameters
dims – Dimension names of all data variables, e.g. (‘time’, ‘lat’, ‘lon’).
shape – Shape of all data variables according to dims, e.g. (512, 720, 1480).
chunks – Chunk sizes of all data variables according to dims, e.g. (128, 180, 180).
attrs – Global dataset attributes.
get_chunk – Default chunk fetching/computing function.
trace_store_calls – Whether to print calls into the
MutableMapping
interface.
-
keys
() → a set-like object providing a view on D's keys¶
-
class
xcube.core.schema.
CubeSchema
(shape: Sequence[int], coords: Mapping[str, numpy.array], x_name: str = 'lon', y_name: str = 'lat', time_name: str = 'time', dims: Sequence[str] = None, chunks: Sequence[int] = None)¶ A schema that can be used to create new xcube datasets. The given shape, dims, and chunks, coords apply to all data variables.
- Parameters
shape – A tuple of dimension sizes.
coords – A dictionary of coordinate variables. Must have values for all dims.
dims – A sequence of dimension names. Defaults to
('time', 'lat', 'lon')
.chunks – A tuple of chunk sizes in each dimension.
-
property
ndim
¶ Number of dimensions.
-
property
dims
¶ Tuple of dimension names.
-
property
x_name
¶ Name of the spatial x coordinate variable.
-
property
y_name
¶ Name of the spatial y coordinate variable.
-
property
time_name
¶ Name of the time coordinate variable.
-
property
x_var
¶ Spatial x coordinate variable.
-
property
y_var
¶ Spatial y coordinate variable.
-
property
time_var
¶ Time coordinate variable.
-
property
x_dim
¶ Name of the spatial x dimension.
-
property
y_dim
¶ Name of the spatial y dimension.
-
property
time_dim
¶ Name of the time dimension.
-
property
shape
¶ Tuple of dimension sizes.
-
property
chunks
¶ Tuple of dimension chunk sizes.
-
property
coords
¶ Dictionary of coordinate variables.
-
classmethod
new
(cube: xarray.Dataset) → xcube.core.schema.CubeSchema¶ Create a cube schema from given cube.
Plugin Development¶
-
class
xcube.util.extension.
ExtensionRegistry
¶ A registry of extensions. Typically used by plugins to register extensions.
-
has_extension
(point: str, name: str) → bool¶ Test if an extension with given point and name is registered.
- Parameters
point – extension point identifier
name – extension name
- Returns
True, if extension exists
-
get_extension
(point: str, name: str) → Optional[xcube.util.extension.Extension]¶ Get registered extension for given point and name.
- Parameters
point – extension point identifier
name – extension name
- Returns
the extension or None, if no such exists
-
get_component
(point: str, name: str) → Any¶ Get extension component for given point and name. Raises a ValueError if no such extension exists.
- Parameters
point – extension point identifier
name – extension name
- Returns
extension component
-
find_extensions
(point: str, predicate: Callable[[Extension], bool] = None) → List[xcube.util.extension.Extension]¶ Find extensions for point and optional filter function predicate.
The filter function is called with an extension and should return a truth value to indicate a match or mismatch.
- Parameters
point – extension point identifier
predicate – optional filter function
- Returns
list of matching extensions
-
find_components
(point: str, predicate: Callable[[Extension], bool] = None) → List[Any]¶ Find extension components for point and optional filter function predicate.
The filter function is called with an extension and should return a truth value to indicate a match or mismatch.
- Parameters
point – extension point identifier
predicate – optional filter function
- Returns
list of matching extension components
-
add_extension
(point: str, name: str, component: Any = None, loader: Callable[[Extension], Any] = None, **metadata) → xcube.util.extension.Extension¶ Register an extension component or an extension component loader for the given extension point, name, and additional metadata.
Either component or loader must be specified, but not both.
A given loader must be a callable with one positional argument extension of type
Extension
and is expected to return the actual extension component, which may be of any type. The loader will only be called once and only when the actual extension component is requested for the first time. Consider using the functionimport_component()
to create a loader that lazily imports a component from a module and optionally executes it.- Parameters
point – extension point identifier
name – extension name
component – extension component
loader – extension component loader function
metadata – extension metadata
- Returns
a registered extension
-
remove_extension
(point: str, name: str)¶ Remove registered extension name from given point.
- Parameters
point – extension point identifier
name – extension name
-
-
class
xcube.util.extension.
Extension
(point: str, name: str, component: Any = None, loader: Callable[[Extension], Any] = None, **metadata)¶ An extension that provides a component of any type.
Extensions are registered in a
ExtensionRegistry
.Extension objects are not meant to be instantiated directly. Instead,
ExtensionRegistry.add_extension()
is used to register extensions.- Parameters
point – extension point identifier
name – extension name
component – extension component
loader – extension component loader function
metadata – extension metadata
-
property
is_lazy
¶ Whether this is a lazy extension that uses a loader.
-
property
component
¶ Extension component.
-
property
point
¶ Extension point identifier.
-
property
name
¶ Extension name.
-
property
metadata
¶ Extension metadata.
-
xcube.util.extension.
import_component
(spec: str, transform: Callable[[Any, Extension], Any] = None, call: bool = False, call_args: Sequence[Any] = None, call_kwargs: Mapping[str, Any] = None) → Callable[[xcube.util.extension.Extension], Any]¶ Return a component loader that imports a module or module component from spec. To import a module, spec should be the fully qualified module name. To import a component, spec must also append the component name to the fully qualified module name separated by a color (“:”) character.
An optional transform callable my be used to transform the imported component. If given, a new component is computed:
component = transform(component, extension)
If the call flag is set, the component is expected to be a callable which will be called using the given call_args and call_kwargs to produce a new component:
component = component(*call_kwargs, **call_kwargs)
Finally, the component is returned.
- Parameters
spec – String of the form “module_path” or “module_path:component_name”
transform – callable that takes two positional arguments, the imported component and the extension of type
Extension
call – Whether to finally call the component with given call_args and call_kwargs
call_args – arguments passed to a callable component if call flag is set
call_kwargs – keyword arguments passed to callable component if call flag is set
- Returns
a component loader
-
xcube.constants.
EXTENSION_POINT_INPUT_PROCESSORS
= 'xcube.core.gen.iproc'¶ The extension point identifier for input processor extensions
-
xcube.constants.
EXTENSION_POINT_DATASET_IOS
= 'xcube.core.dsio'¶ The extension point identifier for dataset I/O extensions
-
xcube.constants.
EXTENSION_POINT_CLI_COMMANDS
= 'xcube.cli'¶ The extension point identifier for CLI command extensions
-
xcube.util.plugin.
get_extension_registry
()¶ Get populated extension registry.
-
xcube.util.plugin.
get_plugins
() → Dict[str, Dict]¶ Get mapping of “xcube_plugins” entry point names to JSON-serializable plugin meta-information.