xcube Developer Guide¶
Version 0.1, draft
IMPORTANT NOTE: Any changes to this doc must be reviewed by dev-team through pull requests.
Preface¶
Gedacht ist nicht gesagt.
Gesagt ist nicht gehört.
Gehört ist nicht verstanden.
Verstanden ist nicht einverstanden.
Einverstanden ist nicht umgesetzt.
Umgesetzt ist nicht beibehalten.
by Konrad Lorenz (translation is left to the reader)
Table of Contents¶
Versioning¶
We adhere to PEP-440.
The current software version is in xcube/version.py
.
While developing a version, we append version suffix .dev<N>
.
Before the release, we remove the suffix.
Main Packages¶
xcube.cli
- Here live CLI commands that are required by someone. CLI command implementations should be lightweight. Move implementation code either intoapi
orutil
.
CLI commands must be maintained w.r.t. backward compatibility. Therefore think twice before adding new or change existing CLI commands.xcube.api
- Here live API functions that are required by someone or that exists because a CLI command is implemented here. API code must be maintained w.r.t. backward compatibility. Therefore think twice before adding new or change existing API.xcube.webapi
- Here live Web API functions that are required by someone. Web API command implementations should be lightweight. Move implementation code either intoapi
orutil
.
Web API interface must be maintained w.r.t. backward compatibility. Therefore think twice before adding new or change existing API.xcube.util
- Mainly implementation helpers. Comprises classes and functions that are used bycli
,api
,webapi
in order to maximize modularisation and testability but to minimize code duplication.
The code in here must not be dependent on any ofcli
,api
,webapi
. The code in here may change often and in any way as desired by code implementing thecli
,api
,webapi
packages.
The following sections will guide you through extending or changing the main packages that form xcube’s public interface.
Package xcube.cli
¶
Checklist¶
Make sure your change
is covered by unit-tests (package
test/api
);is reflected by the CLI’s doc-strings and tools documentation (currently in
README.md
);follows existing xcube CLI conventions;
follows PEP8 conventions;
is reflected in API and WebAPI, if desired;
is reflected in
CHANGES.md
.
Hints¶
Make sure your new CLI command is in line with the others commands regarding command name, option names, as well as metavar arguments names. The CLI command name shall ideally be a verb.
Avoid introducing new option arguments if similar options are already in use for existing commands.
In the following common arguments and options are listed.
Input argument:
@click.argument('input')
If input argument is restricted to an xcube dataset:
@click.argument('cube')
Output (directory) option:
@click.option('--output', '-o', metavar='OUTPUT',
help='Output directory. If omitted, "INPUT.levels" will be used.')
Output format:
@click.option('--format', '-f', metavar='FORMAT', type=click.Choice(['zarr', 'netcdf']),
help="Format of the output. If not given, guessed from OUTPUT.")
Output parameters:
@click.option('--param', '-p', metavar='PARAM', multiple=True,
help="Parameter specific for the output format. Multiple allowed.")
Variable names:
@click.option('--variable',--var', metavar='VARIABLE', multiple=True,
help="Name of a variable. Multiple allowed.")
For parsing CLI inputs, use helper functions that are already in use.
In the CLI command implementation code, raise click.ClickException(message)
with a clear message
for users.
Common xcube CLI options like -f
for FORMAT should be lower case letters and
specific xcube CLI options like -S
for SIZE in xcube gen
are recommended to be uppercase letters.
Extensively validate CLI inputs to avoid that API functions raise
ValueError
, TypeError
, etc. Such errors and their message texts are
usually hard to understand by users. They are actually dedicated to
to developers, not CLI users.
There is a global option --traceback
flag that user can set to dump stack traces.
You don’t need to print stack traces from your code.
Package xcube.api
¶
Checklist¶
Make sure your change
is covered by unit-tests (package
test/api
);is covered by API documentation;
follows existing xcube API conventions;
follows PEP8 conventions;
is reflected in xarray extension class
xcube.api.api.API
;is reflected in CLI and WebAPI if desired;
is reflected in
CHANGES.md
.
Hints¶
Create new module in xcube.api
and add your functions.
For any functions added make sure naming is in line with other API.
Add clear doc-string to the new API. Use Sphinx RST format.
Decide if your API methods requires xcube datasets as inputs,
if so, name the primary dataset argument cube
and add a
keyword parameter cube_asserted: bool = False
.
Otherwise name the primary dataset argument dataset
.
Reflect the fact, that a certain API method or function operates only
on datasets that conform with the xcube dataset specifications by
using cube
in its name rather than dataset
. For example compute_dataset
can operate on any xarray datasets, while get_cube_values_for_points
expects a
xcube dataset as input or read_cube
ensures it will return valid xcube datasets only.
In the implementation, if not cube_asserted
,
we must assert and verify the cube
is a cube.
Pass True
to cube_asserted
argument of other API called later on:
from .verify import assert_cube
def frombosify_cube(cube: xr.Dataset, ..., cube_asserted: bool = False):
if not cube_asserted:
assert_cube(cube)
...
result = bibosify_cube(cube, ..., cube_asserted=True)
...
If import xcube.api
is used in client code, any xarray.Dataset
object will have
an extra property xcube
whose interface is defined by the class
xcube.api.XCubeAPI
. This class is an
xarray extension that is
used to reflect xcube.api
functions and make it directly applicable to the xarray.Dataset
object.
Therefore any xcube API shall be reflected in this extension class.
Package xcube.webapi
¶
Checklist¶
Make sure your change
is covered by unit-tests (package
test/webapi
);is covered by Web API specification and documentation (currently in
webapi/res/openapi.yml
);follows existing xcube Web API conventions;
follows PEP8 conventions;
is reflected in CLI and API, if desired;
is reflected in
CHANGES.md
.
Hints¶
The Web API is defined in
webapi.app
which defines mapping from resource URLs to handlersAll handlers are implemented in
webapi.handlers
. Handler code just delegates to dedicated controllers.All controllers are implemented in
webapi.controllers.*
. They might further delegate intoapi.*
Development Process¶
Make sure there is an issue ticket for your code change work item
Select issue, priorities are as follows
“urgent” and (“important” and “bug”)
“urgent” and (“important” or “bug”)
“urgent”
“important” and “bug”
“important” or “bug”
others
Make sure issue is assigned to you, if unclear agree with team first.
Add issue label “in progress”.
Create development branch named “developer-issue#-title”.
Develop, having in mind the checklists and implementation hints above.
In your first commit, refer the issue so it will appear as link in the issue history
Develop, test, and push to the remote branch as desired.
In your last commit, utilize checklists above. (You can include the line “closes #
” in your commit message to auto-close the issue once the PR is merged.)
Create PR if build servers succeed on your branch. If not, fix issue first.
For the PR assign the team for review, agree who is to merge. Also reviewers must have checklist in mind!Merge PR after all reviewers are accepted your change. Otherwise go back.
Remove issue label “in progress”.
Delete the development branch “developer-issue#-title”.
If the PR is only partly solving an issue:
Make sure the issue contains a to-do list (checkboxes) to complete the issue.
Do not include the line “closes #
” in your last commit message. Add “relates to issue#” in PR.
Make sure to check the corresponding to-do items (checkboxes) after the PR is merged.
Remove issue label “in progress”.
Leave issue open.