Icechunk

The Icechunk key-value store adapter provides access to an Icechunk repository containing a Zarr v3 hierarchy.

URL syntax

  • base-kvstore-url/|icechunk:path/to/array/

  • base-kvstore-url/|icechunk:@branch.BRANCH/path/to/array/

  • base-kvstore-url/|icechunk:@tag.TAG/path/to/array/

  • base-kvstore-url/|icechunk:@SNAPSHOT/path/to/array/

The base-kvstore-url/ must refer to a directory.

  • The icechunk:path/to/array/ syntax is equivalent to icechunk:@branch.main/path/to/array/.

  • The icechunk:@branch.BRANCH/path/to/array/ syntax indicates the latest version of the specified BRANCH.

  • The icechunk:@tag.TAG/path/to/array/ syntax indicates a specific tag.

  • The icechunk:@SNAPSHOT/path/to/array/ syntax indicates a specific snapshot.

Currently, icechunk can only store Zarr v3 hierarchies. Therefore, it is always used in conjunction with the Zarr data format driver.

For example:

  • gs://bucket/path/to/repo.zarr.icechunk/|icechunk:|zarr3:path/to/array/

  • gs://bucket/path/to/repo.zarr.icechunk/|icechunk:@branch.other/|zarr3:path/to/array/

  • gs://bucket/path/to/repo.zarr.icechunk/|icechunk:@tag.v5/|zarr3:path/to/array/

  • gs://bucket/path/to/repo.zarr.icechunk/|icechunk:@4N0217AZA4VNPYD0HR0G/|zarr3:path/to/array/

Note

Consistent with normal URL syntax, any special characters in the path/to/array/, including @ which is used to specify a ref, must be percent-encoded.

Capabilities

Supported capabilities

Byte range reads

Supported.

Key listing

Supported.
Required capabilities of base key-value store

Byte range reads

Required.

Key listing

Required when a tag or snapshot is not specified explicitly.

Auto detection

Directories containing Icechunk repositories are detected automatically based on the presence of the refs and snapshots sub-directories, and by the presence of the refs/branch.main/ZZZZZZZZ.json file.