WDF File I/O / WDFファイル入出力

The generated save and load docstrings are the public WDF API contract. They describe typed Frame data, metadata, and lazy-load behavior. WDF stores a concrete Frame result, not a replayable Recipe; use the Recipe How-to when the artifact should replay operations on another input. Saving and loading WDF need the io extra (pip install "wandas[io]").

生成されたsave／load docstringが、公開WDF API契約の正本です。具体的なFrame結果を保存する WDFと、別入力へ処理を再実行するRecipeの使い分けはRecipe How-to を参照してください。WDFの保存と読み込みにはio extra（pip install "wandas[io]"）が必要です。 wandas/io/の実装不変条件はI/O Contractsで管理します。

wandas.io.wdf_io.save(frame, path, *, compress='gzip', overwrite=False)

Save an exact built-in Frame as WDF 0.4, 0.5 or sparse 0.6.

A time-frequency Frame with a known physical frame origin uses WDF 0.5. Sparse time-frequency results use WDF 0.6 with explicit sparse state. All other Frames retain WDF 0.4. Older readers reject new versions explicitly; this preserves placement instead of silently discarding its additional state.

The artifact retains the Frame label, user metadata, channel labels and metadata (units, references, calibration, and channel extras), source-time offsets, analysis coordinates, and the derived operation_history view. Metadata fields use strict JSON encoding and therefore follow JSON value semantics (for example, a tuple is loaded as a list). It stores the concrete Frame result; previous references and replayable Recipe intent are not persisted.

Dask data is handed directly to xarray and is written synchronously; Wandas does not first materialize the complete tensor with frame._data.compute(). The frame is written to a temporary file first, so a failure while computing or writing it leaves an existing file unchanged and creates no partial file. A new file is then moved into place. An existing file is overwritten in place by copying the finished temporary file into it, so it keeps its hard links, owner, group, permission bits and ACLs, and a symlinked path updates the file it points to. Overwriting a file that is not writable, or an HDF5 file that is open or locked, raises OSError before anything is computed. An interruption of that final copy (for example a full disk or a power loss), or another process opening the file between that check and the copy, can still leave the existing file incomplete.

Saving needs the io extra (pip install "wandas[io]"); without it, ImportError names that install command.

Source code in wandas/io/wdf_io.py

def save(
    frame: BaseFrame[Any],
    path: str | Path,
    *,
    compress: str | None = "gzip",
    overwrite: bool = False,
) -> None:
    """Save an exact built-in Frame as WDF 0.4, 0.5 or sparse 0.6.

    A time-frequency Frame with a known physical frame origin uses WDF 0.5.
    Sparse time-frequency results use WDF 0.6 with explicit sparse state.
    All other Frames retain WDF 0.4. Older readers reject new versions explicitly; this
    preserves placement instead of silently discarding its additional state.

    The artifact retains the Frame label, user metadata, channel labels and
    metadata (units, references, calibration, and channel extras), source-time
    offsets, analysis coordinates, and the derived ``operation_history`` view.
    Metadata fields use strict JSON encoding and therefore follow JSON value
    semantics (for example, a tuple is loaded as a list).
    It stores the concrete Frame result; ``previous`` references and replayable
    Recipe intent are not persisted.

    Dask data is handed directly to xarray and is written synchronously; Wandas does
    not first materialize the complete tensor with ``frame._data.compute()``.
    The frame is written to a temporary file first, so a failure while computing
    or writing it leaves an existing file unchanged and creates no partial file.
    A new file is then moved into place. An existing file is overwritten in place
    by copying the finished temporary file into it, so it keeps its hard links,
    owner, group, permission bits and ACLs, and a symlinked path updates the file
    it points to. Overwriting a file that is not writable, or an HDF5 file that is
    open or locked, raises ``OSError`` before anything is computed. An
    interruption of that final copy (for example a full disk or a power loss), or
    another process opening the file between that check and the copy, can still
    leave the existing file incomplete.

    Saving needs the ``io`` extra (``pip install "wandas[io]"``); without it,
    ``ImportError`` names that install command.
    """
    target = _normalized_path(path)
    if target.exists() and not overwrite:
        raise FileExistsError(f"File {target} already exists. Set overwrite=True to overwrite.")

    # Validate all Frame and JSON state before either importing the storage backend
    # or opening the destination, so invalid state cannot leave a partial artifact.
    dataset = _build_dataset(frame)
    require_h5netcdf("WDF save")
    encoding = (
        {"data": {"compression": compress, "chunksizes": _data_chunksizes(dataset["data"])}} if compress else None
    )
    # Lazy data is computed while writing, so the frame is written to a temporary
    # file first: a failure leaves an existing artifact intact and no partial file.
    destination = Path(os.path.realpath(target))
    if not destination.exists():
        # A fixed-length name stays within NAME_MAX even for a maximal destination name.
        temporary = destination.with_name(f".wdf-{uuid.uuid4().hex}.tmp")
        os.close(os.open(temporary, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o666))
        try:
            _write_dataset(dataset, temporary, encoding)
            os.replace(temporary, destination)
        except BaseException:
            temporary.unlink(missing_ok=True)
            raise
        return

    # Copy the finished file into the existing one so it keeps its inode: hard
    # links, owner, group, permission bits and ACLs stay as they were.
    # Opening it for writing without truncation reports a read-only destination
    # before the frame is computed.
    os.close(os.open(destination, os.O_WRONLY))
    _require_hdf5_not_open(destination)
    try:
        handle, name = tempfile.mkstemp(prefix=".wdf-", suffix=".tmp", dir=destination.parent)
    except PermissionError:
        handle, name = tempfile.mkstemp(prefix="wdf-", suffix=".tmp")
    os.close(handle)
    temporary = Path(name)
    try:
        _write_dataset(dataset, temporary, encoding)
        shutil.copyfile(temporary, destination)
    finally:
        temporary.unlink(missing_ok=True)

wandas.io.wdf_io.load(path)

Load a local WDF 0.4, 0.5 or 0.6 artifact as its exact built-in Frame type.

WDF 0.4 time-frequency Frames have unknown physical center times; their existing zero-based time axes remain available. WDF 0.5 restores the explicit physical origin without inferring it from history. WDF 0.6 also restores the sparse-window permission, retaining gaps in the time axis.

The returned Frame restores the saved label, user metadata, channel labels and metadata (units, references, calibration, and channel extras), source-time offsets, analysis coordinates, and derived operation_history view. Metadata follows strict JSON value semantics; for example, a tuple saved in metadata is loaded as a list. WDF does not restore a previous reference or replayable Recipe intent.

The returned Frame owns access to its source internally. Keep the source path unchanged while that Frame or Frames derived from it are in use. Obtain NumPy values through frame.data without managing the storage backend directly.

Loading needs the io extra (pip install "wandas[io]"); without it, ImportError names that install command.

Source code in wandas/io/wdf_io.py

def load(path: str | Path) -> BaseFrame[Any]:
    """Load a local WDF 0.4, 0.5 or 0.6 artifact as its exact built-in Frame type.

    WDF 0.4 time-frequency Frames have unknown physical center times; their
    existing zero-based time axes remain available. WDF 0.5 restores the
    explicit physical origin without inferring it from history. WDF 0.6 also
    restores the sparse-window permission, retaining gaps in the time axis.

    The returned Frame restores the saved label, user metadata, channel labels
    and metadata (units, references, calibration, and channel extras),
    source-time offsets, analysis coordinates, and derived
    ``operation_history`` view. Metadata follows strict JSON value semantics;
    for example, a tuple saved in metadata is loaded as a list. WDF does not
    restore a ``previous`` reference or replayable Recipe intent.

    The returned Frame owns access to its source internally. Keep the source path
    unchanged while that Frame or Frames derived from it are in use. Obtain NumPy
    values through ``frame.data`` without managing the storage backend directly.

    Loading needs the ``io`` extra (``pip install "wandas[io]"``); without it,
    ``ImportError`` names that install command.
    """
    source = Path(path)
    if not source.exists():
        raise FileNotFoundError(f"File not found: {source}")
    require_h5netcdf("WDF load")

    # CF decoding is disabled because WDF owns dtype/value semantics; foreign CF
    # scale, offset, fill-value, and time attributes must never transform raw data.
    dataset = xr.open_dataset(
        source,
        engine="h5netcdf",
        chunks={},
        decode_cf=False,
        mask_and_scale=False,
        backend_kwargs={"phony_dims": "access"},
    )
    version = dataset.attrs.get("version")
    if version not in (WDF_FORMAT_VERSION, _FRAME_TIME_FORMAT_VERSION, _SPARSE_FORMAT_VERSION):
        got = "missing" if version is None else repr(version)
        raise ValueError(
            "Unsupported WDF format version\n"
            f"  Got: {got}\n"
            f"  Supported: {WDF_FORMAT_VERSION!r}, {_FRAME_TIME_FORMAT_VERSION!r}, {_SPARSE_FORMAT_VERSION!r}\n"
            "Use a compatible Wandas version or resave the file."
        )
    _require_exact_schema(dataset)

    frame_type = dataset.attrs["frame_type"]
    if not isinstance(frame_type, str):
        raise ValueError("Invalid WDF frame_type; expected text")
    constructor = _load_json(dataset.attrs["constructor_json"], field="constructor_json")
    has_origin = isinstance(constructor, dict) and "frame_time_origin" in constructor
    has_sparse = isinstance(constructor, dict) and "allow_sparse" in constructor
    if version == _SPARSE_FORMAT_VERSION:
        if frame_type not in {"SpectrogramFrame", "CepstrogramFrame"} or not has_sparse:
            raise ValueError("Invalid WDF 0.6 sparse schema; expected sparse time-frequency state")
    elif has_sparse:
        raise ValueError("WDF sparse state requires format version 0.6")
    if version == _FRAME_TIME_FORMAT_VERSION:
        if frame_type not in {"SpectrogramFrame", "CepstrogramFrame"} or not has_origin:
            raise ValueError("Invalid WDF 0.5 physical frame-time schema; expected an explicit time-frequency origin")
    elif has_origin and version != _SPARSE_FORMAT_VERSION:
        raise ValueError("WDF physical frame-time state requires format version 0.5")
    metadata = _load_json(dataset.attrs["metadata_json"], field="metadata_json")
    label = _load_json(dataset.attrs["label"], field="label")
    history = _validate_history(_load_json(dataset.attrs["operation_history_json"], field="operation_history_json"))
    if not isinstance(constructor, Mapping):
        raise ValueError("Invalid WDF constructor_json; expected an object")
    if not isinstance(metadata, dict):
        raise ValueError("Invalid WDF metadata_json; expected an object")
    if label is not None and not isinstance(label, str):
        raise ValueError("Invalid WDF label; expected a string or null")
    sampling_rate = _finite_number(dataset.attrs["sampling_rate"], field="sampling_rate")

    data = dataset["data"]
    channel_count = int(dataset.sizes["channel"])
    channel_ids = _text_vector(dataset, "channel", channel_count)
    labels = _text_vector(dataset, "channel_label", channel_count)
    units = _text_vector(dataset, "channel_unit", channel_count)
    refs = _number_vector(dataset, "channel_ref", channel_count)
    factors = _number_vector(dataset, "channel_calibration_factor", channel_count)
    offsets = _number_vector(dataset, "source_time_offset", channel_count)
    extras_json = _text_vector(dataset, "channel_extra_json", channel_count)
    extras = [_load_json(value, field=f"channel_extra_json[{index}]") for index, value in enumerate(extras_json)]
    if not all(isinstance(extra, dict) for extra in extras):
        raise ValueError("Invalid WDF channel_extra_json; expected JSON objects")
    channels = [
        ChannelMetadata(
            label=labels[index],
            calibration=ChannelCalibration(factor=float(factors[index]), unit=units[index], ref=float(refs[index])),
            extra=extras[index],
        )
        for index in range(channel_count)
    ]
    common: dict[str, Any] = {
        "sampling_rate": sampling_rate,
        "label": label,
        "metadata": metadata,
        "channel_metadata": channels,
        "channel_ids": channel_ids,
        "source_time_offset": offsets,
        "operation_history_prefix": history,
    }
    frame = decode_frame(
        frame_type,
        constructor,
        data=data.data,
        common=common,
        stored_dims=cast(tuple[str, ...], tuple(data.dims)),
    )
    coordinates = {
        str(name): np.asarray(coordinate.values) for name, coordinate in dataset.coords.items() if name != "channel"
    }
    restore_frame_coordinates(frame, coordinates)
    return frame