Repository navigation
Conversation
open_mfdataset, too. Also, further improves docstrings for those functions. Also, removes `if grid.source_grid_spec == "Structured"` branch in `_map_dataset_to_ugrid`.
There was a problem hiding this comment.
🟡 Changes recommended
Active issues remain, including a critical unguarded dimension-mapping path that can raise UnboundLocalError, an invalid xarray.open_mf_dataset API reference, and misleading API documentation.
3 open issues
What changed in this PR
This pull request clarifies that ux.open_dataset and ux.open_mfdataset are intended for unstructured data, warns about forthcoming structured-input deprecation, and updates related documentation and tests.
Changes:
- Added structured-grid detection and
FutureWarnings to both dataset-opening APIs. - Improved documentation and migration guidance for grid and data inputs.
- Added warning regression tests and removed an obsolete structured-grid mapping shortcut.
This pull request requires changes.
| File | Review |
|---|---|
uxarray/core/utils.py |
Critical (3 votes): Restore a guard for datasets containing neither structured source dimension; otherwise _map_dims_to_ugrid can raise UnboundLocalError. |
uxarray/core/api.py |
Moderate (4 votes): Correct the nonexistent xarray.open_mf_dataset reference to xarray.open_mfdataset. Nit (4 votes): Keep grid_filename_or_obj documented as the grid source rather than a data-file path. |
test/core/test_api.py |
Adds warning coverage for structured inputs to open_dataset and open_mfdataset. |
Review effort: Lite
|
From internal discussion, @erogluorhan confirmed that And, further clarification: Orhan's preference is to also proceed without a deprecation warning in the ux.open_dataset case as well. I will update the code appropriately and then send a message here once it is done to clarify that the PR is once again ready for reviews. For now it needs updates before it should be reviewed. |
There was a problem hiding this comment.
Hi @Sevans711 , thanks for considering clarifications! Also, thank you for communicating the internal discussion!
Let me bring additional important parts of our internal discussion here for our user & contributor community's reference:
UXarray has native support for only unstructured grids, not for structured grids. Therefore, any support for structured grids should be explicit through only a few specialized APIs, e.g. from_structured(). The core API members such as open_dataset() shouldn't support structured grids in any way.
That said, this PR's title, description, and the work need to be updated accordingly:
- I’d rather NOT have
open_dataset()’s docstrings mention anything about structured grids.- Maybe a short phrase similar to
from_xarray()’s, i.e. “... data residing on an unstructured grid”, or something similar about the grid being an unstructured grid can be added though.
- Maybe a short phrase similar to
and similar for open_mfdataset. See discussion in #1887 for details. Also, some small rephrasing in docstrings
|
@erogluorhan Thank you for providing the additional context here. I believe all your points have been addressed. I also updated the PR title and the PR overview message to clarify the new scope and intentions here. Please let me know if you find anything I missed! |
erogluorhan
left a comment
There was a problem hiding this comment.
Thanks @Sevans711 ! This covers everything I had pointed out before.
I only have a couple inline comments, plus Claude helped me find a few points as below (Despite issue 1, I can see the docs built successfully though, so please take them all with a pinch of salt):
1. ⛔ This breaks the user guide and will fail the docs build
The PR description states "Nothing in the docs currently uses from_xarray() with structured data." This isn't quite right — docs/user-guide/structured.ipynb has a whole section, "Pairing a Structured Grid with an Existing Dataset" (md cell 13, code cells 14 and 16), built on exactly this:
uxgrid = ux.Grid.from_structured(lon=ds_air_temp.lon, lat=ds_air_temp.lat)
uxds = ux.UxDataset.from_xarray(ds_air_temp, uxgrid=uxgrid) # now raises DimensionError
regional = uxds["air"][0].subset.bounding_circle((-100.0, 40.0), 15) # cascadesI ran this on the PR branch and it raises DimensionError. The notebook is listed in docs/userguide.rst:136 and is not in nb_execution_excludepatterns (docs/conf.py:233), so it executes at docs build time.
That section is the documented rendering of #1410 — the same workflow the deleted test covered. It should be rewritten to the UxDataset.from_structured(ds) equivalent (and the prose about "UXarray records the structured source dimensions" dropped) as part of this PR.
2. ⛔ The UxDataArray remedy in the error message doesn't work
Consider UxDataset.from_structured(arr.to_dataset()).to_dataarray() instead.
I verified this: it returns a plain xr.DataArray with a spurious variable dim of size 1 — the uxgrid is gone, which is the opposite of what the user wanted. It also fails outright for unnamed arrays.
The working form is:
UxDataset.from_structured(da.to_dataset())[da.name] # -> UxDataArray, uxgrid attachedError messages that route users to a broken API are worse than no suggestion at all.
3. The dims reported in the error message are the wrong dims
...received a grid with source_grid_spec='Structured' and dims ['n_node', 'n_face', 'n_max_face_nodes']
uxgrid._ds.dims are the post-conversion UGRID dims — they're identical for every grid, so they carry no diagnostic signal and actively mislead ("I passed lat/lon, why is it talking about n_node?"). It also surfaces a private attribute in user-facing text.
Either drop the clause, or print uxgrid._source_dims_dict (e.g. {'n_face': ('lon', 'lat')}), which actually names what the user passed in.
4. open_grid() is left as an unguarded hole
g = ux.open_grid(structured_ds) # still silently succeeds: spec='Structured', n_face=32open_grid is as much a core entry point as open_dataset, and the agreed principle here is that core API shouldn't support structured grids in any way. Post-PR it returns a Grid that is now a dead end — data can no longer be attached to it through any public path.
Guarding open_grid (not Grid.from_dataset, which Grid.from_structured(ds=...) delegates to) would close the issue properly. If deferring is preferred, let's open an explicit follow-up issue rather than leaving it silent.
5. Backwards compatibility should be called out in the release notes
This is a hard break on a path that #1569 added deliberately and covered with a regression test for #1410, in a released version. Skipping the deprecation cycle was agreed, so no objection to the decision itself — but it should be called out in the release notes.
Note that this also makes items 2 and 3 more than cosmetic: the error message is the only migration guidance users will get.
6. Is DimensionError the right error type?
DimensionError is documented as "wrong size(s), name(s), or number of dimensions," which isn't really what happened — the user supplied an unsupported grid kind. GridInvalidError ("data does not correspond to a valid uxarray.Grid") looks like a closer fit, and both subclass ValueError so it's not a compatibility concern.
Minor, but given the effort that went into a curated errors.py (#1556), worth picking precisely.
| must record its source dimensions so ``UxDataset.from_xarray`` flattens the | ||
| structured (lon, lat) data variables onto ``n_face``.""" | ||
| ds = xr.tutorial.open_dataset(ds_name) | ||
| def test_from_xarray_with_grid_from_latlon(): |
There was a problem hiding this comment.
How about adding a "_fails_with" to this test case similar to the one in test_api? The current name reads a bit like expected usage, not a failure case.
| filename_or_obj, chunks=corrected_chunks, **kwargs | ||
| ) | ||
|
|
||
| if _represents_structured_data(uxgrid): |
There was a problem hiding this comment.
A couple points:
- If we need to implement it this way: This is okay, but I am wondering if a simple, one-line check wherever this was needed (without defining a newer internal function) as the following could suffice:
if uxgrid.source_grid_spec == "Structured":
- Maybe more importantly, could we implement this in a way that the unintentional leakage of structured grids this far into the core API could be avoided in the first place (rather than catching them here after all)? For instance, the support for ingestion of structured grids could be separated from the main flow a bit?



Closes #1854
Overview
EDIT: as per messages below, this PR has been updated from its original form (clarify/warn if using structured inputs in ux.open_dataset or ux.open_mfdataset) to instead fully remove support for structured data in ux.open_dataset, ux.open_mfdataset, UxDataset.from_xarray, and UxDataArray.from_xarray. This expands the PR's scope beyond #1854 which only discussed ux.open_dataset.
Fully removing support for structured data in
from_xarrayalso means deleting most of the contents oftest_from_xarray_with_grid_from_latlonwhich was introduced by PR #1569. With my current understanding of the uxarray API, I believe the "correct" solution to the underlying issue solved by that PR (#1410) would have instead been to simply recommend the use of the existing UxDataset.from_structured() method.Additional expansions of scope:
dstoda.grid_filename_or_obj._map_dataset_to_ugridwhich are specific to structured grids and not backed by tests. For more details see: ux.open_dataset currently supports structured data #1854 (comment)Misc. notes:
from_xarray()usage, but that notebook is currently inaccessible in the docs; see Readthedocs userguide broken notebook links: Custom Grid Topology and Spatial Hashing #1883. All examples there are from unstructured data though. Nothing in the docs currently uses from_xarray() with structured data, and the docstring already clearly specifies that the method expects unstructured data.PR Checklist
General
Testing & Benchmarking
Documentation and Examples
docs/api.rst; internal (private) function names start with an underscore (_)AI Disclosure
AI Usage: just GitHub Copilot's inline code suggestions.