Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
39f13a8
Document surface_distance backends, geodesic limits, and add examples…
brendancol Aug 16, 2026
b9278f1
Record the surface_distance documentation sweep in the state CSV
brendancol Aug 16, 2026
80de4c9
Close the surface_distance backend test gaps and fix dask direction (…
brendancol Aug 16, 2026
e4b2da3
Record the surface_distance test-coverage sweep (#3725)
brendancol Aug 16, 2026
fbfd649
Grow the surface_distance Dijkstra heap on demand (#3723)
brendancol Aug 16, 2026
17dcefc
Pin surface_distance against scipy csgraph and document the allocatio…
brendancol Aug 16, 2026
0babfb4
Record the surface_distance reference-validation sweep result
brendancol Aug 16, 2026
4afb223
Address review findings on the heap fix (#3723)
brendancol Aug 16, 2026
43053e9
Merge remote-tracking branch 'origin/main' into deep-sweep-reference-…
brendancol Nov 6, 2026
52cabe8
Merge remote-tracking branch 'origin/main' into deep-sweep-documentat…
brendancol Nov 6, 2026
02652a6
Merge remote-tracking branch 'origin/main' into deep-sweep-test-cover…
brendancol Nov 6, 2026
df5a999
Merge remote-tracking branch 'origin/main' into deep-sweep-performanc…
brendancol Nov 6, 2026
dd54912
Pin the surface_direction docstring example to the north-up bearings …
brendancol Nov 6, 2026
e3adbca
Merge remote-tracking branch 'origin/deep-sweep-test-coverage-surface…
brendancol Nov 6, 2026
2964bba
Merge branch 'deep-sweep-performance-surface_distance-2026-08-16' int…
brendancol Nov 6, 2026
d47ad44
Merge branch 'deep-sweep-reference-validation-surface_distance-2026-0…
brendancol Nov 6, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .claude/sweep-documentation-state.csv
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,5 @@ interpolate-idw,2026-06-26,,MEDIUM,1;5,"Cat1 MEDIUM: idw (only public func in _i
mahalanobis,2026-06-30,3579,MEDIUM,1;2;5,"Cat1 MEDIUM: mahalanobis (only public func) had Parameters/Returns but no Examples section (peers normalize.rescale/standardize do). Cat5 MEDIUM: NaN propagation (any non-finite band -> NaN pixel) and auto-stats N+1 all-finite requirement undocumented. Cat2 LOW: name param default not noted. Doc-only fix on deep-sweep-documentation-mahalanobis-2026-06-30: added Notes (NaN+backends) and a runnable Examples block (executed, output matches incl. NaN), set name default='mahalanobis'. Returns float64/shape and 4-backend claim verified against ArrayTypeFunctionMapping (accurate, left as-is). Cat3 n/a (no prior examples). Cat4 clean (listed in reference/utilities.rst). issue #3579. CUDA available: ran numpy example; cupy/dask covered by test_mahalanobis.py (24 pass).",1/1
pathfinding,2026-07-08,3651,MEDIUM,1;3;5,"Cat1 MEDIUM: multi_stop_search had Parameters/Returns/Raises but no Examples section (a_star_search has 13 example lines); added runnable example pinning waypoint_order/segment_costs/total_cost. Cat3 MEDIUM: a_star_search Examples used '... sourcecode:: python' (three dots, only occurrence in codebase) so the rendered page shows literal text instead of a code block; fixed to '..'; example itself executes clean. Cat5 MEDIUM: NaN surface cells are always impassable (_is_not_crossable, pinned by input_data_with_nans tests) but neither docstring said so (friction NaN rule was documented); documented in summary + barriers entries of both funcs. LOW fixed in passing: 'y: str' missing numpydoc space. Deferred to #3644 (api-consistency sibling, same day): 'values to bin' surface description + empty connectivity description. Clean: params match both signatures (grouped-entry aware check), both funcs in reference/pathfinding.rst, backend table verified by running all 4 backends (CUDA available; dask path returns equally-optimal alternate path, goal cost identical). Added 3 docstring-validation tests to test_pathfinding.py. Fix on deep-sweep-documentation-pathfinding-2026-07-08, issue #3651.",2/2
perlin,2026-06-23,,MEDIUM,2;5,"name param undocumented (Cat2) + float-dtype requirement/ValueError undocumented, no Raises section (Cat5); fixed in deep-sweep-documentation-perlin-2026-06-23; repo has issues disabled so no issue number; example runs and output matches; 1 public func (perlin) listed in reference/surface.rst",1/1
surface_distance,2026-08-16,3714;3713,HIGH,1;2;5,"severity_max HIGH is the separately-filed code bug #3713; the documentation findings themselves top out at MEDIUM. 3 MEDIUM: zero >>> example blocks in the whole module (Cat 1); no backend-support statement although all 4 backends dispatch and run (Cat 5); method='geodesic' documented with no caveat but raises NotImplementedError on cupy, dask+numpy and dask+cupy (Cat 5). Fixed in PR for #3714: backend paragraph, geodesic caveat, runnable Examples on all 3 public funcs, plus 13 docstring guard tests. 3 LOW, not fixed: surface_allocation/surface_direction fold 6 signature params into one 'x, y, target_values, ... See surface_distance' entry (numpydoc parses it, nothing misleading); the UserWarning emitted on the unbounded dask path is undocumented; pre-existing flake8 F841 at test_surface_distance.py:236 left alone. Cat 4 clean: all 3 funcs appear in docs/source/reference/proximity.rst. Separately filed #3713 (HIGH, code bug not doc bug): surface_direction returns wrong bearings on the dask iterative path because _finalize_direction subtracts block-local pixel indices from the global src_row/src_col written by _run_tile; 105/144 pixels wrong on a 12x12 chunks=(6,6) case. CUDA available, all 4 backends executed.",3/3
templates,2026-06-26,3541,MEDIUM,3;5,"Cat3/Cat5 MEDIUM: from_template docstring example 'from_template(""FRA"", preserve=""shape"").attrs[""crs""]' showed 32631/UTM 31N but code returns 32630/UTM 30N (GADM FRA bbox includes overseas territories, centroid lon -2.98 -> UTM 30N; code correct per documented centroid-UTM contract). Doc-only fix on deep-sweep-documentation-templates-2026-06-26; issue #3541, PR pending. Both public funcs (from_template, list_templates) fully documented and in reference/templates.rst (no Cat1/Cat2/Cat4). CUDA available: ran all docstring examples incl. preserve paths; 63/63 test_templates.py pass.",2/2
2 changes: 1 addition & 1 deletion .claude/sweep-performance-state.csv
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ resample,2026-04-15T12:00:00Z,SAFE,compute-bound,0,false-positive,Downgraded. GP
sieve,2026-04-14T12:00:00Z,WILL OOM,memory-bound,0,false-positive,False positive. Memory guards already in place on both dask paths. CCL is inherently global — documented limitation. CuPy CPU fallback is deliberate and documented.
sky_view_factor,2026-03-31T18:00:00Z,SAFE,compute-bound,0,,
slope,2026-03-31T18:00:00Z,SAFE,compute-bound,0,,
surface_distance,2026-03-31T18:00:00Z,SAFE,memory-bound,0,1128,Memory guard added to dd_grid allocation.
surface_distance,2026-08-16,RISKY,compute-bound,0,3723,"CRITICAL #3723/PR: _dijkstra and _dijkstra_geodesic sized the lazy-deletion heap height*width; peak occupancy hit 1.09x cap at 40% target density, _heap_push wrote OOB (SIGABRT 'double free or corruption'); fixed by doubling on demand rather than cost_distance's static (n_neighbors+1) bound, which would have raised the 80 B/px guard to ~272. MEDIUM (documented, not fixed): dask iterative path recomputes each source and elev block 6x (2 in _preprocess_tiles_sd, 4 across sweeps, 1 in _assemble_sd) - measured on 40x40/80x80/120x120; MEDIUM: cupy Bellman-Ford relaxation is 6-27x slower than the numpy Dijkstra (512^2: 0.315s vs 0.051s, 373 kernel launches each with a device sync), and DIRECTION mode round-trips to host for _vectorized_calc_direction; MEDIUM: map_overlap depth guard tests pad < max(chunks), so depth may approach chunk size (~9x redundant work) and exceed the smallest chunk when chunking is uneven. Memory guard is correctly chunk-scoped on dask (test_dask_path_bounded_per_chunk) - the morphology #3401 false-MemoryError hazard is NOT present. _sd_relax_kernel has ~8 float64 locals, no register pressure. OOM verdict RISKY: memory stays chunk-bounded but the whole iterative Dijkstra runs eagerly and single-threaded on the client at graph-build time."
terrain,2026-03-31T18:00:00Z,RISKY,compute-bound,0,,
terrain_metrics,2026-03-31T18:00:00Z,SAFE,memory-bound,0,,
viewshed,2026-04-05T12:00:00Z,SAFE,memory-bound,0,fixed-in-tree,Tier B memory estimate tightened from 280 to 368 bytes/pixel (accounts for lexsort double-alloc + computed raster). astype copy=False avoids needless float64 copy.
Expand Down
1 change: 1 addition & 0 deletions .claude/sweep-reference-validation-state.csv
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@ convolution,2026-07-02,3619,MEDIUM,CONVENTION-DIFF,0.0 exact (float64),scipy 1.1
edge_detection,2026-07-18,3677,,MATCHES,"1e-6 rel (observed ~1e-13 interior, bit-exact vs ndi.correlate)",scipy 1.16.1; sobel_x/sobel_y/prewitt_x/prewitt_y/laplacian interior match ndi.sobel/prewitt/laplace and ndi.correlate with same stencils; boundary nan/nearest/reflect/wrap == scipy modes nearest/reflect/wrap incl. edges; tilted plane analytic exact (sobel=8g prewitt=6g laplacian=0); coords/cellsize independent as documented; cupy+dask+numpy+dask+cupy bit-identical to numpy (CUDA run); golden-value test added via #3677
geotiff,2026-07-02,,,MATCHES,rtol=1e-6 float64; float32 codecs exact,"rasterio 1.4.4 (GDAL 3.10.3); gdal-cli 3.11.4; tifffile 2026.3.3; richdem-unavailable; osgeo-python-unavailable. Reader/writer round-trip vs rasterio/GDAL: codecs zstd/deflate/lzw/packbits/none + predictor 2&3 dmax=0; CRS EPSG:32611 & 4326 round-trip exact; GeoTransform north-up, non-square px, tiepoint & ModelTransformation all correct; nodata float-NaN->sentinel + int16/uint8/int32/uint16 sentinels exact; BigTIFF magic+data ok; windowed reads match rasterio.Window incl coord alignment; multiband round-trips (band-last is xrspatial's documented convention); COG valid per 'rio cogeo validate'; overview mean matches numpy block-mean to float32 ulp; tifffile tag check: Compression/Predictor/SampleFormat/GeoKeyDir all correct. Existing repo parity suite parity/test_reference.py 31 passed (incl dask+cupy). No divergence; golden_corpus already pins parity."
pathfinding,2026-07-08,3655,,MATCHES,1e-12 rel (observed <=2e-15),"a_star_search vs scipy 1.16.1 sparse.csgraph.dijkstra + skimage 0.26.0 MCP_Geometric + analytic open-grid truth; 64x64 open/maze/friction/NaN-friction/anisotropic inputs, 4+8 conn; path chains adjacency+increment consistent; backend parity numpy/dask+numpy/cupy/dask+cupy confirmed (CUDA host); no prior golden test -> added golden-value tests (#3655); gdal/richdem/wbt not applicable to pathfinding"
surface_distance,2026-08-16,3726,MEDIUM,MATCHES,1e-6 rel (observed 0.0 exact vs scipy on all backends),"scipy 1.16.1 (sparse.csgraph.dijkstra over an explicit grid graph with sqrt(horizontal^2+dz^2) edge weights) used as the reference; gdal-unavailable richdem-unavailable whitebox-unavailable; skimage 0.26.0 MCP_Geometric not applicable (per-pixel cost model, not 3D edge length). cuda True. numpy, dask+numpy (bounded map_overlap and unbounded iterative), cupy and dask+cupy all reproduce the scipy reference bit for bit (max_abs=0.0) on a Gaussian hill, a tilted plane, a seeded rough surface and a NaN-barrier scene, at connectivity 4 and 8, with and without max_distance and target_values. Analytic tilted-plane ground truth matches to float32 eps (3.9e-8 rel). surface_direction bearings match the reference bearing convention on north-up rasters (N=360, E=90). Geodesic mode matches an independent haversine graph at R=6378137 exactly; that is the library-wide radius (proximity uses the same), and the spherical-vs-WGS84-ellipsoid difference is +-0.17% locally, +0.003% mean bias - convention, not a defect, though the docstring does not state the sphere. Two CRITICAL defects surfaced during validation were already filed by parallel sweep agents, so no duplicate issue was opened: #3722 (heap sized H*W overflows, glibc abort; reproduced here at 40x40 with a 50% dense source mask, peak occupancy 1.07x H*W) and #3721 (cupy max_iterations=H+W truncates relaxation). Added new evidence to #3721: with no barriers at all, only steep ridges, cupy returns finite distances up to 173.8% too large on 1314/1600 pixels - silent, not NaN. #3723 flagged as a duplicate of #3722. Filed #3726 (MEDIUM): surface_allocation/surface_direction resolve exact ties differently per backend (29/4096 px numpy vs cupy, 25/4096 numpy vs dask) with no documented rule; PR documents the tie-break as unspecified and adds golden-value reference-parity tests (no golden test existed before)."
Loading
Loading