Skip to content

point splitter - #954

Open
HansVRP wants to merge 12 commits into
masterfrom
hv_split_points_jobmanager
Open

HansVRP wants to merge 12 commits into
masterfrom
hv_split_points_jobmanager

Conversation

@HansVRP

@HansVRP HansVRP commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

While working on the ML blueprint I figured it should not be hard to extend the spitting function towards point geometries as well.

@VincentVerelst you are quite familiar with this; can you please chekc and validate as well?

For me the results look good and respect tilegrids as well:

image

@HansVRP

HansVRP commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

@VincentVerelst can you take a look?

@HansVRP HansVRP self-assigned this Oct 21, 2026
@HansVRP

HansVRP commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

@soxofaan could you have a look as well (I do still need to fix the unit tests)

@soxofaan soxofaan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't go deep here, just some quick observations

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Input features can be duplicated or lost, and custom tile grids do not satisfy the advertised interface contract.

3 open issues
What changed in this PR

Adds point-based job splitting to the existing job-management utilities, supporting sample extraction workflows alongside area-based splitting.

Changes:

  • Introduces split_points with optional feature limits, spatial subdivision, and tile-grid grouping.
  • Exports the helper publicly and documents its usage.
  • Adds tests for supported inputs, grouping, reprojection, and validation.
File Description
tests/​extra/​job_management/​test_job_splitting.py Tests point splitting and tile-grid behavior.
openeo/​extra/​job_management/​_job_splitting.py Implements point grouping and recursive subdivision.
openeo/​extra/​job_management/​__init__.py Exports split_points publicly.
docs/​cookbook/​job_manager.rst Adds point-extraction batching examples.
CHANGELOG.md Announces the new helper.

Review effort: Balanced

@soxofaan

Copy link
Copy Markdown
Member

FYI: any unit test failures you see about jsonschema are fixed with #960, so make sure to merge master to get rid of that problem

@HansVRP
HansVRP force-pushed the hv_split_points_jobmanager branch from a1ac215 to 30f9394 Compare November 10, 2026 09:51
Co-authored-by: HansVRP <57984106+HansVRP@users.noreply.github.com>
Co-authored-by: HansVRP <57984106+HansVRP@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The change breaks existing split_area(aoi=...) keyword calls and uses a Shapely 2-only API that causes predefined-grid grouping to fail on supported Shapely 1.8 installations.

2 open issues
3 resolved since last review

Review effort: Balanced

@HansVRP

HansVRP commented Nov 11, 2026

Copy link
Copy Markdown
Contributor Author

Some additional info to the commit; I'll add some figures to illustrate the caveats

Grouping strategies:

  • max_geometries: recursive median bisection along the longer axis of
    the representative points; groups stay spatially clustered with sizes
    between ~max/2 and max.
image
  • tile_size + projection: features are binned per regular tile, then
    each tile is bisected. The grid is anchored at the CRS origin
    (floor(x / tile_size)), so adding an outlier does not change the
    grouping of existing features.

  • tile_grid (e.g. Sentinel-2 MGRS): features are assigned to the tile
    containing their representative point; features outside all tiles are
    grouped separately. Without max_geometries: one group per tile.

    image

Caveats (TODO):

  • Groups are limited by feature count, not by extent unless a tile_gris is added. On 550 synthetic
    points (3 clusters + noise), the largest group extent is ~41 km
    without a grid and ~21 km with tile_size=20 km.

    image
  • Polygons are never split: large polygons extend beyond their tile, and
    groups are not balanced by area (use split_area for a single large AOI).

image
  • Multi-part geometries stay in one group, placed by a single
    representative point. 40 MultiPolygon farms (3 distant parts each),
    max_geometries=10: 4 groups each spanning the whole AOI, versus
    16 compact groups (sizes 7..8) after explode(). The original index is
    kept, so exploded parts can be linked back to their rows.
image
  • Overlapping tiles: the first matching tile in the grid wins (not the
    one with the largest overlap)

Input validation:

  • Null or empty geometries raise JobSplittingFailure instead of being
    silently dropped.
  • Duplicated index labels are supported: grouping is position-based and
    the original index is preserved.
  • tile_grid cannot be combined with tile_size / projection.

Copilot AI and others added 2 commits November 11, 2026 08:25
Co-authored-by: HansVRP <57984106+HansVRP@users.noreply.github.com>
Co-authored-by: HansVRP <57984106+HansVRP@users.noreply.github.com>
@HansVRP

HansVRP commented Nov 11, 2026

Copy link
Copy Markdown
Contributor Author

@VincentVerelst @soxofaan ready for another look; I added a comment above with the caveats I know of.

@soxofaan soxofaan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

short on time here, so only this small note

Comment thread CHANGELOG.md


## [0.54.0] - 2026-10-28
- Add public `group_geometries` helper in `openeo.extra.job_management` for geometry-grouping based job splitting, including optional tile-grid co-grouping.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrong place to put this changelog entry

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants