Skip to content

Atera reader (code review of #426) - #444

Draft
LucaMarconato wants to merge 29 commits into
mainfrom
atera-reader
Draft

LucaMarconato wants to merge 29 commits into
mainfrom
atera-reader

Conversation

@LucaMarconato

Copy link
Copy Markdown
Member

Atera support is implemented by @stephenwilliams22 in #426. Such PR is from a fork form an organization, and this doesn't allow spataldata-io maintainers to edit it directly. I will therefore work on this branch. It can then be merged into the #426, or merged into main (closing #426).

This PR implements code changes arising from the code review of #426.

stephenwilliams22 and others added 29 commits September 29, 2026 11:35
Adds a native reader for 10x Genomics Atera datasets: cell-by-gene
expression table, cell/nucleus segmentation labels and boundary
polygons, per-transcript locations, and morphology images.

- Table (`cell_feature_matrix.zarr.zip`/`csc_cell_feature_matrix.zarr.zip`)
  is read directly with `anndata`'s own zarr IO (`anndata.io.read_elem`/
  `anndata.io.sparse_dataset`), with `X` lazily backed by dask.
- Cell/nucleus labels are lazily backed by dask; boundary polygons are
  read via `shapely.from_ragged_array`, with an optional tiled/pyramided
  partial-read path (`read_cell_boundaries`) for large bundles.
- Transcripts are read lazily per spatial tile via dask-delayed.
- Morphology images are read via `tifffile`'s native OME-TIFF tile grid,
  avoiding materializing full-resolution planes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A tile can hold 100M+ transcripts. Each dask partition was a whole tile, so
writing a large dataset ran many multi-GB partitions at once and swapped to
disk. atera_to_parquet likewise decoded a whole tile per batch.

- _read_transcript_tile takes start/stop and reads only those rows; the
  per-row gene index comes from searchsorted over gene_offset.
- _get_points makes one partition per 5M-row chunk of a tile.
- atera_to_parquet reads and writes each tile in the same chunks.
- Add tests that chunked reads match whole-tile reads across gene
  boundaries, and that partitions are bounded.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Removed code for calculating the number of genes detected per cell as this is now in the atera loader
- Replace zarr.Group.get_array/get_group (zarr >= 3.3 only) with small
  helpers that work on any zarr 3.x; we don't pin zarr>=3.3 for now.
- Raise a clear FileNotFoundError when the bundle directory doesn't
  exist, instead of a misleading pixel-size error.
@review-notebook-app

Copy link
Copy Markdown

Review Notebooks

Notebooks changed:

Review visual diffs and comment on any notebook cell. All comments sync back to this PR.


Powered by ReviewNB

@LucaMarconato LucaMarconato changed the title Atera reader Atera reader (code review of #426) Nov 20, 2026
@codecov-commenter

codecov-commenter commented Nov 20, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.55556% with 231 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.57%. Comparing base (722d3da) to head (ec169cf).
⚠️ Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
src/spatialdata_io/readers/_atera_common.py 47.00% 168 Missing ⚠️
src/spatialdata_io/readers/atera.py 76.00% 48 Missing ⚠️
src/spatialdata_io/writers/atera.py 95.37% 15 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #444      +/-   ##
==========================================
+ Coverage   66.60%   68.57%   +1.97%     
==========================================
  Files          26       30       +4     
  Lines        3345     4290     +945     
==========================================
+ Hits         2228     2942     +714     
- Misses       1117     1348     +231     
Files with missing lines Coverage Δ
src/spatialdata_io/__init__.py 100.00% <ø> (ø)
src/spatialdata_io/_constants/_constants.py 100.00% <100.00%> (ø)
src/spatialdata_io/converters/atera_to_parquet.py 100.00% <100.00%> (ø)
src/spatialdata_io/writers/atera.py 95.37% <95.37%> (ø)
src/spatialdata_io/readers/atera.py 76.00% <76.00%> (ø)
src/spatialdata_io/readers/_atera_common.py 47.00% <47.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

"### 2. Clone the branch with the Atera reader\n",
"\n",
"```bash\n",
"git clone --branch atera-reader https://github.com/10XGenomics/spatialdata-io\n",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will need to be updated

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants