Skip to content

Support reading records from fsspec's memory:// - #629

Open
Ivorforce wants to merge 1 commit into
MIT-LCP:mainfrom
Ivorforce:memory-uris
Open

Ivorforce wants to merge 1 commit into
MIT-LCP:mainfrom
Ivorforce:memory-uris

Conversation

@Ivorforce

@Ivorforce Ivorforce commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Note

This code was implemented by AI.

Adds support for memory:// to rdrecord, rdsamp and rdheader, addressing #491 (comment). Supersedes #491.

Adding this was already almost done via fsspec. What was needed was Windows support for path-joining POSIX paths and adding it to the supported protocols list.

Besides that the PR adds documentation and tests.

Add memory:// to the protocols handled through fsspec, so that records
can be read without touching the disk, for example from an upload form.

Build signal and segment paths with forward slashes for URIs. They used
to contain backslashes on Windows, which also affected cloud URIs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Ivorforce Ivorforce changed the title Support reading records from fsspec's :memory:// Support reading records from fsspec's memory:// Nov 8, 2026
@tompollard

Copy link
Copy Markdown
Member

Thanks for picking this up! A couple of minor things:

memory://100 doesn't work yet. rdheader and _rdrecord split the path first and then look for the protocol in the directory part (record.py:1864 and record.py:2205). With no directory, that part is just memory:, so it's treated as a local path and you get an error:

FileNotFoundError: ... '<cwd>/memory:/100.hea'

I think checking record_name for the protocol instead, in rdheader, _rdrecord and _infer_sig_len, would fix it. Then we could drop the paragraph in docs/io.rst that asks people to use a directory.

The test fixture's cleanup can fail. If nothing was uploaded, fs.rm("/wfdb-test", recursive=True) raises FileNotFoundError. When a test fails, you then also get an error from the cleanup that hides the real problem. Adding an fs.exists("/wfdb-test") check first should sort it.

Please could you take a look at these?

A couple of other minor things that we could address here or in new PRs:

  • rdheader and _infer_sig_len still choose between posixpath.join and os.path.join themselves, so they could use the new _join_path too.
  • The threading warning in the docs is correct, because fsspec gives every reader the same BytesIO. Later we might read memory files with fs.cat_file() into a new BytesIO, so each reader gets its own copy and the warning can go.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants