Skip to content

Hsqldb2 - #520

Merged
jimjag merged 14 commits into
trunkfrom
hsqldb2
Nov 20, 2026
Merged

jimjag merged 14 commits into
trunkfrom
hsqldb2

Conversation

@jimjag

@jimjag jimjag commented Nov 9, 2026

Copy link
Copy Markdown
Member

Upgrade bundled HSQLDB from 1.8.0.11 to 2.5.0

  • maintain backwards compatibility
  • passes test suite

Pedro Giffuni and others added 11 commits November 8, 2026 06:56
Upgrade the bundled HSQLDB database engine from 1.8.0.11 to 2.5.0.

While newer versions exist, 2.5.0 is chosen for two key advantages:
  - It still opens older 1.8.x databases.
  - It doesn't require Java 11 or alternative hacks like newer versions.

Changes to build configuration:
  - main/hsqldb/version.mk: Update version numbers (1.8.0 -> 2.5.0)
  - main/hsqldb/makefile.mk: Update TARFILE_NAME, TARFILE_ROOTDIR,
    TARFILE_MD5 to match upstream hsqldb-2.5.0.zip. Remove obsolete
    CONVERTFILES entry for Library.java (removed in 2.5.0). Change
    Ant target from 'jar' to 'hsqldb'. Clear PATCH_FILES (all old
    patches are obsolete).
  - main/external_deps.lst: Update MD5 and name for hsqldb-2.5.0.zip
  - main/hsqldb/prj/d.lst: Update jar delivery path for new directory
    structure (hsqldb-2.5.0/hsqldb/lib/).
  - main/configure.ac: Update version check for system hsqldb to accept
    2.5.0+.

API compatibility changes for connectivity drivers:
  - StorageAccess.java: Implement org.hsqldb.lib.Storage instead of
    org.hsqldb.persist.ScaledRAInterface (removed in HSQLDB 2.0+).
  - StorageFileAccess.java: Update openOutputStreamElement() to
    accept a boolean 'append' parameter per the new FileAccess
    interface. Update renameElement() to accept a boolean
    'copyIfFailed' parameter.

Native code compatibility:
  - HStorageMap.cxx: Fix StreamHelper destructor to properly close
    read-only streams.
  - HStorageAccess.cxx: Add missing include for Unicode handling.
  - HDriver.cxx: Add shutdown required prompt and database conversion
    handling for HSQLDB 2.x database compatibility.
  - hsqldb_res.hrc: Add STR_CONVERT_TO_NEW_VERSION and
    STR_SHUTDOWN_REQUIRED resource string IDs.
  - conn_shared_res.src: Add resource strings for database upgrade
    and shutdown prompts.

Patch removals:
  - Remove obsolete patches: i121754.patch, runfinalizers.patch,
    script.patch, text_tables.patch
The patch now throws when the storage class cannot be loaded instead of
falling back to the file system.
…'No'

dbaccess never passed the document's InteractionHandler to the driver.
With no answer at all it still converts, since headless the default handler
answers nothing.
The prompt holds the data source mutex; a connect from another thread can deadlock.
With the type check first, a NULL DOUBLE was equal to an INTEGER 0.
* Replace std::hash_map/hash_set with std::unordered_map/unordered_set on trunk (C++11)

GCC 15 removed std::hash_map/std::hash_set (GNU extensions) from libstdc++.
Replace all 628 occurrences across 477 files with std::unordered_map/
std::unordered_set, updating #include <hash_map>/<hash_set> to

Affects: vcl, crashrep, extensions, dbaccess, scripting, sw, io, rdbmaker,
codemaker, unodevtools, and many other modules.

* Migrate remaining std::hash_map/std::hash_set to std::unordered_map/std::unordered_set

Replace TR1/backport hash containers with C++11 standard equivalents
in source files not yet covered by trunk's migration.

* Replace std::hash_multimap with std::unordered_multimap in propcontroller.hxx

std::hash_multimap is a GNU extension removed from GCC 13's libstdc++,
causing a build failure in the pcr module (propcontroller.obj).
std::unordered_multimap is the C++11 standard equivalent already used
elsewhere in this file and supports all operations performed on
m_aDependencyHandlers.

* Replace std::hash_multimap with std::unordered_multimap in oleobjw.hxx

std::hash_multimap is a GNU extension removed from GCC 13's libstdc++.
The functors hashOUString_Impl/equalOUString_Impl are already used with
std::unordered_map on line 74 of the same file, and all operations on
m_mapComFunc (find, end, insert) are supported by std::unordered_multimap.

---------

Co-authored-by: Pedro F. Giffuni <pfg@apache.org>
@pgiffuni

pgiffuni commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

I don't have time to rebuild and test this but let me make some notes of the stuff I figured out in the past.
You may want to refer to the Bugzilla issue (i121837) in the commit:

Basic expectation:
(This was broken on the plain 2.5.0 without patches, it appears 2.2.0 still has the complete compatibility code)

  • You should be able to create a new database, save it, generally use it.
  • You should be able to populate it using SQL commands.
    Compatibility:
    Old 1.8.1 database can be opened.
    Conversion can happen automatically and without asking but you cannot save.
    You can still use Save As, and the new database will be in the new format.

This last behavior is consistent because I didn't find any code in the old branch that would trigger a dialog.
If you did get dialogs working, then kudos!

Upstream HSQLDB (Fred Toussi) is willing to workout maintaining support for AOO in future releases.

@jimjag

jimjag commented Nov 19, 2026

Copy link
Copy Markdown
Member Author

Thanks Pedro, those notes help.

New databases: create, save, reopen and populate via SQL all work, including LOBs, backup and recovery after a kill -9. Tested on macOS with JDK 8, 11, 17 and 21, and on Linux with JDK 21.

Old 1.8 databases open, both clean ones as well as "dirty" ones (not shut down cleanly). And yes, the dialog works :) Opening a 1.8 database asks whether to convert it. Yes converts it. No refuses the open and leaves the file untouched. After converting, a plain Save works and writes a 2.5.0 database, so you don't need Save As.

One thing I still need to look at: during a UI test, a converted file got rewritten as 2.5.0 even though I closed it without saving. Since the conversion is one-way, that will go in the release notes if I don't track it down first.

I'll reference i121837 when this gets merged.

Fred's offer is great news. 2.5.0 is the newest release that still opens 1.8 files, so it's as far as we can go for now. If upstream brought back the storage hooks and kept 1.8 read support, we could drop most of our patch and move to current releases. Do you want to contact him, or should I?

Cheers!

@pgiffuni

Copy link
Copy Markdown
Contributor

You reminded me .. it appears the older AOO never closed the database cleanly. :-).

I sent Fred and you an email so you can arrange what would be missing for future support. Great job!

@fredtoussi

fredtoussi commented Nov 20, 2026 •

Copy link
Copy Markdown

Thank you for your work.

There are 2 legacy features that were finally removed in version 2.5.0:

  1. opening and converting a 1.8 database, this feature was fully working in 2.5.0 and I will probably revive it
  2. the storage hooks and conditional code to operate with the Storage interface from OOo, this feature may have regressed between the unreleased v. 1.9.0 (on which it was developed) and 2.5.0

For your current development, it is important to ensure the zipped hsqldb database files within the .odb file constitute a valid database file, which can be unzipped+renamed and opened and modified using the unmodified standalone version 2.5.0 jar.

For future developments, I would suggest a different approach which allows HSQLDB to handle the storage itself.

This was lost when the JDBC bridge was ported to Java (956476d), so AutoValue columns got no IDENTITY and would cause errors in the Build Wiz.
@jimjag
jimjag merged commit 7bb36ca into trunk Nov 20, 2026
1 check passed
@jimjag

jimjag commented Nov 20, 2026

Copy link
Copy Markdown
Member Author

Merged into trunk (7bb36ca) and backported to AOO50X (21ecb0b..1d87e02).

Fred, thanks for looking at this. On your points:

  1. Opening and converting 1.8 databases is what keeps us on 2.5.0. The later releases I tried (2.5.1, 2.7.0, 2.7.4) refuse the files our users already have. If you bring it back, we can move to a current release.
  2. The storage hooks had regressed. Our patch (main/hsqldb/patches/aoo_integration.patch) puts them back, so the data, LOB and backup files go through the AOO storage again.
  3. I checked the database inside saved .odb files against the unmodified 2.5.0 jar from the release zip. Six documents: a new database, 50,000 rows, CLOB/BLOB columns, several store/checkpoint rounds, a converted 1.8 file, and repeated store/edit. In each one the unzipped and renamed files open, every table reads back with the same rows and LOB bytes, a new table with a BLOB can be added, and the result reopens, also after SHUTDOWN COMPACT.

Letting HSQLDB handle the storage itself sounds like the better long-term route. Happy to work out what that would take, here or on the thread Pedro started.

Cheers!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants