Instance ID tests fail on hosts without a readable MAC address #413

❌ This issue is not open for contribution. Visit Contributing guidelines to learn about the contributing process and how to find suitable issues.

Observed Behavior

Eight tests that change MORANGO_SYSTEM_ID and expect a new instance ID fail on hosts where Morango cannot read a MAC address. They pass on hosts where it can. In CI, the same tests pass and fail between runs on the same commit.

The tests expect InstanceIDModel.get_or_create_current_instance(clear_cache=True) to create a new instance after MORANGO_SYSTEM_ID changes. On a host without a readable MAC address, it returns the existing instance with created=False:

• tests/testapp/tests/test_api.py::MorangoInfoTestCase::test_id_changes_id_hash_changes
• tests/testapp/tests/test_uuid_utilities.py::InstanceIDModelTestCase::test_instance_id_caching
• tests/testapp/tests/sync/test_controller.py::SerializeIntoStoreTestCase::test_last_saved_instance_updates
• tests/testapp/tests/sync/test_controller.py::RecordMaxCounterUpdatesDuringSerialization::test_new_rmc_for_non_existent_model
• tests/testapp/tests/sync/test_operations.py::QueueStoreIntoBufferV1TestCase::test_fsic_counters
• tests/testapp/tests/sync/test_operations.py::QueueStoreIntoBufferV1TestCase::test_fsic_specific_id
• tests/testapp/tests/sync/test_operations.py::QueueStoreIntoBufferV2TestCase::test_fsic_counters
• tests/testapp/tests/sync/test_operations.py::QueueStoreIntoBufferV2TestCase::test_fsic_specific_id

Expected Behavior

The tests give the same result on every host, whether or not Morango can read a MAC address on that host.

Steps to Reproduce

The failure depends on the host, so these steps simulate a host without a readable MAC address.

1. Unset MORANGO_SYSTEM_ID and MORANGO_NODE_ID in your shell.

2. Save this pytest plugin as no_mac.py in a directory outside the repository:

   # Simulates a host with no readable MAC address: ifcfg finds no interfaces, and
   # uuid.getnode returns a random node, which has the multicast bit set
   import uuid
   
   
   def pytest_configure(config):
       import ifcfg
   
       uuid.getnode = lambda: 0x010203040506
       ifcfg.interfaces = lambda: {}

3. From the repository root, run the tests with the plugin:

   PYTHONPATH=<plugin directory> python -O -m pytest -p no_mac \
     tests/testapp/tests/test_api.py::MorangoInfoTestCase::test_id_changes_id_hash_changes \
     tests/testapp/tests/test_uuid_utilities.py::InstanceIDModelTestCase::test_instance_id_caching \
     tests/testapp/tests/sync/test_controller.py::SerializeIntoStoreTestCase::test_last_saved_instance_updates \
     tests/testapp/tests/sync/test_controller.py::RecordMaxCounterUpdatesDuringSerialization::test_new_rmc_for_non_existent_model \
     tests/testapp/tests/sync/test_operations.py::QueueStoreIntoBufferV1TestCase \
     tests/testapp/tests/sync/test_operations.py::QueueStoreIntoBufferV2TestCase
   

4. The run reports 8 failed, 25 passed.

5. To compare, make the plugin return a real MAC address: uuid.getnode returns 0x001122334455, and ifcfg.interfaces returns {"eth0": {"device": "eth0", "ether": "00:11:22:33:44:55"}}. The same run reports 33 passed.

Minimal Reproduction Environment / Conditions

• Commit: reproduced on release-v0.9.x@799d70d and on the branch of Reorder filters produced by Scope #410. The results are the same on both.
• Environment: Linux, Python 3.10, SQLite. The CI failures below are from a tox py3.9 job.
• CI trigger (unconfirmed): some CI runners probably give no usable network interface to ifcfg, and uuid.getnode() falls back to a random node. This explains why the failures come and go.

Errors, Logs & Telemetry

From a CI run:

>       self.assertNotEqual(m_info.data["instance_hash"], old_id_hash)
E       AssertionError: 'fifag-gokiv' == 'fifag-gokiv'
tests/testapp/tests/test_api.py:1151: AssertionError

>           self.assertTrue(created)
E           AssertionError: False is not true
tests/testapp/tests/test_uuid_utilities.py:200: AssertionError

>       self.assertNotEqual(new_id.id, self.current_id.id)
E       AssertionError: '0ff26305271e1133dcfd3350ab7fd63f' == '0ff26305271e1133dcfd3350ab7fd63f'
tests/testapp/tests/sync/test_controller.py:337: AssertionError

>           assert i.id not in buffer_ids
E           AssertionError: assert '055e7be12e154c9a7a9c2974f70d803c' not in <QuerySet [...]>
tests/testapp/tests/sync/test_operations.py:66: AssertionError

====== 8 failed, 614 passed, 31 skipped, 2 warnings in 1619.20s (0:26:59) ======

Deliverables & Testing Requirements

• The tests detect once whether Morango gets a node ID for the host, from a MAC address or from MORANGO_NODE_ID.
• If the host has a node ID, the eight original tests run against the real host identity.
• If the host has no node ID, the eight original tests are skipped. A copy of each test runs instead, with a fake MAC address.
• The copies apply the fake MAC address in setUp. Instances that setUp creates then have the same host identity as instances that the test creates. A copy cannot pass only because the MAC address changed.
• On every host, exactly one version of each of the eight tests runs, and it passes.
• All other tests keep the real host identity. The suite does not use a fake MAC address for every test, so the original tests still cover hosts with a readable MAC address.
• The fix changes only tests. It does not change production code.

These items are out of scope:

• A test that records the production behavior described under Notes.
• The failure of InstanceIDModelTestCase::test_consistent_0_5_instance_id when MORANGO_NODE_ID is set.

Technical Pointers & Root Cause Clues

• Legacy instance lookup: get_or_create_current_instance (morango/models/core.py:154-163) first returns any current instance that matches the 0.4 system parameters. Those parameters do not include MORANGO_SYSTEM_ID.
• Node ID sources: the 0.4 node_id (get_0_4_system_parameters, morango/models/utils.py:19-47) is empty when uuid.getnode() returns a random node. The 0.5 node_id (get_0_5_mac_address, morango/models/utils.py:213-238) is also empty when ifcfg finds no usable interface and uuid.getnode() is random.
• Match on empty node IDs: when both node IDs are empty, the stored instance matches the 0.4 lookup. A change to MORANGO_SYSTEM_ID then does not create a new instance.
• Partial mocks: some of the tests mock ifcfg.interfaces, but none mock uuid.getnode. Instances created in setUp, or by other code before the mock applies, use the real host values.
• Related: Switch ifcfg utility for get-mac to find MAC addresses #216 (replace ifcfg for MAC address lookup).

Notes & Edge Cases

• Production behavior (unconfirmed impact): on a device without a readable MAC address, a change to MORANGO_SYSTEM_ID does not change the instance ID. The code comment says the legacy lookup must not "mess with" a matching legacy instance, so this can be intended. This issue does not require a change to it.
• Environment variables: if you set MORANGO_NODE_ID to hide the failures locally, InstanceIDModelTestCase::test_consistent_0_5_instance_id fails instead. That test expects a node ID from its mocked ifcfg value.
• Data integrity: no data impact. Only the tests are affected.

Metadata

Severity / Priority: P2 (CI fails at random. Rerunning the job is the workaround.)
Area: bug, tests, CI

AI Usage

I used Claude Code to look into these failures while working on #410. Claude found that the failures depend on the host MAC address, reproduced them with the pytest plugin above, and showed the same results on release-v0.9.x@799d70d and on the #410 branch. It then drafted this issue, and I reviewed it before filing.