Repository navigation
HBASE-30539 CreateTableProcedure ignores failed table descriptor write (branch-2) - #8879
Open
Umeshkumar9414 wants to merge 2 commits into
Open
Umeshkumar9414 wants to merge 2 commits into
Umeshkumar9414 wants to merge 2 commits into
Conversation
FSTableDescriptors.writeTableDescriptor retried 10 times, logged every failure at DEBUG and then returned null. CreateTableProcedure.createFsLayout ignored the result, so a failed .tableinfo write still let the procedure go on to ADD_TO_META and ASSIGN_REGIONS. Region servers then failed every open with "Missing table descriptor", the TRSP retried forever and the create procedure stayed in ASSIGN_REGIONS holding the table lock. Make writeTableDescriptor throw IOException (with the last failure as cause) once all attempts are exhausted, and log failed attempts at WARN. The create procedure now stays in WRITE_FS_LAYOUT and retries, and the other callers that ignored the result (clone snapshot, snapshot manifest, MasterRegion, backup) no longer silently continue either. createTableDescriptorForTableDirectory returns false only when an equal descriptor already exists. Backport of the master change (apache#8877) to branch-2. Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
…riptorWriteFails Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport of the master change #8877 to
branch-2.JIRA: https://issues.apache.org/jira/browse/HBASE-30539
FSTableDescriptors.writeTableDescriptorused to returnnullafter 10 failed attempts, andCreateTableProcedureignored that, so the create moved on tohbase:metaand region assignment with no.tableinfoon HDFS. The region server then failed every open withMissing table descriptorand the assign procedure retried forever.writeTableDescriptornow throwsIOExceptionafter the last attempt (failed attempts logged at WARN), so the create procedure stays inWRITE_FS_LAYOUTand retries.The cherry-pick from master had three cosmetic conflicts in
FSTableDescriptors.java(thehbase:metalog line and two javadoc hunks), resolved by keeping this branch's log line and taking the new javadoc. The main logic is identical to master. In the test,HBaseTestingUtilis renamed toHBaseTestingUtility, which is the name on this branch.Tests run locally on this branch (
-Dhadoop.profile=3.0) and passing:TestCreateTableProcedureDescriptorWriteFailure(3),TestFSTableDescriptors,TestCreateTableProcedure,TestCloneSnapshotProcedure,TestMasterRegionInitialize.Co-authored-by: Claude Sonnet 5.5 noreply@anthropic.com