Repository navigation
Upgrade Guava to 32.0.0-jre and bump minimum CDAP to 6.12.0-SNAPSHOT - #683
AbhishekKumar9984 wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request updates the Guava dependency version across multiple POM files to use the ${guava.version} property. The review feedback points out that specifying the version in the submodules is redundant due to parent dependency management. Additionally, the selected Guava version (32.0.0-jre) is vulnerable to CVE-2023-2976, and it is recommended to upgrade to a secure version like 32.1.3-jre and ensure consistency across all profiles.
2a7fcbb to
f10f18c
Compare
deepika-lalwani123
left a comment
There was a problem hiding this comment.
This depends on cdapio/cdap#16279. The currently published 6.12.0-SNAPSHOT still has Guava 13, so this PR cannot be validated or merged until that PR merges and CDAP (and Twill) are published. Please call out the dependency in the PR description.
|
@deepika-lalwani123 Updated the PR description to explicitly call out the dependency on cdapio/cdap#16279, and addressed all inline review comments. |
shruthi713
left a comment
There was a problem hiding this comment.
LGTM. Please add a line to the description on why io.cdap.plugin.db.* was added to the exports in aurora-mysql-plugin and generic-database-plugin.
|
@shruthi713 Thanks for the approval! Updated the PR description to explain why |
|
The build check on a6b9719 failed on 18 Nov: 24 tests in 12 suites, with NoSuchMethodError: ...Service.stopAndWait() from the CDAP test framework. A new CDAP 6.12.0-SNAPSHOT was published today, so please re-run build. If it still fails, this can't merge until CDAP publishes a snapshot built with Guava 32. |
fixes3 fixes34 fixes343 fixes
a6b9719 to
ca6310c
Compare
There was a problem hiding this comment.
A few more points on the latest commit:
-
Build is failing on the head commit (
ca6310c): 24 of 202 tests. The failures come from the CDAP test framework, not plugin code:NoSuchMethodError: io.cdap.cdap.internal.app.worker.sidecar.ArtifactLocalizerService.stopAndWait()atTestBase.finish(TestBase.java:623), plusIncompatibleClassChangeError: Implementing class. So the6.12.0-SNAPSHOTthis build resolved still calls Guava 13 APIs. Note that cdapio/cdap#16279 was merged into thecs_guava_upgradebranch, notdevelop. A new snapshot was published on 19 Nov at 15:48 UTC, after this build ran (14:57 UTC). Please re-run the build. If it still fails, this can't merge until a CDAP snapshot built with Guava 32 is published. -
Parent artifact range is not tightened.
pom.xmlL379-380 still has[6.11.0-SNAPSHOT,7.0.0-SNAPSHOT)whilecdap.versionis now6.12.0-SNAPSHOT. Please raise the lower bound to6.12.0-SNAPSHOT, as was done in the ftp, servicenow and salesforce PRs. -
maven-bundle-pluginin the cloudsql modules. The reply on my earlier thread says the block was restored, but at head the firstmaven-bundle-plugindeclaration is still removed fromcloudsql-mysql-plugin/pom.xmlandcloudsql-postgresql-plugin/pom.xml; only the second, pre-existing declaration remains. That may well be harmless since the second one already set the specific_exportcontents, but please paste theExport-Packageline from the built jar before and after to confirm. -
io.cdap.plugin.db.*export. It is added only toaurora-mysql-pluginandgeneric-database-plugin. What failure showed it was needed, and why doesn't it apply to the other modules that extend the samedatabase-commonsclasses (for examplecloudsql-mysql-plugin, which still exports onlydb.source.*anddb.sink.*)?
|
@deepika-lalwani123 @shruthi713 Thanks for the reviews and approvals! Addressed all points:
|
Depends on: cdapio/cdap#16291 (follow-up to cdapio/cdap#16279 targeting
develop)Issue :
Resolved security vulnerabilities (CVE-2023-2976 / CVE-2020-8908) associated with older versions of
com.google.guava:guava(13.0.1/31.x) pulled in directly and transitively across database plugins.Root Cause :
Root and submodule POMs referenced older Guava versions (
13.0.1,31.0.1-jre,31.1-jre), and CDAP, Hadoop, and CloudSQL socket-factory dependencies transitively brought in conflicting older versions of Guava causing compilation and test issues.Fix :
<guava.version>to32.0.0-jreand pinnedcom.google.guava:guavain root<dependencyManagement>(removing redundant submodule version overrides and thee2e-testsprofile override).maven-enforcer-plugin(requireUpperBoundDeps) forcom.google.guava:guava.<cdap.version>from6.11.0to6.12.0-SNAPSHOTand tightened<cdapArtifacts>parent ranges to[6.12.0-SNAPSHOT,7.0.0-SNAPSHOT).io.cdap.plugin.db.*to<_exportcontents>inaurora-mysql-plugin/pom.xmlandgeneric-database-plugin/pom.xmlbecauseAuroraMysqlSourceandDatabaseSourceare the only two source plugins that inheritAbstractDBSource#getDBRecordType()returningio.cdap.plugin.db.DBRecord.class(all other 14 modules overridegetDBRecordType()with a module-specific*DBRecordin an already-exported package), which Hadoop'sDBConfiguration#getInputClass()loads dynamically viaExport-Package.Testing evidence: https://docs.google.com/spreadsheets/d/1Cw6Z_k3wlks7sAC4-Kg6FTIMRL7-zxLlsJIXvVJLrb8/edit?gid=0#gid=0