Repository navigation
Conversation
Collaborator
Author
|
@JulienVig I think we need to refactor a bit the architecture of disco. The goal would be to decouple some components a bit more and make it easier to implement different algorithms in the future. PR #1289 already does some refactoring in this direction, but still there are some smells, like algorithm-specific parameters being thrown in the trainer. I will use this PR to come up with a design doc for a new architecture which we can review together before moving forward with it (if you agree) |
Collaborator
|
Yes, refactoring the architecture makes a lot of sense. Let me know when the document is ready for review. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Disco currently models collaborative learning variations primarily through a single
aggregationStrategyaxis while embedding algorithm-specific state and update clipping directly insideTrainer. This coupling prevents composing client-side optimization strategies (such as FedProx, SCAFFOLD, or Ditto) with robust or private aggregators (ByzantineRobustAggregator,SecureAggregator) and forcesTrainerandModelto accumulate ad-hoc branches for every new algorithm. This PR introduces a design document (docs/TRAINING_ARCHITECTURE.md) detailing these architectural limitations and proposing a modular target architecture.Technical plan
Add
docs/TRAINING_ARCHITECTURE.mddocumenting the current responsibility leaks acrossTrainingInformation,Trainer,Model, andAggregator, and specifying the proposed three-axis decomposition (intra-round local training strategy, post-round update transforms/privacy, and inter-client weight aggregation). The document defines concrete TypeScript contracts for composableRoundHooklifecycle stages, backend-agnosticModellocal objective options operating strictly ontrainableWeights, and a phased migration path.Deviations from plan
None.
Todo
Model.trainableWeightsandTrainerRoundHookpipeline).trainingStrategyfromaggregationStrategyinTrainingInformation,cli, andwebapp).ByzantineRobustAggregator).