tff.aggregators.DifferentiallyPrivateFactory

View source on GitHub

UnweightedAggregationFactory for tensorflow_privacy DPQueries.

Inherits From: UnweightedAggregationFactory

tff.aggregators.DifferentiallyPrivateFactory(
    query: tfp.DPQuery,
    record_aggregation_factory: Optional[tff.aggregators.UnweightedAggregationFactory] = None
)

Used in the notebooks

The created tff.templates.AggregationProcess aggregates values placed at CLIENTS according to the provided DPQuery, and outputs the result placed at SERVER.

A DPQuery defines preprocessing to perform on each value, and postprocessing to perform on the aggregated, preprocessed values. Provided the preprocessed values ("records") are aggregated in a way that is consistent with the DPQuery, formal (epsilon, delta) privacy guarantees can be derived. This aggregation is controlled by record_aggregation_factory.

A simple summation (using the default tff.aggregators.SumFactory) is usually acceptable. Aggregations that change the records (such as compression or secure aggregation) may be allowed so long as they do not increase the sensitivity of the query. It is the users' responsibility to ensure that the mode of aggregation is consistent with the DPQuery. Note that the DPQuery's built-in aggregation functions (accumulate_preprocessed_record and merge_sample_states) are ignored in favor of the provided aggregator.

The state of the created AggregationProcess contains a DPEvent released by the DPQuery that can be extracted using differential_privacy. extract_dp_event_from_state.

Methods

create

View source

create(
    value_type: factory.ValueType
) -> tff.templates.AggregationProcess

Creates a tff.aggregators.AggregationProcess without weights.

The provided value_type is a non-federated tff.Type, that is, not a tff.FederatedType.

The returned tff.aggregators.AggregationProcess will be created for aggregation of values matching value_type placed at tff.CLIENTS. That is, its next method will expect type <S@SERVER, {value_type}@CLIENTS>, where S is the unplaced return type of its initialize method.

gaussian_adaptive

View source

@classmethod
gaussian_adaptive(
    noise_multiplier: float,
    clients_per_round: float,
    initial_l2_norm_clip: float = 0.1,
    target_unclipped_quantile: float = 0.5,
    learning_rate: float = 0.2,
    clipped_count_stddev: Optional[float] = None
) -> tff.aggregators.UnweightedAggregationFactory

DifferentiallyPrivateFactory with adaptive clipping and Gaussian noise.

Performs adaptive clipping and addition of Gaussian noise for differentially private learning. For details of the DP algorithm see McMahan et. al (2017) https://arxiv.org/abs/1710.06963 The adaptive clipping uses the geometric method described in Thakkar et al. (2019) https://arxiv.org/abs/1905.03871

The adaptive clipping parameters have been chosen to yield a process that starts small and adapts relatively quickly to the median, without using much of the privacy budget. This works well on most problems.

gaussian_fixed

View source

@classmethod
gaussian_fixed(
    noise_multiplier: float, clients_per_round: float, clip: float
) -> tff.aggregators.UnweightedAggregationFactory

DifferentiallyPrivateFactory with fixed clipping and Gaussian noise.

Performs fixed clipping and addition of Gaussian noise for differentially private learning. For details of the DP algorithm see McMahan et. al (2017) https://arxiv.org/abs/1710.06963

tree_adaptive

View source

@classmethod
tree_adaptive(
    noise_multiplier: float,
    clients_per_round: float,
    record_specs: Collection[tf.TensorSpec],
    initial_l2_norm_clip: float = 0.1,
    restart_warmup: int = 128,
    restart_frequency: int = 1024,
    target_unclipped_quantile: float = 0.5,
    clip_learning_rate: float = 0.2,
    clipped_count_stddev: Optional[float] = None,
    noise_seed: Optional[int] = None
) -> tff.aggregators.UnweightedAggregationFactory

DifferentiallyPrivateFactory with adaptive clipping and tree aggregation.

Performs clipping on client, averages clients records, and adds noise for differential privacy. The noise is estimated based on tree aggregation for the cumulative summation over rounds, and then take the residual between the current round and the previous round. Combining this aggregator with a SGD optimizer on server can be used to implement the DP-FTRL algorithm in "Practical and Private (Deep) Learning without Sampling or Shuffling" (https://arxiv.org/abs/2103.00039).

The standard deviation of the Gaussian noise added at each tree node is l2_norm_clip * noise_multiplier. Note that noise is added during summation of client model updates per round, before normalization (the noise will be scaled down when dividing by clients_per_round). Thus noise_multiplier can be used to compute the (epsilon, delta) privacy guarantee as described in the paper.

The l2_norm_clip is estimated and periodically reset for tree aggregation based on "Differentially Private Learning with Adaptive Clipping" (https://arxiv.org/abs/1905.03871).

tree_aggregation

View source

@classmethod
tree_aggregation(
    noise_multiplier: float,
    clients_per_round: float,
    l2_norm_clip: float,
    record_specs: Collection[tf.TensorSpec],
    noise_seed: Optional[int] = None,
    use_efficient: bool = True,
    record_aggregation_factory: Optional[tff.aggregators.UnweightedAggregationFactory] = None
) -> tff.aggregators.UnweightedAggregationFactory

DifferentiallyPrivateFactory with tree aggregation noise.

Performs clipping on client, averages clients records, and adds noise for differential privacy. The noise is estimated based on tree aggregation for the cumulative summation over rounds, and then take the residual between the current round and the previous round. Combining this aggregator with a SGD optimizer on server can be used to implement the DP-FTRL algorithm in "Practical and Private (Deep) Learning without Sampling or Shuffling" (https://arxiv.org/abs/2103.00039).

The standard deviation of the Gaussian noise added at each tree node is l2_norm_clip * noise_multiplier. Note that noise is added during summation of client model updates per round, before normalization (the noise will be scaled down when dividing by clients_per_round). Thus noise_multiplier can be used to compute the (epsilon, delta) privacy guarantee as described in the paper.