Notes from the 9/9/2022 meeting of TFF collaborators

  • Discussion of Jeremy’s proposal, continued
  • What to cover specifically - walk through both + verify against the understanding of TFF
  • Brief recap for new audience:
    • Right now, all communication initiated by the server / coordinator to clients
    • In many scenarios, clients can’t be addressed, they have no ingress endpoints
    • Want a setup with server-side endpoint to connect to
    • Desirable addition to the ecosystem, relevant to many application scenarios
  • Problem identified in Jeremy’s proposal - the concept of a task store where all responses are uploaded is at odds with the privacy properties we’re trying to preserve. Flow of data to the server must be mediated by the federated operators, and should’nt be happening at the granularity of individual TFF executor requests/responses.
  • (discussion of the TFF executor protocol)
  • (a few minutes of conceptual intro to the executor interface in this YouTube recording)
  • TFF supports deployment in two regimes:
    • Stateful clients.
      • General TFF executor interface is designed to support this mode.
      • Clients host executors.
      • Handles returned in response to executor requests hold client-side state.
      • Passing those handles to subsequent executor requests supports client-side operations and pipelining.
      • This is certainly possible with client-initiainited connections, albeit there is no component currently in the TFF repo designed for this.
      • With client-initiated connections, control is still top-down, driven by the executor on the server side.
      • Whereas mechanisms to orchestrate the exchange of requests and responses can vary depending on which party initiates communication, whether connections are long running, etc., at a logical level requests are still issued by the server.
      • Client can contact the server repeatedly to feed responses and ask for subsequent requests.
      • Client still retains state locally as it keeps contacting the server.
      • Loss of state on the client or timeout on the sever still results in failure of the entire computation (same as in the regular executor setup).
    • Stateless clients.
      • Not compatible with general TFF executor protocol, as per the above.
      • But, it can be supported by the MapReduce compiler - there’s a library function in TFF in the tff.mapreduce.backends module to translate classes of TFF computations into a MapReduce-like form that can operate in the stateless client regime.
  • Next steps: Jeremy’s proposal can be salvaged (but needs to incorporate statefulness on the client side)