We will now consider a different machine-learning objective: statistical learning. Here, it is assumed that each data point is drawn i.i.d. from some unknown underlying distribution. The goal is to process all of this data and then output some hypothesis; this hypothesis should have small expected loss on a new
data point drawn from the same distribution. This loss is specified by a loss function given in advance. Luckily, we will be able to adapt Mechanism 3and the results already obtained for regret minimization in order to solve this problem.
2.3.1 Model
Our data points are objects z ∈ Z. We are given a hypothesis class H which we will assume is
parameterized by vectors Rdbut more broadly can be any Hilbert space endowed with a norm k · k; for
convenience we will treat elementsh∈ H as vectors which can be added, scaled, etc. We are also given a
loss function`:H × Z →Rthat is convex inH. We assume throughout the paper that the loss function
is1-Lipschitz in h; that is, for anyz∈ Z and any h, h0 ∈ H we have|`(h, z)−`(h0, z)| ≤ kh−h0k.
In many common scenarios, Z is the space of pairs (x, y)from the cross product X × Y, withx the
feature input and y the label, though in our setting Z can be a more generic object. For example, in the canonical problem oflinear regression, we have that Z=X × Y =Rd×R, the hypothesis class is
vectorsH=Rd, and the loss function is defined according to squared error `(h,(x, y)) := (h>x−y)2.
Thedata-purchasing statistical learning problemis parameterized by the data spaceZ, hypothesis spaceH, loss function`, number of arriving data points T, and expected budget constraint B. Aproblem instance consists of a distributionDon the set Z and a sequence of pairs(c1, z1), . . . ,(cT, zT) where
eachztis a data point drawn i.i.d. according to Dand each ct∈[0,1] is the private cost associated with
that data point. The costs may be arbitrarily chosen,i.e. we consider a worst-case model of costs. (For
case of our setting.)
In this problem, the task is to design a mechanism implementing the operations “post”, “receive”, and
“predict” and interacting with the problem instance as follows. • For each time step t= 1, . . . , T:
1. The mechanism posts a pricing function πt:Z →R, where πt(z) is the price posted for data
pointz.
2. Agent tarrives, possessing (ct, zt).
3. If the posted priceπt(zt)≥ct, then agent taccepts the transaction: The mechanism pays
πt(zt) to the agent andreceives (ct, zt). Ifπt(zt)< ct, agent t rejects the transaction and
the mechanismreceives a null signal.
• The mechanism outputs a prediction ¯h∈ H.
The risk or predictive error of a hypothesis is
L(h) = E
z∼D`(h, z)
and the goal of the mechanism is to minimize the risk L(¯h) of its final hypothesis ¯h. The benchmark
is the optimal hypothesis in the class, h∗ = arg minh∈HL(h). The mechanism must guarantee that, for every input sequence(c1, z1), . . . ,(cT, zT), it spends at most B in expectation over its own internal
randomness.
Note the differences from regret minimization: The mechanism is not required to post a hypothesis at each time step, but only a single final hypothesis ¯h. And rather than being judged on performance
against the arriving sequence of data, the mechanism is measured by how well this final ¯h performs on a
new data point.
To recap, the mechanism is given the parameters Z,H, `,T, and B, but the problem instance is
completely unknown to the mechanism prior to to the arrivals. The design problem of the mechanism is how to choose the pricing function πt topost at each time, how to update based on receiving data, and
2.3.2 Results
The key idea is to use a standard tool known as the “online-to-batch conversion,” where we may leverage an online learning algorithm for use in a “batch” setting. A sketch of this technique is as follows, and further details can be found in, e.g., Shalev-Shwartz[2012].
Given a batch of i.i.d. data points, feed them one-by-one into the no-regret algorithm. For each data pointz, feed the algorithm the loss function f(·) =`(·, z). Because the algorithm has low regret, its
hypotheses had low loss on average over the arriving data. But since each data point was drawn i.i.d., this means that these hypotheses on average predict well on an i.i.d. draw from the distribution. Thus it suffices to take the mean of the hypotheses to obtain low risk.
Lemma 2.3.1(Online-to-BatchCesa-Bianchi et al.[2004]). Suppose the sequence of convex loss functions
f1, . . . , fT are drawn i.i.d. from a distributionF and that an online learning algorithm with hypotheses
h1, . . . , hT achieves expected regret R(T). Let L(h) = Ef∼Ff(h) and h∗ = arg minh∈HL(h). For
¯ h1:T = T1 PTt=1ht, we have E f1,...,fT, alg L(¯h1:T)≤ L(h∗) + 1 TR(T).
We note that this conversion continues to hold in the data-purchasing no-regret setting, as all that is required is that the algorithm output a hypothesis ht at each step and that there is a regret bound on
these hypotheses.
Given this result, the idea is to simply run the regret-minimization Mechanism3on the arriving agents. At each stage, Mechanism 3posts a hypothesis ht. We then aggregate these hypothesis by averaging to
obtain our final prediction. This is summarized in Mechanism4.
Algorithm 4 Mechanism for statistical learning data-purchasing problem.
1: Input: parameters K, η, access to OLA
2: Identify each data pointz with the loss function f(·) =`(·, z)
3: Run Mechanism3with parameters η, K and access to OLA
4: Leth1, . . . , hT be the resulting hypotheses
5: Output¯h= T1 P
Theorem 2.3.1. Mechanism4 guarantees spending at most B in expectation and EL(¯h)≤ L(h∗) +O max n g √ B , q 1 T o ,
where g=qγT ,A·γT,maxA, assuming that γT ,A and γT ,maxA are known in advance up to a constant factor.
If one assumes approximate knowledge respectively of γT ,A, of ¯c= T1 Pt√ct, or of µ= T1 Ptct,
then the guarantee holds with respectively g=√γT ,A,g=√¯c, org=µ1/4.
Proof. By Theorem 2.2.2, Mechanism3 guarantees an expected regret of O
max n T √ Bg , √ T o
when run with the specified prior knowledge for the specified values ofg. Therefore, the online-to-batch
conversion of Lemma2.3.1 proves the theorem.