PROPUESTA A PRIORI Y A POSTERIORI DEL ESTADO DE COSAS INCONSTITUCIONAL

of the marginal distribution into the regularization framework. We believe that this framework will extend to other structures that may constrain the learning task and bring about effective learnability. One important example of such structure is invariance under certain classes of natural transforma- tions, such as invariance under lighting conditions in vision. Some ideas are presented in Sindhwani (2004).

Acknowledgments

We are grateful to Marc Coram, Steve Smale and Peter Bickel for intellectual support and to NSF funding for financial support. We would like to acknowledge the Toyota Technological Institute for its support for this work. We also thank the anonymous reviewers for helping to improve the paper. References

R.A. Adams. Sobolev spaces. Academic Press New York, 1975.

N. Aronszajn. Theory of reproducing kernels. Transactions of the American Mathematical Society, 68(3):337–404, 1950.

M. Belkin. Problems of Learning on Manifolds. PhD thesis, The University of Chicago, 2003. M. Belkin, I. Matveeva, and P. Niyogi. Regularization and semi-supervised learning on large graphs.

M. Belkin and P. Niyogi. Laplacian eigenmaps for dimensionality reduction and data representation.

Neural Computation, 15(6):1373–1396, 2003a.

M. Belkin and P. Niyogi. Using manifold structure for partially labeled classification. Advances in

Neural Information Processing Systems, 15:929–936, 2003b.

M. Belkin and P. Niyogi. Towards a theoretical foundation for Laplacian-based manifold methods.

Proc. of COLT, 2005.

M. Belkin, P. Niyogi, and V. Sindhwani. On manifold regularization. Proceedings of the Tenth

International Workshop on Artificial Intelligence and Statistics (AISTAT 2005), 2005.

Y. Bengio, J.F. Paiement, P. Vincent, and O. Delalleau. Out-of-sample extensions for LLE, isomap, MDS, Eigenmaps, and spectral clustering. Advances in Neural Information Processing Systems, 16, 2004.

K. Bennett and A. Demiriz. Semi-supervised support vector machines. Advances in Neural Infor-

mation Processing Systems, 11:368–374, 1999.

A. Blum and S. Chawla. Learning from labeled and unlabeled data using graph mincuts. Proc. 18th

International Conf. on Machine Learning, pages 19–26, 2001.

A. Blum and T. Mitchell. Combining labeled and unlabeled data with co-training. Proceedings of

the eleventh annual conference on Computational learning theory, pages 92–100, 1998.

O. Bousquet, O. Chapelle, and M. Hein. Measure based regularization. Advances in Neural Infor-

mation Processing Systems, 16, 2004.

M. Brand. Continuous nonlinear dimensionality reduction by kernel eigenmaps. Int. Joint Conf.

Artif. Intel, 2003.

O. Chapelle, B. Sch ¨olkopf, and A. Zien, editors. Semi-supervised Learning. MIT Press, 2006. O. Chapelle, J. Weston, and B. Scholkopf. Cluster kernels for semi-supervised learning. Advances

in Neural Information Processing Systems, 15:585–592, 2003.

O. Chapelle and A. Zien. Semi-supervised classification by low density separation. Proceedings of

the Tenth International Workshop on Artificial Intelligence and Statistics, pages 57–64, 2005.

F.R.K. Chung. Spectral Graph Theory. American Mathematical Society, 1997.

RR Coifman, S. Lafon, AB Lee, M. Maggioni, B. Nadler, F. Warner, and SW Zucker. Geomet- ric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps.

Proceedings of the National Academy of Sciences, 102(21):7426–7431, 2005.

A. Corduneanu and T. Jaakkola. On information regularization. Proceedings of the Ninth Annual

Conference on Uncertainty in Artificial Intelligence, 2003.

F. Cucker and S. Smale. On the mathematical foundations of learning. American Mathematical

O. Delalleau, Y. Bengio, and N. Le Roux. Efficient non-parametric function induction in semi- supervised learning. Proceedings of the Tenth International Workshop on Artificial Intelligence

and Statistics (AISTAT 2005), 2005.

M.P. Do Carmo. Riemannian Geometry. Birkhauser, 1992.

D.L. Donoho and C. Grimes. Hessian eigenmaps: Locally linear embedding techniques for high- dimensional data. Proceedings of the National Academy of Sciences, 100(10):5591–5596, 2003. S. Dumais, J. Platt, D. Heckerman, and M. Sahami. Inductive learning algorithms and representa-

tions for text categorization. Proceedings of the Seventh International Conference on Information

and Knowledge Management, 11:16, 1998.

T. Evgeniou, M. Pontil, and T. Poggio. Regularization networks and support vector machines.

Advances in Computational Mathematics, 13(1):1–50, 2000.

G. Fung and O.L. Mangasarian. Semi-supervised support vector machines for unlabeled data clas- sification. Optimization Methods and Software, 15(1):99–05, 2001.

A. Grigor’yan. Heat kernels on weighted manifolds and applications. Cont. Math, 398, 93-191, 2006.

J. Ham, D.D. Lee, and L.K. Saul. Semisupervised alignment of manifolds. Proceedings of the

Annual Conference on Uncertainty in Artificial Intelligence, Z. Ghahramani and R. Cowell, Eds,

10:120–127, 2005.

M. Hein, J.Y. Audibert, and U. von Luxburg. From graphs to manifolds-weak and strong point- wise consistency of graph Laplacians. Proceedings of the 18th Conference on Learning Theory

(COLT), pages 470–485, 2005.

T. Joachims. Transductive inference for text classification using support vector machines. Proceed-

ings of the Sixteenth International Conference on Machine Learning, pages 200–209, 1999.

T. Joachims. Transductive learning via spectral graph partitioning. Proceedings of the International

Conference on Machine Learning, pages 290–297, 2003.

C. Kemp, T.L. Griffiths, S. Stromsten, and J.B. Tenenbaum. Semi-supervised learning with trees.

Advances in Neural Information Processing Systems, 16, 2004.

R.I. Kondor and J. Lafferty. Diffusion kernels on graphs and other discrete input spaces. Proc. 19th

International Conf. on Machine Learning, 2002.

S. Lafon. Diffusion Maps and Geometric Harmonics. PhD thesis, Yale University, 2004.

K. Nigam, A.K. Mccallum, S. Thrun, and T. Mitchell. Text classification from labeled and unlabeled documents using EM. Machine Learning, 39(2):103–134, 2000.

R.M. Rifkin. Everything Old Is New Again: A Fresh Look at Historical Approaches in Machine

S.T. Roweis and L.K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Sci-

ence, 290(5500):2323, 2000.

B. Scholkopf, C. Burges, and V. Vapnik. Extracting support data for a given task. Proceedings,

First International Conference on Knowledge Discovery & Data Mining, Menlo Park, 1995.

B. Scholkopf and A.J. Smola. Learning with Kernels. MIT Press Cambridge, Mass, 2002.

M. Seeger. Learning with labeled and unlabeled data. Inst. for Adaptive and Neural Computation,

technical report, Univ. of Edinburgh, 2001.

V. Sindhwani. Kernel machines for semi-supervised learning. Master’s thesis, The University of Chicago, 2004.

V. Sindhwani, M. Belkin, and P. Niyogi. The geometric basis of semi-supervised learning. In O. Chapelle, A. Zien, and B. Sch ¨olkopf, editors, Semi-supervised Learning, chapter 12, pages 217–235. MIT Press, 2006.

V. Sindhwani, P. Niyogi, and M. Belkin. Beyond the point cloud: from transductive to semi- supervised learning. In Proceedings, Twenty Second International Conference on Machine Learn-

ing, 2005.

A. Smola and R. Kondor. Kernels and regularization on graphs. Conference on Learning Theory,

COLT/KW, 2003.

M. Szummer and T. Jaakkola. Partially labeled classification with Markov random walks. Advances

in Neural Information Processing Systems, 14:945–952, 2002.

J.B. Tenenbaum, V. Silva, and J.C. Langford. A global geometric framework for nonlinear dimen- sionality reduction. Science, 290(5500):2319, 2000.

A.N. Tikhonov. Regularization of incorrectly posed problems. Sov. Math. Dokl, 4:1624–1627, 1963. I. W. Tsang and J. T. Kwok. Very large scale manifold regularization using core vector machines.

NIPS 2005 Workshop on Large Scale Kernel Machines, 2005.

V.N. Vapnik. Statistical Learning Theory. Wiley-Interscience, 1998.

J.P. Vert and Y. Yamanishi. Supervised graph inference. Advances in Neural Information Processing

Systems, 17:1433–1440, 2005.

U. von Luxburg, M. Belkin, and O. Bousquet. Consistency of spectral clustering. Max Planck

Institute for Biological Cybernetics Technical Report TR, 134, 2004.

G. Wahba. Spline models for observational data. Society for Industrial and Applied Mathematics Philadelphia, Pa, 1990.

D. Zhou, O. Bousquet, T.N. Lal, J. Weston, and B. Scholkopf. Learning with local and global consistency. Advances in Neural Information Processing Systems, 16:321–328, 2004.

X. Zhu, Z. Ghahramani, and J. Lafferty. Semi-supervised learning using Gaussian fields and har- monic functions. In Proceedings of the 20th International Conference on Machine Learning, 2003.

X. Zhu, J. Kandola, Z. Ghahramani, and J. Lafferty. Nonparametric transforms of graph kernels for semi-supervised learning. Advances in Neural Information Processing Systems, 17, 2005.