Unsupervised Learning and the Natural Origins of Content

Avant, Vol. X, No. 3/2019, doi: 10.26913/avant.2019.03.11
published under license CC BY-NC-ND 3.0

Tomasz Korbak
Institute of Philosophy and Sociology, Polish Academy of Sciences
University of Warsaw
tomasz.korbak @ gmail.com

Published Online First 12 September 2019   Download full text

Abstract: In this paper, I evaluate the prospects and limitations of radical enactivism as recently developed by Hutto and Myin (henceforth, “H&M”) (2013, 2017). According to radical enactivism, cognition does not essentially involve content and admits explanations on a semantic level only as far as cognition is scaffolded with social and linguistic practices. I investigate their claims, focusing on H&M’s criticism of the predictive processing account of cognition (dubbed the bootstrap hell argument) and their own account of the emergence of content (the natural origins of content). I argue that H&M fail on two fronts: unsupervised learning can arrive at contentful representations and H&M’s account of the emergence of content assumes an equivalent bootstrapping. My case is illustrated with Skyrms’ evolutionary game-theoretic account of the emergence of content and recent deep learning research on neural language models. These arguments cast a shadow of doubt on whether radical enactivism is philosophically interesting or empirically plausible.

Keywords: hard problem of content; radical enactivism; predictive processing; neural language models; deep learning; bootstrap hell; semantic information


References

Adams, R. A., Huys, Q. J. M., & Roiser, J. P. (2015). Computational Psychiatry: Towards a mathematically informed understanding of mental illness. Journal of Neurology, Neurosurgery & Psychiatry, jnnp-2015-310737.
https://doi.org/10.1136/jnnp-2015-310737
Alksnis, N. (2015). A Dilemma or a Challenge? Assessing the All-star Team in a Wider Context. Philosophia, 43(3), 669-685.
https://doi.org/10.1007/s11406-015-9618-2
Barrett, L. F. (2018). How emotions are made: The secret life of the brain (Paperback edition). London: PAN Books.
Bergstrom, C. T., & Rosvall, M. (2011). The transmission sense of information. Biology & Philosophy, 26(2), 159-176.
https://doi.org/10.1007/s10539-009-9180-z
Bickhard, M. H. (2009). The interactivist model. Synthese, 166(3), 547-591.
https://doi.org/10.1007/s11229-008-9375-x
Bouchacourt, D., & Baroni, M. (2018). How agents see things: On visual representations in an emergent language game. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 981-985. Retrieved from http://aclweb.org/anthology/D18-1119
https://doi.org/10.18653/v1/D18-1119
Clark, A. (2016). Surfing uncertainty: Prediction, action, and the embodied mind. Oxford ; New York: Oxford University Press.
https://doi.org/10.1093/acprof:oso/9780190217013.001.0001
Dennett, D. C. (2017). From bacteria to Bach and back: The evolution of minds (First edition). New York: W.W. Norton & Company.
Dretske, F. I. (1983). Knowledge & the flow of information. Cambridge, Mass.: MIT Press.
https://doi.org/10.1017/S0140525X00014631
Grice, P. (1957). Meaning. The Philosophical Review, 66, 377-388.
https://doi.org/10.2307/2182440
Harvey, M. I. (2015). Content in languaging: Why radical enactivism is incompatible with representational theories of language. Language Sciences, 48, 90-129.
https://doi.org/10.1016/j.langsci.2014.12.004
Hohwy, J. (2013). The Predictive Mind.
https://doi.org/10.1093/acprof:oso/9780199682737.001.0001
Hutto, D. D. (2018). Getting into predictive processing’s great guessing game: Bootstrap heaven or hell? Synthese, 195(6), 2445-2458.
https://doi.org/10.1007/s11229-017-1385-0
Hutto, D. D., & Myin, E. (2012). Radicalizing Enactivism: Basic Minds without Content. https://doi.org/10.7551/mitpress/9780262018548.001.0001
https://doi.org/10.7551/mitpress/9780262018548.001.0001
Hutto, D. D., & Myin, E. (2017). Evolving enactivism: Basic minds meet content. Cambridge, Massachusetts: MIT Press.
https://doi.org/10.7551/mitpress/9780262036115.001.0001
Isaac, A. M. C. (2019). The Semantics Latent in Shannon Information. The British Journal for the Philosophy of Science, 70(1), 103-125.
https://doi.org/10.1093/bjps/axx029
Jelinek, F., & Mercer, R. (1980). Interpolated estimation of Markov source parameters from sparse data. Pattern Recognition in Practice. Proc. Workshop Amsterdam, May 1980, 381-397, 401.
Kolchinsky, A., & Wolpert, D. H. (2018). Semantic information, autonomous agency and non-equilibrium statistical physics. Interface Focus, 8(6), 20180041.
https://doi.org/10.1098/rsfs.2018.0041
Korbak, T. (2015). Scaffolded Minds And The Evolution Of Content In Signaling Pathways. Studies in Logic, Grammar and Rhetoric, 41(1).
https://doi.org/10.1515/slgr-2015-0022
Lewis, D. (1969). Convention: A Philosophical Study. Cambirdge: Harvard University Press.
Martinez, M. (2019). Representations are Rate-Distortion Sweet Spots. Philosophy of Science.
https://doi.org/10.1086/705493
Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributed Representations of Words and Phrases and their Compositionality. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, & K. Q. Weinberger (Eds.), Advances in Neural Information Processing Systems, 26 (pp. 3111-3119). Retrieved from http://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf
Miłkowski, M. (forthcoming). Thinking about Semantic Information.
Miłkowski, M. (2015). The Hard Problem Of Content: Solved (Long Ago). Studies in Logic, Grammar and Rhetoric, 41(1), 73-88.
https://doi.org/10.1515/slgr-2015-0021
Millikan, R. G. (1984). Language, thought, and other biological categories: New foundations for realism. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&scope=site&db=nlebk&db=nlabk&AN=49593
Pattee, H. H., & Rączaszek-Leonardi, J. (2012). Laws, language and life: Howard Pattee’s classic papers on the physics of symbols with contemporary commentary. Dordrecht ; New York: Springer.
https://doi.org/10.1007/978-94-007-5161-3
Pearl, J., & Mackenzie, D. (2018). The book of why: The new science of cause and effect (First edition). New York: Basic Books.
Pelevina, M., Arefiev, N., Biemann, C., & Panchenko, A. (2016). Making Sense of Word Embeddings. Proceedings of the 1st Workshop on Representation Learning for NLP, 174-183.
https://doi.org/10.18653/v1/W16-1620
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018). Deep Contextualized Word Representations. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), 2227-2237.
https://doi.org/10.18653/v1/N18-1202
Rączaszek-Leonardi, J., Nomikou, I., Rohlfing, K. J., & Deacon, T. W. (2018). Language Development From an Ecological Perspective: Ecologically Valid Ways to Abstract Symbols. Ecological Psychology, 30(1), 39-73.
https://doi.org/10.1080/10407413.2017.1410387
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language Models are Unsupervised Multitask Learners. Preprint.
Rathkopf, C. (2017). What Kind of Information is Brain Information? Topoi.
https://doi.org/10.1007/s11245-017-9512-6
Seth, A. K., Suzuki, K., & Critchley, H. D. (2012). An Interoceptive Predictive Coding Model of Conscious Presence. Frontiers in Psychology, 2.
https://doi.org/10.3389/fpsyg.2011.00395
Shannon, C. E. (1948). A Mathematical Theory of Communication. Bell System Technical Journal, 27(3), 379-423.
https://doi.org/10.1002/j.1538-7305.1948.tb01338.x
Shea, N., Godfrey-Smith, P., & Cao, R. (2017). Content in Simple Signalling Systems. The British Journal for the Philosophy of Science, 69(4), 1009–1035.
https://doi.org/10.1093/bjps/axw036
Skyrms, B. (2010). Signals: Evolution, learning, & information. Oxford ; New York: Oxford University Press.
https://doi.org/10.1093/acprof:oso/9780199580828.001.0001
Tani, J. (2017). Exploring robotic minds: Actions, symbols, and consciousness as self-organizing dynamic phenomena. Oxford ; New York: Oxford University Press.
https://doi.org/10.1093/acprof:oso/9780190281069.001.0001
Thompson, E. (n.d.). (2018). Evolving Enactivism: Basic Minds Meet Content. Notre Dame Philosophical Reviews.
Wiese, W., & Metzinger, T. (2017). Vanilla PP for Philosophers: A Primer on Predictive Processing. In T. Metzinger & W. Wiese (Eds.), Philosophy and Predictive Processing.
https://doi.org/10.7551/mitpress/9780262036993.003.0008
Yogatama, D., d’Autume, C. de M., Connor, J., Kocisky, T., Chrzanowski, M., Kong, L., … others. (2019). Learning and Evaluating General Linguistic Intelligence. ArXiv Preprint ArXiv:1901.11373.


“Avant” journal – the task financed under the contract 711/P-DUN/2019 from the funds of the Minister of Science and Higher Education for the dissemination of science.
Czasopismo „Avant” – zadanie finansowane w ramach umowy 711/P-DUN/2019 ze środków Ministra Nauki i Szkolnictwa Wyższego przeznaczonych na działalność upowszechniającą naukę.

Comments are closed.